Artificial Neural Networks for Pattern Recognition

1. Artificial Neural Networksfor Pattern Recognition Jack Breese Computer Systems Quarter 4, Pd. 7

2. What is a Neural Network? • Interconnected neurons • Weights • Output

3. Uses of Neural Networks • Pattern Recognition • Face Recognition • OCR

4. Neurons • Add up each weighted input • Use an activation function to determine output • Pass on output to next layer

5. Training Neural Networks • Large input set • Outputs are verified, weights adjusted along a gradient based on these results.For each neuron in the network: For each connection to the neuron: weight = random_value() Until desired accuracy is reached: For each example in the training data: actual_out = run_network(example) exp_out = calculate_expected(example) error = exp_out – actual out For each neuron in the network: calculate_delta_weights(error) adjust_weights(neuron)

6. Program Information • Neural Network Library written in C • Currently capable of initializing a two-layer perceptron with working, weighted connections. • Capable of loading images and propagating data through the network. • Can load images up to 500x500 pixels in size.

7. Data Structure typedef struct _connection { float weight; struct _neuron * from; } connection; typedef struct _neuron { //TODO: Implement a neuron which supports connections. float d; connection * cons; }neuron; neuron* mkneuron(int c) { neuron* n = malloc(sizeof(neuron)); n->d = 0; connection * a = malloc(c*sizeof(connection));; n->cons = a; return n; }

8. New Progress • Load PGM Images • Create TrainingInfo structs • Begin Training • Perform Backpropagation

9. Training and Propagation Algos. Calculating Neuron Values For each neuron in the previous layer: Sum += neuron_weight*neuron_value neuron_value = activation_function(sum)TrainingFor each neuron in the network: For each connection to the neuron: weight = random_value() Until desired accuracy is reached: For each example in the training data: actual_out = run_network(example) exp_out = calculate_expected(example) error = exp_out – actual out For each neuron in the network: calculate_delta_weights(error) adjust_weights(neuron)‏

10. New Data Structures • TrainInfo • pImg

11. Testing • Memory Usage was tested • Training was attempted • Values for known images and random weights propagated through.

12. Problems Encountered • Initially thought memory usage was low. • Forgot to reset counter in nested for loops to 0. • That was dumb. • Corrected problem, memory usage went up • Decided to scale back network size/interconnectedness • Issues with String arrays in C • Prevented progress with training.

13. Conclusion • Works as a valid header file • Many methods • Useful for further exploration