1 / 24

IE 585

IE 585. Introduction to Neural Networks. High Cost / Low Error. Low Cost / High Error. Unarticulated Wisdom. Articulated Qualitative Models. Empirical Categorical Models. Empirical Continuous Models. Theoretic (First Principles) Models. Modeling Continuum.

wimberly
Download Presentation

IE 585

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IE 585 Introduction to Neural Networks

  2. High Cost / Low Error Low Cost / High Error Unarticulated Wisdom Articulated Qualitative Models Empirical Categorical Models Empirical Continuous Models Theoretic (First Principles) Models Modeling Continuum

  3. Rise of Empirical Models • Sensoring - lots of data • Fast computing • Computing available on site • More complicated systems - do not adhere to simple models • Easy to use software

  4. Typical Empirical Models • linear regression • splines • nearest neighbor clustering • neural networks

  5. What is a Neural Net? • An NN is a network of many simple processors (“units, neurons”), each possibly having a small amount of local memory. The units are connected by communication channels (“connections”) which usually carry numeric data, encoded by any of various means. The units operate only on their local data and on the inputs they receive via the connections. Usenet newsgroup comp.ai.neural-nets

  6. What is a Neural Net? • An NN is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: 1. Knowledge is acquired by the network through a learning process. 2. Interneuron connection strengths known as synaptic weights are used to store the knowledge. Haykin (1994)

  7. Objectives of Neural Nets • High Computing Speed • Large Memory Capacity • Adaptive Learning • Fault Tolerance

  8. Neural Network Predictive Models - Advantages • Can accommodate non-linear relationships with interactions among variables • Generalize well even for noisy and imprecise data • No assumption of analytical function or theoretic relation needed • User friendly software available • Computationally very fast, once built

  9. Neural Network Predictive Models - Disadvantages • Strongly data dependent • No statistical interpretation of significance or confidence • Difficult to build and validate properly - too many choices, too little general guidance, misleading validation results

  10. How Do Neural Networks Work? • Inspired by the biological brain • Consist of small, but numerous, parallel elements - weighted connections (synapses) and summing nodes (neurons) • “Learn” relationships through repeated calculations called “training” • Remain fixed after training to be applied to new data

  11. Biological Neuron

  12. How are Signals Transmitted?

  13. x1 w1 w2 y = f (  wixi ) x2 y . . . i wn Simple SummingNode (Neuron) xn 1 y Non-linear Transfer Function 0.5 0  w x i i Elements of Neural Networks

  14. Error Feedback During Training I N P U T S Weighted Synapses Neural Network Output Output Layer Hidden Layer Input Layer Typical Neural Network

  15. Terminology • Neurons / nodes / units / cells / processing elements (PEs) • Transfer / activation function • Connections / links / synapses • Weights / bias (fixed input of 1) • Feedforward / feedback • Input / output vectors / patterns • Self organizing (unsupervised) / supervised • Training / testing data sets

  16. Biological vs Artificial Neural Networks • Biological neurons are all excitatory (positive) or inhibitory (negative) - ANN neurons can be mixed • Biological neurons operate asynchronously - ANN neurons usually synchronize by layer • Biological neurons transmit signals at varying rates but ANN use a single rate

  17. Biological vs Artificial Neural Networks • There are many specialized biological neurons - ANN neurons tend to be generic • Biological neurons work through chemical / electrical transmission (“wet ware”) • Biological neurons are much slower but there are many, many more of them (~ 1011 neurons with 104 synapses per neuron!)

  18. Types of Neural Nets • Supervised • Unsupervised • Associate • Optimization

  19. Common Neural Net Applications • Pattern classification / recall • medical • defense • manufacturing quality • machine vision / postal • speech recognition • security detection • noise removal

  20. Common Neural Net Applications • Clustering / compression • data mining • signal processing • space exploration applications • speech recognition

  21. Common Neural Net Applications • Prediction / simulation • financial / stock market • music composition • utility usage • fault / degradation detection • sunspots

  22. Common Neural Net Applications • Control - real time / on line • robots • vehicles • manufacturing • Control - off line • batch manufacturing • process optimization

  23. Common Neural Net Applications • Optimization • traveling salesman • routing • scheduling • facility location

  24. Cool Neural Net Web Sites http://www.csse.monash.edu.au/~app/CSE5301/index.html Detailed class notes and some matlab code. http://www.geocities.com/CapeCanaveral/1624/ C source code for lots of neural nets.

More Related