Learning

1. Learning Department of Computer Science & Engineering Indian Institute of Technology Kharagpur

2. Paradigms of learning • Supervised Learning • A situation in which both inputs and outputs are given • Often the outputs are provided by a friendly teacher. • Reinforcement Learning • The agent receives some evaluation of its action (such as a fine for stealing bananas), but is not told the correct action (such as how to buy bananas). • Unsupervised Learning • The agent can learn relationships among its percepts, and the trend with time

3. Decision Trees • A decision tree takes as input an object or situation described by a set of properties, and outputs a yes/no “decision”. Decision: Whether to wait for a table at a restaurant. • Alternate: whether there is a suitable alternative restaurant • Lounge: whether the restaurant has a lounge for waiting customers • Fri/Sat: true on Fridays and Saturdays • Hungry: whether we are hungry • Patrons: how many people are in it (None, Some, Full) • Price: the restaurant’ price range (★, ★★, ★★★) • Raining: whether it is raining outside • Reservation: whether we made a reservation • Type: the kind of restaurant (Indian, Chinese, Thai, Fastfood) • WaitEstimate: 0-10 mins, 10-30, 30-60, >60.

4. Sample Decision Tree Patrons? None Some Full No Yes WaitEstimate? >60 1-10 30-60 10-30 No Alternate? Hungry? Yes No Yes Yes Reservation? Fri/Sat? Yes Alternate? No Yes No Yes No Yes Lounge? Yes No Yes Yes Raining? No Yes No Yes No Yes No Yes

5. Decision Tree Learning Algorithm FunctionDecision-Tree-Learning( examples, attributes, default ) ifexamples is empty thenreturndefault else if all examples have the same classification then return the classification else ifattributes is empty then return Majority-Value( examples ) else best  Choose-Attribute( attributes, examples ) tree  a new decision tree with root test best for each value vi of best do examplesi { elements of examples with best = vi } subtree  Decision-Tree-Learning( examplesi, attributes – best, Majority-Value( examples )) add a branch to tree with label vi and subtree subtree end returntree

6. Neural Networks • A neural network consists of a set of nodes (neurons/units) connected by links • Each link has a numeric weight associated with it • Each unit has: • a set of input links from other units, • a set of output links to other units, • a current activation level, and • an activation function to compute the activation level in the next time step.

7. Basic definitions ai = g( ini ) aj Wj,i g ini Input Links Output Links ai S • The total weighted input is the sum of the input activations times their respective weights: • In each step, we compute: Input Function Activation Function Output

8. Learning in Single Layer Networks • If the output for a output unit is O, and the correct • output should be T, then the error is given by: • Err = T – O • The weight adjustment rule is: • Wj  Wj +  x Ij x Err • where  is a constant called the learning rate Oi Wj,i Ij • This method is unable to learn all types of functions • can learn only linearly separable functions • Used in competitive learning

9. Two-layer Feed-Forward Network Oi Output units Wj,i aj Hidden units Wk,j Ik Input units

10. Back-Propagation Learning • Erri = (Ti – Oi) is the error at the output node • The weight update rule for the link from unit j to output unit i is: Wj,i Wj,i +  x aj x Erri x g'(ini) where g' is the derivative of the activation function g. • Let i = Erri g'(ini). Then we have: Wj,i Wj,i +  x aj x i

11. Error Back-Propagation • For updating the connections between the inputs and the hidden units, we need to define a quantity analogous to the error term for the output nodes • Hidden node j is responsible for some fraction of the error i each of the output nodes to which it connects • So, the i values are divided according to the strength of the connection between the hidden node and the output node, and propagated back to provide the j values for the hidden layer • The weight update rule for weights between the inputs and the hidden layer is: Wk,j Wk,j +  x Ik x j

12. The theory • The total error is given by: • Only one of the terms in the summation over i and j depends on a particular Wj,i, so all other terms are treated as constants with respect to Wj,i • Similarly: