stem/AI/Learning.md

Supervised

• Dataset with inputs manually annotated for desired output
  • Desired output = supervisory signal
  • Manually annotated = ground truth
    • Annotated correct categories

Split data

• Training set
• Test set Don't test on training data

Top-K Accuracy

• Whether correct answer appears in the top-k results

Confusion Matrix

Samples described by feature vector Dataset forms a matrix

Un-Supervised

• No example outputs given, learns how to categorise
• No teacher or critic

Harder

• Must identify relevant distinguishing features
• Must decide on number of categories

Reinforcement Learning

• No teacher - critic instead
• Continued interaction with the environment
• Minimise a scalar performance index

• Critic
  • Converts primary reinforcement to heuristic reinforcement
  • Both scalar inputs
• Delayed reinforcement
  • System observes temporal sequence of stimuli
  • Results in generation of heuristic reinforcement signal
• Minimise cost-to-go function
  • Expectation of cumulative cost of actions taken over sequence of steps
  • Instead of just immediate cost
  • Earlier actions may have been good
    • Identify and feedback to environment
• Closely related to dynamic programming

Difficulties

• No teacher to provide desired response
• Must solve temporal credit assignment problem
  • Need to know which actions were the good ones

Fitting

• Over-fitting
  • Classifier too specific to training set
  • Can't adequately generalise
• Under-fitting
  • Too general, not inferred enough detail
  • Learns non-discriminative or non-desired pattern

ROC

Receiver Operator Characteristic Curve