Introduction to Machine Learning
What is Machine Learning?
Learning: the process of improve automatically with experience → Machine Learning: a computer program learn from experience with respect to some tasks, and performance measure.
Three components:
Tasks: e.g. classification, pattern recognition…
Performance measure: e.g. accuracy and error… represented as matrices
Experience: e.g. training dataset with known class labels
Design Learning System
most ML learning process follows this procedures
1. Choose training experience (E)
how well can training experience represents the test examples (when the system has to make the final decisions i.e. support business decisions)
most machine learning theories assume that training examples are identical to the distribution of the test examples, but this is rarely the reality
2. Choose representation of target function
we need to know how to represent what we want to achieve from the end results, representation of target function is closely tied with the algorithm choice
there is a tradeoff between expressibility and the amount of training data required
more expressive representation means that more training data required to choose among alternative hypothesis
e.g. simply said, linear algebra y = ax + b can be treated as a target function (which is extended to linear regression in ML), other functions can be artificial neural network (ANN) …
3. Choose learning algorithm
similar to how human learn from previous experience, target function algorithm needs constant updates while being trained, the goal is to minimize the error
two examples of learning algorithm:
minimizing sum of squared errors (SSE) is used in Regression
least mean squared (LMS) is used in Artificial Neural Network (ANN)
Learning as Search
Learning can be seen as the process of search through hypothesis space and find the best hypothesis that is consistent with the training dataset.
What is the hypothesis space?
choice of values for weights in the linear regression
can be represented as finding values to fill in matrices
How to search?
relate to concepts in Artificial Intelligence Local Search, it is the process of finding the optimal value among all hypotheses
e.g. use hill-climbing searching to find local optima
e.g. ANN use genetic algorithm which is a kind of local beam search