top of page
Upgrade to Get Unlimited Access
($10 One Off Payment)

Introduction to Machine Learning

What is Machine Learning?

Learning: the process of improve automatically with experience → Machine Learning: a computer program learn from experience with respect to some tasks, and performance measure.

Three components:

  • Tasks: e.g. classification, pattern recognition…

  • Performance measure: e.g. accuracy and error… represented as matrices

  • Experience: e.g. training dataset with known class labels

Design Learning System

most ML learning process follows this procedures

1. Choose training experience (E)

  • how well can training experience represents the test examples (when the system has to make the final decisions i.e. support business decisions)

  • most machine learning theories assume that training examples are identical to the distribution of the test examples, but this is rarely the reality

2. Choose representation of target function

  • we need to know how to represent what we want to achieve from the end results, representation of target function is closely tied with the algorithm choice

  • there is a tradeoff between expressibility and the amount of training data required

  • more expressive representation means that more training data required to choose among alternative hypothesis

  • e.g. simply said, linear algebra y = ax + b can be treated as a target function (which is extended to linear regression in ML), other functions can be artificial neural network (ANN) …

3. Choose learning algorithm

  • similar to how human learn from previous experience, target function algorithm needs constant updates while being trained, the goal is to minimize the error

  • two examples of learning algorithm:

  1. minimizing sum of squared errors (SSE) is used in Regression

  2. least mean squared (LMS) is used in Artificial Neural Network (ANN)

Learning as Search

Learning can be seen as the process of search through hypothesis space and find the best hypothesis that is consistent with the training dataset.

What is the hypothesis space?

  • choice of values for weights in the linear regression

  • can be represented as finding values to fill in matrices

How to search?

  • relate to concepts in Artificial Intelligence Local Search, it is the process of finding the optimal value among all hypotheses

  • e.g. use hill-climbing searching to find local optima

  • e.g. ANN use genetic algorithm which is a kind of local beam search

Recent Posts

See All


bottom of page