Introduction to AI &ML(21CS752)
MODULE 4
CHAPTER 3 - BASICS OF LEARNING THEORY
3.1 INTRODUCTION TO LEARNING AND ITS TYPES
Learning is a process by which one can acquire knowledge and construct new ideas or
concepts based on the experiences.
The standard definition of learning proposed by Tom Mitchell is that a program can
learn from E for the task T, and P improves with experience E.
There are two kinds of problems – well-posed and ill-posed. Computers can solve only
well-posed problems, as these have well-defined specifications and have the following
components inherent to it.
1. Class of learning tasks (T)
2. A measure of performance (P)
3. A source of experience (E)
Let x- input, χ-input space, Y –is the output space. Which is the set of all possible
outputs, that is yes/no,
Let D –dataset for n inputs.Consider, target function be: χ-> Y , that maps input to
output.
Objective: To pick a function, g: χ-> Y to appropriate hypothesis f.
Fig: Learning Environment
Learning model= Hypothesis set + Learning algorithm
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 1
Introduction to AI &ML(21CS752)
Classical and Adaptive ML systems.
Classic machines examine data inputs according to a predetermined set of rules, finding
patterns and relationships that can be used to generate predictions or choices. Support
vector machines, decision trees, and logistic regression are some of the most used
classical machine-learning techniques.
A class of machine learning techniques called adaptive machines, commonly referred
to as adaptive or deep learning, is created to automatically learn from data inputs
without being explicitly programmed. By learning hierarchical representations of the
input, these algorithms are able to handle more complex and unstructured data, such
as photos, videos, and natural language.
Adaptive ML is the next generation of traditional ML – the new, the improved, the better.
Even though traditional ML witnessed significant progress.
Learning Types
3.2 INTRODUCTION TO COMPUTATION LEARNING THEORY
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 2
Introduction to AI &ML(21CS752)
These questions are the basis of a field called ‘Computational Learning Theory’ or in
short
(COLT).
3.3 DESIGN OF A LEARNING SYSTEM
3.4 INTRODUCTION TO CONCEPT LEARNING
3.4.1 Representation of a Hypothesis
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 3
Introduction to AI &ML(21CS752)
3.4.2 Hypothesis Space
Hypothesis space is the set of all possible hypotheses that approximates the target
function f.
The subset of hypothesis space that is consistent with all-observed training instances is
called as Version Space.
3.4.3 Heuristic Space Search
Heuristic search is a search strategy that finds an optimized hypothesis/solution to a
problem by iteratively improving the hypothesis/solution based on a given heuristic
function or a cost measure.
3.4.4 Generalization and Specialization
Searching the Hypothesis Space
There are two ways of learning the hypothesis, consistent with all training instances
from the large hypothesis space.
1. Specialization – General to Specific learning
2. Generalization – Specific to General learning
Generalization – Specific to General Learning This learning methodology will search
through the hypothesis space for an approximate hypothesis by generalizing the most
specific hypothesis.
Specialization – General to Specific Learning This learning methodology will search
through the hypothesis space for an approximate hypothesis by specializing the most
general hypothesis.
3.4.5 Hypothesis Space Search by Find-S Algorithm
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 4
Introduction to AI &ML(21CS752)
Limitations of Find-S Algorithm
3.4.6 Version Spaces
List-Then-Eliminate Algorithm
Candidate Elimination Algorithm
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 5
Introduction to AI &ML(21CS752)
The diagrammatic representation of deriving the version space is shown below:
Deriving the Version Space
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 6
Introduction to AI &ML(21CS752)
Dr. Sudhamani M J, Professor, Dept. of CSE,RNSIT 7
