Institute for Advanced Management Systems Research
Department of Information Technologies
Åbo Akademi University
Neuro-fuzzy classifiers - Tutorial
Robert Fullér
Directory
• Table of Contents
• Begin Article
c 2010 [email protected]
November 29, 2010
Table of Contents
1. Fuzzy IF-THEN rules for classification
2. An adaptive-network-based fuzzy classifier
3
1. Fuzzy IF-THEN rules for classification
Conventional approaches of pattern classification involve clus-
tering training samples and associating clusters to given cate-
gories. The complexity and limitations of previous mechanisms
are largely due to the lacking of an effective way of defining
the boundaries among clusters. This problem becomes more
intractable when the number of features used for classification
increases.
On the contrary, fuzzy classification assumes the boundary be-
tween two neighboring classes as a continuous, overlapping
area within which an object has partial membership in each
class. This viewpoint not only reflects the reality of many ap-
plications in which categories have fuzzy boundaries, but also
provides a simple representation of the potentially complex par-
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 4
tition of the feature space. In brief, we use fuzzy IF-THEN
rules to describe a classifier.
Assume that K patterns xp = (xp1 , . . . , xpn ), p = 1, . . . , K
are given from two classes, where xp is an n-dimensional crisp
vector. Typical fuzzy classification rules for n = 2 are like
If xp1 is small and xp2 is very large then
xp = (xp1 , xp2 ) belongs to C1
If xp1 is large and xp2 is very small then
xp = (xp1 , xp2 ) belongs to C2
where xp1 and xp2 are the features of pattern (or object) p, small
and very large are linguistic terms characterized by appropriate
membership functions.
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 5
The firing level of a rule
<i : If xp1 is Ai and xp2 is Bi
then xp = (xp1 , xp2 ) belongs to Ci
with respect to a given object xp is interpreted as the degree of
belogness of xp to Ci .
This firing level, denoted by αi , is usually determined as
αi = min{Ai (xp1 ), A2 (xp2 )}.
As such, a fuzzy rule gives a meaningful expression of the qual-
itative aspects of human recognition.
Based on the result of pattern matching between rule antecedents
and input signals, a number of fuzzy rules are triggered in par-
allel with various values of firing strength.
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 6
Individually invoked actions are considered together with a com-
bination logic. Furthermore, we want the system to have learn-
ing ability of updating and fine-tuning itself based on newly
coming information. The task of fuzzy classification is to gen-
erate an appropriate fuzzy partition of the feature space.
In this context the word appropriate means that the number of
misclassified patterns is very small or zero.
Then the rule base should be optimized by deleting rules which
are not used. Consider a two-class classification problem shown
in Figure 1. Suppose that the fuzzy partition for each input fea-
ture consists of three linguistic terms
{small, medium, big}
which are represented by triangular membership functions.
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 7
Both initial fuzzy partitions in Figure 1 satisfy 0.5-completeness
for each input variable, and a pattern xp is classified into Class
j if there exists at least one rule for Class j in the rule base
whose firing strength with respect to xp is bigger or equal to
0.5.
So a rule is created by finding for a given input pattern xp the
combination of fuzzy sets, where each yields the highest degree
of membership for the respective input feature.
If this combination is not identical to the antecedents of an al-
ready existing rule then a new rule is created.
However, it can occur that if the fuzzy partition is not set up cor-
rectly, or if the number of linguistic terms for the input features
is not large enough, then some patterns will be misclassified.
Toc JJ II J I Back J Doc Doc I
terms for the input features is not large enough, then
some patterns will be missclassified.
Section 1: Fuzzy IF-THEN rules for classification 8
x2
B3 1
B2 1/2
B1
A1 A2 A3
1
1/2 1 x1
Figure 1: Initial fuzzy partition with 9 fuzzy subspaces and 2 misclassified
Figure
patterns. Closed and1openInitial
circles fuzzy partition
represent the given with
pattens9 from
fuzzy
Class 1
subspaces
and Class and 2 misclassified patterns. Closed and
2, respectively.
Toc
openJJ
circlesII
represent
J the given
I pattens
Back Jfrom
Doc
Class
Doc I
Section 1: Fuzzy IF-THEN rules for classification 9
The following 9 rules can be generated from the initial fuzzy
partitions shown in Figure 1:
<1 : If x1 is small and x2 is big then x = (x1 , x2 ) belongs to C1
<2 : If x1 is small and x2 is medium then x = (x1 , x2 ) belongs to C1
<3 : If x1 is small and x2 is small then x = (x1 , x2 ) belongs to C1
<4 : If x1 is big and x2 is small then x = (x1 , x2 ) belongs to C1
<5 : If x1 is big and x2 is big then x = (x1 , x2 ) belongs to C1
<6 : If x1 is medium and x2 is small then xp = (x1 , x2 ) belongs to C2
<7 : If x1 is medium and x2 is medium then xp = (x1 , x2 ) belongs to C2
<8 : If x1 is medium and x2 is big then xp = (x1 , x2 ) belongs to C2
<9 : If x1 is big and x2 is medium then xp = (x1 , x2 ) belongs to C2
where we have used the linguistic terms small for A1 and B1 ,
medium for A2 and B2 , and big for A3 and B3 .
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 10
However, the same rate of error can be reached by noticing that
if ”x1 is medium” then the pattern (x1 , x2 ) belongs to Class 2,
independently from the value of x2 , i.e. the following 7 rules
provides the same classification result
<1 : If x1 is small and x2 is big then x = (x1 , x2 ) belongs to C1
<2 : If x1 is small and x2 is medium then x = (x1 , x2 ) belongs to C1
<3 : If x1 is small and x2 is small then x = (x1 , x2 ) belongs to C1
<4 : If x1 is big and x2 is small then x = (x1 , x2 ) belongs to C1
<5 : If x1 is big and x2 is big then x = (x1 , x2 ) belongs to C1
<6 : If x1 is medium then xp = (x1 , x2 ) belongs to C2
<7 : If x1 is big and x2 is medium then xp = (x1 , x2 ) belongs to C2
As an other example, Let us consider a two-class classification
problem. In Figure 2 closed and open rectangulars represent
the given from Class 1 and Class 2, respectively. If one tries to
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 11
classify all the given patterns by fuzzy rules based on a simple
fuzzy grid, a fine fuzzy partition and (6 × 6 = 36) rules are
required.
However, it is easy to see that the patterns from Figure 3 may be
correctly classified by the following five fuzzy IF-THEN rules
<1 : If x1 is very small then Class 1,
<2 : If x1 is very large then Class 1,
<3 : If x2 is very small then Class 1,
<4 : If x2 is very large then Class 1,
<5 : If x1 is not very small and x1 is not very large
and x2 is not very small and x2 is not very large
then Class 2
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 12
gure 2 A two-dimensional classification problem
Figure 2: Fuzzy partition with 36 fuzzy subspaces.
Toc JJ II J I Back J Doc Doc I
Section 1: Fuzzy IF-THEN rules for classification 13
0.5
B6 1
R25 R45
X2
R22 R42 B2
B1 0
A1 A4 A6
0.5
0 X1 1
Figure 3 Fuzzy partition with 36 fuzzy subspaces.
Figure 3: Fuzzy partition with 36 fuzzy subspaces.
Toc However,
JJ it II
is easy toJsee thatIthe patterns
Back from
J DocFig-Doc I
Section 2: An adaptive-network-based fuzzy classifier 14
2. An adaptive-network-based fuzzy classifier
Sun and Jang propose an adaptive-network-based fuzzy clas-
sifier to solve fuzzy classification problems. Figure 4 demon-
strates this classifier architecture with two input variables x1
and x2 . The training data are categorized by two classes C1 and
C2 . Each input is represented by two linguistic terms, thus we
have four rules.
• Layer 1 The output of the node is the degree to which
the given input satisfies the linguistic label associated to
this node. Usually, we choose bell-shaped membership
functions
2
1 u − ai1
Ai (u) = exp − ,
2 bi1
Toc JJ II J I Back J Doc Doc I
Section 2: An adaptive-network-based fuzzy classifier 15
Layer 1 Layer 2 Layer 3 Layer 4
A1 T
x1 S C1
A2 T
B1 T
S C2
x2
B2 T
Figure 4 An adaptive-network-based fuzzy classifier.
Figure 4: An adaptive-network-based fuzzy classifier.
• Layer 1 The output of the node is the degree
to which the given input satisfies the linguistic
label associated to this node. Usually, we choose
Toc JJbell-shaped
II membership
J functions
I Back J Doc Doc I
Section 2: An adaptive-network-based fuzzy classifier 16
to represent the linguistic terms, where
{ai1 , ai2 , bi1 , bi2 },
is the parameter set.
As the values of these parameters change, the bell-shaped
functions vary accordingly, thus exhibiting various forms
of membership functions on linguistic labels Ai and Bi . In
fact, any continuous, such as trapezoidal and triangular-
shaped membership functions, are also quantified candi-
dates for node functions in this layer. The initial values of
the parameters are set in such a way that the membership
functions along each axis satisfy -completeness, normal-
ity and convexity. The parameters are then tuned with a
descent-type method.
• Layer 2 Each node generates a signal corresponing to the
conjuctive combination of individual degrees of match.
Toc JJ II J I Back J Doc Doc I
Section 2: An adaptive-network-based fuzzy classifier 17
All nodes in this layer is labeled by T , because we can
choose any t-norm for modeling the logical and operator.
The nodes of this layer are called rule nodes.
We take the linear combination of the firing strengths of the
rules at Layer 3 and apply a sigmoidal function at Layer 4 to
calculate the degree of belonging to a certain class. If we are
given the training set {(xk , y k ), k = 1, . . . , K} where xk refers
to the k-th input pattern and
(
k
(1, 0)T if xk belongs to Class 1
y =
(0, 1)T if xk belongs to Class 2
then the parameters of the hybrid neural net (which determine
the shape of the membership functions of the premises) can be
learned by descent-type methods.
Toc JJ II J I Back J Doc Doc I
Section 2: An adaptive-network-based fuzzy classifier 18
This architecture and learning procedure is called ANFIS (adaptive-
network-based fuzzy inference system) by Jang.
The error function for pattern k can be defined by
1 k
Ek = (o1 − y1k )2 + (ok2 − y2k )2
2
where y k is the desired output and ok is the computed output by
the hybrid neural net.
Toc JJ II J I Back J Doc Doc I
