04_Artificial Intelligence (1)

8/13/2019 04_Artificial Intelligence (1)

1/171

Projekt wspfinansowany ze rodkw Unii Europejskiej w ramachEuropejskiego Funduszu Spoecznego

ROZWJ POTENCJAU I OFERTY DYDAKTYCZNEJ POLITECHNIKI WROCAW SKIEJ

Wrocaw University of Technology

Advanced Informatics and Control

A. J. Koshkouei, O. C. L. Haas

THEORY AND PRACTICEOF ARTIFICIAL INTELLIGENCEFOR CONTROL

Artificial Intelligence for Control

Wrocaw 2011


2/171

Wrocaw University of Technology

Advanced Informatics and Control

A. J. Koshkouei, O. C. L. Haas

THEORY AND PRACTICEOF ARTIFICIAL INTELLIGENCE

FOR CONTROL Artificial Intelligence for Control

Wrocaw 2011


3/171

Copyright by Wrocaw University of TechnologyWrocaw 2011

Reviewer: K. Burnham

ISBN 978-83-62098-36-1

Published by PRINTPAP d, www.printpap.pl


4/171

Thi b i e f a e ie f Ma e e e e hich ha bee d ced f a gh d ei hi a c c e de ig ed f Ad a ced I f a ic a d C . The e c c e

de e e f a c ab a i be ee C e U i e i , U i ed Ki gd , a d W c aU i e i f Tech g , P a d. The e c e ec g i e he c e i f e a d e e gi gad a ced ech gie i i f a ic a d c , a d each e i a ched he ic c e ed ia i di id a a gh d e. The ce f ch f he a e ia c ai ed i each e i de i edf ec e e , hich ha e e ed e he ea , c bi ed i h i a i e e a e , hich

a e ha e bee ed b a he a h f i i a e . Whi he ce f he a e iaa be a , a e ha a be f d a e he e e ibi i f he a h .

I e ige ech i e a e a i g a i c ea i g i a e i e gi ee i g a d cie ce ha ie ed f a ecia i ed e ea ch bjec ai ea a ied e ea ch a d c e cia

d c . Thi b f c e he ic f f gic, a ificia e a e (ANN) a d gea g i h (GA ). Thi b e e a b ief he e ica i d c i each bjec bef ede c ibi g i i g e a e dea i g i h de i g a d c . M f he i i a ed

he c e f a e ac age MATLAB .

A ificia Ne a Ne (ANN ) Ne a Ne (NN) a e i i a he bi gica e i he e e f i f a i ce i g a adig a hei f c i a d c e i i a he bi gica e a e c i i g f a e f a a e i hich

c ec i e . H a a ea f he a e e a d gai e e ie ce. S ch e e ie ce aea i g i b e e ed fi d i f f e b e f i i a c e i .

ce e a e a ied c ea e a ANN. I hi ega d, ANN a e c fig ed ia a ea i g cei g a ai ab e da a b ai ed f a e e ie ce a d/ fac f a a g i h

c e b e . The abi i f b e i g f ANN , ha bee ec g i ed a e f he a d effec i e e h d hich a e ed i a i di ci i e i c di g ec ic , bi edica e ,

a e ec g i i , da a c a ifica i , e gi ee i g a d he bjec a ea . Ad a cedc a i a ech gie a d f a e ha e e ha ced he abi i f ANN ega d e f he

f da a a d hei c e i . The i e i f hi b i ide a i a g i h a he a ica f a d c a i a i e ige ce a adig f a ech i e . H e e ,

a ech i e a d a g i h a e e e ed ch ha he eade de a d headi a d a e ab e a he e e h d f i g e a i e c e b e i hei

Ma a ica i i i f a ic a d c i h a cia ed MATLAB fi e a e ided.

F gic he i ba ed f e he i d ced b Zadeh i 1965, a d ie d aa i a e i f a b e b ge e a i g a f a i , fac , gic e a aa d he a ai ab e i f a i . F gic ha a ide a ge f a ica i i a a ea i c di

i c i e ia i i a i , edica diag e a d cie ce , a a i , edic i f a e a d e gi ee i g. I h d be e ha i ed ha a e h d a be efi f b h f gic

a d ANN ech i e hich a e ab e e a c e b e .

The a ica i f ge e ic a g i h ha e bee i e e ed i g he Ge e ic A g i h bde e ed i Sheffie d b Chi e fie d, F e i g, P hei a d F eca beca e i i f ee a ai ab e d ad a d i a ia e f he e.


5/171

The b ief i e f hi b i a f :

Cha e 1 a i h a i d c i f he ba ic c ce f f gic i h a ica i ce gi ee i g i i g f e a i a d he e h d f def ifica i . The La

f he Cha e f c e f c de ig e h d i c di g Ma da i a d Ta agi S gec e .

Cha e 2 e e e a e i c di g e he e ica bac g d ed b e ea d MATLAB e a e . A be f c e a e a chi ec e a e de c ibed i c di g

i g e a d i a e e ce (MLP), Radia ba i f c i a e a Ga ia adia f c i a d ge e a i ed eg e i e a e . T ai i g a g i h ba ed bac agaa d ea a g i h a e di c ed a d de a ed i g MATLAB e a e . Fi a a e a e i ed high igh he i a ce f e i g a d eg a i a i .

Cha e 3 e e ge e ic a g i h a a e h d f i i a i a d de a e i e h gMATLAB e a e a ied i a + i eg a + de i a i e c e i g a d ide ifica i .

Thi b ha bee i e ch ha c ce a e i d ced h gh e a e . A bib i g a hided f eade i hi g gai e de ai ed i f a i a ic a a ec f he a e

e e ed i hi b .


6/171

C

..................................................................................................................................................... ii

1 F L gic a d a ica i i c ........................................................................................... 2

1.1 B ief hi ............................................................................................................................. 2

1.2 F Se a d i a c ce ............................................................................................ 2

1.3 Re e e a i f f e ................................................................................................... 6

1.3.1 Di c e e i e f e e a e .................................................................................... 6

O e a i a d ........................................................................................................ 7

1.3.3 F e a d e a i f c i ca e ............................................................. 101.3.4 O e a i a d f he c i ca e ............................................................... 11

1.3.5 C a ica a d f e a i .......................................................................................... 12

1.3.6 O e a i f e a i ........................................................................................ 12

1.4 The c e f e be hi f c i : ..................................................................... 13

1.4.1 S a d b da ie f e be hi ...................................................................... 15

1.4.2 C e f e ............................................................................................................ 16

1.4.3 The heigh f a f e ................................................................................................ 161.4.4 P jec i f a e a i ................................................................................................ 16

1.4.5 C i d ica e e i f f e ................................................................................. 17

1.4.5.1 A~ C i i f a f e i h a e a i ............................................................ 18

1.4.6 C i i f e a i ........................................................................................ 19

1.5 A i a e ea i g a d def ifica i e h d .......................................................... 20

1.5.1 Li g i ic a iab e a d hedge....................................................................................... 20

1.5.2 Li g i ic hedge ........................................................................................................... 21

1.5.3 M d e i fe e ce che e ................................................................................. 23

1.6 The Ma da i i ica i ...................................................................................................... 24

1.7 Def ifica i Me h d ....................................................................................................... 25

1.7.1 The ce e f a ea e h d (ce e f g a i ) .............................................................. 26

1.7.1.1 Di c e e ca e .............................................................................................................. 26

1.7.1.2 C i ca e ........................................................................................................ 27

1.7.2 The ce e f e h d .......................................................................................... 29

1.7.2.1 Di c e e ca e .............................................................................................................. 29


7/171

1.7.2.2 C i ca e ........................................................................................................ 30

1.7.3 The ce e f he heigh e h d .................................................................................. 30

1.8 Def ifica i e h d a d deci i a i g ce ........................................................ 31

1.9 F C e ................................................................................................................... 34

1.9.1 R e f a , i ica i a d i fe e ce ......................................................................... 35

1.10 Ta agi S ge (T S) f c e .................................................................................... 40

1.10.1 O e e ........................................................................................................ 40

1.10.2 Ta agi S ge f c : C ed e ....................................................... 41

1.10.3 Di c e e i e Ta agi S ge f e ................................................................... 42

1.10.4 N e ica e a e (a i e ed e d ) ................................................................ 431.11 F a d PID c e c fig a i .............................................................................. 46

1.12 C c i f c e ........................................................................................... 47

1.12.1 Ta agi S ge F c e h d ........................................................................... 47

1.12.2 Ma da i e h d .......................................................................................................... 48

2 Ne a Ne ............................................................................................................................ 50

2.1 Bac g d a d i i ia i a i ....................................................................................... 50

2.2 Hi f e a e .................................................................................................... 512.3 A i e a ificia e a e ......................................................................................... 51

2.3.1 Pe ce ..................................................................................................................... 55

2.3.2 T a e a d De a R e ................................................................................. 57

2.3.3 U da i g eigh f a ge e a ca e: The De a R e.................................................... 61

2.3.4 S a ........................................................................................................................ 62

2.4 Ne a e ea i g........................................................................................................ 62

2.4.1 Machi e ea i g ........................................................................................................... 63

2.4.2 Lea i g a egie ........................................................................................................ 63

2.4.3 Machi e ea i g a g i h ......................................................................................... 64

2.4.4 Te i g a e .......................................................................................................... 64

2.4.5 Acc ac ea e e f a e ........................................................................... 65

2.4.6 Li i a i f i g e a e e ce ......................................................................... 65

2.4.7 S i egi .............................................................................................................. 67

2.5 M i a e e a e ................................................................................................... 68

2.5.1 XOR b e ................................................................................................................. 72


8/171

2.6 Bac aga i Me h d ...................................................................................................... 74

2.6.1 Bac aga i a g i h ( a )......................................................................... 76

2.6.2 The e ce c e ge ce he e ......................................................................... 76

2.7 Ga ia adia ba i f c i e ................................................................................ 79

2.8 Diffe e ce be ee a RBF a d he a da d MLP ................................................................ 81

2.8.1 Ga ia adia ba i f c i (GRBF) e a e ................................................ 83

2.8.2 E i a i f he eigh a i .................................................................................... 85

2.9 Ge e a i ed Reg e i Ne a Ne (GRNN) ................................................................. 88

2.10 K ea a g i h ................................................................................................................ 89

2.10.1 I a ce f ea a g i h . .............................................................................. 892.10.2 The ea a g i h ................................................................................................ 92

2.10.2.1 Fi ea a g i h ......................................................................................... 92

2.10.2.2 Sec d ea a g i h .................................................................................... 93

2.10.2.3 The f ea a g i h ................................................................................. 94

2.11 S e i ed e ec i f ce e ............................................................................................. 95

2.12 Ne a e f c e .................................................................................... 99

2.12.1 Rec e e a e (RNN ) ............................................................................... 992.12.2 Ti e De a Ne a Ne (TDNN ) ........................................................................ 100

2.13 Ne a e a egie ................................................................................................... 101

2.14 Ne a e i g MATLAB ........................................................................................... 104

2.14.1 Ac i a i F c i .................................................................................................... 104

2.14.2 MATLAB de ......................................................................................................... 104

2.14.3 MATLAB .......................................................................................................... 105

2.14.4 NN De ig .................................................................................................................... 105

2.14.5 T ai i g A g i h ..................................................................................................... 106

2.14.6 Pa a e e O i i a i .............................................................................................. 107

2.14.7 P e a d P ce i g ............................................................................................. 107

2.14.8 Radia Ba i Ne .................................................................................................. 108

2.15 C ea i g a ge e a i ab e e a e i g MATLAB .................................................. 109

2.15.1 Ba e ia Reg a i a i............................................................................................... 110

2.16 C a ida i : Ea i g .......................................................................................... 111

2.16.1 C a ida i f RBF ANN .................................................................................... 111


9/171

2.16.2 P i ci a C e A a i ..................................................................................... 112

2.16.3 O i i i g Ne Si e ............................................................................................. 112

2.16.4 S a i ica A a i ....................................................................................................... 113

2.17 NN i C ..................................................................................................................... 113

2.18 A A M ................................................................................ 115

2.18.1 W i g i h Bac aga i e h d i g MATLAB ............................................. 115

3 Ge e ic a g i h ...................................................................................................................... 117

3.1 I d c i he i ic ech i e a d c e e ea ch ............................................... 118

3.1.1 O igi f Ge e ic a g i h ....................................................................................... 119

3.1.2 Ge e ic a g i h e i g .................................................................................. 1193.2 C cha ac e i ic f GA .......................................................................................... 121

3.2.1 Fi e f c i ........................................................................................................... 122

3.2.1.1 P i a fi e ................................................................................................. 122

3.2.1.2 Li ea ca i g ........................................................................................................... 123

3.2.1.3 Ra i g e h d ..................................................................................................... 123

3.2.2 Pa e Ra i g ............................................................................................................ 125

3.2.2.1 Pa e a i g e ec i e h d .......................................................................... 1263.3 Se ec i .............................................................................................................................. 128

3.3.1 R e e hee e ec i ............................................................................................. 129

3.3.2 Re ai de cha ic a i g i h e ace e ..................................................... 130

3.3.3 S cha ic i e a a i g...................................................................................... 130

3.4 Ge e ic ea ch e a .................................................................................................... 131

3.4.1 C e e a .................................................................................................... 131

3.4.1.1 Si g e i c e ............................................................................................. 131

3.4.1.2 M i i c e .............................................................................................. 131

3.4.1.3 U if c e ................................................................................................... 132

3.4.1.4 Sh ff e c e ..................................................................................................... 132

3.4.1.5 Red ced ga e .................................................................................................. 132

3.4.1.6 Re a he c e e a ...................................................................... 133

3.4.2 M a i e a ..................................................................................................... 133

3.4.2.1 Bi a a i ....................................................................................................... 133

3.4.2.2 Di c e e a i .................................................................................................... 134


10/171

3.4.3 Sea ch e a f ea a e GA ............................................................................. 134

3.4.3.1 Li e ec bi a i .................................................................................................. 134

3.4.3.2 I e edia e ec bi a i ................................................................................... 134

3.4.3.3 C i a i ............................................................................................... 135

3.5 GA: a he e ica e ec i e .............................................................................................. 135

3.6 W ed e a e: ................................................................................................................ 137

3.6.1 O i i a i f a ad a ic c f c i ................................................................... 137

3.7 E e ci e .............................................................................................................................. 151

3.7.1 Re i i f GA e i g a d f c i a i ........................................................... 151

3.7.2 S i g a e a i ..................................................................................................... 1533.7.3 S e ide ifica i i g ge e ic a g i h .......................................................... 154

3.7.4 T i g PID c e i h a GA .................................................................................. 156

3.8 MATLAB c de f GA a d c f c i i e e a i ............................................... 157

3.9 Bib i g a h Ge e ic a g i h a d E i a egie .............................................. 161


11/171

1

1.1 B The c ce f f e he a i d ced i 1965 b L fi Zadeh ( Zadeh, 1965. H e e , a hebegi i g, he de e e f hi idea a e . I 1972, he fi i g g f

e a e ab i hed i Ja a b T hi Te a (S ge a d, 2005). Zadeh b i hed a a eab f a g i h i 1973 (Zadeh, 1973). Af e hi da e, i e e i f he a ed

eadi g he de e e f a a g i h (Ve b gge , 1999) ge he i h hei a ica i a ide a ge f a ica i d ai . Ea a ica i i c ded c f a ea e gi e

e (Ma da i, 1974), a e e e f a a ica e a a i (Ha Zi e a , 1977),c f a ce e i e (S id h e a , 1980), a e ea e c e cche ica i jec i , c i g f he b a Se dai a a i e (Ja e , 2007), che

a bac ga g a (Ha Be i e , 1999). Si ce hi da e, a be f g a a d a g i hca ab e f bea i g d c a h a a e ha e bee de e ed f b h che a d bac ga .

The fi f gic chi a de e ed b Ma a i T gai a d Hi e Wa a abe a AT & T BeLab a ie (USA) i 1985 (Ha Zi e a , 1993). B 1987 a f gic a ica ii c di g c ai e c a c , e e ca a i , de i g b a d a a ed ai c af

a di g, had bee i d ced.

T gai I f aL gic I c. a he fi f c a e ab i hed i I i e (USA), 1987. I 1989 heLab a f I e a i a F E gi ee i g Re ea ch (LIFE) i Ja a a e ab i hed. I 1990F L gic S e I i e (FLSI), ed b P fe Ta e hi Ya a a a, de e ed he f echi i BiCMOS (bi a c e e a e a ide e ic d c ) ech g hich faci i a eha d i e cha ac e ec g i i i hi e ic ec d i g a i g e f e chi a d cha chi i CMOS ech g i (1991) a d (1992), e ec i e . I 1991, he I e ige S eC Lab a a c ea ed i Sie e (Ge a ) a d a e a he F A ificia I eP i Ce e i Ja a . Si ce 1992 a e e , i e i a d a ica i c ce i g f

gic a d i a ica i ha e a ea ed.

Ha i g e e ed a b ief i d c i he igi a d e de e e i f gic, he ai de f he cha e i c ed a f . Sec i 1.2 i d ce he c ce f f

i a ed i h e a e de ai i g h a f e ca be c c ed bef e de c ibi g cf e a i ch a i , a a d e a i a e. The i a c ce ch a e bef c i a d f gic i fe e ce a d i ica i a e de c ibed i Sec i 1.3. The c ce def ifica i a d he e h d f def ifica i a e e e ed i Sec i 1.4. Fi a fc e a e i d ced a d he diffe e ce be ee he Ma da i a d Ta agi a d S gec e a e i a ed. Th gh Sec i 1 e a e a e ed i a e he i c

a de c ibed.

1.2 I hi ec i he c ce f he f e a d hei a ica i a e i d ced b i g ie a e . The e be hi f c i a d e a ed c ce ch a c e , a d c e adefi ed.


12/171

C ea , he c a f a ea be hich a e ch g ea e ha 1, he c a fbea if e , " he c a f a e ," d c i e c a e e i he a

a he a ica e e f he e e . (Zadeh, 1965])Zadeh C ai

A c i e i a e defi ed c a f bjec e e e ch ha a defi i i e i a be gi ee e e a be defi ed ba ed e . F e a e he a e f e f he da f he eeca be defi ed a a e , = S da , M da , T e da , F ida , Sa da

he e

i a bi a i ch e c ai fi e e e e . The de f he e e e i i a

he ef e he e a be i e a = T e da , M da , S da , F ida , Sa daA he ib e da f a ee c i e a e hich i e ed he i e e f di c e, i

i e e i e a e . The i e a e defi i g he da f he ee i he ef e:

= S da , M da , T e da , Wed e da , Th da , F ida , Sa da

A e a e a b e f he i e a e . The i e a e i he a ge e ha he a ciae e e be g . The i e a e de e d he a e f he e e e a d i i c de a

e e e bjec ha a e c ide ed.F e a e, a i g ha B i he e f a a be e ha 10:

B= 1, 2, 3, 4, 5, 6, 7, 8, 9

The he i e a e i he e f a a a be .

A e ha i a e a d defi e a f c i C ch ha :

( ) 1C x = if x i a e e e fC a d

( ) 0C x = if x i a e be fC

I f haC i a f c i f he i e a eU i { }1,0

{ }: 0,1C U

a d he e i defi ed a :

{ }: ( ) 1C C x U x= =

=C x

C x xC if 0

if 1)(


13/171

i hich C i ca ed he e be hi f c i . I hi ca e e e e e e f he i e a e iei he i i .

F e a e, f he e f da f he ee , , (Monday) 1 A = a i i e e i he e A h e e

(Wednesday ) 0 A = beca e i i e e .

Gi e he eB, (5) 1 B = a d (15) 0 B = , he a ge f he e be hi f c i ca he bedefi ed a he e{ }1,0 .

N c ide he c ce ch a ea ee e d be ea 6 . I hi ca e, e be hi f he e i i c ea ed a d he a ge f he e i { }0,1 . I fac , he a ge f he e

i c de a e i he i e a [0, 1],

he e he e be hi f c i i i hi he i e a 10 A , i h he a e i he i e a [0, 1]de e di g he defi i i f he e be hi f c i .

E 1.2.1 (D ): The f i g a e e i ab he da f a ee a de be hi a e g he c d be a cia ed i h:

I T e da a ee e d da ? The a e i 0 beca e i i a fa e a e e .

I F ida a ee e d da ? I i a a c ec , h e e , gi e ha i i ea he ee e d,i.e. he a e i e . S he a e h d be 0, f e a e i i c ide ed a 0.

I Sa da a ee e d da ? I i a e a e e , beca e i i he ee e d. S hee be hi a e ca be 1.

I S da a ee e d da ? I ca be c ide ed a a e a e e , h e e , i i a c e M da hich i a i g da . B c a Sa da , i i f ed b a ee e

da a d he e be hi a e c d be e ha e, a 0.95.

I M da , T e da Wed e da a ee e d da ? The a e a d e e if M da i c e he ee e d i a he begi i g f he i g ee , e a e e fa f he

e ee e d. The e be hi a e f each f he e e e e c d be e 0.

I Th da a ee e d da ? I i , h e e , i i ea i g he ee e d. Gi e ha i i a c e he ee e d a F ida , a e be hi a e f 0.2 c d be e ec ed.

1.2.2 ( ):

The e a e f ea a d he ea he f each ea d e a cha ge f he idd e f aea . The ef e, he ica ea he ha de c ibe a ea i a d he idd e f h

ea . The ea he he g ad a cha ge f e ec g i ed ea he a e he e eThe a a defi i i f he e be hi f c i f he ea a e h i Fig e 1.2.1. Each

[ ]: 0,1 A U


14/171

ea ca he ef e be e ee he eighb i g i e

F

1.2.3 ( :

I i a da d c a if he e a a ed i he f i g g

a ed ha he a i ifeach f he af e e i ed g

ea d e i idd e age, aBa ed he e a i , h

g, idd e age, d ae be hi f c i e e ee cei ed. F e a e, idd e

d be a ed b ha e h

F

ed b a i e a e hich ea a he idd ea e i dica i g he he adjace ea ea .

1.2.1

):

a i acc di g i age. Le a e ha a: e g, g, idd e age, d,

f a a ic a a i i 100 ea . Ba ed a e be hi f c i a be defi ed. I i

e b bab i e g a d a 100 ea e e be hi f c i f he e g,

e d a iab e a e defi ed, ee Fig e 1. ec ed cha ac e i e h he age f a eage d ha e a fai ide ba e he ea

e h d .

1.2.2: .

f each ea i h

a i c d bed e d. I i he defi i i f

a ed ha a 50d e i e d.g, d,

.1.The ha e f he i a a i i g e d


15/171

E 1.2.4 ( c i f c i f i e. Th

e ecifica i . The ee e h e h d a cha ge ia e ge e ic a ach ca be ae ec ed be ia g a ef e

F 1.2.3:

1.3 Le be a i e a e a d A

ai defi e a : {( , ( ) : A A x x =%

i hich f a x U A , ( A x b U a d [ ]0, 1 , e ec i e . Ii ca ed e be hi deg ee f

1.3.1

E 1.3.1:

Le { }0, 1, 2, 3, 4, 5 ,6U = be

{(0, A =%

i hich (0) 0.1, A = (1) A (6) 0. A =

The e be hi f c i ca b

x 0

0.1 ( ) A%

): The e e a e e e f ae a i a d i i e e a e e

e e e ca be c ide ed be , ah he e c ide ed. H e e , f a fd ed. I hi e a e, he ha e f he ec he i ea cha ge be ee e e e e e

U . A f e de ed A% he i e a e

} x U

) 0= . The d ai a d a ge f he e be hif ha : [ ]: 0,1 A U he e 0 1 A . F

x .

he i e a e a d A U = . Defi e a f e

}.1),(1,0.3),(2,0.4),(3,1),(4,0.7),(5,0.5),(6,0)

0.3 ,= (2) 0.4 , A = (3) 1 , A = (4) 0.7 , A =

i e a

1 2 3 4 5

0.3 0.4 1 0.7 0.5

c e i ae de e d he

high. The e egic e ec i e hebe hi f c i ae Fig e 1.2.3.

.

i a e f de ed

f c i a e gi e

each x U , ( ) A x

A U = a f :

(5) 0.5 A = a d


16/171

Zadeh ed he f i g a i e e e he ab e e be hi f c i :

0.1 0.3 0.4 1 0.7 0.5 00 1 2 3 4 5 6 A

= + + + + + + %

F : Le a d A be he e a d he c e e f he e , e ec i e , he

F a

F a

If X Y he

F a

E 2.3.2:

Le { }4,3,2,1,0,1,2 =U be he i e a e , { }1,0,1 X = a d { }1,0,2Y = .

Defi e he f i g e c e di g he c i e X a d Y a

0 0.3 0.5 0.7 0 0 0

2 1 0 1 2 3 40 0.1 0.6 0 1 0 02 1 0 1 2 3 4

X

Y

= + + + + + + = + + + + + +

%

%

The e f e a be i e a

0.3 0.5 0.71 0 1

X = + +

%

0.1 0.6 11 0 2

Y = + +

%

N e ha he e e e i h 0 e be hi deg ee a be i c ded. I fac a e e e f he

i e a e hich d a ea i he e e e a i ha e 0 e be hi deg ee.

1.3.2

The e a e c defi i i f a d. Le a d B be c i e , a d,( ) A x a d( ) B x a e hei e be hi deg ee, e ec i e . The he e be hi deg ee he e A B i

defi ed a

)(1)( , x xU x A A =

1)( , = xU x U

(x) (x) Y X

, ( ) 0 x U x =


17/171

{ } ( ) min ( ), ( ) A B A B x x x =

( ) ( ) ( ) A B A B x x x = a d he e be hi deg ee he e A B i defi ed a

I f gic, he defi i i f a d i g he i a d a e a i a e a ed a da e he ef e ad ed f hi b .

The e a e c defi i i f a d. Le A a d B be c i e , a d,( ) A x a d( ) B x a e hei e be hi deg ee, e ec i e . The

0 0.3 0.5 0.7 0 0 02 1 0 1 2 3 4

0 0.2 0.6 0 0 0 .9 12 1 0 1 2 3 4

A

B

= + + + + + + = + + + + + +

%

%

.

2.4.1 :

C ide he i e a e{ }2, 1, 0, 1, 2, 3, 4U = a d he b e { }A -1, 0, 1= a d

{ } B 1, 0, 3, 4= . Defi e he f e A a d B a f :

+++++

+

=

+++++

+

=

41

39.0

20

10

06.0

12.0

20~

40

30

20

17.0

05.0

13.0

20~

B

A

The e be hi deg ee f A B% %

( A B% %

) i gi e b he a i ( i i ) be ee hee be hi deg ee A% a d B% .

0.3 0.6 0.7 0.9 1 0.2 0.5,

1 0 1 3 4 1 0 A B A B = + + + + = + % %% %

A B% % i b ai ed b a i g he a i deg ee f e be hi be ee A%a d B% f eache e e f he i e a e . The e e e f he i e a e a e e e e ed b hede i a f A B% % he ea he deg ee f e be hi i i he e a . The e be hideg ee f a he e e e ha d be g B i e . F e a e 2 a d 2 d be g

ei he A B hei e be hi i e . N e ha he he e be hi deg ee i e , he i i

)()()()()( x x x x x B A B A B A +=

{ })(),(max)( x x x B A B A =


18/171

ece a i c de i i he e A B% % . Thi i e i a e a i g ha he e be hi deg eef e e e e e i he e i e .

Si i a B% % i b ai ed b a i g he i i be ee he e A%a d B% . M e e e a ee a d he ce d eed be i c ded i he e i g e .

2.4: 2 :

Le { }2, 1, 0, 1, 2, 3, 4U = be he i e a e a d

be the fuzzy e he c i e

{ } { } { }A -1, 0, 1 , B 1, 0, 3, 4 , 2, 0, 2 ,C = = =

e ec i e . The ( ) B C % %% i gi e b he i i be ee B% a d C% f ed b hea i be ee he e a d A%. Si i a ( ) A B C % %% i gi e b he a i be ee A%

a d B% f ed b he i i be ee he e a dC%

0.3 0.6 0.7( )

1 0 1

0.6( )

0

A B C

A B C

= + +

=

% %%

% %%

N e ha e a i a d a e bi a . I addi i , he e a i a f c i e a bea id f f e a i i c di g a d e a i .

N he di ib i a d a cia i e a a e e a i ed f he ab e f e . F he abe a e, he c e e f A i defi ed a { }4,3,2,2== AU A a d i ce he f

e be hi deg ee i defi ed a )(1)( x x A A = , he ef e,

1 0.7 0.5 0.3 1 1 1 2 1 0 1 2 3 4

A = + + + + + + %

0 0.3 0.5 0.7 0 0 0,

2 1 0 1 2 3 40 0.2 0.6 0 0 0.9 1

,2 1 0 1 2 3 4

0.4 0 0.8 0 1 0 02 1 0 1 2 3 4

A

B

C

= + + + + + + = + + + + + + = + + + + + +

%

%

%


19/171

O he he ha d, acc di g f i g i b ai ed

A%

A%

N e ha acc di g a da d, A A A A U = =

Thi ea ha he a da d

1.3.3

Le X U he e he i e aF e a e he i e a e hf e X i defi ed a

{ X =%

2.5.1 :

A e ha { : 0 1 X x x= ad ed he he f e X

X = %

F

he defi i i f e a i a d

1 0.7 0.5 0.7 1 1 12 1 0 1 2 3 4

A = + + + + + + %

0 0.3 0.5 0.3 0 0 02 1 0 1 2 3 4

A = + + + + + +

%

e he he f i g e d be b ai

e f e he a e a id f f e .

e U i a b e f he ea be i h cd be he i f i e a a a b e f ea

( ) } ( ) , ( ) : : X X x

x x x X x X

x

=

%%

} 2 a d a Ga ia e be hi f c i i ai gi e b :

( )26

2x x, ( ) : x and ( )

x

x x X x e = % %

1.3.1: G .

, he

d

ab e e e e .be . The he

ed i Fig e 1.3.1 i


20/171

1.3.4

2.5.2:

Le { }: 0 12 A x x= a d B

Defi e he f e a bi a i

A = %

B = %

U i g he f e A% a d B%, a

e ec i e , a d hei ace aThe e f he ab e e (

e g h f he e .

F 1.3.2:

( ) A x % , ( ) B %

{ }: 0 6 y y .

e ec ed a Ga ia e be hi f c i :

( )26

2A A, ( ) : x and ( )

x

x x A x e = % %

( )

2

2

B B, ( ) : y and ( )

y

y y B x e = % %

he e be hi deg ee f he e f e

{ } ( ) ( ) max ( ), ( ) A B BC A x x x x = =% % %

{ }( ) ( ) min ( ), ( ) A B D B A x x x x = =% % %

h i Fig e 1.3.2. e a i ), i.e. ( )C x % a d ( ( ) D x % , a e a c

A~ B%

a d D% a e defi ed

a ed he fi i g

.


21/171

1.3.5

Le X a d Y be e . The he c a ica e a i f X i Y i defi ed a f :

2.6.1:

A e ha { }1, 2, 3, 4 X = a d { } 1, 2, 3Y = . The

{ }(1,1), (1,2), (1, 3), (2,1), (2,2), (2, 3), (3,1), (3,2), (3,3), (4,1), (4,2), (4,3) X Y =

N defi e a f e be hi deg ee he e a i Y X i g he f i g a

1 if x-y 0

0.7 if x-y 1( , )

0.2 if x-y 2

0 otherwise

R x y

==

= =

Whe e x-y de e he ab e a e a d e e e he di a ce be ee a d . F e a e

a i g =4 i hi X a d =1 i hi Y, hex-y 3= . Ba ed he e be hi f c i defi i i

( , ) R x y gi e ha x-y i e a 0, 1 2 he(4,1) 0 R = , de ed i he e R a0

(4,1) .The ef e, he f e a i c c ed f he c i e a iY X i

1 1 1 0.7 0.7 0.7 0.7 0.2 0.2 0.2 0 0.7

(1,1) (2, 2) (3,3) (1, 2) (2,1) (3, 2) (2,3) (1,3) (3,1) (4, 2) (4,1) (4,3)

R = + + + + + + + + + + +

Thi f e a i ca be h i he f f a a a

1 2 31 1 0.7 0.2

2 0.7 1 0.7:

3 0.2 0.7 1

4 0 0.2 0.7

Y

R X

=

1.3.6

O e a i a d ca a be defi ed f f e a i . U i g he e a i , ee a i a e b ai ed b c ide i g he a i he i i f he c e di g e e e

f he e a i 1 R a d 2 R . Le 1 R a d 2 R be f e a i he he e e e f he e a i

( ){ }, : , X Y x y x X y Y =


22/171

21 R R a e b ai ed f he i i f he e ie hich a e he a e c a d

. Si i a he e e e f 21 R R a e ca c a ed b a i g he a i f c e di g

e ie d. F e a e, ifija a d ijb a e he e e e f1 R a d 2 R he h a d h c ,

e ec i e he he h a d h c f he f e a i 21 R R a d 21 R R a e

{ }min ,ij ija b a d { }max ij ija , b , e ec i e .

E 2.6.1:

C ide he f i g f e a i

==

7.06.03.0

3.026.066.0

165.027.075.04.055.0

:

y y

,

9.02.01

16.02.0

4.045.07.025.07.03.0

:

y y

4

3

2

1

2

321

4

3

2

1

1

321

x

x

x x

R

y

x

x

x x

R

y

The he f e b ai ed i g he e a i a d a e

1 2 3 1 2 3

1 1

2 21 2 1 2

3 3

4 4

y y y y

0.3 0.4 0.25 0.55 0.7 0.75

0.27 0.45 0.4 0.7 0.65 1: , =:0.2 0.26 0.3 0.66 0.6 1

0.3 0.2 0.7 1 0.6 0.9

y y

x x

x x R R R R x x

x x

=

1.4 :The e a e a c e f f c i hich a e ed a e be hi f c i . Thi b ide c ibe ia g a , a e ida a d Ga ia e be hi f c i .

:

Thi e f e be hi f c i i ge e a a h i Fig e 1.4.1. The a he a ica e a i de c ibe he e be hi f c i a h i Fig e 1.4.1 i :

1

11 2

2 1

32 3

3 2

3

0

( )

0

A

x a

x aa x a

a a x

a xa x a

a a

x a

< 0.

2. Se ec a ai f ai i g a e i , a ge ( a d ) i feedf a d di ec i a dc e he a .

3. Se ec he ea i g a e a d a d ca c a e

4. Ca c a e he e eigh

kj kj kj

kj kj kj

v v v

w w w

= + = +

%

%

5. G e 2, i.e. e ec he e ai f ai i g a e i , a ge a d fi d a e. Re ea hi c c e f i e i a ai i g a e i a d a e ed.I hi e e c c e (e ch) i c e ed.

6. Ca c a e he a e ( )2

1

12

m

j j j

E t y=

= . If E i fficie a , i.e. E < he e 0 > i a de i ed a a e. The c e eigh e he e a d he e a e

ai i g ce i c e e. O he i e, e he fi a eigh b ai ed f e 5 af ec e i f he a e ch a i i ia a e a d e ea e 2 6.

A ai i g c c e e ch i c e ed each i e he i i ia eigh a d he a ai i g a ei a d a e ed. I addi i , i i a a ib e b ai he de i ed e

e ai i g c c e. Se e a ai i g c c e , he ca ed e ch , a be e i ed b aif a ia e eigh a d he ef e he c ec .

2.6.2

If he i a e i ea e a ab e, he e ce i fi d a deci i b da hich c ecdi ide he i .

E 2.6.1: A e ha he ac i a i f c i i he gi ic f c i

( ) ( )( ) ( ) ( )

kj k k k j

i i j k k k kjk

v t y f u h

w x f z t y f u v

= =


86/171

1( )

1 x f x

e=

+

The (1 ( )) ( )df f x f xdx

= a d he da ed a e eigh a e b ai ed f

( ) ( )(1 ( ))kj kj kj

kj k k k k j

v v v

v t y f u f u h

= + = %

a d he da ed hidde a e eigh a e

2 ( )(1 ( )) ( ) ( )(1 ( ))

ji ji ji

ji i j j k k k k kjk

w w w

w x f z f z t y f u f u v

= + = % %

The e da i g ce e a e c e ed he a ai f ai i g a e i a d a ge a

ed a d he a e ( )2

1

12

m

j j j

E t y=

= i fficie a , i.e. i a e ea i i i a e.

The f i g e a e i a e h he bac aga i e h d i a ied a d e e heeffec he be f e a d he be f e ch ha i g f he e .

3.15.2 : U i g he bac aga i e h d f he i

= 0.2 0.6 1.7 1.60.9 0.3 1.2 020.2 0.1 0.4 2 a d he a ge = [0.1 0.3 0.6 0.2], he fi a a ia e eigh a e b ai ed af e 187e ch ,ee Fig e 2.6.1.

For 10 hidden nodes,187training cycles are required toobtain the desired error.

0 20 40 60 80 100 120 140 160 180 2000.1

0.11

0.12

0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2Backpropagation Training

The number of epochs

S u m o f

S q u a r e d

E r r o r

F 2.6.1: .


87/171

The be f hidde de a d he be f e ch f hi e a e a e gi e i Tab e 2.6.1 a di h hich he de a e e ec ed, e e ch a e e i ed f he e ai i g.Fig e 2.6.2 de ic he ce f a e a e a g i h .

The be f hidde de a d he be f e ch f hi e a e a e gi e i Tab e 2.6.1. Ih ha he fe a de a e c ide ed e e ch a e e i ed each he

de i ed a ge .

N be f hidde e 1 2 3 4 5 6 7 8 9

The be f e ch e i ed

each he de i ed i g

188 170 165 166 169 172 180 183 186

2.6.1: .

N e c ea i g a MATLAB f c i i g he bac aga i a g i h

T defi e a d ai a MLP i g he BP a g i h , ac i a i f c i h d be fi ecified. Thebe f ac i a i f c i de e d he be f a e . The ac i a i f c i a

a gi ic f c i ( ig id f c i , MATLAB c de ). Thi i i he e he i e a [0 1].

Si ce b e ha e a ge ide f hi a ge, a i ea f c i i e ec ed a he ac i a( ai i g) f c i f he a e . A i ea a e a a ge f a ag i d

eached. A bac aga i MATLAB fi e i gi e i A e di A, a ed BP .. .


88/171

Collect Data

Select Trainingand Test Sets

Select Neural Network Architecture

Initialize Weights

SSE GoalMet?

Y

SSE GoalMet?

Y

N

Run Test Set

Done

Reselect TrainingSet or CollectMore Data

Change Weightsor

Increase NN Size

N

F 2.6.2: Overview of neural network training method

2.7

The adia ba i f c i (RBF) e i a i a e a e ha ha bee ide The ac i a i f c i f he hidde a e i a a Ga ia f c i , i ch ca e, he ei e ed a Ga ia RBF (GRBF).

The cha ac e i ic f a RBF e ca be a i ed a f :

1. a a e e hich ha diffe e e f e i he hidde a e a d he a e

2. The hidde a e hich c e d a MLP hidde a e i a i ea a i g.3. The hidde a e c ai adia ba i f c i e .

4. The ac i a i f c i i a a Ga ia f c i .

5. The ac i a i f c i a e ce ed e a ea i he i .

The Ga ia ac i a i f c i f a RBF Ne i

2( )

2

( )

x c j

j j g x e

=

NN ed a d ai i g c e ed


89/171

he e i he i ec , i

id h f he ece i e fie d. N

2.7.1 h a Ga ia eai ed i a e i ed a e .

N e ha he i f a GRBF de e i e he a ge f he gi

A RBF e ha he f i

1. The a e i a a

2. T ai i g a g i h a e

3. AS MLP, he a e

4. If a i ec x iee i be ac i a ed

5. If a i ec ie bid h () he he hidd

6. Whe he i eca d he RBF i e

The c ce f ca a d g ba e id h a d a ge

RBF a d he a da d MLP i

c

he ce e f a egi efe ed a he ece i

e ha ( ) j g x i he f he

h e

i h he ce e a 0 a d he id h 1. A RBF i a

F 2.7.1: A G .

c c a e , he ca ed a ece i e da a.

g fea e :

f a da d i ea e .

ba ed a i e a i e e h d.

a , a , c ai bia e .

ea he ce e f a ece i e fie d (), he. S a i ab e ce e be ide ified f each

e ee ece i e fie d ce e , b i ide e a e b h a ia ac i a ed.

ie fa f a ece i e fie d he e i hia he a e bia a e .

e i be e ai ed a d he effe id h i be di c ed. I addi i , he di

be i e iga ed. N e ha a RBF e f a

e fie d a d j i he

f he GRBF. Fig e

ca e ha i

fie d, hich

he RBF hidde a ee .

hei ece i e fie d

de a e ac i a i

c f e i hffe e ce be ee a

ca a i g if


90/171


91/171

N e ha a i ie i hi a ece i e fie d a d ha e a e e a i h he ecefie d ; he id h eed be a ea e a he di a ce be ee i f c a e g a a

ha each i i e be f a ea e c a (i.e. ece i e fie d).B ! N . a d B ! N . h

he de ed a d ac a i e a i hi f e .B ! N . h i a ed ha i g a RBF i h a a ge id h f 20, hich i c de ai , gi e be e fi be ee he da a a d he de he ea he a RBF id h a e ed he

e i ab e edic he acc a e . The ef e, he e ec i f a id h fe i a i a i e a d h d be e ec ed ch ha i c e f a i ie i hece i e fie d. B h a ge id h e ha fficie a id h e ha a a ia e

bad ed e i h g a a ee ha he i f ee da a.

F 2.8.1: RBF 20.

-15 -10 -5 0 5 10 15-250

-200

-150

-100

-50

0

50

100Testing the RBF Network

Input

O u t p u t width( )=20

Ac a i

E i a e f he i


92/171

F 2.8.2: RBF 5.

2.8.1 ( B )

The GRBF i a a e e a e i hich he ac i a i f c i f he hidde a e f e i Ga ia . The e a i hi be ee he i a d f a RBF e i h

Fig e 2.8.3 i hich 1 2, , , M x x xK a e i ,1 2, , , M c c cL a e he ce e a d x c i dica e hedi a ce be ee he i a d he ce e he e

( ) ( ) ( )

1 1

2 2 22 21 1 2 2

2

2 2, c , , M M

M M

x c x c

x c v x x c x c x c x c y e e

x c

= = = + + = =

LM M

Fig e 2.8.4 de ic he GRBF e diag a i hich ( , ), 1, , ,i i i i g g r i = = L a e heGa ia e f he hidde a e . The f he GRBF a e

1 1 1 1 11 2 2 2 12 1 1 1

2 1 1 1 21 2 2 2 22 2 2 2

1 1 1 1 2 2 2 2

1 1 1 1

( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , )

( , )

j j j j

j j j j

k k k j j j kj k k

P P

y g r w g r w g r w g r w b



y g r w

= + + + + + += + + + + + +

= + + + + + +

=

L L

L L

ML L

M

2 2 2 2( , ) ( , ) ( , ) P j j j Pj P p g r w g r w g r w b + + + + + +L L

hich ca be e e e ed i c ac f a :

( )

j=N

j=1 ( , ) , 1k j j j kj k y g r w b k p = +

-15 -10 -5 0 5 10 15-150

-100

-50

0

50

100Testing the RBF Network

Input

O u t p u t

width( )=5

Ac a i

E i a e f he i


93/171

i a i f :

[ 1 1 1 2 2 2( , ) ( , ) g r g r

The eigh be dified he eigh a i f he f

F 2.8.3

F

[ 1 1 1 2 2 2( , ) ( , ) g r g r

i hich [ ]1 2, P t t t L a e he

] [

11 21 1

12 22 2

1 2

1 2

1 2

( , ) 1

P

P

P

p

w w w

w w w

g r y yw w w

b b b

=

LL

M M O MLLL

ch ha he f he e bec e he i g e a i :

: I GRBF

2.8.4: A GRBF .

] [

11 21 1

12 22 2

1 2

1 2

1 2

( , ) 1

P

P

P

p

w w w

w w w

g r t t w w w

b b b

=

LL

M M O MLLL

a ge . The e de (000) ca e e e

] P yL

a ge . The ef e,

.

] P t L

(2.8)

a


94/171

11 21 11 11 1 2 12 2 1

12 22 21 21 1 2 22 2 2

1 21 1 1 2 2 2

1 2

( , ) ( , ) ( , ) 1

( , ) ( , ) ( , ) 1

( , ) ( , ) ( , ) 1

P

P

P Q Q Q

pG

W

w w w g r g r g r

w w w g r g r g r

w w w g r g r g r

b b b

LL

LL

M M O MM M O M ML

LL144444444424444444443

14442 {

1

2

T

T

T Q

T

t

t

t

=

M

4 44443

i he i ified f GW T = he e . N e ha , i ge e a 1+ Q .2.8.2

N i i e i ed fi d he eigh a iW ch haGW T = . A e ha he a e a iTG G i i e ib e. Si ceT T G GGW T = he

( ) ( )-1 -1T T T TG G G GG GW G T =

hich i ie ha ( )-1T TG GW G T = . N a e haW i a e i a e fW . I e i ed i g

ha = GW T i e . T hi e d, c ide

( ) ( )( )( )

2 T = =

=

=

T

T T T

T T T T T T

GW T GW T

W G T GW T

W G GW T GW W G T T T

+

N he de i a i e2 i h e ecW i ca c a ed

( )2 T

= = 2

=2 2

T T T T

T T

G GW T G G T W W

G GW G T

T b ai he i i , e a e e

2 T= = 2 2 0

T T G GW G T

W W =

hich ie d

( )1 T T W G G G T

=

N e ha1 2

( ) ( ) ( ) ( )=

W W W W

L .

E 3.20.1:


95/171

Using MATLAB, ca c a e he

A e : See he e d f E a e

OR P

The GRBF e diag a f X

F 2.8

The e i b ai ed

i hich

he e 1c a d 2c a e he ce eSe ec f he f i a

id h, , i b ai ed f

igh a i ( )1 T T W G G G T

= f he XOR b

3.20.2.

GRBF

OR i e e ed iB ! N

.5: GRBF OR.

he f i g e a i :

( )

( )

21

21

1 1 1 1 1 1 2

22

22

2 2 2 2 2 1 2

, , [ ]

, , [ ]

r

g r e r c u u

r

g r e r c u u

= =

= =

. he e ia ce e , a ,[ ]1 1 1c = a d 2c

maxd

C =

( ) ( )1 1 1 1 2 2 2 2, , y w g r w g r b = + +

.

. .

[ ]0 0= . A i ab e


96/171

he e C he be f he ce e a daxd i he a i di a ce be ee he e ec edce e Si ce he be f he ce e i 2C = a d he a i di a ce be ee ce e i

2 2max (0 1) (0 1) 2d = + = . The ef e,

max 2= 12

d

C = =

A a i ie hich a e ed f b ai i g he eigh a e gi e i 2.2.

1 x 2 x 1r ( )1 1 1, g r 2r ( )2 2 2, g r t

0 0 2 0.1353 0 1 0

0 1 1 0.3678 1 0.3678 1

1 0 1 0.3678 1 0.3678 1

1 2 0 1 2 0.1353 0

2.2: GRBF OR.

S b i i g he gi e a e i ab e i ( ) ( )1 1 1 1 2 2 2 2, , y w g r w g r b = + + ie d

1 2

1 2

1 2

1 2

0.1353 0

0.3678 0.3678 1

0.3678 0.3678 1

0.1353 0

w w b

w w b

w w b

w w b

+ + =+ + =+ + =

+ + =

i he a i f

1

2

0.1353 1 1 0

0.3678 0.3678 1 1

0.3678 0.3678 1 1

1 0.1353 1 0

ww

b

=

Which ca e e e ed i he ed ced f

1

2

0.1353 1 1 0

0.3678 0.3678 1 1

1 0.1353 1 0

w

w

b

=

The i a e


97/171

The ef e he e i

i Tab e 3.20.1. N e ha , he

The ef e

( )1 T T W G G G T

= =

2.9 A GRNN ha h ee a e i c diA e ha he ac i a i f c2.9.1.

F 2.9.1: G

The f a GRNN i gi e b

1

2

2.5027

2.5027

2.8413

w

W w

b

= =

2 2

1 22.5027 2.5027 2.8413r r y e e = + i hic

eigh ca be f d i g ( )1 T T W G G G T

= h

0.1353 1 1

0.3678 0.3678 1

0.3678 0.3678 1

1 0.1353 1

G

=

00.6727 1.2509 1.2509 1.8292

1 1.8292 1.2509 1.2509 0.6727

1 0.9202 1.4202 1.4202 0.9202

0

=

( )

g hidde a e , a i a d a e , i f he hidde a e i Ga ia he e

R N N

2

2

2

2

11

e

e

n n

n

j j

j

x c

nn x c

j

j j

vv

v

==

= =

1r a d 2r a e gi e

e

2.5019

2.5019

2.8404

e .Fig e 2.9.1 a h i Fig e

.


98/171

The f 1, , = L

2

2

2

2

1

1

1

esummationsummation

e

j j

j

j j

j

x c

pj j

p pj j x c j

j

wS y w v

D

=

=

=

= = =

2.10

2.10.1 .

Si ce he ce e f he ece i e fie d f da a i GRBF a d GRNN high affec he e a ec fig a i , a e a ic e h d i e i ed ie d a ia e ce e . The fi eide if i g he ce e i c a if he da a. The ce e f he adia e h d hede e i ed ch ha he i i hi each c a (c e ) a e he e be f a c a h edi a ce f he c a ce e a e e ha he c e . The e f c e ce e i e

e e e he a a di ib i f he ai i g ca e . Theea a g i h a ig he adiace e f adia e he fi hidde a e i he e . See D da a d Ha (1973); Ma d Da i (1989); Bi h (1995) f de ai . Lebe he be f adia e (a i ca ed

de i ). ea a ig each ai i g i e f he c e ch ha each c ei e e e ed b i ce e f i h e a e e be f he c e , a d each i i c e

he ce e f i c e ha he ce e f a he c e .ea a g i h a e ba ed a i e a i e ced e , a d he f i g e h hced e f b ai i g he fi a c e a d ce e i g a a g i h :

(a) Fi he c e a e a d a ig ed a e a bi a i c ed b e ec i g ca e .

(b) The i g a a ia e e h d he ce e f each c e i ca c a ed.

(c) Each i i e ed ee he he he ce e f a he c e i c e i ha he

ce e f i c e ; if , he i i ea ig ed.(d) The e c e a e f ed a d he e ce e a e eca c a ed.

(e) The a g i h e ea , i.e. i i a f e (c).

The i e a i i c e ed if he ea ig e f he ca e i ib e, i.e. each i i c ehe ce e f i c e ha he ce e f a he c e .

N e ha he e i f a f f c e ge ce f hi a g i h , a h gh i ac ice i c e ge ic . N a i e e a e i e e ed h h heea a g i h .


99/171

7.1.1 : S e he ebe c a ified ba ed he a

h i he f i g ab e:

Fi ce e a e a bi a i ch e a he ce e f he c

N he di a ce f a i ff i g ab e:

Tab e D0: Di a ce

Di a ce f C1

Di a ce f C2

F e a e The di a ce f hf :

(2.0,

(3.5,

The a ig i g 0 a d 1, i i de

i i ea he ce e 0 i a ie a a i ab e

a e i diffe e che ica d c A, B, C, D, F a f i g edie b a ce . The e

e ec ed. F e a e he ai C1(2.0, 1.5) a d C e .

he e ce e a e ca c a ed. The e di

f each ai f he ce e C1 a d C2

A B C D F E

0.00 1.35 3.81 1.50 1.30 4.85

3.81 2.64 0.00 2.44 4.01 1.00

e ai c e di g d c B f he ce

( ) ( )

( ) ( )

2 2

2 2

1.5) (3.0, 2.4) 3.0 2.0 2.4 1.5

1.35

5.0) (3.0, 2.4) 3.0 3.5 5.0 2.4

=2.64

= + =

= +

e i ed he he he i a e ea e

ed, he i e a ig 1. See he f i g ab

d E, a d he e f ea e e a e

2(3.5, 5.0) a e

a ce a e h i

e a e ca c a ed a

f he ce e . If he

hich i e ed he


100/171

Tab e G0: The e

Di a ce f C1

Di a ce f C2

Ba ed he di a ce f h

he da hed i e be g c e di g 0 i be g c e .

N he e ce e a e ea ica ed i he a e c e . S

1

2

2 3

3.5

C

C

+ = =

N he di a ce f he i f

Tab e D1 : Di

Di a ce f

Di a ce f

The he e a a i ab e f e

a a i f he di a ce f he ce e C1 a

A B C D F E

0 0 1 0 0 1

1 1 0 1 1 0

e ce e C1 a d C2, c e a e a ig ed.

e c e a d he he e a he c he a e c e a d he d c a ig ed

ed hich a e he ea a e (a e age) f hehe e ce e a e

( )

( )

2.1 3.2 1.5 2.4 3 12.56 1.98

4 4

3.8 5 63.65 5.5

2

+ + + + + =

+ =

he e ce e a e c ed a h i

ce f f each ai f he ce e 1 a d

A B C D F E

0.74 0.61 3.16 1.12 1.17 4.21

4.32 3.17 0.52 2.94 4.52 0.52

di a ce i

d C2

The e e e i hi

e . The d c1 i be i he he

a i hich a e

he f i g ab e.

2


101/171

Tab e G1: The e a a i f di a ce f he ce e 1 a d 2

A B C D F E

Di a ce f 1 0 0 1 0 0 1

Di a ce f 2 1 1 0 1 1 0

Si ce he e ea ig e i e i ed, i.e. Tab e G1 a d G0 a e he a e, cha ge hee be f c e a e eeded a d Tab e D1 de e i e he c e . The fi c e

c i e f he d c A, B, D a d F ( ee Tab e D1 G1 i hich he di a ce f he e d cf he ce e a d i e a a i ha e bee e c ed b ec a g e ) a d he e ai i g

d c , i.e. d c C a d D be g he ec d c e . The ce e a d he ca cia ed i h hi e a e a e h i Fig e 2.10.1.

F 2.10.1: .

2.10.2

He e diffe e ea a g i h a e e e ed.

.10. .1 1. I Se ec a d , i de e de , ec be he i i ia ce e f he RBF f c i

0 1 2k c ( ); k ; ; : : : ; K, K = <

1 1.5 2 2.5 3 3.5 40

1

2

3

4

5

6

7

1

2


102/171

he e i he be f i ec , i he be f ce e a di he be fc e .

2. Se ec a ec a a d f he e f i ec

{ }1 2, , , u u uL

3. C The ce e be e e e i g he c e a be b ai ed i g hei i di a ce E c idea c i e i :

arg min ( )k k k(u(i)) u(i) c i=

he e k c (i) i he ce e f he h RBF a i e a ii. k c (i) i i i ed a he h edi a ce f a d ide ifie he c e hichbe g . N e ha a g i i edf ca e i hich he e i i e i i . (a g i a e e f he i i a

a d if he e i e ha e i i .)

4. Cha ge he ce e i g he da e e.

( )c if1

otherwisek k

k k

(i) u(i)- c (i) k k (u)c (i )

c (i)

+ =+ =

he e i i ab e ea i g a e i he a ge0 1< < .

5. C I c ea e b 1, e e 2, he e ea i iceab e cha ge cc ihe i i f he ce e . Thi i e i a e i i i i g he bjec i e f c i

2

1 1

k Q K k q k

k q

F = u - c= =

he e K i he be f c e a dQk i he be f he i ec i h he

e e ek qu be gi g he c e.

.10. . The ec d ea a g i h e e a i i a ced e f b ai i g a e f a ia ece e :

1. I Ch e a d , i de e de , ec be he i i ia ce e f he RBFf c i , 0 1 2 ,k c ( ); k ; ; K, K =


103/171

arg min ( )k k k(u(i)) u(i) c i=

he e k c (i) i he ce e f heh

c e a he i e a i .

3. C I c ea e b 1, e e 2, he e ea 2 i a he i ec ha ebee a ca ed a c e .

4. Ca c a e he ce id. If ik lqu heh e e e f heq h ec a ig ed ce ek ,

he (i + 1) h ce id ( e ce e) f hek h c e i gi e b

1 2 ( )( ) ( )1

1 2( )1 1 1 1

( 1) ( ) ( ) ( ) ,k

k

Q iQ i Q i K k k k

k q q mq k Q iq q q k

c i u i u i u i Q = = = =

+ = =

L

he e k Q (i) i he be f i ec ( )k lqu i a ig ed hek

h ce e a hei h

i e a i .

.10. . T e a a g i h f he f ea , fi he f i g a ec h d be c ide ed:

(a) The e be hi f c i a e [ ]( ) 0 1k n nu

(b) F a1

, ( ) 1 K

k k n n n

k

u u =

= ; k nu i b ai ed f

1

1

2

1( ) ( )( )

( ) ( )

K n k k

n n j n j

u i c iu

u i c i

=

=

he e 1 > i he f e e de e i i g he deg ee f f i e .

(c) The ce id da e e a i (CUE) i b ai ed f

1

11 2

1 1 1( )( 1)

n

k k k k k k

k n n n n n Mnn q q

k unn

c i u u u

=

= = = + =

L

N he f ea a g i h ca be e e ed a f :

1. I i ia i e Ch e a d , i de e de , ec be he i i ia ce e f he RBf c i

0 1 2k c ( ); k ; ; ; K, K = .


104/171

3. A ig e be hi deg ee:k nu i ca c a ed f each i ec i g CUE.

4. Ne ce e a e ca c a ed i g CUE.

5. C I c ea e b 1, e e 2, he e ea i iceab e cha ge cc ihe i i f he ce e . Thi i e i a e i i i i g he bjec i e f c i

2

1 1

K k n n k

k n

F = u - c= =

The f ea a g i h ha he f i g e ie :

a be e e f a ce ha he a da dea a g i h .

I i i e e ed i MATLAB i g he f gic b .

Nei he f he e h d i g a a eed fi d he i a ce e .

Diffe e i i ia c di i i ie d diffe e ce e .

Se ec i gN i i ia ce e (i.e. he e i e e f i ec ) g a a ee a e a e f bjec i e f c i .

A c i e i i h c ide ab e ce e i ge e a de i ed.

N e ha f a a g i h , a a e f he i g c i e i f c i (f e a e he he ee ha 0.001) i e ec ed gi e ea ab e c e ge ce.

2.11 Defi e he e ea e

= 12 W i e i e f he RBF f c i = 12 12 ,

Diffe e ia e . . .


105/171

= , ,= , ,

= , ,= , ,

Ob ai he eigh da e e a i f he g adie de ce a g i h

+1 = = , ,

Defi e he i i ia c di a e f he ce e

0 = 000 = 0 0 00 0 00 0 0 Ca c a e he da e e a i f he ce e

+1 = =

Diffe e ia e i h e ec


106/171

= = 12 =

Diffe e ia e i h e ec

= = 12 =

he e

= , = , /= 2 /

Le

= 2 , The


107/171

+1 = +2 , , ,

= 1+1+1+

Si if

= 2 /

The da e e a i

+1 = +2 = +2 ,

T e f he i e a i , he f i g e a i a e e i ed:

+1 = = , ,


108/171

+1 = =

+1 = +2 = +2 ,

The e f a ce f he e i ed e ec i f ce e i be e ha he he a g i hd ci g i a RBF e . T e he a g i h e h d

(a) Se ec ce e a d ead a a d .

(b) Ca c a e he eigh , he i e a e.

N e ha he a d eigh a e de e de b h he ce e a d he ead , he a d eigh i be c edi each .

2.12 I hi ec i he e f e a e i c , E a ec e e a e , i e

e a e , a d e h d f i e e i g e a e c e a e died. De a e e i e e hich a he e e hibi e a beha i . I c

e de ig i g e a e , he e beha i i de e de ei , b a a i . The e a e aj c a e f d a ic e :

(a) Rec e Ne a Ne (RNN)

(b) Ti e De a ed Ne a Ne (TDNN).

2.12.1 ( )

The c RNN a e a f :


109/171

(a) The E a e (Ede a d i

(b) The J da e (J a e a he ha

N e ha he RNN a e ea

2.12.2

TDNN ca ea e a behaig a . The e a e feedbace a chi ec e i a a

RBF

PNN (P babi i ic Ne

GRNN (Ge e a i ed Reg

he feedf a d e

F 2.12.1:

A i e de a e a e

a 1990): The hidde a e a e fed bacde . See Fig e 2.12.1.

da 1986): I i i i a he E a ehe hidde a e . See Fig e 2.12.1.

ai d e he feedbac c ec i .

( )

i b i g he e e a d a i i ge , i i ea i ai ed i h a da d a

a da d MLP b i ca a be a

Ne )

e i Ne a Ne )

a chi ec e .

E J N , .

de i h i Fig e2.12.2.

h gh a e e

i feed bac he

de a ( ) i he ii h . The e a

,


110/171

E a e 2.12.1: Le [4,6,9,1 x =

E 7.5.1:(a) W i e a fi e2, 3 a d 5 a e . (b) A e

= 0.05 1/2rand1f 100 200. C ideb ai ed f ig a .U i, ( )d t f x ) i a i g e fig e he

ca c a i g a i ea eigh ed a

2.13

The e a e a i e h d f i

(a) S e i ed C

(b) Di ec I e e C

(c) Ne a Ada i e C

(d) Bac aga i Th

(e) Ada i e c i ic Me h d

A a ia e e h d i ee f a ce. S e i ed c

F 2.12.2: A DNN .

]1,14,17,20,21,23,25 T a d i e de a be 3. The

4 6 9 116 9 11 14

9 11 14 17

11 14 17 20

14 17 20 21

17 20 21 23

20 21 23 25

d x

=

MATLAB ca c a i g x defi ed i E a e 2 ha a i e ig a i gi e b

0.3sin 2 +0.5co0.01 = 5 a a de a , = [0.1 0.3 0.5 0.1]g MATLAB ca c a e d y x W = a d ( ,d x 1 length( ( ) 1)d t f x . N e ha hi e

age. (c) Se ec diffe e de a a d a

e e i g e a e c e i c di

h Ti e

c ed ba ed a ai ab e i f a i hee a e c a d a e h d i hic

d x i defi ed a

.12.1 he de a a e

a d d x he i) a e a (

fi e he i e bia e eigh

g

e a d de i edhe e a e


111/171

i ai ed e f he a ea d a c di i . O ce he

2.13.1.

F 2

Ne a e a egie a di e b ch a Hi e (199

I hi ec i a i e a ei d ced. The i i a i fa g i h ha bee e e ed The bac aga i e h d

c a ified a d he ce e f eachea a g i h . GRBF e

c i a a he c e ( echa ica he a c e i e ai ed, i e ace he

.13.1: S N C .

ATLAB g ide i e f e a e a d f).

, i c di g i g e a e e ce , MLPi g e a e e ce ha e bee ide ified

fi d a ia e eigh hich a e e i ed fie d a g ba i . T c c a GRB, a

c a (c e ) be de e i ed. Thi a ca gi e a ca i a b e .

a ) f gi e ie . See Fig e

gic ca be f d

a d GRBF ha e bee. A bac aga i

i g a e .gi e da a be

be achie ed i g a


112/171

R

[1] Si Ha i ,N N L M, Thi d edi i , Pea H :N J , 2009.

[2] Michae Neg e i , A I : A G I , Sec d edi i ,Addi We e (Pea Ed ca i Li i ed), E g a d, 2005.

[3] A d ie E ge b ech , C I : A I , , J h Wi e& S , L d, E g a d. 2007

[4] Michi S ge a d Ta ehi a O i a a,F D M (Obi a :

P fe T hi Te a ), 4(4), 255 256, Ne he a d : S i ge Scie ce+ B i e Media, 2005.[6] Ha J ge Zi e a ,F I A , Ma ach e : K e

Acade ic P b i he , F h edi i , 2001

[7] D L e, Ada i e RBF i ea i ie , a d he b e f ge e a i a i , 1 IEE IC A N N , 171 175, L d , 1989.

[8] C A Micche i, I e a i f ca e ed da a: Di a ce a ice a d c di i a defi i e f c i ,C A , 2, 11 22, 1986.

[9] R O D da a d P E Ha , C A, Ne Y : Wi e , 1973.

[10] J C Be de , Pa e Rec g i i i h F Objec i e F c i A g i h , Ne Y : P eP e , 1981.

[11] D We che ec a d T Die e ich, I i g he e f a ce f RBF e b ea ice e ca i , A N , 4, 1133 1140, Sa Ma e ,CA: M ga Ka f a , 1992.

[12] L. We e Hi e ,F N A E , Ne Y : J h Wi e & S ,1997

[13] J. M d a d C. Da e Fa ea i g i e f ca ed ce i g i . Nc a i , i 1, 281 294. 1989.

[14] C. Bi h ,N N , C a e d P e , O f d, 1995

[15] K.H i , M. S i chc be a d H. Whi e, M i a e feedf a d e a e i e aa i a ,N N 2, 359 366, 1989.

[16] C be , A i a i b S e i i f a Sig ida F c i ,M . C , 2, 303 314, 1989.

[17] J. W. Hi e , MATLAB e e F a d e a e i e gi ee i g, J h Wi e S , I c, Ne Y , 1997


113/171

2.14 A ABNe a e de ig i e ch a he i ic cie ce. The MATLAB Ne a Ne b

a a ea ab e e ec i f NN de ig be ea i ed i h e i i g addi i a H e e , de ig f a a ia e NN a chi ec e f a a ic a b e a a a be

aigh f a d. I i a e i ed i e fi e hich a he fi d he be c bi af be a d e f e , he be eigh a d bia e , a d e a a ea i g a g i h .

2.14.1 A

The MATLAB NN b c i f ac i a i f c i ch ha he ca be ed b a eeadi . F e a e, a f he ha d i hea i ide f c i ca be b ai ed a f :

= 5:0.1:5; ( , ha d i ( ))

. Si i a g a h ca be b ai ed f

e i , g ig, a ig a d adba f c i . MATLAB a e ha he a a e e i e a 1 i

= , y = tanh =, = logsig = A adj e g adie be ade e ici . F e a e,

= logsig2 = 11+

he MATLAB c a d bec e: = 5:0.1:5 ; ( , g ig(2* )).

The a ic eigh a e adj ed d i g he MATLAB ai i g ce i i e he effec f c i a d he ef e, he e f f c i be a ia e e ec ed. The i a i

e ha diffe e f he RBF f c i .

2.14.2 A AB

De hich a e a ai ab e i he NN b h ha h e ca e MATLAB f de ig i g aNN. T a he de e:

de

a d c ic he e e a ab . Si e e a e e e ed i de e e e he g a hicThe NN de a e a ia e f gi i g a fa f NN he , e ecia he c ce f ca ag ba i i a i eigh ace. I i g a fe diffe e i i ia i a i eigh h he deg ee

hich NN ai i g e ie cha ic i.e. a d ce e . I ai i g a NN, if he fi a e cce f , a cce i e ia a be e i ed b ai a ia e i .

The effec f i i i g he be f hidde e i de a ed e b d11g . Se ec i

f a i ab e be f e i e i a a d i h d be ed ha a g d e deh d i i i e he be f e .


114/171

2.14.3 A AB

A i e a d e f ie d GUI Ne /Da a Ma age i d ca be ge e a ed i h . Ti h he XOR f c i . I

= [0 0 1 1; 0 1 0 1]i a2 4a i , a 2 4 ec . The e i ed NN he ef e be de ig ed ha e 2i e a d 1 e . C ic I i a a i a d a a a ge . Thec ic Ne Ne a e e . Ch e a e c fig a i i g he Ne

T e d d e . T MATLAB, a Pe ce i a a a i g e a e e i h ha d e . MATLAB ca MLP a Feed F a d, Bac aga i Ne . Each e

e ha a i i ed be f T a fe a d Lea i g f c i a cia ed i h i . C ic he

f i d d e . C ic gi e a i a ge [0 1; 0 1]. The be f i eha a a ica bee e 2. C ic i g Vie gi e a b c diag a f NN. N e, a

i , ha e i dica ed h a e a e e i ed.

The defa i 1, b ca cha ge hi a ( a a ) be i h. If i d e a ch hi e f he a ge , h e e , ai i g i be a ed c e ce. Re he Ne /Da

Ma age i d , fi c ic e a e, he T ai . Se ec i a d a gec . C ic T ai i g Pa a e e . L a he defa a a e e a d cha ge e f hee i ed. C ic T ai Ne a d he ai i g c ee i a ea . Chec he e e f he a

NN b e i g he Ne /Da a Ma age i d , c ic i g Ne O : Si a eC ic Si a e, ecif , he abe he i h a e a e. C ic i g Si a e Ne

d ce a e a iab e. D b e c ic i g i gi e he a e f he ai ed NN. c e a e he he a ge a e ? Y ca e he a d e he MATLAB c

i e ace b fi e i g he Ne /Da a Ma age i d he c ic i g EHigh igh i g he a d e ec f ed b E i c he he E e i e i g i h diffe e NN a chi ec e i c i ce h i a i i g

ac ica e e ie ce i de NN de ig effec i e .

2.14.4

A fe f he NN a chi ec e ca be i e iga ed i g . A MLP i fi c ide ed hiMATLAB ca hi a Feed f a d, Bac aga i NN. T c ea e hi i MATLAB, i

= [0 0 1 1; 0 1 0 1]e = e ff([ 1 2; 0 5],[3,1], ' a ig',' e i ' ,' ai gd')

Thi ha c ea ed a e MLP i h 2 i , 3 hidde a ig e a d 1 i ea eI (1) ha a a ge [ 1, 2], i (2), [0, 5]. ' ai gd' i dica e ha ba ch g adie de ce i.e.

a da d BP i be ed f ai i g. The e i c a de ibe a e i ed i he ie a ib e . The a e e ca be ge e a ed i g

e = e ff( i a ( ),[3,1], ' a ig',' e i ' ,' ai gd');


115/171


116/171

ai bfg B de , F e che , G dfa b, Sha (BFGS) A g i h . Achie e fa e c e gei g a i Ne ( ec d de eca ) e h d. A a i a e He ia f i e

i

ed, he eN i he be f f ee a a e e i he NN. ai Le e be g Ma a d A g i h . Achie e fa e c e ge ce b a i a i

Ne ' e h d b a Jac bia a i . Me e i e e i c ea e i h he be f NNa a e e AND i e f ai i g da a.

The e a e a e a g i h a ai ab e. I i ad i ed a i ea de (LM) fi hich iec ac a be e (1000 i e fa e ha BP i h e ) e e f a d

a a e e i g 3000 ai i g ai a a ge a d LM i i e fa d . BFGS h d bee , i h CG RBP f . T c a i g he effec i e e f each a g i h i h he a

i / da a. Be ee each ai i g i

e =i i ( e );

i de ei i ia i e e i h a e e f a d eigh ; a e a i e g bac e ff a d gieach NN a e abe .

2.14.6

N e ha he a e f i i ia i a i eigh ca a e a big diffe e ce he c e f a ai i. I i g d ac i e, he ef e, e ea each e e a i e i g he a e ai i

a a e e , b i h e i i ia i a i eigh , ee i g a ec d f he ea a ed e (MSEf each . The fi e f a a chi ec e i he ca c a ed a he a e age f he e . If ide i ed fi d be c bi a i f a e f ea i g a e ( ) a d e () a a e e

hi e ee i g he NN a chi ec e c a , i i ece a e he a e i i ia i a i eigheach , ha he ai i g i affec ed b he a d a e .

2.14.7

T ai i g i f e ade e efficie b e ca i g i a d be i hi a a ge i ab e f he ac i a i f c i bei g ed. The c e h d i a i e he i be i a a ge [ 1, 1], hich i c a ib e i h he a e hich a e ea i ha d ed b gi ic

f c i . The a ge a e e ca ed acc di g he f c i ed he a e . F e aif he e a e g ig, a e ca e [0.25, 0.75] e e ha he a ge a e e i hi he e

f he e . U e f a addi i a i ea a e b ia e he e i e e f e ca i g, b i c ea e ai i g i e . MATLAB ha b i i f c i hich ca

a i a i ca c a i f .

[ , i , a , , i , a ]= e ( , );ff e = 2;

gai = 0.25;e = e ff( i a ( ),[3,1], ' a ig',' g ig' ,' ai gd');

e = ai ( e , ,gai *( + ff e ));


117/171

e i a e ai ed a e ha e bee ch e i

e: = i ( e , )./gai ff

= ( , i ,

ce e hec e e e ia ha a f he

e i ec be e

e = a ( e ,

2.14.8 B

MATLAB ide de ig fi d ced i g he i

e = e be( , , )

e = e b( , ,g, )

The f e a ig a RBF ea ed de e i e he i

RBF f c i he a e0:8326/ f i ce e, he

e a be ee adjace ee , i e i ab de e di g

h d be g ea e ha he adi a ce ac he a i a i e, i g each i ec

a e e e , i ch ea he ea ch h gh he ee . The ce i e ea ed

i h a ec bei g ed ade ig . e be ha he ad a a

e a ai i g ec ibe e , d e he i e a i

i g he fc c a d (a g i h (MATLAB efe c ia g i h de c ibed i he ecba ed i di a ce f a b i h hi a g i h , e

i g he e ce ed i a d ec he g ig a e . T d ce he e

e ;

a );

e i g he a a e e c ea ede a e ed i de e ca e he f

ce ed bef e bei g ed a a i e . T

i , a );

c i f he ba ic RBF e : e be a d

each i ec , he eb a i g e aih f he a ea i i ace hich each e

0.5 f a i ec a a E c idea di ai he bia f he RBF. The a ge he a e f

. Fi di g he be a e i be e chhe ag i de f he e e e f i ec

e age E c idea di a ce be ee i ecace. i he e g a f e b. e b c ea ea a ib e ce e. The be ce e i.e. h

. If he e a e i e ha g, he NN iai i g i ec i ade f he ce e gi

i he e g a i achie ed. If he g a i ce e , hich i he a e a e be, b ae f bei g fa . The di ad a age a e ha he

he high babi i f ge e a i a i . ee ce . A a a e a i e he 'i h e' Nhich i ca ed i he f b ) i e

ead f ). The diffe e ce be ee hi a d e i ha a f e be hi deg ee i a ig ed

ec i e ce e. The ec a e i each a igec i a ce e, i e be hi deg ee i

. The ff e a d gai e e ,

e . I i , f e . Si i a , a

e c de i

e b. The RBF NN

i g e . The a e e d . Each

ce f / , he g ea e hea e f ia a d

. Ce ai , h gh, a d e ha he

RBF e e ae d ci g he

fi i hed. O he i e,i g he e ea , a e d i g ch ge e ha a a

b, c e e , caN b a g i h ,e a f eahe a da d ea

each i eced a c e be ,hi he c e i < 1


118/171

(a h gh a be a edeg ee f each ec i ed a

[ce , e , b] = fc ( , );

he e i he be f ce edi a e , a a i , e , f e

f c i a each i e a i . T ca

1 = adba ( e d(d

he e i a ec f ead a

The i ea a e eigh a e ca c

I e iga e de b1 a d dede a e he a chi ec e

hich a e a ia i he RBFai i g. The f e i ef f

E e ci e1. I e iga e diffe e e

2. C ea e a5 4000 a i TTU i h a f c i a e a iia . Add a a a

he ec d a i , he eFi d he be a chi ec e

2.15

NN ha e bec e a a ad

he i e i e aH e e , if e hi b hbe be e dif he NN eac ce f ge e a i a i a d

i i e NN a chi ec e i g

a deg ee). A he a g i h da e he cea eigh i g i he ca c a i . T i e e ,

e i ed. Thi c a d i a a i , cbe hi deg ee a d a ec , b, hich ec

c a e he f he RBF a e d e a i

i (ce , ), ))

e , ca c a ed i g .

a ed i gWb = /[ 1 ; e (1, e g h 1 ]

b3 f ba ic RBF e de . def Ge e a i ed Reg e i Ne a d P ba

e a d ca he ef e be de ig ed i hf c i a i a i , he a e f c a ifica

e he AND, NAND, OR, NOR b e .

i h e e e f a a d di ib i . Ca chi he fi . P ibi i ie i c de e f a d Ga ia i e ( e a i ab e

hi a he i , i h T a he a ge , a ide c ibe he i a i g.

A AB i i he c e a e a . Pa f hei

i hi e edge fe a i g ( ica a a i fi g a e hi i acc . N ha e be

eg a i a i e ca a h ea i eMATLAB .

e , he e be hie

e , f ce e c he bjec i e e

g 1 a d dei i ic ANN , b h f

he e i e e fi f da a.

a e a ec d a ie ia , i ida

i e f a d ) NN c fig a i .

a ea i he abi ia e i a ai ab e.

h e ), i ae i d ced he

he e c ce a d


119/171

2.15.1 B

Ba e ia eg a i a i i be i e e ed i MATLAB f e i h ab 400 a a e ei g he ai i g f c i ai b , hich c bi e he LM a g i h i h Ba e ia eg a i a i

Thi a a ica cha ge he a a e e i he e a i

MSE = +1 a ai i g g e e , he eM e e e he be f ai i g ai a dN he be f

e eigh . We i c ea e a NN hich a i a e a i i e a e:

a d(' a e', (100*c c ))

= [ 1:0.05:1];

= i (2* i* )+0.1* a d ( i e( ));

e = e ff( i a ( ),[20 1], ' a ig',' e i ' ,' ai b ');

e . ai Pa a . h = 10;

e . ai Pa a .e ch = 100;

[ e , ] = ai ( e , , );i = i ( e , );

fig e

( , ,' *', , i ,' ')

If he ab e c de, he ai i g e f a ce i be h 3 g a h . E a e h e f SSE a d SSW ( f a ed e a d f a ed eigh ), e ec i e . The hi

g a h h he effec i e be f f ee a a e e ac a ed i ai i g he e . If hc e ge a fig e ig ifica e ha he be ed i e de ig , i ea

eed i e . The a e a e a i e hi e abi i e, a he a g i hi i i e he NN eigh a he e e e f he e , he i c ea e he i e f eigh i i i e he e a e . The e i g i ab e f i h e .

F a ge e , i i e i ed e ace a a e e i e i i g MATLAB ai i g a gThi i i e e ed b fi i i g

e . e f Fc = ' e eg';

cha ge he MSE f c i be f he f gi e a he a f hi Sec i . The a e f i

fi ed f he c e e ai i g ce b i i g


120/171

e . e f Pa a . a i =

he e 0< < 1 e.g. f e a eigh i g be ee MSE a d , e =0.5.

2.16 : C a ida i i ca ied b de ig i g a a chi ec e i g a a ia e a g i h i h a

ai i g e he af e a d ca c a i g e e a e i ee a ida i e . Fa g i h i g c e i g, he i a a e fk i he c ide ed be ha hich d ce he

i i e .The igi a i da a = 1:0.05:1 a e e a ed ch ha e i f i a e e ec ed.P =

1:0.2:1 .The a ida i i he b ai ed b igh a e i g he a ge i g a da ia i : a .T = i (2* i* a .P) + 0.1* a d ( i e( a .P));

T ai i g i i e e ed a a , i h he addi i f he a c e. Le e be g Ma a i d e he a i e f he e :

e = e ff( i a ( ),[20 1], ' a ig',' e i ' ,' ai g ');

e . ai Pa a . h = 10;

e . ai Pa a .e ch = 100;

e . ai Pa a . = 1;

e . ai .Pa a . _dec = 0.8;

e . ai .Pa a . _i c = 1.5;

[ e , ] = ai ( e , , ,[ ],[ ], a );

i = i ( e , );

fig e

( , ,' *', , i ,' ')

The MSE a e i , b he e a c e i a h a ha d ced b Ba eeg a i a i . N e ha he a ida i e i a ed f ac he h e a ge f he da a.

i e i a he i g c a ida i . The a a e e a e ch e e e ha ai c e ge a id . MATLAB a ec e d i g ai cg a d ai i h ea

a h gh i he , a a g i h ca be ed.

2.16.1 B A

Thi i ca ied b de ig i g a a chi ec e i g a a ia e a g i h i h a ai i g he af e a d ca c a i g e e a e i ee a ida i e . F a a g

i g c e i g, he i a a e fi he c ide ed be ha hich d ce he i i

e .


121/171

2.16.2 A

The e a e e ca e he e e a f he i ec c e a e high c e a ed. Ib i ad a age i ch ci c a ce ed ce he i e f he ec , a hi a

c e di g ed c i i he NN a chi ec e. If he da a i e e i e, i i be i e c i gif h gh each ai f c e a a i g c a ia ce, he e h d f P i ci a C e

A a i (PCA) i ed i ead. Thi e eige he ca c a e he e a i e i a ce h g a i ed i ec f a f e a a i , ee Ha i (2009). MATLAB ha a b

i e i e e i g PCA:

, , = ( );

= 0.01;

, M = ( , );

The fi i e a i e he i ec ha e e ea a d a ia ce = 1. i he c ib ibe hich e e i e he ec be e i i a ed. If hi i e e.g. 0.01, i ci a c ec ib i g e ha 1% he a a ia ce f he da a i be e ed f he da a. a

he a f ed i da a, a Ma he PCA a f a i a i , hich be ed a f a b e e i da a bef e a i g hi he NN. The e i ed c de i

e = a d( e , ea , d );

e a = a ca( e , a Ma );

= i ( e , e a );

PCA h d be ed i h i ec ed ce he be f i ed b he ee i ie i i i e he e i e.

2.16.3

The be f hidde e c ec i a MLP i f da e a he abi i f he e a i a e a f c i . A i e b effec i e i d c i he c ce ca be f d i MATLAB

d11g c a d.

I a ca e , a chi ec e i be a ge a d/ c e . T a id edi a d e e i i e h ai e e i , i i efficie i e a fi e hich h gh he a i a chi ec e fThe fi e h d ca a he ea he f i g ced e :

1. S a i h a a e , addi g e e a a i e a hidde a e .

2. Re ea ai i g a be f i e i h each a chi ec e Ha i (2009) gge 20 e ea .

3. Sa e each ai ed NN a c e.


122/171

4. Rec d he i i MSE achie ed d i g each ai i g OR he c e e ai i g . Ai di e i a a i (M Nb ) h d be ed a e he e he eM i he

be f hidde a e e ,N i he be f e ea ed ai i g a di hei i MSE achie ed a d R i he be f ai i g .

5. Sa e he e a a i a d he e c e a . a fi e.

Ti e c ai a a a a e be f ai i g e ea , a d, a he hidde ag , e e be ha each e ch i be c e di g ge . If ai i fi ed a

e , he c e e a be i fficie a d diffic ie a be e c e ed a hee g .

2.16.4 A

The a e age f he i i MSE achie ed i each i e f he ib e i dica f a NN'i abi i f e. The i e i h ch ea e i ha i a acc da e f ie a d

ead a i g affec ed b a e e e a i e a ge ai i g e . Thi i e eciaca e i h LM, a he ai i g i e f e e i a e i a ca i a. I i he ef e i f a i echec he a ia ce f each a chi ec e' ai i g e , a d a be ejec he e if

ch e ha he e . U i g he i i MSE f each e e b e a i ide e d h a a e ed f each e e b e. The f i g MATLAB c a d ca

e f eg e i a a i he NN c a ed i h he a ge f ca a :

= i ( e , );

[a,b, ] = eg( , );

he e a, b a e he e a d i e ce f he i ea eg e i i e e a i g a ge NN F e e , a =1 a d b = 0. i he c e a i c efficie be ee a d a ge . F e , = 1.

2.17 P a ide ifica i f i e i a ia a i ea i ca ied i g ANN . The i e

e h d i e he a ed i a d f a a , he ai he a de he e. If i e de a a e i e e ed, hi c ea e he fa i ia de f he f

( ) ( ( ))t F u t =

hi i ha e ha e bee i g i h fa . If, h e e , edic i f f e a ie i ed, ha he de i


123/171

i h he c h i , he he a be f i he e i + he e ihe i e f a i ec a dhe i e f a ec . The a ge i eed be hif ed b

+ ace i de c ea e a NN hich edic he a . If ch e de ig a de he e he i e a ed i i , i.e.

he MATLAB i ca ab e f b i di g i i e de a i he a chi ec e. A e ade a e he e h d. F i a ce,

e = e ff( i a (U),[5 1]

04_Artificial Intelligence (1)

Documents

Transcript of 04_Artificial Intelligence (1)