Dynamical entropy for systems with stochastic perturbation
12
Click here to load reader
Transcript of Dynamical entropy for systems with stochastic perturbation
PHYSICAL REVIEW E AUGUST 2000VOLUME 62, NUMBER 2
Dynamical entropy for systems with stochastic perturbation
Andrzej Ostruszka,1,* Prot Pakon´ski,1,† Wojciech Słomczyn´ski,2,‡ and Karol Zyczkowski1,3,§
1Instytut Fizyki im. Mariana Smoluchowskiego, Uniwersytet Jagiellon´ski, Reymonta 4, 30-059 Krako´w, Poland2Instytut Matematyki, Uniwersytet Jagiellon´ski, Reymonta 4, 30-059 Krako´w, Poland
3Institute for Plasma Research, University of Maryland, College Park, Maryland 20742~Received 7 June 1999!
Dynamics of deterministic systems perturbed by random additive noise is characterized quantitatively. Sincefor such systems the Kolmogorov-Sinai~KS! entropy diverges if the diameter of the partition tends to zero, weanalyze the difference between the total entropy of a noisy system and the entropy of the noise itself. We showthat this quantity is finite and non-negative and we call it the dynamical entropy of the noisy system. In theweak noise limit this quantity is conjectured to tend to the KS entropy of the deterministic system. In particu-lar, we consider one-dimensional systems with noise described by a finite-dimensional kernel for which theFrobenius-Perron operator can be represented by a finite matrix.
PACS number~s!: 05.45.2a, 05.40.Ca, 05.10.2a
arsayriis,caem
b
ge
n
th
zeen
en
-thed,ithles
per,on-the
itsse abe
ofius-
i-
s.i-
eirex-d in
pylary
s-the
icsoss
I. INTRODUCTION
Stochastic perturbations are typical for any physical reization of a given dynamical system. Also round-off erroinevitable in numerical investigation of any dynamics, mbe considered as a random noise. Quantitative charactetion of dynamical systems with external stochastic noisesubject of several recent studies@1–4#. On the other handthe influence of noise on various low-dimensional dynamisystems and the properties of random dynamical systhave been extensively studied for many years@5–10#.
Consider a discrete dynamical system generatedf :X→X, whereX is a subset ofRd, in the presence of anadditive noise
xn115 f ~xn!1jn , ~1!
wherej1 ,j2 , . . . areindependent random vectors fulfillin^jn&50 and^jnjm&5s2dmn . The case with vanishing noisstrengths→0 will be called the deterministic limit of themodel. Properties of such stochastic systems have recebeen analyzed by means of the periodic orbit theory@11#.Convergence of invariant measures of the noisy systemthe deterministic limit has been broadly discussed inmathematical literature~see, for instance,@12–20#!.
A dynamical system generated byf is called chaotic if itsKolmogorov-Sinai~KS! entropy is positive@21#. Such a defi-nition is not applicable for stochastic systems, characteriby infinite entropy. In this case the partition-dependenttropy diverges if the partitionA of the spaceX is madeincreasingly finer.
In this paper we propose a generalization of the KStropy for systems with additive noise~1!. Since entropy di-
*Email address: [email protected]†Email address: [email protected]‡Email address: [email protected]§Present address: Centrum Fizyki Teoretycznej PAN, Lotnik´w
32/46, 02-668 Warszawa, Poland. Email [email protected]
PRE 621063-651X/2000/62~2!/2018~12!/$15.00
l-,
za-a
ls
y
tly
ine
d-
-
verges also for the pure noise@with the trivial deterministiclimit f (x)5I (x)5x for xPX], we study the difference between the total entropy of the system with noise andentropy of the noise itself. First, we set the partition fixeand then we take the supremum over all finite partitions wregular cell boundaries. In this way our definition resembthe coherent states dynamical entropythat two of us pro-posed several years ago@22–24# as a measure of quantumchaos. The entropy of the noise, discussed in this paplays the role of entropy of quantum measurement, cnected with the overlap of coherent states and linked toHeisenberg uncertainty relation.
Even though our definition is suitable ford-dimensionalsystems with an arbitrary additive noise, we demonstrateusefulness on simple one-dimensional systems. We choospecific kind of distribution defining the noise, which canexpanded in a finite basis ofN functions in both variablesxand y. This condition allows us to express then-step prob-abilities, required to compute the entropy, as a productcertain matrices. Moreover, we represent the FrobenPerron operator of the system with noise by an (N11)3(N11) matrix, and obtain its spectrum by numerical dagonalization. The deterministic limits→0 requiresN→`,which resembles the classical limit of quantum mechanic
This paper is organized as follows. In Sec. II the dynamcal entropy for noisy systems is defined and some of thproperties are analyzed. One-dimensional systems withpandable noise and their invariant measures are analyzeSec. III, while different methods of computing the entroare presented in Sec. IV. The entropy for some exempsystems with noise~Renyi map, logistic map! is studied inSec. V. The paper is concluded by Sec. VI, while an illutrative iterated function system, used for computation ofentropy of noise, is provided in Appendix A.
II. DYNAMICAL ENTROPY FOR SYSTEMS WITH NOISE
A. Dynamical entropy for deterministic systems
Let us consider a partitionA5$E1 , . . . ,Ek% of X into kdisjoint cells. The partition generates the symbolic dynamin the k-symbol code space. Everyn-steps trajectory can be:
2018 ©2000 The American Physical Society
m
to
rti
a
md
onites
akdu
ity
ap
gue
-
w-en
eor-em-in-
rtur-nt
y-
s we
of
ns
th-e-n
e
e
PRE 62 2019DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
represented by a string ofn symbols, n5$ i 0 , . . . ,i n21%,where each letteri j denotes one of thek cells. Assuming thatinitial conditions are taken fromX with the distributionm finvariant with respect to the mapf, let us denote byPi 0 , . . . ,i n21
the probability that the trajectory of the systecan be encoded by a given string of symbols, i.e.,
Pi 0 , . . . ,i n215m f„$x:xPEi 0
, f ~x!PEi 1, . . . ,
f n21~x!PEi n21%…. ~2!
Thepartial entropies Hn are given by the sum over allkn
strings of lengthn
Hnª2 (i 0 , . . . ,i n2151
k
Pi 0 , . . . ,i n21ln Pi 0 , . . . ,i n21
, ~3!
while thedynamical entropy of the system f with respectthe partitionA reads
H~ f ;A!ª limn→`
1
nHn . ~4!
The above sequence is decreasing and the quantity
H152(i 51
k
m f~Ei !ln@m f~Ei !#, ~5!
which depends onf only via m f , is just theentropy of thepartition A. We denote it byHA(m f). TheKS entropyof thesystemf is defined by the supremum over all possible pations A @21#
HKS~ f !ªsupA
H~ f ;A!. ~6!
A partition for which the supremum is achieved is calledgeneratingpartition. Knowledge of akg elements generatingpartition for a given map allows one to represent the tievolution of the system inkg-letters symbolic dynamics anto find the upper bound for the KS entropy:HKS( f )< ln kg .In the general case it is difficult to find a generating partitiand one usually performs another limit, tending to zero wthe diameter of the largest cell of a partition, which implithe limit k→`. We shall denote this limit byA↓0.
B. Entropy for systems with stochastic perturbation
For simplicity we consider the one-dimensional case ting X5@0,1#, imposing periodic boundary conditions, anjoining the interval into a circle. We denote the Lebesgmeasure onX by m, settingdx5dm(x) @clearly m(X)51].The noisy system introduced in Eq.~1! will be denoted byf s . From now on we assume that all the random vectorsjn(nPN) in Eq. ~1! have the same distribution with the densPs . Then the probability density of transition fromx to yunder the combined action of the deterministic mapf and thenoise is given byPs„f (x),y…5Ps„f (x)2y…, wherex,yPXand the difference is taken mod1. In the pure noise c( f 5I ) this density depends only on the length of the jumand equals toPs(x,y)5Ps(x2y).
-
e
h
-
e
se
We assume thatf s has a unique invariant measurem f s,
which is absolutely continuous with respect to the Lebesmeasurem ~i.e., it has a densityr f s
). Clearly,m I s5m, and
sor I s[1. Moreover, we assume that the measurem f s
tends
weakly to m f , for s→0, wherem f is some invariant measure for the deterministic systemf. In Sec. III B we discussthe situation, where the above assumptions are fulfilled.
Now, let us fix a partitionA of X. We define thetotalentropy Htot( f s ;A) of the noisy systemf s by formulas~3!and ~4!, analogously to the deterministic case. Note, hoever, that in this case the initial conditions should be takfrom X with the measurem f s
. As we shall see below thisentropy grows unboundedly withk. Hence, we cannot definpartition independent entropy of the noisy system using fmula ~6!, as the supremum in~6! is, in this case, equal to thinfinity. On the other hand, there are two kinds of randoness in our model: the first is connected with the determistic dynamics; the second comes from the stochastic pebation. Accordingly, we split the total partition-dependeentropyHtot of a noisy systemf s given by Eqs.~3! and ~4!into two components: thenoise entropyand thedynamicalentropy. The latter quantity characterizes the underlying dnamicsf s and is defined by
Hdyn~ f s ;A!ªHtot~ f s ;A!2Hnoise~s,A!, ~7!
where the entropy of the noiseHnoise(s,A) reads
Hnoise~s,A!5Htot~ I s ;A!, ~8!
and I s is a stochastic system given by Eq.~1! with f 5I~pure noise!. Although both the quantitiesHtot and Hnoisemay diverge in the limit of fine partitionA↓0 (k→`) for anonzero noise strength, one can make their differenceHdynbounded, taking an appropriate sequence of partitions, ashall see in Sec. II C.
In order to keep away from the ambiguity in the choicea partition, we eventually define thedynamical entropy off s as
Hdyn~ f s!ªsupA
Hdyn~ f s ;A!, ~9!
the supremum being taken over all finite partitioA5$E1 , . . . ,Ek% such thatm(Ei)51/k and m(]Ei)50 foreachi 51, . . . ,k, kPN. We will call such partitionsuniform.The restriction to uniform partitions is necessary, since oerwise we may encounter various ‘‘pathologies’’ in the dterministic limit @25#. Note, that the uniformity assumptiomay be omitted in the case when all the measuresm f s
, m f ,andm coincide.
It seems that in many cases the entropy of the noise~8!tends to zero in the deterministic limits→0, and the dy-namical entropy off s converges to the KS entropy of thcorresponding deterministic systemf ~for partial results inthis direction see@15# and @22#; for numerical evidence seSec. V D!.
ica
ef
is
a
oy
d
ge
in
pyth
ls
ical
inh
enpyd
eor
y-stic
y
hethe
in-is
is
eing-an
yinsof
sewo
s toh-
ys-
2020 PRE 62ANDRZEJ OSTRUSZKAet al.
C. Boltzmann-Gibbs entropy and boundsfor dynamical entropy
In this section we discuss the behavior of the dynamentropy in another limit,A↓0 (k→`). We first introduce theBoltzmann-Gibbs~BG! entropyof the noise
HBG~s!ª2EXdm I s
~x!EXdyPs~x2y!ln Ps~x2y!
52EXdjPs~j!ln Ps~j!. ~10!
For interpretation and generalizations of this quantity, somtimes calledcontinuous entropy, consult the monographs oGuiasu @26#, Martin and England@27#, Jumarie @28#, orKapur @29#. In the simplest case of the rectangular nogiven byPb(x)ªQ(b/22x)Q(x1b/2)/b, for 1>b.0 andxPX, the BG entropy is equal to lnb. Note that this quantityvanishes for the noise uniformly spread over the entire sp(b51), becomes negative forb,1, and diverges to minusinfinity in the deterministic limitb→0.
For the systemf s combining the deterministic evolutionfand the stochastic perturbation, the probability densitytransition from x to y during one time step is given bPs„f (x),y…5Ps„f (x)2y… for x,yPX. The BG entropy forthis system can be defined as
HBG~ f s!52EXdm f s
~x!EXdyPs„f ~x!,y…ln Ps„f ~x!,y….
~11!
Due to the homogeneity of the noise and due to the perioboundary conditions the integral overy in Eq. ~11! does notdepend onx. Therefore for any systemf perturbed by a non-zero noise (s.0) one obtains
HBG~ f s
an
ondnm-c
si
ldy
h
nd
thr
oe
en
is
ioemhe
h
nt
e
-it
thes-otem
py
nfor
eb-
to
ofdise.
aos-
ex-seethin
tion
PRE 62 2021DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
tem and the corresponding deterministic system,investigated its dependence on the noise strengths and thenumber of the time stepsn.
In order to avoid problems with the unbounded growththe total entropy for sufficiently fine partitions Gaspard aWang studiede entropy@31#, where the supremum is takeover the class of the partitions, for which the minimal diaeter of a cell is larger thane. This quantity can be numerically approximated by the algorithm of Cohen and Procac@32#. The e entropy diverges logarithmically in the limite→0; the character of this divergence may be used to clasvarious kinds of random processes@31,33#.
The dependence of the dynamical entropy on time yieanother interesting problem. For discrete deterministic stems the KS entropy is additive in time:HKS( f T)5THKS( f ). On the other hand, it follows from Eq.~19! thatthe dynamical entropy of a noisy system fulfillsHdyn( f s
T)<2HBG(s), for each timeT. Thus for a nonzeros the ratioHdyn( f s
T)/T tends to zero in the limitT→`, while for thedeterministic dynamics@HKS( f T)#/T5HKS( f ). The symbolf s
T represents the same deterministic systemf, subjected tothe stochastic perturbation only once forT time steps. Therelated issue has been recently raised by Fox@34# in thecontext of deterministic evolution of Gaussian densities. Tdiscontinuity of Hdyn( f s
T)/T in the limit s→0 is a conse-quence of the fact that the other limits, time to infinity anoise strength to zero, do not commute.
In some sense this resembles the noncommutativity oflimits time→` and\→0 in quantum mechanics, crucial foinvestigation of the so-calledquantum chaos~see, e.g.,@35,36#!. Continuing this analogy even further, the entropynoise corresponds to the entropy of quantum measurem@22,23#, while the Boltzmann-Gibbs entropyHBG plays therole of the Wehrl entropy@37#, recently used by two of us~W.S., K.Z.! to estimate the coherent states dynamicaltropy @24#.
D. Systems with rectangular noise
We now discuss the computation of the entropy of nofor the rectangular noisePb ~see Sec. II C! with the periodicboundary conditions imposed. Computation of the transitprobabilities in Eq.~3! reduces to simple convolutions of threctangular noise and is straightforward for the first few tisteps. For largern the calculations become tedious, and tconvergence in the definition of entropy~4! is rather slow~not faster than 1/n). It is hence advantageous to consider tsequence ofrelative entropies GnªHn2H (n21) which con-verge much faster to the same limitH( f ;A) @30,32#. Forsome systems the exponential convergence of this quawas reported@38–40#.
In our analytical and numerical computations we usrelative entropiesGn . In all of the cases studied, the termG7gives the entropy limn→`Gn with a relative error smallerthan 1025. For a partition consisting of two equal cells (k52) and the rectangular noisePb we obtained an explicitexpression forG4, as a function of the noise widthb. Ana-lytical results obtained in@25# that are too lengthy to reproduce here give an approximation of the entropy of noise wprecision 1024.
d
f
-
ia
fy
ss-
e
e
fnt
-
e
n
e
e
ity
d
h
We analyzed the dynamical entropy of the Re´nyi mapf s(x)5@sx#mod1 ~with integer parameters) subjected to therectangular noise. Independently of the noise strength,uniform distribution remains the invariant density of this sytem. For a large noiseb;1 the dynamical entropy is close tzero, since the difference between the noise and the syswith noise is hardly perceptible. The dynamical entrogrows with the decreasing noise widthb and in the determin-istic case seems to tend to the KS entropy of the Re´nyi mapHKS( f s)5 ln s. For more involved systems the computatioof the dynamical entropy becomes rather difficult eventhis simple rectangular noise. In order to avoid calculatingkn
different integrals in Eq.~3!, in the subsequent section wintroduce the class of noises for which computing of proabilities Pi 0 , . . . ,i n21
for any dynamical system reducesmultiplication of matrices.
III. SYSTEMS WITH SMOOTH NOISEOF DISCRETE STRENGTHS
A. Model distribution of noise
In this section we define the particular discrete familythe probability distributionsPN representing the noise anstudy properties of dynamical system subjected to this noAs above, we consider one-dimensional spaceX5@0,1) andimpose periodic boundary conditions. We shall look forkernelP(x,y) homogeneous, periodic, and being decompable in a finite basis
P~x,y
a-
on
rte,
am
b
r
ra
the
be-
el
hed
2022 PRE 62ANDRZEJ OSTRUSZKAet al.
wherexPX and r ,l 50, . . . ,N. We do not require their or-thonormality. Expanding cosine as a sum to theNth power inEq. ~21! we find that the (N11)3(N11) matrixA, definedin Eq. ~20!, is for this noise diagonal
Alr 5CNS Nl D d lr . ~24!
The parameterN controls the strength of the noise mesured by its variance
s251
2p2C8S N
211D5
1
2p2 S (k5(N/2)11
`1
k2D , ~25!
whereC8 stands for the derivative of the digamma functi@41#.
Figure 1 presents the densities of the noise forN510, 20,50, and 100. The deterministic limit is obtained by lettingNtend to infinity. SinceN determines the size of the Hilbespace, in which the evolution of the densities takes placcan be compared with the quantum numberj ;1/\ used inquantum mechanics. Note that for each value of the pareter N the probability function PN(x).0 for xÞ1/2~mod1!, so the analyzed perturbation is notlocal in the senseof Blank @18#.
It is worthwhile to mention that the properties~20! arepreserved for the kernelPN„f (x),y… describing the dynamicsof the system with noise~1!. The expansion matrixA is thesame, if one uses the modified basis functions defineduk(x)ªuk„f (x)…, for xPX, which explicitly depend on thedeterministic dynamicsf. To illustrate some features of oumodel we plot in Fig. 2 the transition kernelPN„f (x),y… forthe logistic map perturbed by the noise defined in Eq.~21!with N520.
B. Invariant measure for systems with stochastic perturbation
The density of the invariant measurerP of the system~1!is given as the eigenstate of the Frobenius-Perron ope
FIG. 1. Probability density of the noiseP(x) for N510, 20, 50, and 100.
it
-
y
tor
M P corresponding to the largest eigenvalue equal 1. Fordeterministic system, the invariant densityr fulfills the for-mal equation
r~y!5E0
1
d„f ~x!2y…r~x!dx. ~26!
In the presence of a stochastic perturbation this equationcomes
rP~y!5„M P~rP!…~y!
5E0
1E0
1
P~x8,y!d„f ~x!2x8…rP~x!dx8dx
5E0
1
P„f ~x!,y…rP~x!dx, ~27!
whereM P is theFrobenius-Perron~FP! operatorconnectedwith the noisy system~1!. Let us assume that the kernsatisfies the conditions~20! listed in Sec. III A and so it canbe expanded asP„f (x),y…5( l ,r 50
N Alr ur„f (x)…v l(y). Thenwe have
M P~r!~y!5E0
1
(l ,r 50
N
Alr ur„f ~x!…v l~y!r~x!dx
5 (l ,r 50
N
Alr F E0
1
ur„f ~x!…r~x!dxGv l~y!
5(r 50
N F E0
1
ur„f ~x!…r~x!dxG v r~y! ~28!
for yPX, wherev r5( l 50N Alr v l . Thus, any initial density is
projected by the FP operatorM P into the vector spacespanned by the functionsv r ;r 50, . . . ,N, and so its imagemay be expanded in the basis$v l% l 50, . . . ,N . This statementconcerns also the invariant densityrP . ExpandingrP
FIG. 2. Transition kernelPN„f (x),y… for the logistic mapf (x)54x(12x) with the noise characterized byN520. The darkercolor denotes a higher value of the kernel according to the attacscale. The variablex is periodic;x5xmod1.
he
th
ea
cesrn
e
fo
f
ral
bi
prer
in
op-f an
lex, inan
ity.the
rac-
xts
ofe
PRE 62 2023DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
rP5(l 50
N
q~P! l v l ~29!
with unknown coefficientsq(P) l and inserting this intoEq. ~28! we obtain the eigenequation for the vector of tcoefficientsq(P)5$q(P)0 , . . . ,q(P)N%
q~P!5Dq~P!. ~30!
The FP operator is represented here by(N11)-dimensional matrixD5:BA, whereA is given byEq. ~24!, and the entries of the matrixB are given by
Brm5E0
1
ur„f ~x!…vm~x!dx ~31!
for n,m50, . . . ,N. Observe thatA does not depend on thdeterministic dynamicsf, while B depends on the noise vithe basis functionsu andv. Furthermore, note that
Drm5E0
1
ur„f ~x!…vm~x!dx ~32!
for r ,m50, . . . ,N, and
M PS (l 50
N
ql v l~y!D 5(l 50
N
~Dq! l v l~y! ~33!
for each vectorq5$q0 , . . . ,qN%PRN11. It follows fromEqs.~28! and~33! that there is a one-to-one correspondenbetween the eigenvectors ofD and the eigenfunctions of thFP operatorM P . The latter has a one-dimensional eigenpace corresponding to the eigenvalue 1, since the keP(x,y) vanishes only forx2y51/2 ~mod1!, which impliesthat the two-step probability*P(x,z)P(z,y)dz.0 for x,yPX ~see@17#, Th. 5.7.4!. Thus equation~30! has the uniquesolutionq(P) fulfilling
(r 50
N
q~P!rE0
1
v r~y!dy51, ~34!
or equivalently^q(P),t&51, where the vectortPRN11 isgiven byt5„*Xv0(y)dy,*Xv1(y)dy, . . . ,*XvN(y)dy…. Wefind it diagonalizing numerically the matrixD. The functionrP given by Eq.~29! is then the invariant density for thsystem with noisef s .
This technique was used to find the invariant measurethe logistic map given byf (x)54x(12x) for xPX in thepresence of noise. Figure 3 presents the invariant densitythe logistic map with noise parameterN520. It can be com-pared to the histogram showing the density of the 11th itetion of one million uniformly distributed random initiapoints.
In the deterministic limits→0 the size of the matrixN11 grows to infinity. We believe that our approach canused to approximate the invariant measure of the determistic system by decreasing the noise strength. Figure 4sents a plot of invariant densities for the logistic map pturbed with the trigonometric noise forN510, 20, 50, and100, compared with the invariant measure for the determistic caseN→` given by @42#
e
e
-el
r
or
-
en-e--
-
r~y!51
pAy~12y!~35!
for yPX.
C. Spectrum of randomly perturbed systems
The spectral decomposition of the Frobenuis-Perronerators corresponding to classical maps is a subject ointensive current research@43–47,17,34,48#. The spectrumof a FP operator is contained in the unit disk on the compplane and depends on the choice of a function spacewhich acts the FP operator. If the dynamical system hasinvariant density, the largest eigenvalue is equal to the unThe radius of the second largest eigenvalue determinesrate of convergence to the invariant measure. To cha
FIG. 3. Invariant density for the logistic mapf (x)54x(12x)subjected to the trigonometric noise (N520): the solid line repre-sents the density related to the leading eigenvector of the matriD;the histogram is obtained by iteration of one million of initial poinby the noisy map.
FIG. 4. Invariant densities of the logistic map for parametersthe trigonometric noiseN510, 20, 50, and 100, together with thdeterministic limitN→`.
n
s
und
rovhrv
liz
Tian
a
rss
re
thha
imyo
gee
rba-
on
g
r
ini-
deer
eed
thee
.
and
.
2024 PRE 62ANDRZEJ OSTRUSZKAet al.
terize the spectrum one definesessential spectral radius r. Itis the smallest non-negative number for which the elemeof the spectrum outside the disk of radiusr, centered at theorigin, are isolated eigenvalues of finite multiplicity. It washown @49# that for one-dimensional piecewiseC2 expand-ing maps and the FP operator defined on the space of ftions of bounded variations, the spectral radius is relatethe expanding constant.
We analyzed the spectral properties of the FP operatothe perturbation of the logistic map, for which the Lyapunexponents equals ln~2! ~in analogy to the tent map for whicr 51/2 we draw such a circle to guide the eye and obsethat besides the leading eigenvalues, all others are locainside this circle!. The interval@0,1# is joined into a circle tokeep the system conservative in the presence of noise.FP operator of the system subjected to the shift-invaradditive perturbationPN is represented by the matrixD ofthe sizeN11. We obtained its spectrum by the numericdiagonalization. The largest eigenvaluel1 of D was equal tothe unity up to a numerical error of order 10210. The abso-lute value of the second eigenvaluel2 was found to ap-proachr51/2 in the deterministic limitN→`. Since the ma-trix D has real entries, its eigenvalues are real or appeaconjugate pairs. Figure 5 presents the largest eigenvaluethis system forN510, 20, 50, and 100. All other eigenvalueare so small that they coincide with the origin in the pictuObserve that eigenvalues do not tend to the valueslm51/4m21 for m51,2, . . . found for the deterministic systemin @34#.
Our results show that the structure of the spectrum ofFP operator of a deterministic system depends on the cacter of the method used to approximate it. Introducingrandom noise may be considered as a possible approxtion, since it enables us to represent the FP operator bmatrix of a finite dimension. In other words, the presencethe noise predetermines a certain space, in which the eistates live. This numerical finding corresponds to the rec
FIG. 5. Spectra of the FP operator for the logistic mapf (x)54x(12x) subjected to the noisePN with N510, 20, 50, and 100
ts
c-to
of
eed
het
l
inof
.
er-
aa-afn-
nt
results of Blank and Keller@50#, who showed the instabilityof the spectrum for some maps subjected to certain pertutions.
IV. COMPUTING ENTROPY FOR SYSTEMSWITH EXPANDABLE NOISE
A. Matrix formulation of probability integrals
Our aim is to compute probabilities entering the definitiof the total dynamical entropy of a noisy systemPi 0 , . . . ,i n21
Pi 0 , . . . ,i n215E
Ei 0
rP~x0! dx0EEi 1
dx1•••
3EEi n21
dxn21P„f ~x0!,x1…P„f ~x1!,x2…•••
3P„f ~xn22!,xn21…. ~36!
Introducing (n21) times the expansion~20! applied to thekernel P„f (x),y… and interchanging the order of summinand integration we arrive at
Pi 0 , . . . ,i n215tT @D~ i n21!•••D~ i 1!D~ i 0!#q~P!, ~37!
where A, t, and q(P) are defined in Sec. III B,D( i )5B( i )A, and matricesB( i ) are given by the integral ovethe cell Ei , i.e., B( i ) rl 5*Ei
ur„f (x)…v l(x)dx for i51, . . . ,k; r ,l 50, . . . ,N in the analogy to Eq.~31!.
The above formula provides a significant simplificationthe computation of entropy. Instead of performing multidmensional integrals in Eq.~36!, we start from computing thematricesD( i ) for any celli 51, . . . ,k, and receive the desireprobabilities by matrix multiplications. By this method thprobabilities may be efficiently obtained even for largnumbers of the time stepsn. The only problem consists inthe number of terms in Eq.~3!, equal tokn, which for largernumber of cellsk becomes prohibitively high. To overcomthis difficulty we apply in this case the technique of iteratfunctions systems presented below.
B. Computation of entropy via IFS
In this section we present a method of computingdynamical entropy~7!, which is especially useful when thnumber of cellsk of the partition of the spaceX is large. Weuse the concept of iterated function systems~IFSs!, discussedin detail in the book of Barnsley@51#. Consider the set ofkfunctionspi :Y°R1 and mapsFi :Y°Y defined as@52,53#
pi~z!5t TD~ i !z, i 51, . . . ,k, zPY,~38!
Fi~z!5D~ i !z
pi~z!, i 51, . . . ,k, zPY,
where the vectort and the matricesD( i ) are defined in SecsIII B and IV A, respectively, andY,RN11 is the convexclosure of the set of all vectors of the formu„f (x)… forxP@0,1#.
Let us stress that the spacesX5@0,1# and the(N11)-dimensional spaceY are different. The normalizationof the kernel*XP„f (x),y…dy51 for xPX leads to the con-dition ( i 51
k pi(z)51 for any zPY. Therefore the functionspi can be interpreted as place-dependent probabilitiestogether with the functionsFi form an IFS. It is uniquely
eate
ot
row
thle
se
he
he
is
py
y
ye
s,s-
-
llsarepy
-
r
PRE 62 2025DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
determined by the dynamical systemf with the noise givenby the densityP and a specifick-element partitionA. Thus,the number of cellsk determines the size of the IFS. It can bshown @53# that the entropy of the considered dynamicsystem with noise is equal to the entropy of the associaIFS.
The IFS generates a Markov operatorM acting on thespace of all probability measures onY. For any measurablesetS,Y the following equality holds:
~Mn!~S!5(i 51
k EFi
21(S)pi~w!dn~w!. ~39!
It describes the evolution of the measuren transformed byM. If the functionsFi fulfill the strong contraction condition@51#, there exists a unique attracting invariant measurem forthis IFS
Mm5m, ~40!
which, in general, displays multifractal properties@54#. Thetotal entropy can be computed as the Shannon entrhk(p1 , . . . ,pk)52( i 51
k pi ln pi averaged over the invarianmeasure@23,52#
Htot~ f s ;A!5EYhk„p1~y!, . . . ,pk~y!…dm~y!. ~41!
The calculation of such an integral from the definition coresponds to the matrix method presented previously. Hever, the existence of the attracting invariant measurem andthe Kaijser-Elton ergodic theorem@55,56# assures that
Htot~ f s ;A!5 limn→`
1
n (l 50
n21
hk~yl !, ~42!
where $yl% is a generic random sequence produced byIFS. Such a method of computing an integral is often cala random iterated algorithm@51#. We start computationsfrom an arbitrary initial pointy0, iterate it by the IFS, andcompute the average~42! along a random trajectory. Toavoid transient dependence on the initial pointy0 one shouldnot take into account a certain number of initial iterationNote that the computing time grows only linearly with thnumber of cellsk and one does not need to perform tburdensome time limit~4!.
We used a similar method to compute the quantum coent states entropy@23# and the Re´nyi entropies for certainclassical deterministic maps@54#.
V. DYNAMICAL ENTROPY FOR NOISYSYSTEMS — EXEMPLARY RESULTS
In this section we will study the entropy of the Re´nyi mapand the logistic map perturbed by the trigonometric nogiven by Eq.~21!. We will consider the partitionsAk of theinterval @0,1# into k equal subintervals. We putH(k)ªH(Ak), Htot(N,k)ªHtot( f N ;Ak), Hnoise(N,k)ªHnoise( f N ;Ak), Hdyn(N,k)ªHdyn( f N ;Ak), andHdyn(N)ªHdyn( f N).
ld
py
--
ed
.
r-
e
A. Boltzmann-Gibbs entropy
A simple integration allows us to obtain the BG-entroHBG(N) for this kind of noise,
HBG~N!52E0
1
dj CN cosN~pj!ln@CN cosN~pj!#
5N
2 FCS N
2 D2CS N11
2 D G112 ln CN , ~43!
whereC denotes the digamma function@41# and the normal-ization constantCN is given by Eq.~22!.
It follows from Eq. ~43! that in the deterministic limit(N→`) the BG entropy diverges to minus infinity, namel
limN→`
2HBG~N!
ln N5
1
2. ~44!
This relation shows how the maximal dynamical entrop2HBG @see Eq.~18!#, admissible by a certain level of thnoise, grows logarithmically in the deterministic limit.
B. Entropy of the noise
We used the matrix method of computing probabilitiewhich led to partial entropiesHn and the relative entropieGn . Fast ~presumably exponential! convergence of the sequenceGn allowed us to approximate the entropy byG7with accuracy of order'1026. Figure 6 presents the dependence of the entropy of the noiseHnoise(N,k) on the numberof cells k in the partitionAk for two different amplitudes ofnoise (N510 and 20). The data for large number of ce(k>20) are obtained by the technique of IFS. The resultscompared with the upper and lower bounds for the entroof the noise which occurred in Eq.~14!. It follows from Eq.~14! that the entropy diverges logarithmically with the num
FIG. 6. Dependence of the entropy of the noiseHnoise(k) on thenumber of cellsk in partition for N510 (s) and 20 (h). Opensymbols are obtained with the matrix method, while the data fok>20 are received with the IFS technique~closed symbols!. Thesolid line represents the upper boundH(k)5 ln k, while two dashedlines provide lower bounds given by Eq.~14!.
-as
sistate
th
ye
em
it
n-
he
toise
n
s
ofndestmi-y
KS
pyousrges
2026 PRE 62ANDRZEJ OSTRUSZKAet al.
ber of cellsk in the partition. For a fixed partition it decreases to zero with decreasing strength of the stochperturbation~increasing parameterN).
C. Entropy for the noisy Renyi map
The Renyi map f (s)(x)5@sx#mod 1 (sPN), with explicitlyknown metric entropyHKS( f (s))5 ln s, is particularly suit-able to test changes of the dynamical entropy with stochaperturbation. Results obtained for the trigonometric no~21! are much more accurate than these obtained for recgular noise and reviewed briefly in Sec. II D. Data presenbelow are received for the Re´nyi map with s56 ~we put f5 f (6)). Dependence of the total entropyHtot(N,k) on thenumber of cellsk is presented in Fig. 7~a! for four levels ofnoise (N510, 20, 50, and 100). The solid line representsentropy of the partitionH(k)5 ln k ~upper bound! and thedashed line provides theN-dependent lower bound given bH(k)1HBG(N) ~for N510), while the stars denote th
FIG. 7. Entropies for the Re´nyi map f (x)5@6x#mod1 perturbedby the noise withN510 (s), 20 (h), 50 (1), and 100 (n)as a function of the number of cellsk: ~a! The total entropyHtot(k).The solid line represents the upper bound (HAk
) while the dashedline provides the lower bound~13! for N510. ~b! The dynamicalentropyHdyn(k). The maximum of each curve givesHdyn as rep-resented on the right side.
tic
ticen-d
e
partition-dependent entropy of the deterministic systgiven by f. It saturates at the generating partitionkg56 andachieves the valueHKS( f )5 ln(6). It seems that this valuegives another lower bound for the total entropyHtot . Thetotal entropy and the entropy of the noise diverge in the limof fine partitionA↓0 (k→`), but their difference remainsbounded.
Figure 7~b! shows the differenceHdyn(N,k)5Htot(N,k)2Hnoise(N,k) necessary for computation the dynamical etropy ~9!. This quantity tends to zero fork51 andk→` ~17!and achieves its maximum—giving a lower bound for tdynamical entropyHdyn(N)—close to the number of cellskgin the generating partition. Dynamical entropy is equalzero for N50 and increases with the decreasing nostrength. In the limitN→` it is conjectured to tend to theKS entropy of the deterministic systemHKS5 ln(6) repre-sented by a horizontal line.
D. Entropy for the noisy logistic map
A similar study was performed for the logistic map giveby f (x)54x(12x) for xP@0,1# perturbed by the trigono-metric noise~21!. As before we treat the intervalX as a circlesetting f (x)5 f (xmod1). Numerical data produce pictureanalogous to those obtained for the Re´nyi map with s56.Instead of presenting them here, we supply a compilationthe results for both systems. Computing total entropy aentropy of the noise for several partitions we took the largdifference between them as an approximation of the dynacal entropy~9!. Figure 8 shows how the dynamical entropchanges with the noise parameterN for both systems. It isconjectured to tend to the corresponding values of theentropy @ln~2! for the logistic map and ln~6! for the Renyimap# in the deterministic limitN→`.
VI. CONCLUDING REMARKS
The standard definition of the Kolmogorov-Sinai entrois not applicable for systems in the presence of a continurandom noise, since the partition-dependent entropy dive
FIG. 8. Dynamical entropyHdyn for the Renyi map (s) and thelogistic map (h) depicted as functions of the noise parameterN.Horizontal lines represent the values ln~6! and ln~2! of the KS en-tropy for the corresponding deterministic maps.
ofuny
s,ete
zeanblce
hem
reds
thno
ea
ocehis.bb
rl
igbthtr
ticae
crin
e
a
e,as
be
tw
mitlts
chave
-a
eslso
ion
ture
PRE 62 2027DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
in the limit of a fine partition. We generalize the notionthe KS entropy for dynamical systems perturbed by ancorrelated additive noise. The total entropy of a random stem is split into two parts: thedynamical entropyand theentropy of the noise. In the deterministic limit~the varianceof the noise tends to zero! the entropy of the noise vanishewhile the dynamical entropy of the stochastically perturbsystem is conjectured to tend to the KS entropy of the deministic system.
The continuous Boltzmann-Gibbs entropy characterithe density of the distribution of the noise. It providesupper bound for the maximal dynamical entropy observaunder the presence of this noise. If the BG entropy is equazero such a noise may be called disruptive because onenot draw out any information concerning the underlying dterministic dynamics. Investigating properties of the dynamcal entropy we find that the two limits, the diameter of tpartition to zero, and the noise strength to zero do not comute, and we point out some consequences of this fact.
Computation of the dynamical entropy becomes easieone assumes that the density of the noise can be expanda finite ~N11!-element basis consisting of continuous bafunctions. In this case we find a simple way of computingprobabilities of trajectories passing through a given sequeof the cells in the partition. The calculations are basedmultiplication of matrices of sizeN11 and the computingtime grows linearly with the length of a trajectoryn. On theother hand, diminishing the noise strength causes an incrof the matrix dimension.
For each dynamical system perturbed by this kindnoise and for a givenk-element partition of the phase spawe construct an associated iterated function system, wconsists ofk functions with place-dependent probabilitieand acts in a certain (N11)-dimensional auxiliary spaceEntropy of the dynamical system with noise is shown toequal to the entropy of IFS, which can be easily computedthe random iterated algorithm. This method is particulasuitable for large number of cellsk, for which the number ofpossible trajectories grows in time askn.
We study some one-dimensional maps perturbed by trnometric noise, for which the basis functions are giventrigonometric functions. In this case we can representFrobenius-Perron operator for the noisy system by a maof size (N11). Diagonalizing this matrix numerically wefind the spectrum of this operator. Analyzing the logismap subjected by such a random perturbation we indicthat the invariant measure tends to the invariant measurthe deterministic system in the limitN→`. On the otherhand, the spectrum of the Frobenius-Perron operator desing the noisy system need not to tend to the correspondcharacteristics of the deterministic system.
The deterministic limitN→` resembles in a sense thsemiclassical limit of quantum mechanics\→0. For ex-ample, if one discuss the quantum analogs of classical mon the sphere@57#, the size of the Hilbert space 2j 11 be-haves as 1/\, wherej is the spin quantum number. Thereforit would be interesting to analyze such two-dimensional clsical systems in the presence of noise~in the case of two-dimensional ‘‘trigonometric’’ noise the FP operator canrepresented by a matrix of the sizeN2) and to compare howthe spectrum of a given classical map is approached in
-s-
dr-
s
letoan--i-
-
ifin
eecen
se
f
ch
ey
y
o-yeix
teof
ib-g
ps
-
o
complementary limits: the semiclassical limitj→` of thecorresponding quantum map and the deterministic liN→` of a classical noisy system. Some preliminary resuon the related issue of truncating the infinite matrix whirepresents the FP operator of a deterministic system hbeen achieved very recently@58,59#.
ACKNOWLEDGMENTS
One of us~K.Z.! would like to thank Ed Ott for the hospitality he received at the University of Maryland, wherepart of this work was done. A.O. gratefully acknowledgsupport by the Subsydium FNP 1/99. This work was asupported by the Polish KBN Grant No. P03B 060 13.
APPENDIX: EXEMPLARY ITERATED FUNCTIONSYSTEM
To illustrate the IFS method we discuss the computatof the entropy of the noise given by Eq.~21! for N52 andfor the partition of the interval@0,1# into k54 equal cells. Inthis case Eq.~38! gives the IFS consisting ofk54 functionsacting in a three-dimensional spaceY,@21,1#3. The prob-abilities pi are place dependent,
p1~x,y,z!5x~p12!
16p1
y
8p1
z~p22!
16p,
p2~x,y,z!5x~p22!
16p1
y
8p1
z~p12!
16p,
~A1!
p3~x,y,z!5x~p22!
16p2
y
8p1
z~p12!
16p,
p4~x,y,z!5x~p12!
16p2
y
8p1
z~p22!
16p,
while the functions read
FIG. 9. Fractal support of the invariant measurem! of the IFSassociated with the trivial dynamical systemf (x)5x in the pres-ence of the trigonometric noise withN52. The number of cellsk54 determines the number of functions in the IFS and the strucof the depicted set.
in
2028 PRE 62ANDRZEJ OSTRUSZKAet al.
F1~x,y,z!51
p1~x,y,z! Fx~813p!
32p1
3y
16p1
z
32;
3x
8p1
y
161
z
8p;
x
321
y
16p1
z~3p28!
32p G ,F2~x,y,z!5
1
p2~x,y,z! Fx~3p28!
32p1
y
16p1
z
32;
x
8p1
y
161
3z
8p;
x
321
3y
16p1
z~3p18!
32p G ,~A2!
F3~x,y,z!51
p3~x,y,z! Fx~3p28!
32p2
y
16p1
z
32; 2
x
8p1
y
162
3z
8p;
x
322
3y
16p1
z~3p18!
32p G ,F4~x,y,z!5
1
p4~x,y,z! Fx~3p18!
32p2
3y
16p1
z
32; 2
3x
8p1
y
162
z
8p;
x
322
y
16p1
z~3p28!
32p Gfor (x,y,z)PY.
Figure 9 presents the support of the invariant measurem for this IFS. Applying the random iteration algorithm we obtain this case the entropy of the noiseHnoise51.1934. Ironically, a less interesting~more contracting! fractal picture leads to afaster convergence of the averages~42!.
ca
ys
.
J.
s
ticI,
s
cs
r
r,
-or-
,
@1# G. Paladin, M. Serva, and A. Vulpiani, Phys. Rev. Lett.74, 66~1995!.
@2# V. Loreto, G. Paladin, M. Pasquini, and A. Vulpiani, PhysiA 232, 189 ~1996!.
@3# A. Namenson, E. Ott, and T. M. Antonsen, Phys. Rev. E53,2287 ~1996!.
@4# V. Loreto, G. Paladin, and A. Vulpiani, Phys. Rev. E53, 2087~1996!.
@5# J. P. Crutchfield, D. J. Farmer, and B. A. Huberman, PhRep.92, 45 ~1982!.
@6# J. M. Deutsch, J. Phys. A18, 1457~1985!.@7# K. Matsumoto and I. Tsuda, J. Phys. A18, 3561~1985!.@8# T. Kapitaniak,Chaos in Systems with Noise~World Scientific,
Singapore, 1988!.@9# A. R. Bulsara, E. W. Jacobs, and W. C. Schieve, Phys. Rev
42, 4614~1990!.@10# L. Yu, E. Ott, and Q. Chen, Phys. Rev. Lett.65, 2935~1990!.@11# P. Cvitanovic, C. P. Dettmann, R. Mainieri, and G. Vattay,
Stat. Phys.93, 981 ~1998!.@12# Yu. Kifer, Izv. Acad. Nauk SSSR Mat.38, 1091~1974! @Izv.
Acad. Sci. USSR, Math8, 1083~1974!#.@13# M. Boyarsky, Trans. Am. Math. Soc.257, 350 ~1980!.@14# L.-S. Young, Erg. Theor. Dyn. Sys.6, 311 ~1986!.@15# Y. Kifer, Random Perturbation of Dynamical System
~Birkhauser, Boston, 1988!.@16# V. Baladi and L.-S. Young, Commun. Math. Phys.156, 355
~1993!.@17# A. Lasota and M. C. Mackey,Chaos, Fractals and Noise
~Springer-Verlag, New York, 1994!.@18# M. Blank, Discreteness and Continuity in Problems of Chao
Dynamics~American Mathematical Society, Providence, R1997!.
@19# M. Blank and G. Keller, Nonlinearity10, 81 ~1997!.@20# A. Boyarsky and P. Go´ra, Laws of Chaos~Birkhauser, Boston,
1997!.@21# I. P. Cornfeld, S. V. Fomin, and Ya. G. Sinai,Ergodic Theory
~Springer-Verlag, Berlin, 1982!.@22# W. Słomczynski and K. Zyczkowski, J. Math. Phys.35, 5674
~1994!.
.
A
@23# J. Kwapien, W. Słomczyn´ski, and K. Zyczkowski, J. Phys. A30, 3175~1997!.
@24# W. Słomczynski and K. Zyczkowski, Phys. Rev. Lett.80, 1880~1998!.
@25# A. Ostruszka, Master thesis, Krako´w, 1997~unpublished!.@26# S. Guias¸u, Information Theory with Applications~McGraw-
Hill, New York, 1977!.@27# N. F. G. Martin and J. W. England,Mathematical Theory of
Entropy ~Addison-Wesley, Reading, MA, 1981!.@28# G. Jumarie,Relative Information~Springer-Verlag, Berlin,
1990!.@29# J. N. Kapur,Measures of Information and Their Application
~John Wiley & Sons, New York, 1994!.@30# J. P. Crutchfield and N. H. Packard, Physica D7, 201 ~1983!.@31# P. Gaspard and X.-J. Wang, Phys. Rep.235, 291 ~1993!.@32# A. Cohen and I. Procaccia, Phys. Rev. A31, 1872~1985!.@33# P. Gaspard,Chaos, Scattering and Statistical Mechani
~Cambridge University Press, Cambridge, 1998!.@34# R. F. Fox, Chaos5, 619 ~1995!.@35# G. Casati and B. Chirikov, inQuantum Chaos. Between Orde
and Disorder, edited by G. Casati and B. Chirikov~CambridgeUniversity Press, Cambridge, 1995!.
@36# A. V. Andreev, B. D. Simons, O. Agam, and B. L. AltshuleNucl. Phys. B482, 536 ~1996!.
@37# A. Wehrl, Rep. Math. Phys.16, 353 ~1979!.@38# P. Sze´pfalusy and G. Gyo¨rgyi, Phys. Rev. A33, 2852~1986!.@39# M. Misiurewicz and K. Ziemian, inProceedings of the Con
ference on Ergodic Theory and Related Topics II, Gegenthal, 1986, edited by H. Michel~Teubner, Leipzig, 1987!,pp. 147–159.
@40# K. Zyczkowski and W. Słomczyn´ski, Phys. Rev. E52, 6879~1995!.
@41# I. S. Gradshteyn and I. M. Ryzhik,Table of Integrals, SeriesProducts~Academic, New York, 1965!.
@42# E. Ott, Chaos in Dynamical Systems~Cambridge UniversityPress, Cambridge, 1993!.
@43# P. Gaspard, J. Stat. Phys.68, 673 ~1992!.@44# I. Antoniou and S. Tasaki, Physica A190, 303 ~1992!.@45# H. H. Hasegawa and W. C. Saphir, Phys. Rev. A46, 7401
~1992!.
PRE 62 2029DYNAMICAL ENTROPY FOR SYSTEMS WITH . . .
@46# P. Gaspard, Chaos3, 427 ~1993!.@47# H. H. Hasegawa and D. J. Driebe, Phys. Rev. E50, 1781
~1994!.@48# R. F. Fox, Chaos7, 254 ~1997!.@49# G. Keller, Commun. Math. Phys.96, 181 ~1984!.@50# M. Blank and G. Keller, Nonlinearity11, 1351~1998!.@51# M. F. Barnsley,Fractals Everywhere, 2nd ed. ~Academic,
Boston, 1993!.@52# W. Słomczynski, Chaos, Solitons and Fractals8, 1861~1997!.@53# W. Słomczynski ~unpublished!.@54# W. Słomczynski, J. Kwapien´, and K. Zyczkowski, Chaos10,
180 ~2000!.@55# T. Kaijser, Rev. Roum. Math. Pures Appl.26, 1075~1981!.@56# J. H. Elton, Ergod. Theor. Dynam. Sys.7, 481 ~1987!.@57# F. Haake,Quantum Signatures of Chaos~Springer, Berlin,
1991!.@58# J. Weber, F. Haake, and P. Sˇeba, LANL Report No. nlin.CD/
0001013, 2000.@59# S. Fishmann, inSupersymmetry and Trace Formulae, edited
by I. V. Lerner, J. P. Keating, and D. E. Khmelnitskii, NATOAdvanced Study Insitute, Series B: Physics Vol. 370~KluwerAcademic/Plenum Publishers, New York, 1999!.
