Signal processing in complex networks : many faces of

Warsaw University of TechnologyFaculty of Physics

Koszykowa 75, PL-00-662 Warsaw, PolandTel: (48) 22 6607267; fax: (48) 22 6282171; http://www.if.pw.edu.pl

Wojciech Zduniak Wojciech Zduniak andandAndrzej KrawieckiAndrzej Krawiecki

SignalSignal processingprocessing inin complexcomplex networksnetworks: : many many facesfaces ofof

structuralstructural stochasticstochastic multiresonancemultiresonance

SummerSummer SolsticeSolstice 2011 2011 InternationalInternational ConferenceConference on on DiscreteDiscrete ModelsModels ofof ComplexComplex SystemsSystems

W. Zduniak and A. Krawiecki Signal processing in complex networks: many faces of structural stochastic multiresonance

Summer Solstice 2011 International Conference on Discrete Models of Complex Systems 6-10 June 2011, Turku, Finland

Stochastic resonance consists in the maximization of the response of a (nonlinear) system to a periodic stimulation by addition of noise at the input [R. Benzi, A. Sutera, and A. Vulpiani, J. Phys. A14, L453 (1981).]

( )ttDtt

xxxV

ttAdx

dV

dt

dxs

′−=′

+−=

++−=

δξξ

ξω

2)()(41

21

)(

)(sin

42

Example Overdamped particle in a bistable potential well

For optimum noiseintensity D the position ofthe particle in the twopotential wells issynchronized with theperiodic signal

Escape rate from theright or left potential wellis given by the Kramersformula and isasymetrically modulatedby the periodic signal

( )

,sin

exp2

1

∆−

=±

D

tAV

tr

s

K

ωπ

m



SP(ωωωωS)

SN(ωωωωS)

SPA

>+<−

=0)(dla1

0)(dla1)(

tx

txty

Characterization of stochasticresonance

Output signal

Measures:• Signal-to-noise ratio (SNR)• Spectral power amplification (SPA)

( )( )

( )2

SPA

log10SNR

A

S

S

S

SP

SN

SP

ωωω

=

=

Typical power spectrum density of the outputsignal from a system exhibiting stochasticresonance (top) and dependence of the SNR andSPA on the noise intensity D (bottom).



• Coupling bistable elements driven by a periodic signal an d noise can lead to the enhancement of stochastic resonance (e.g., array enha nced stochasticresonance in spatially extended systems).

• Can suitably chosen structure of the coupling lead to qualitatively newphenomena in stochastic resonance?

• Example: structural stochastic multiresonance can occur in the Ising model on certain scale-free networks (the curves SPA vs. T show double maxima) [A. Krawiecki, European Phys. J. B69, 81 (2009)].

• Example: structural stochastic multiresonance can also occur in systems ofcoupled passive threshold elements on hierarchical netwo rks , e.g., tree-likenetworks or Ravasz-Barabási networks [M. Kaim and A. Krawiecki, Phys. Lett. A374, 4814 (2010) ].

Stochastic multiresonance (concept): J.M.G. Vilar, J.M. Rubi, Phys. Rev. Lett. 78, 2882 (1997); Physica A264, 1 (1999).

Motivation for the study of stochastic resonance in system s with the structureof complex networks



• Periodic signal: oscillating magnetic field,• Noise: thermal fluctuations (proportional to the temperature),• Output signal: the time-dependent order parameter (e.g., magnetization).

Stochastic resonance in the Ising model

The Ising model is treated as a complex system which consists of coupled bistableelements (spins), and its response to the periodic signal is studied as a function ofthe temperature and frequency of the magnetic field.

Exemplary results

• SR in the 1-dimensional Ising model (the paramagnetic phase) J.J. Brey and A. Prados, Phys. Lett. A216, 240 (1996); U. Siewert and L. Schimansky-Geier, Phys. Rev. E58, 2843 (1998);

• SR in the 2- and 3-dimensional Ising model (Monte-Carlo simulations and theory in the mean-field approximation): Z. Neda, Phys. Rev. E51, 5315 (1995), K.-T. Leung and Z. Neda, Phys. Lett. A246, 505 (1998) et al.;

• Connection with dynamical phase transitions: B.J. Kim et al., Europhys. Lett. 56, 333 (2001);

• SR in the Ising model on complex networks: H. Hong et al., Phys. Rev. E66, 011107 (2002)(Watts-Strogatz small-world networks), A. Krawiecki, Int. J. Modern Phys. B18, 1759 (2004)(Barabasi-Albert scale-free networks).

• Structural stochastic multiresonance in the Ising model on scale-free networks: A. Krawiecki, Eur. Phys. J. B69, 81 (2009).

Regular lattice Complex network

node

edge

Quantity of interest: distribution of connectivity pk (= probability distribution that a randomly selected node has connectivity k)

Connectivity of the node i: the number of edges attached to the node i

( ) ( )4δδ =∝ zpk

( )

= −

K

γσ

Ak

kG

pk

,



Complex networks

• Networks with complex topology are ubiquitous in real world . Animportant class of complex networks are scale-free networks whichlook similar at any scale; e.g., the distribution of connectivity k obeys a power scaling law , pk ∝ k-γ.

• Examples of scale-free networks comprise, e.g.,

• the internet activity,

• the www links,

• networks of cooperation (between scientists, actors, etc.),

• traffic networks (airplane & railway connections, city transport schemes),

• biological networks (sexual contacts, protein interactions, certain neuralnetworks), etc.



Snapshot view ofinternet connections

Imported from:http://www.nd.edu/~networks/gallery.htm

hubs



Examples of scale-free networks: internet connections

Imported from:http://www.nd.edu/~networks/gallery.htm

Examples of scale-free networks: Map of protein-protein interactions.The colour of a node signifies the phenotypic effect of removing the corresponding protein (red, lethal; green, non-lethal; orange, slow growth; yellow, unknown).



Hamiltonian

∑ ∑−−=ji i

ijiij sthssJk

H,

00 sin1 ω Jij=J if there is an edge between nodes i,j,

Jij=0 otherwise

The local field acting on spin i

( ) ∑ +=j

jiji thsJk

tI 00 sin1 ω

( ) ( )

−=T

tIssw i

iii tanh12

1

Glauber dynamics (heat bath algorithm)

The order parameter

( ) ∑=i

ii skkN

tS1

( ) ( )∑−

=→∞−==

1

0012

0

2

1 explimT

tT

titSPh

PSPA ω

Spectral Power Amplification (SPA)

The transition rate between two spin configurations which differ by a single flip of one spin, e.g., that in node i

The spins si are located in the nodes i and edges between the nodes correspond to non-zero exchange interactions between the corresponding spins.

Ferromagnetic phase transitionCritical temperature: maximumfluctuations of the order parameter

222 SSS −=δ



The Ising model on complex networks

• The algorithm starts with a small number m of fully connected nodes.

• Evolving network: New nodes are added step by step,

• Preferential attachment: From each new node m new links are created to theexisting nodes, and the probability to create an edge to the node i is

where ki is the actual number of edges attached to the node i.

• The network grows until a given number of nodes N is added.

• Evolving network + preferential attachment = scale-free network. The distribution ofthe degrees of nodes is

( ) ( )∑ −>++=i

iii mBBkBkp ;

( ) mBkBkpkk += →+∝ −→∞

− 3, γγγ

[A.-L. Barabási and R. Albert, Science 286, 509 (1999) (for B=0, γ=3); S.N. Dorogovtsev, J.F.F. Mendes, Adv. Phys. 51, 1079 (2002)]



Creating scale-free networks

Signal: S(t)N = 10 000,m = 5,h0= 0.01,(a) B = - 4 (γ =2.2),(b) B = 10 (γ =5)

(a)

(b)

Multiresonance(two maxima of the SPA)



Structural stochastic multiresonance in the Ising model o n scale-free networks

• It can be seen that the occurrenceof stochastic multiresonancedepends on the structure ofconnections, thus in (a) structuralstochastic multiresonance occurs,• Structural stochasticmultiresonance occurs for 2 < γ < 3provided that the power-law tails ofthe distribution of the degrees ofnodes are fully developed, i.e., no artificial constraint is imposed on the maximum degree.



cutting rewiring

Creating coupled scale-free networks

Network ANA nodes,exponent γA

Network BNB nodes,exponent γB

Subnetwork A,Rewiring probability π(A)

NA nodes, exponent γA

Subnetwork B,Rewiring probability π(B)

NB nodes, exponent γB

Rewiring probability (in a given network) = = number of rewired edges (in a given network) / number of edges (in a given network).

Cutting and rewiring do not change the distribution of degrees of nodes in the subnetworks.

For small rewiring probabilities the composite network consists of separate, densely connectedsubnetworks with few inter-subnetwork links, so it has modular structure; For increasing rewiring probabilities the network structure becomes more and more uniform



The Ising model on coupled scale-free networksThe spins si are located in the nodes i and edges between the nodes correspond to non-zero exchange interactions between the corresponding spins. The sysyem is subject to the oscillatingmagnetic field

( ) ∑+=

=

=BA NNN

iii sk

kNtS

1

1

Order parametersFor the composite network For the subnetworks

( ) ( ) ∑∑==

==BA N

iii

BBB

N

iii

AAA sk

kNtSsk

kNtS

11

1,

1

• Phase transitions in the Ising model on coupled scale-free networks (Barabási-Albert ones) inthe absence of the magnetic field were considered by K. Suchecki, J.A. Hołyst, Phys. Rev. E74, 011122 (2006), Phys. Rev. E80, 031110 (2009).• In general, there is ferromagnetic transition as the temperature decreases (spins in bothsubnetworks are aligned in parallel), although special preparation of the initial condition can leadto antiparallel alignement of spins in the two subnetworks.• In the analysis based on the mean-field and linear response theory approximations it will be assumed that the ordered stationary state is the ferromagnetic one.

( )2

,

,

2

,

BA

BABAc

k

kJT =

( ) ( )( )( )

( ) ( )( )( )BABA

BABAcBA

NBA

BABA

BABAcBA

JT

NJTBA

BA

BA

,,

2,,

,

1

3

,,,

2,,

,

31

23

31

232

,

,

,

γγγ

γ

γγγ

γ γγ

−−−

=⇒>

∞→−−

−=⇒<<

∞→−

−

Critical temperatures for the uncoupled subnetworks (ferromagnetic transition)



• The Master equation for theprobability that at timet thesystem is in the spin configuration (s1, s2, ..., sn)

( ) ( ) ( )

( ) ( )tssssPsw

tssssPswtsssPdt

d

nj

N

jjj

nj

N

jjjn

;,,,,,

;,,,,,;,,,

211

211

21

KK

KKK

−−+

+−=

∑

∑

=

=

( ) ( ),tanh1

2

1

−=T

tIssw i

iii( ) ∑ +=

jjiji thsJ

ktI 00 sin

1 ω

• Multiply both sides by si and perform an ensemble average (denoted by ⟨ ⟩), i.e., summation over allpossible spin configurations of the composite network

( )BA

ii

i NNNiT

tIs

dt

sd+==

+−= ,,2,1,tanh K

• The aboveN equations can be formally separated into two groups ofNA andNB equations, for the meanvalues of spins belonging to the sunbentworksA, B, respectively

( )( )

( )( ) ( )( )

( )( )B

BiB

i

Bi

A

AiA

i

Ai

NiT

tIs

dt

sdNi

T

tIs

dt

sdKK ,2,1,tanh;,2,1,tanh =

+−==

+−=

• Make the following mean-field approximation

Mean field approximation

( )( ) ( )( )( ) ( )

thssk

JtItI

Bi

Ai j

jj

jBA

iBA

i 00KK

,, sin ω+

+=≈ ∑∑

∈∈

( )( ) ( )( )

≈

⇒

T

tI

T

tIBA

iBA

i

,,

tanhtanh

Equations of motion for the order parameter



• Divide the nodes of the subnetworksaccording to their degreesk and assume that the average values ofspins located in the nodes belonging to the class with degree k are equal to ⟨sk⟩• Replace the sums over the nodes of the subnetworkswith sums over the classes of nodes, e.g.

( )( )( )( )

( )( ) ( )( )( )( )

( )( )

( )( ) ( )( )( )( )

( )( ) ( )( )( )

( )( )

( )( ) ( )( ) ( ) ( )( )[ ] ( )( )

( )( ) ( )( ) ( )( ) ( ) ( )( ) ,1~

,

,sin~

sin1

sin1

,

0000

00

maxmax

maxmax

tStStSk

kJJ

thtSk

kJthtStS

k

kJ

thtsk

kpts

k

kp

k

JktI

tsk

kptSts

k

kptS

BAAAAAA

A

A

iABAAA

A

iA

Bk

k

mk B

BkAA

k

k

mk A

AkAiA

i

Bk

k

mk B

BkBA

k

k

mk A

AkA

BA

BA

ππ

ωωππ

ωππ

+−≡≡

+=++−=

=+

+−=

==

∑∑

∑∑

==

==

and similarily for the networkB.In the above equations it was taken into account that the probability that a nodei with degreeki belongingto the subnetworkA is connected to a node with degreek, belonging to

• subnetworkA is • subnetworkB is

and similarily for a nodei belonging to the subnetworkB.

( )( )( )

A

AkA

k

kpπ−1( )

( )

B

BkA

k

kpπ



• Multiply both sides of the equations of motion for ⟨si(A,B)⟩ by ki , perform the sum over all nodes of the

respective subnetwork and replace it with a sum over the classes of nodes,

( )( )

( )( )( )

( )

( )( )

( )( )( )

( )

∑

∑

=

=

++−=

++−=

B

A

k

mk

B

B

B

B

BkB

B

k

mk

A

A

A

A

AkA

A

tT

htS

Tk

kJ

k

kpS

dt

Sd

tT

htS

Tk

kJ

k

kpS

dt

Sd

max

max

00

00

sin~

tanh

,sin~

tanh

ω

ω

Stationary values of the order parameters and magnetizations for the subnetworks

( )( )

( )( )( )

( )( )

( )( )( )

∑

∑

=

=

=

=

⇒=B

A

k

mk

B

B

B

B

BkB

k

mk

A

A

A

A

AkA

tSTk

kJ

k

kpS

tSTk

kJ

k

kpS

hmax

max

00

00

0~

tanh

,~

tanh

0



The case of two coupled scale-free networks

( ) ( )( )( )

( ) ( )( )( ) 1

max1

1

1

max

1

max1

1

1

max

1,,2,

1,,2,

+−+−

−−

+−+−

−−

−−==<=

−−==<=

BB

BB

AA

AA

B

BB

BA

Bk

A

AA

AA

Ak

kmBmNkBkp

kmAmNkAkp

γγγγ

γγγγ

γγ

γγ

Replacing summation with integration, one gets

( )( )( )

( )( )( )

( ) ( )( ) ( )

( )( )( )

( )( )( )

dkTk

SkJBkM

dkTk

SkJAkM

Tk

SkJpM

dkTk

SkJ

k

BkS

dkTk

SkJ

k

AkS

B

B

A

A

A

BB

AA

k

m B

BBB

k

m A

AAA

A

AA

k

mk

Ak

A

k

m B

BB

B

B

k

m A

AA

A

A

∫

∫

∑

∫

∫

=

=→

→

=

=

=

−

−

=

+−

+−

max

max

max

max

max

0

0

0

0

0

0

01

0

01

0

~tanh

~tanh

~tanh

~tanh

,~

tanh

γ

γ

γ

γ

For non-zero rewiring probability there is a commoncritical temperature for both subnetworks Tc. For T<Tc the mean-field stationary values of the order parameters deviate from zero and there is a ferromagnetic transition, but the critical behaviour inthe vicinity of the critical point for both subnetworks canbe different. Below the critical point the magnetizations in bothsubnetworks are parallel.

( ) ( )00

, BA SS



Linear response theory( )

( )( )

( )( )( )

( )( )

( )( )( )

( )

∑

∑

=

=

++−=

++−=

B

A

k

mk

B

B

B

B

BkB

B

k

mk

A

A

A

A

AkA

A

tT

htS

Tk

kJ

k

kpS

dt

Sd

tT

htS

Tk

kJ

k

kpS

dt

Sd

max

max

00

00

sin~

tanh

,sin~

tanh

ω

ωIn order to solve theequations of motion

( )( ) ( ) ( )( )( ) ( ) ( )

( )( ) ( )( )( ) ( )

( ) ( )

( )( ) ( )( )( ) ( )

( ) ( )

( )( )( )

( )( )( )

∑∑

∑

∑

=

−

=

−

−−

−

=

−

−−

−

=

−

=

=

−

−=

−−=

−

−=

−−=

+−−=

+−−=⇒

+=

+=

BA

B

A

k

mk B

BBB

k

B

B

k

mk A

AAA

k

A

A

BBB

B

BA

k

mk B

BBB

k

B

BBBB

AAA

A

AB

k

mk A

AAA

k

A

AAAA

B

BA

A

BB

BB

A

AB

A

AA

AA

BBB

AAA

Tk

SkJkp

kQ

Tk

SkJkp

kQ

Tk

SkJkp

kT

J

Tk

SkJkp

kT

J

thT

Q

dt

d

thT

Q

dt

d

tStS

tStS

maxmax

max

max

0202

1

1

1

0222

1

1

1

0222

00

00

0

0

~cosh

1,

~cosh

1

11

,~

cosh11

,11

,~

cosh11

sin

sin

τπ

πτπτ

τπ

πτπτ

ωτξ

τξξ

ωτξ

τξξ

ξξ

let us assume that the external magnetic field forces the order parameters of the subnetworks to performsmall oscillations around their stationary values



The case of two coupled scale-free networks

( )( )( )

( )( )( )

( ) ( )

( )

( )( )( )( )

( )( )( )

( ) ( ) ( )( )

( )

( )

−+

−

=

−−−+

+

−

−=

+−+−

−

+−+−

AA

A

A

A

AA

A

AAAA

AA

A

AA

A

A

A

AAAA

A

AA

A

A

A

AA

SJ

M

Tk

SmJm

Tk

SkJk

SJ

ATQ

S

S

Tk

SkJk

Tk

SmJm

Sk

A

AA

AA

0

0010max1

max

0

1

0

0

0max2

max02

0

~1~

tanh~

tanh

21~1

~tanh

~tanh

1

γ

γπ

πτ

γγ

γγ

In the ordered state, for T < Tc

In the disordered state, for T > Tc

( )( )( )

( )

1

11 2

21

=

=

−−=

−

A

A

AAc

AcAA

AA

Q

k

kJT

T

T

k

kπτ

The parametersτBB, QB are obtained in a similar way.

(critical temperatue for the subnetworkA in the case with no rewiring)



Solution of the equations of motion in the linear response approximation is

( ) ( )

( ) ( ) ( )TSPAN

NTSPA

N

NTSPA

hTSPASPA

hTSPASPA

hT

Q

hT

Q

tt

tt

BB

AA

BBAA

B

A

BBBA

BBBA

ABAA

ABAA

B

A

+=

+==+==

=

−

−

+=+=

−−

−−

−−

−−

20

22

21

20

22

21

0

0

2

1

2

1

10

10

11

110

10

1

0201

0201

4,

4

0

0

0

0

0

0

cossin

cossin

ββαα

ββαα

τωτωττ

ττωτωτ

ωβωβξωαωαξ

where the coefficients can be obtained from the system of linear equations

Spectral power amplification SPA as a function of temperature

Spectral power amplification for the subnetworks is

and for the composite network



Structural stochastic multiresonancein the Ising model on two coupledscale-free networks

Networks with the same number of nodes, NA = NB, and different exponents γA ≠γB.

• Coupling networks with different criticaltemperatures results in the curves SPA vs. Twith two distinct maxima.• As the rewiring probability increases themaxima approcah each other and themaximum at lower temperature graduallydisappears.

Signal: S(t)NA = NB = 5 000,m = 5,ωs=2π/512, h0= 0.01,for different probabilities of rewiringπ (A) = π (B) = ν (seelegends);Subnetwork A:B = 10 (γ A=5),Subnetwork B:B = -3 (γΒ =2.5)(a) Theoretical results in the mean-field and linear responsetheory approximations,(b) Numerical results from Monte Carlo simulations,



The same system as on the previous slide, but apart from SPA vs. T for the whole networkalso SPAA,B vs. T for the two subnetworks areshown. • For small rewiring probability the network hasmodular structure and the curves SPAA,B vs. Tresemble those for uncoupled networks. Twomaxima of SPAB vs. T are observed for thenetwork with γB <3.• As the rewiring probability increases and thedensity of inter-subnetworks links becomeshigher the curves SPA vs. T for the twosubnetworks approach each other and double maxima of the SPAA appear also for thenetwork with γA>3,• The curve SPA vs. T for the compositenetwork resembles in general that for thesubnetwork with higher critical temperature.

Black symbols/line –SPA vs. T for the whole network,Blue symbols/line –SPAA vs. T,Red symbols/line –SPAB vs. T,Upper panels: Theoretical results in the mean-field and linearresponse theory approximations,Lower panels: Numerical results from Monte Carlosimulations

π(A) = π(B) =0



π(A) = π(B) =0.06 π(A) = π(B) =0.3



Networks with different number of nodes, NA ≠ NB, and the same exponents γA =γB.

• Coupling networks with different criticaltemperatures can result in the curves SPA vs. T with two or even three distinct maxima for small to moderate rewiring probability,• Two maxima are located close to the criticaltemperatures for the two uncoupledsubnetworks, and one deep in theferromagnetic phase,• As the rewiring probability rises the twoformer maxima usually merge into one, located close to the higher of the two criticaltemperatures

NA = 3000, NB = 5 000, m = 5, ωs=2π/512, h0= 0.01,π (A) = 0.02, π (B) = 0.012

B = -3 (γA= γΒ =2.5)(a) Theoretical results in the mean-field and linear responsetheory approximations,(b) Numerical results from Monte Carlo simulations,

Black symbols/line –SPA vs. T for the whole network,Blue symbols/line –SPAA vs. T,Red symbols/line –SPAB vs. T.



• Stochastic multiresonance can be observed in the Ising model on scale-free networks with 2 <γ < 3, for small and moderate frequencies of the oscillating magnetic field and for a large enoughnumber of interacting spins N. One maximum of the curve SPA vs. T (a trivial one) appears atT ≈ Tc and the other one usually at T << Tc. .

• The necessary condition for the occurrence of stochastic multiresonance is the presence of thefully developed power-law tails in the distribution of the degrees of nodes pk (structuralstochastic multiresonance).

• Coupling scale-free networks with different critical temperatures (different number of nodes orexponents γ) also leads to the occurrence of structural stochastic multiresonance. Curves SPAvs. T exhibit two or three maxima.

• The maxima of the SPA vs. T for the composite network can be related to those for theuncoupled subnetworks if the rewiring probability is small to moderate and the compositenetwork has modular structure.

• As the rewiring probability increases and the structure of the composite network becomesmore uniform the curve SPA vs. T for the composite network becomes similar to that for thesubnetwork with higher critical tempetrature.

Conclusions

ThankThank youyou for for youryour attentionattention

Signal processing in complex networks : many faces of

Documents

Transcript of Signal processing in complex networks : many faces of