[IEEE International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06) -...

HGSNash Strategy as the Decision-Making Methodfor Water Resource Systems with External

Disagreement of InterestsJoanna Kołodziej

University of Bielsko-BiałaInstitute of Mathematicsand Computer Science

ul. Willowa 2, Bielsko-Biała, PolandEmail: [email protected]

Krzysztof JauernigUniversity of Bielsko-Biała

Institute of Mathematicsand Computer Science


Aleksander CieslarUniversity of Bielsko-Biała

Institute of Mathematicsand Computer Science


Abstract— In this paper we define the decision making problemfor water resource systems with external disagreement of interestsas an -person non-cooperative game. Solving this decisionproblem is equivalent to the Nash equilibria detection for thegame. We apply the parallel evolutionary strategy HGSNash todetect these equilibrium points. The paper ends with some simplenumerical results and conclusions.

I. INTRODUCTION

A water resource system can be defined as a system ofthe water management in a given area. There exist someimportant decision problems connected with the system. Themost important and usually very difficult to solve problemsare the coexistence of many users and the interactions amongthem, especially in case of disagreement of their interests.

The disagreement can be internal or external. In case ofthe internal disagreement some decision support systems areusually applied. The external disagreement of interests isusually modeled as an -person non-cooperative game withthe Nash equilibrium as the solution (see[11]).

Game Theory can be understood as a mathematical theoryof making decisions in conflicts among decision-makers calledplayers. A game is a model of strategic interactions amongthe players. Each player tries to choose the strategy whichminimizes his cost of playing the game. The final result ofthe game depends on the strategies chosen by each player.A solution of the game is a systematic description of theoutcomes that may appear in the game. The most commonlyencountered solution concept in the game theory is the Nashequilibrium (see [8]).

The problem of detecting the Nash equilibria of a finitestrategic game remains a challenging global optimizationproblem. To solve the problem we apply in this paper theHGSNash strategy introduced in [1].

The remainder of the paper is organized as follows. Sections2 and 3 introduce notions and state the problem. In section4 we define the HGSNash strategy. Results of performednumerical experiments are reported in section 5. The paperends with some final remarks.

II. NASH EQUILIBRIUM IN -PERSON GAMES

Let us assume that players take part in agame. An - person game can be defined by the tuple

where:

is the set of players,; ; are sets of strategies

for the players,; are players’

cost functions.

In case of a non-cooperative game players are not allowedto cooperate and we can usually define the solution of sucha game as an -dimensional vector of strategiessatisfying the following condition for all :

(1)

This vector is called the Nash equilibrium point for thegame. It can be also interpreted as a steady state of theplay of a strategic game, in which each player holds correctexpectations concerning the other players behavior.

In this paper we consider only games with non-zero sums.The Nash equilibria for such games can be also formulated interms of the global minimization.

Let us first define the players’ response functions for allin the following way:

(2)

where and. The response function defines

the optimal strategy for the player. Let us also define the gamecost function by the following formula:

(3)

The game cost function has non-negative values and itsglobal minimum equals zero. Thus, it is clearly seen that the

Proceedings of the International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06)0-7695-2554-7/06 $20.00 © 2006

...........

Main unit

Subordinate unit

minQ

minQ1 minQn

Fig. 1. Two-steps procedure of finding of the Nash equilibria

vector of strategies for whichmust be the Nash equilibrium (see [5]).

The problem of global optimization of the function ishard to solve. The minimizing procedure must be a parallelalgorithm composed of two cooperated units:

the Main unit - which solves the problem of the globalminimization of the function,the Subordinate unit - which solves the problems ofthe minimization of the players’ cost functions .

We need to minimize first the cost functions of the playersand then we can compute the values of the game cost function.The ”Subordinate unit” could be a parallel algorithm designedfor the numerical optimization of the real functions of severalvariables. The scheme of the general procedure for finding theNash equilibria is shown in Figure 1.

The gradient computation for the function is impossiblein many cases. Thus the non-gradient global optimizationalgorithms are recommended as the main and subordinate unitalgorithms (for example Powell algorithm or Hooke-Jeevesalgorithm).

Some interesting propositions of parallel algorithms definedfor the minimization problem are given by Slepowronskain [9]. She applied the non-gradient method of the exteriorpenalty function and Advanced Controlled Random Searchalgorithm as the main unit algorithms. She used in the subor-dinate unit also non-gradient global optimization algorithms:Hook-Jeeves and Powell methods. Also genetic algorithmsbecame lately very popular and effective method of solvinghard optimization problems. We applied Hierarchical GeneticStrategy (HGS) defined by Kołodziej et al.(see [2],[3]) as themain mechanism of the optimization process of the game costfunction .

III. PROBLEM FORMULATION

Let us consider that many water users pour their effluentsin the common sewage treatment plant. The additional tax forthe emission of effluents can be the reason of the disagreementof interests among the users.

The decision variable (strategy) of the particular user can bedefined as the current pollution level of their effluents poured

in the sewage treatment plant. The aim of the -th player isto minimize their costs connected with the responsibility forthe pollution of the natural environment. The objective (cost)function of the -th player can be defined in the following way:

(4)

where:

- the increasing and continuous function of thewater purification cost paid by the -th user,

- the difference between the maximum ( )and the current ( ) pollution level of water poured in thesewage treatment plant by the -th user ( ),

- the -th user tax for thewater pollution ( - the tax coefficient, - the amountof effluents discharged by the -th user, - thelevel of water pollution in the whole sewage treatmentplant)

- the -th user fine for exceeding themaximum pollution level

The function is defined as follows:

ifotherwise

(5)

The values of parameters , and are estimated by thedepartment responsible for the natural environment protection.These parameters are optimal for the users if the followingcondition is satisfied:

(6)

It was shown in [2] that parameters , and should beestimated according to the following rules:

(7)

(8)

The problem of the optimal estimation of these parametersis the main task for the management of the water resourcesystem. The optimal solution for this problem should implythe optimal users’ decisions, which decrease the values of theircost functions. To find this solution we use the theory of non-cooperative games briefly described in the previous sectionand the concept of the Nash equilibrium formulated in termsof the global minimization of the multi-cost function givenby (3) with the players’ cost functions given by (4).


IV. THE IDEA OF HGSNASH

HGSNash is a parallel evolutionary strategy especially de-signed for the detection of the Nash equilibria as the solutionsof the non-cooperative game. It is based on the HierarchicalGenetic Strategy (HGS) (see [2]). HGS is a very effectivetool in solving ill-possed global optimization problems withmultimodal and weakly convex objective functions (see [3]).High efficiency of the strategy comes from the concurrentsearch in the optimization landscape by many small popu-lations. The sequences of these populations are defined asthe evolutionary dependent processes. HGS was successfullyapplied as a method of solving some practical engineeringproblems, for example for the estimation of the geometricerrors of the Coordinate Measure Machine (see [4]).

HGSNash can be defined as a simple adaptation of the Hi-erarchical Genetic Strategy (HGS) to the global optimizationof the multi-cost game function defined by the formula(3). This adaptation requires a hybridization of the mainevolutionary mechanism in HGS due to the parallel structureof the multi-cost function optimization procedure.

The main idea of HGSNash is a set of dependent evolution-ary processes running in parallel. The relations among theseprocesses have a tree structure. Every process is modeled asa sequence of evolving populations of individuals:

(9)

where - denotes the order of an active process, - is thenumber of active processes of the given order.

Phenotypes of the individuals in populations in HGSNashare -dimensional vectors of the players’ decision variables(strategies). We can define genotypes of the individuals bycoding their phenotypes into binary strings or using their realrepresentation. Thus, we can analyze, as for HGS, the binaryor real implementation of HGSNash.

The basic mechanism of evolution in HGSNash is dependedon the implementation type. We apply a hybrid simple geneticalgorithm in case of the binary HGSNash implementationand a hybrid simple evolutionary algorithm in case of itsreal implementation. A fixed number of iterations of thesealgorithms defines a metaepoch of the given period. The roughscheme of the -periodic metaepoch procedure is presented inFigure 2.

The fitness function is the ”measure of quality” ofindividuals in the populations. It can be defined for ourproblem in the following way :

(10)

where is the multi-cost function defined by the formula (3)and , where and isthe domain of .

To compute the values of we have to compute the valuesof . But to do this we have to minimize the player’s costfunctions first. That was the main reason of the hybridizationof the simple genetic and simple evolutionary algorithms.Another genetic operators defined originally for HGS, i.e.

Start

Initial population

MAIN UNIT

(SGA, SEA)

SUBORDINATE

UNIT;max;1;0 NiiQ ��

Eval F for the population

Min Q1

Eval Q for the population

Min QN

Genetic operations

Resulting population

i>k

Stop

Yes

;1�� ii

No

Fig. 2. A metaepoch of fixed period k

sprouting operator and prefix comparison operator (see [2] and[7] for details) can be applied also in HGSNash.

In this paper HGSNash was applied to solve the decision-making tasks for the systems with external disagreement ofinterests. We performed some numerical experiments andcompared their results with the results of some similar testsobtained for the other parallel algorithms usually applied forthese problems (see [9]).

V. EXPERIMENTS

In this section we reported the results of some simplenumerical experiments performed for the verification of theefficiency of HGSNash. We divided our test problems into twomain groups. In the first one we define very hard theoreticalproblem from the game theory just for the illustration of thegood properties of the presented method in comparison withother very effective parallel algorithm especially designed forthe detection of the Nash equilibria. In the second groupwe consider a decision-making problem of a water resourcesystem in case of two users with external disagreement of theirinterests. We solved that problem for three different variantsof the input parameters values.

A. Parameters of HGSNash

For solving the test problems defined in this section weapplied 3-levels HGSnash strategy in both binary and realimplementations. Binary coded individuals have the genotypesof the different lengths. The genotypes were elongated inpopulations on the higher levels while the sizes of thesepopulations and the mutation rate were decreased. The valuesof all input parameters for the binary implementation ofHGSNash are presented in Table I.


TABLE I

VALUES OF PARAMETERS FOR THE BINARY IMPLEMENTATION OF 3-

LEVELS HGSNASH

Parameter Level 1 Level 2 Level 3

Population size 20 10 10

Binary code length 6 12 18

Mutation probability 0,1 0,05 0,025

Crossover probability 0,9 0,9 0,9

Metaepoch period 10 10 10

TABLE II

VALUES OF PARAMETERS FOR THE REAL IMPLEMENTATION OF 3- LEVELS

HGSNASH

Parameter Level 1 Level 2 Level 3

Population size 20 10 10

Mutation parameter 1 0,5 0,25

Metaepoch period 10 10 10

In the real implementation of HGSNash we applied theGaussian mutation operator with the standard deviation definedas a mutation parameter. The values of parameters for the3-levels strategy are presented in Table II.

In both implementations of HGSNash we applied as thesubordinate unit the Powell optimization algorithm. As thestop criterion for our strategies we accepted the maximalnumber of metaepochs executed in the single run of thealgorithm, which was 20 in every experiment.

B. The efficiency of HGSNash in the detection of multipleNash equilibria

Test problem

In this experiment we define a 3-person game in which eachplayer has two decision variables. The loss functions for theplayers are defined below:

(11)

where .

TABLE III

THE VALUES OF (NR) AND (AVG) AFTER 20 METAEPOCHS

Strategy nr avg

HGSNash binary 8 0,1

HGSNash real 20 0,1

TABLE IV

THE AVERAGE NUMBER OF ITERATIONS NEEDED FOR FINDING THE NASH

EQUILIBRIA

Strategy avg. it. no

HGSNash binary 5500

HGSNash real 5300

ARCS 7274

Par HJ 6270

In this case we have to find 16 optimal points:

(12)

Results of the experiments

Every experiment was repeated 30 times. Table III reportsthe number of runs in which the global optimum was found(parameter(nr)) along with the average objective values of thebest individuals found in the end of each run (parameter(avg)).The real implementation of HGSNash is better in finding thesolutions than the binary one. We compared the results ob-tained for both HGSNash implementations with the results ofthe similar experiments performed for the parallel algorithmsARCS and parallel HJ (Par HJ) defined in [9].

One iteration of the ARCS and HJ algorithm was equivalentto the single genetic period of metaepoch in HGSNash. Theobtained results show that HGSNash in both implementationsis better than applied parallel algorithms ARCS and parallelHJ. More results of similar experiments can be found in [1].


C. The efficiency of HGSNash in solving the decision-makingproblem of water resource system

Test problems

We assume that there are only two water users. We can usethe model of -person game with the following water purifyingcost functions for both users:

(13)

(14)

It follows from the above definition that the values of thecost function for the -st user are always higher than for hisopponent. The main purpose of choosing such functions wasto check the behavior of both players in this situation. Thedifferent values of the users’ cost functions can be the reasonof a conflict of the users’ interests.

All experiments were performed for three different sets ofthe input parameters values , , and .

Case 1

In this case the cost of water purifying is low for bothplayers.

Case 2

In this case we have exchanged the coefficients in the taxrates and we changed the capacity of effluence poured by theusers. These changes should show the greatest difference inthe players’ behavior.

Case 3

In the last case we again exchanged the coefficients inthe tax rates. The capacity of effluence is greater maximalpollution level for both users.

The game cost function defined for the decision-makingproblem of water resource system is very hard to optimize. Ithas many local optima and the global one is usually situatednear the admissible domain boundary. Figures 3,4 and 5 showthe plots of the function in three considered cases.

Most of the algorithms used for the optimization of thefunction are time consuming or just stop after reachingthe boundary points. Some interesting results can be found in

Fig. 3. Plot of the game cost function in Case 1.


[9]. The promising results of the minimization of with theapplication of HGSNash strategy are presented below.

Results of the experiments

Every experiment was repeated 30 times. Tables III andIV report experimental results obtained for binary and realimplementations of HGSNash strategy. The parameter (nr)means the number of runs in which the global optimum wasfound and (avg) means the average objective values of the bestindividuals found at the end of each run. We assumed thatthe global optimum was found if the best adapted individualfrom all populations obtained after 20 metaepochs (the stopcriterion) was in the sufficiently small neighborhood of theglobal minimal point. In our experiments we defined the circlewith the radius 0,05 as that neighborhood.

The obtained results show that both implementations ofHGSNash can be applied for the precise detection of small



TABLE V

THE VALUES OF (NR) AND (AVG) AFTER 20 METAEPOCHS FOR BINARY

HGSNASH

Cases nr avg

Case 1 22 0,156

Case 2 20 0,0299

Case 3 15 0,5444

TABLE VI

THE VALUES OF (NR) AND (AVG) AFTER 20 METAEPOCHS FOR REAL

HGSNASH

Cases nr avg

Case 1 29 0,0055

Case 2 27 0,0019

Case 3 24 0,014

neighborhoods of the global optima of the game cost function. The efficiency of real implementation of the strategy was

higher than binary one. The most difficult case was Case 3.The binary HGSNash found the solution only in 15 runs.

HGSNash should be compared with others algorithms. Inour future research we want to perform more experiments withthe application of non-gradient parallel algorithms similar tothe methods presented in [9].

VI. CONCLUSIONS

A water resource system can be defined as a system ofmethods of the water management on the given area.Some important decision problems connected with thesystem can be very difficult to solve, especially in caseof disagreement of the interests of the water users.The external disagreement of interests of the deci-sion makers is usually modeled as an -person non-cooperative game with the Nash equilibrium as the solu-tion. This model can be used in practice because it doesnot oblige the decision makers to reveal their interests andit does not impose the centralized manner of the decisionmaking.Finding Nash equilibria of the strategic games can bedifficult and tedious. Algorithms for solving games havebeen studied from the beginning of game theory butusually they are very time consuming. HGSNash strategypresented in this paper is an evolutionary optimizationmethod based on the Hierarchical Genetic Strategy. Itis very effective in solving 2-persons games with verycomplicated rules and it is not worse than other parallelalgorithms in solving -person games with multiple Nashequilibria.The experimental results show that HGSNash could bealso effective in finding the optimal solutions for thedecision-making problems of water resource system incase of coexistence of two users with external conflict

of their interests. This method should be examined insome more difficult cases. We want to apply HGSNashfor solving other engineering problems modeled as thenon-cooperative games.The obtained results confirm the good effectiveness, adap-tation properties and flexibility of Hierarchical GeneticStrategy as a global optimization method.

REFERENCES

[1] K. Jauernig, J. Kołodziej and M. Stysło: HGSNash Evolutionary Strategyas an Effective Method of Detecting the Nash Equilibria in -Person Non-Cooperative Games, Proc. of KAEiOG’06, Murzasichle, 30.05-1.06.2006,pp..

[2] J. Kołodziej, R. Gwizdała and J. Wojtusiak: Hierachical Genetic Strategyas a Method of Improving Search Efficiency,Advances in Multi-AgentSystems, R. Schaefer and S. Sedziwy eds., UJ Press, Cracow 2001,Chapter 9, pp. 149-161.

[3] J. Kołodziej J.: Hierarchical genetic Strategy as a New Method in ParallelEvolutionary Computation, Proc. of the 2-nd International Conf. onFormal Methods and Intell. Techn. in Control, Decision, Multimedia andRobotics, Polish-Japanese Institute of Inf. Technology, Warszawa, 2001,pp. 50-58.

[4] J. Kołodziej, W. Jakubiec, M. Starczak and R. Schaefer: HierarchicalGenetic Strategy Applied to the Problem of the Coordinate MeasuringMachine Geometrical Errors. Accepted to the Proc. of the IUTAM’02Symposium on Evolutionary Methods in Mechanics, 24-27 September2002, Cracow, Poland, Kluver Ac. Press, 2004.

[5] N. G. Pavlidis et al: Computing Nash Equilibria Through ComputationalIntelligence Methods, Journal of Computational and Appl. Math., 175(2005), pp.113-136.

[6] L.A. Petrosian and W.W. Zacharow: An Introduction to the Mathematicalecology (in Russian), University of Leningrad Press, Lenigrad 1986.

[7] R. Schaefer and J. Kołodziej: Genetic Search Reinforced by The Popula-tion Hierarchy, Foundation of Genetic Algorithms VII, Morgan Kaufmann2002, pp. 369-385.

[8] P. Straffin: Game Theory. Scholar Press (Polish ed.), Warszawa 2004.[9] K. Slepowronska: A parallel algorithm for the Nash equilibria detection

(in Polish) MScThesis. Warsaw Technical University Press, Warszawa1996.

[10] B. Wierzba, A. Semczuk, J. Kołodziej and R. Schaefer: HierarchicalGenetic Strategy with real number encoding. Proc. of KAEiOG’03. Łagow Lubuski, 26-28.05.2003, pp.231-239.

[11] A. Wozniak: Selected Decision-Making Methods for Water ResourceSystems with Disagreement of Interests (in Polish), Warsaw TechnicalUniwersity Press, Warsaw 1995.


[IEEE International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06) -...

Documents

Transcript of [IEEE International Symposium on Parallel Computing in Electrical Engineering (PARELEC'06) -...