TECHNICALTRANSACTIONS

AUTOMATIC CONTROL

CZASOPISMOTECHNICZNEAUTOMATYKA

ISSUE 1-AC (2)YEAR 2013 (110)

ZESZYT 1-AC (2)ROK 2013 (110)

Chairman of the Cracow University of Technology Press

Editorial BoardJan Kazior

Przewodniczący Kolegium Redakcyjnego Wydawnictwa Politechniki Krakowskiej

Chairman of the Editorial Board Józef GawlikPrzewodniczący Kolegium Redakcyjnego Wydawnictw Naukowych

Pierwotną wersją każdego Czasopisma Technicznego jest wersja on-linewww.czasopismotechniczne.pl www.technicaltransactions.com

© Politechnika KrakowskaKraków 2013

Automatic Control Series Editor Piotr Kulczycki Redaktor Serii Automatyka

Scientific Council

Jan BłachutTadeusz BurczyńskiLeszek Demkowicz

Joseph El HayekZbigniew Florjańczyk

Józef GawlikMarian Giżejowski

Sławomir GzellAllan N. HayhurstMaria Kusnierova

Krzysztof MagnuckiHerbert Mang

Arthur E. McGarityAntonio Monestiroli

Günter WoznyRoman Zarzycki

Rada Naukowa

Section Editor Dorota Sapek Sekretarz SekcjiTypesetting Anna Basista Skład i łamanie

Cover Design Michał Graffstein Projekt okładki

Automatic Control series

Editorial Board

Editor-in-Chief: Piotr Kulczycki, Poland

Editorial Board: Krassimir Atanassov, Bulgaria

Valentina Bălaş, Romania Tadeusz Burczyński, Poland

Antoni L. Dawidowicz, Poland Józef Gawlik, Poland

Aleksander Gegov, England Abdelali El Aroudi, Spain Janusz Kacprzyk, Poland

Jerzy Klamka, Poland Erich P. Klement, Austria László T. Kóczy, Hungary

Heikki Koivo, Finland Józef Korbicz, Poland

Zdzisław Kowalczuk, Poland Krzysztof Kozłowski, Poland Krzysztof Malinowski, Poland

Radko Mesiar, Slovakia Andrian Nakonechnyy, Ukraine

Witold Pedrycz, Canada Leszek Rutkowski, Poland

Dominik Sankowski, Poland Václav Snášel, Czech Republic Ryszard Tadeusiewicz, Poland

Lipo Wang, Singapore Rafael Wisniewski, Denmark

Executive Editors: Dominika Gołuńska, Poland

Piotr A. Kowalski, Poland Szymon Łukasik, Poland

* SzymonŁukasik, Ph.D.,Department ofAutomaticControl and InformationTechnology, Facultyof Electrical and Computer Engineering, Cracow University of Technology; Systems ResearchInstitute,PolishAcademyofSciences.

** Marcin Haręza,Marcin Kaczor, Department ofAutomatic Control and Information Technology,FacultyofElectricalandComputerEngineering,CracowUniversityofTechnology.

SZYMONŁUKASIK*,MARCINHARĘZA**,MARCINKACZOR**

DOCUMENTCONTENTMININGFORAUTHORS’IDENTIFICATIONTASK

EKSPLORACJATREŚCIDOKUMENTÓWWPROBLEMIEIDENTYFIKACJIAUTORÓW

Ab s t r a c t

This paper dealswith automatic authorship attribution throughdocuments content analysis.Thisapproachisbasedonselectingsetsofsuitablefeaturesrelyingonspecificuseofgrammar,punctuation or vocabulary and in the next step – executing given classification algorithm.Thecontributionfirstoverviewsvarious textcharacteristicswhichcanbeemployedfor thatpurpose, then presents the results of experiments involving feature selection and examinesclassifierperformanceforauthoridentificationproblem.Thepaperconcludeswithdiscussionandproposalsforfurtherresearch.

Keywords: author identification, feature selection, classification

S t r e s z c z e n i e

Przedmiotemniniejszegoartykułujestproblemidentyfikacjiautoranapodstawieanalizytreścidokumentów.Podejścietoopierasięnawyborzeodpowiednichcechzwiązanychzespecyficz-nymużyciemstrukturgramatycznych,interpunkcjiorazsłownika,anastępnie–użyciewybra-negoalgorytmuklasyfikacji.Wartykuleprzedstawiononajpierwróżnecharakterystykitekstu,któremogąbyćużytewomawianymzagadnieniu,anastępniezałączonowynikieksperymen-tówobliczeniowychobejmującychwybórcechibadanieskutecznościklasyfikacjiwproble-mieidentyfikacjiautorów.Artykułpodsumowanownioskamiorazpropozycjamidalszychpracwrozważanejtematycebadawczej.

Słowa kluczowe: identyfikacja autora, wybór cech, klasyfikacja

TECHNICAL TRANSACTIONSAUTOMATIC CONTROL

1-AC/2013

CZASOPISMO TECHNICZNEAUTOMATYKA

4

1. Introduction

Author identification is a taskcommonlyperformed inhistorical research, archeologyandcriminology.Historicallyitwaspredominantlyconsideredinthecontextofhandwrittentexti.e.takingintoaccountauthor’swritingstyle.Nowadays,asmostofdocumentsarebeingstoredintheirelectronicversions,itisimpossibletocompletetheidentificationprocedureemploying solely graphical features of the text.Moreover, novel types of textual contentlikecomputerprogrammingsourcecode,posenewproblems,byexcludingthepossibilityofusinggraphicaltextrepresentationintheframeworkofautomaticauthorshipattribution.

Thetaskofselectingauthorofagiventextfromaknownlistofauthorscanbeperceivedasaclassificationorpatternrecognitionproblem.Itisacommonlyknownissueindatamining[23],withabroadrangeofareaswhereittranspires,e.g.biometrics,medicaldiagnosticsorintrusiondetectionsystems [6,14].Classification isa taskofassigningelements fromsocalledtestingset,denotedbymatrixY:

Y Y Y Yn= [ ]1 2 ... (1)

whichncolumnsrepresentfeaturesofmtestobjectsbelongingtothisdataset,tooneoftheknownC classes.Usuallyasetofrepresentativeelementsforthoseclassesisadditionallygiven,intheformofatrainingdataset:

X X X X n= [ ]1 2 ... (2)

withmtrainelementshavingclasslabelsexplicitlydefined.Thetaskforaclassifieristolearnhowtopredictclassassignmentfortestingdatasetusingknowledgeacquiredfromtrainingset.Incaseoftheauthorsidentificationtasks,classlabelcorrespondstoauthor’sidentifier,trainingsettoasetofdocumentswithknownauthorsandtestingsettoagroupofdocumentswhich authorship needs to be identified. To successfully perform such tasks one shouldselectcapableclassificationalgorithmand,evenmoreimportantly,definesuitabledocumentrepresentation,intheformofndistinctivefeatures.

Thispaperinvestigatesapossibilityofemployingvariousnon-graphicalsetsoffeaturesfor author identification task.Among otherswe take into account presence of distinctivegrammaticalstructures,quantitativeanalysisofpartsofspeech,anddiversityofwordsorpunctuation. It is usability of aforementioned features that is experimentally assessed forselectedauthorsidentificationtask.

Thereexistsavastamountofstudiesintheareaofthiscontribution,howeverusuallynot- -involvingsuchabroadrangeofcharacteristicsbeingexaminedatthesametime.Previousworkinthisareainvolveusing:lexicalfeatures(e.g.functionalwords),characterfeatures,including alphabetic or digit characters count, uppercase and lowercase characters, letterfrequencies,etc.[5],syntacticfeatures[11],semanticfeatures[1]andapplicationspecificcharacteristics, like the use of greetings, signatures, etc. [24]. The problem of languagespecificissues isalsowidelystudied[4]. Interestingapplicationsofauthorshipattributionincludemicrobloggingpostsauthoridentification[15],genderrecognition[3]orcombiningauthor classification with opinionmining [18].Accomplished surveys of techniques andstrategiescommonlyemployedforauthorshipattributiontaskscanbefoundin[9,13,22].

5

The paper is organized as follows. First, we give a detailed description of documentscharacteristics, useful for their representation in authorship attribution task. It is followed byexperimentalsetup,employedtoassesstheefficiencyofclassificationalgorithmusedwithvariousdocumentsfeaturessets.Finally,thediscussionandproposalsforfurtherresearcharegiven.

2. Document content representation

Documentcontentwillbedescribedherebyfivegroupsoffeaturescorrespondingtothefollowingcharacteristicsofatext,giveninitscomputer-writtenrepresentation:a) document grammatical composition, represented by features’ groupGR, composed of

R+9characteristicsg1,g2,...,gR+9withRbeingaparameterrelatedtothesizeofanalyzedsetofgrammaticalstructures;

b) usedpunctuationmarks,representedbyfeaturesgroupPwhichincludes16characteristicsp1,p2,...,p16;

c) wordslength,givenbyfeaturesgroupL : l1,l2,...,l4;d) documentformattingdefinedbyfeaturesgroupF : f1,f2,...,f5;e) wordsusagedescribedbyfeaturesgroupV : v1,v2,...,v9.

Thesamesetoffeaturescanbeusedtocharacterizenumerousdocuments–belongingtoboth,trainingandtestingdatasets.Albeit,adocumentDdescriptionmightbeforexamplegivenbyD = {GR,V}–itwouldmeanthatitischaracterizedonlybyindicatorsrelatedtogrammaticalcompositionandusedwordsstatistics.Thestudyofapplicabilityofdifferentsetoffeaturesforauthor’sidentificationtaskwillbeconductedinthenextSectionofthepaper.Wewillnowprovideadescriptionoffeatureswhichwereassignedtogroupslistedabove.

2.1.Documentgrammaticalcomposition

Grammarisalinguisticconceptconcerningboththeshapeofwordsandhowwords(andphrases)canbecombined[2].Grammar,inessence,consistsoftwocomponents:morphology,i.e.studyofhowwordsareformedoutofsmallerunits(morphemes)andthesyntax,whichisasystemofrulesspecifyinghowlexicalitemsoughttobecomposedtogether[21].Ofthosetwo,whenconsideringwritingstyleanalysis,thesyntaxisofmoreimportance.

Toanalyzesyntacticstructureofsentencesoneshoulddefineanotationtoconvenientlyrepresentcontentofagiventext.Treestructureiscommonlyusedforthatpurpose.Itsbranchnodes represent non-terminal syntactic symbols (with sentence as a root) and leaves areequivalent to lexical tokensof thesentence.Toobtainstructure in thisformtextneeds tobeparsed.This language-dependent taskisoneof themost important innatural languageprocessing.Here,weassumethatanalyzedtextcorpusisparsedwithpopularStanfordparser[12].

Toidentifyapresenceofselectedtreecomponents,weestablished256mostcommonlyusedsyntacticstructuresinEnglish,extractedfromtheWallStreetJournalcorporacontainedbyPennTreebankproject[19].Itcanbefoundonthewebsite[16].Onthatbasis,afeatureset:

g g gR1 2, , , (3)

6

wascreated.EachcharacteristicfromthissetindicateshowmanytimestopRstructuresfromtherankingwereusedintheanalyzeddocument.Forexampleg1correspondstothenumberof“preposition+nounphrase”structuresfoundinthetext.Ingeneral,forrankingsizeRonecanassumeanyvaluebetween1and256.

Set of featuresmentioned above corresponds to the presence of selected grammaticalconstructs in analyzed document.Next attributegR+1 describes concentration of syntacticelements listed in the ranking. Let N gR ii

R==∑ 1 denote occurrence of constructs from

theselectedsetofRgrammaticalstructuresinthegiventextandletNGtoindicateoverallnumber of structures pointed out by syntactic parser for the analyzed document. Then gR+1canbewrittenas:

gNNR

R

G+ =1 (4)

SubsequentattributesgR+2,gR+3,...,gR+9describetheoccurrenceoftheindividualpartsofspeechinthetext.HeregR+2correspondstotheincidenceofnouns, gR+3 –pronouns,gR+4 –adjectives,gR+5–verbs,gR+6–adverbs,gR+7–prepositions,gR+8–conjunctionsandgR+9–interjections.

2.2.Documentpunctuation

Punctuation is a practice of inserting standardized signs to clarify themeaning andseparatelanguagestructuralunits[20].Amongfourteenmostcommonlyusedpunctuationmarks in English, at parsing stage the following symbols are identified here: full stop,exclamation mark, question mark, comma, semicolon, colon, apostrophe, quotationmark, ellipsis, dash, hyphen, slash plus additionally multiple exclamation marks andmultiplequestionmarks.UseofthosesignsisindicatedbyfeaturesgroupPformedof16characteristics.

Thefirstattributep1,denotedby:

pNN

P

C1 = (5)

representsnumberofpunctuationmarksNplistedabovefoundinthetext,dividedbytotalnumberofcharactersNc.

Secondfeaturefromthissetp2describesvarietyofpunctuationmarks,byemployingthefollowingformula:

p NP2 =* (6)

where NP* symbolizesoverallnumberofdifferentpunctuationsignsfoundintheanalyzed

document.Finally, featuresp3,p4, ...,p16 refer to thenumberof timeseachofpunctuationmarks

listedabovewasusedinthetext,scaledbytotalnumberofpunctuationmarksNp.

7

2.3.Documentwordlengthstatistics

FeaturesgroupLaimsatcapturingauthors’tendencytouseshortwords,longwordsoringeneral–wordsofapproximatelysimilarlength.LetusdenotebySasetofallsentencesinthetext,andbyWsasetofallwordsformingagivensentences ∈ S.Consequently,bysize(w)wherew ∈ Wsweunderstandlengthofaselectedwordwwhileusingcard(S)andcard(Ws)atthesametimetorepresentcardinalitiesofbothsetsdefinedabove.

Firstfeaturel1introducedhereisreferringtotheaveragewordlengthinthesentencesformingthetext,andcanbewrittenas:

l

w

WS

w W

ss S

s

1 =

∈

∈

∑∑

size

cardcard

( )

( )( )

(7)

Secondfeaturel2describesaveragewordlengthinthedocumentandisrepresentedby:

l

w

Sw Ws S

s S

s2 =

∈∈

∈

∑∑∑

size

card

( )

( ) (8)

Twootherfeaturesfromthisgrouprefertothenumberofshortwords(consistingoflessthan4characters)andlongwords(madeupofmorethan6characters).Bothattributesarerelativetothesumofwordsusedinthetext,andcanbewrittenasfollows:

lW

W

ss S

ss S

3 =∈

∈

∑∑

card

card

short( )

( ) (9)

withW w w W ws sshort size= ∈ ∧

8

However,mostoftextrepositoriescompriseonlycarriagereturnsoradditionalsoftreturns.Thefollowingsetoffeaturescaptureswriters’individualpreferencestousethoseformattingelementsintheirdocuments.

The first feature f1 considered here, refers directly to the number of paragraphs, i.e.sectionsofthetextwithfirstlinebeingindented.Nextattributef2capturesaveragenumberofsentencesincludedinoneparagraph,whichcanbewrittenas:

f SN2

=card

par

( ) (11)

withNparrepresentingtotalnumberofparagraphsintheanalyzeddocument.Features f3and f4characterizeaveragenumberofwordsandcharactersperparagraph.

Theycanbewrittenasfollows:

fW

N

ss S

3 =∈∑card

par

( ) (12)

and

fNN

C4 =

par (13)

Finally,lastfeaturefromthisgroup,f5describeswriter’stendencytoformatthetextwithemptylines.Itisrepresentedbyrelativenumberofblanklines:

fNN

LE

L5 = (14)

withNLEbeingtotalamountofemptylinesinthetextandNL–alllinesintheanalyzeddocument.

2.5.Wordsusestatistics

Authors usually differ in the sizes and structures of their vocabularies.Therefore theanalysis of vocabulary richness and its concentration could be useful in the context ofauthorshipattribution[7].FeaturesetVbeingintroducedhereaimstoquantitativelyexpressthosecharacteristics.

Firstfeatureunderconsiderationv1refersdirectlytothenumberofdistinctwordsusedinthetextNv.Fivefeatureswhichfollow,employtheconceptofhapax legomenon(gr.saidonce)thatiswordsthatonlyoccuronceinthetext[10]anddis legomenon–wordsappearingtwice. Let us denote byNhl,Ndl number ofhapaxes found in the document. First five ofaforementionedfeaturesarenowdefinedasfollows:

v

NW

hl

ss S

2 =

∈∑card( ) (15)

9

vNN

hl

v3 = (16)

v

NW

dl

ss S

4 =

∈∑card( ) (17)

vNN

dl

v5 = (18)

vW

NN

ss S

hl

v

6

10100

1=

−

∈∑log ( )card

(19)

withthelattertwoknownfromtheliteratureunderthenamesofSichel(v5)andHonore(v6)measures[3].

Definingadditionaltextcharacteristics–Nilrepresentingnumberofwordsoccurringinthetextpreciselyi-times,allowsustoformulatethreelastfeaturesinthisset,recognizedasYule(v7),Simpson(v8)andentropy(v9)measures[3]:

vW

N iWs

s S

ili

N

ss S

v

74

1

2

10 1= − +

∈=

∈∑ ∑ ∑card card( ) ( )

(20)

v Ni

Wi

Wili

N

ss S

ss S

v

81

11

=−

−=∈ ∈

∑ ∑ ∑card card( ) ( ) (21)

v Ni

Wi

Wil ss S

i

N

ss S

v

9 101

= −

∈=

∈∑∑ ∑

log( ) ( )card card

(22)

3. Experimental studies

Experimentsperformedweredesignedtoevaluateusefulnessoffeaturessetslistedaboveforauthorshipattributiontasksandtostudyperformanceofclassifiersemployingcarefullyselectedgroupsofattributes.

ExperimentalstudieswereconductedusingThomsonReutersTextResearchCollection(known as TRC2 dataset). The dataset includes almost 2 million news reports collectedbetweenJanuary2008andFebruary2009.Wehavechosenrandomlydocumentsauthored

10

by 20 writers (around 100 contributions each). The first part consisting of 10 writerscontributionswasusedforfirstpartofexperiments–involvingfeaturesetevaluation,andwillbereferredtoasthetrainingdataset.Therestofexperimentaldata(labeledasthetestingdataset)wasemployedinthesecondpartofnumericalstudies,morethoroughlyinvestigatingtheperformanceofselectedclassificationalgorithms.

As classifiers Naïve Bayes, feed-forward Neural Network with back-propagationlearning,classick-NearestNeighbor(withk=3)andRandomForestswerechosen,withthelattertwoonlybeingemployedinthelastpartofthisstudy.Optimalvaluesofparametersforinvestigatedtechniquesweredeterminedthroughasetofpilotruns.Formoredetailsonthosealgorithmsonecouldreferto[6].

3.1.Featuresetselection

First we examined usability of features listed in the previous Section for authorshipattribution.ForthatpurposeSequentialForwardSearch(SFS)orSequentialBackwardSearch(SBS)algorithmwereexecuted,withk-nearestneighborclassifierbeingusedforevaluation.Bothtechniquesconstitutesimilarsupervisedfeatureselectionparadigms[17].Firstalgorithmstartswithemptycandidatefeaturesubsetandateachiterationaddsthefeaturewhichmaximizesclassifieraccuracy.SBSistheoppositestrategy–itstartswiththeentiresetofavailablefeatures,and then iterativelyremovesonefeatureat the time,as longasaforementionedmeasureofaccuracy improves,where the feature removedmaximizes this improvement [8].Here, thediscriminativepowerofagivenfeaturesetwasdeterminedthroughcross-validation.

FeatureselectionalgorithmswereexecutedwithR=256andfivedifferentschemes: – Basic Feature Selection (BFS) – feature selection is performed on all groups ofcharacteristicslistedinSection2.

– TwoPhaseFeatureSelection(TPFS)–featureselectionalgorithmisexecutedonnon-grammaticalcharacteristics(allbutGR).ObtainedreducedfeaturesetismergedwithGR andfeatureselectionisperformedagain.

– ParallelFeatureSelection(PARFS)–featureselectionisconductedinparallelonnon-grammaticalandgrammaticalcharacteristics,obtainedreducedfeaturesetsaremergedafterwards.

– Grammatical Feature Selection (GFS) – feature selection is conducted only oncharacteristicsfromthegrammaticalranking.

– Non-Grammatical Feature Selection (NGFS) – feature selection is executed only oncharacteristicsnotlistedingrammaticalranking.Table1illustratesanumberoffeaturesobtainedfordifferentvariantsoffeatureselection.

Itisevidentlyobservedthatforwardsearchprincipallyresultsinmorecompactreducedsetofcharacteristics,whichisnaturallycausedbylocalsearchnatureofthisalgorithm.Individualcharacteristicsmostfrequentlychoseninexperimentsinvolvingdifferentfeatureselectionstrategies, for both, features listed (top ten) and not listed (top ten) in the grammaticalrankingareshownonFig.1.Itcanbeobservedthatmostusefulfeaturesarefoundwithingrammaticalranking,especiallybetweenfeaturesg20andg100.Furthermorefeaturesdescribingincidenceofpronouns(gR+3),adverbs(gR+6)andprepositions(gR+7)appearedfrequently inthereducedfeaturesets.Amongnon-grammaticalcharacteristicsratioofpunctuationmarksandoccurrenceofexclamationmark,dashandhyphenwerefoundparticularlyimportant;howeveringeneralselectionratioforthosefeaturesseemstobesignificantlylower.

11Ta b l e 1

Obtained features set sizes

Feature selection scheme

Sequential feature selection

Sequential Forward Search (SFS)

Sequential Backward Search (SBS)

BFS 20 222TPFS 26 194

PARFS 37 240GFS 24 233

NGFS 13 7

Fig.1.Tenmostfrequentlychosencharacteristicsforallvariantsoffeatureselectionandfeatureslistedandnotlistedinthegrammaticalranking

Rys.1.Dziesięćnajczęściejwybieranychcharakterystykdlawszystkichwariantówalgorytmuwyborucechicechzispozarankingugramatycznego

3.2.Performanceanalysis

Nextseriesofexperimentswasdevotedtostudyingperformanceofselectedclassifierswithdifferentfeaturesetsobtainedthroughfeatureselectionandvaryingsizeofgrammaticalranking(whereapplicable).Thetrialswereexecutedfortrainingdatasetfirst,todeterminethebest-performingclassifierandthemostappropriatefeatureset,withfiverunsinvolvingcross-validationandaverageclassificationaccuracybeingreportedhere.

Tables2–5sumupobtainedresultsfork-nearestneighbor,NaïveBayes,feed-forwardNeuralNetworkandRandomForestsclassifiers.

12Ta b l e 2

Classification accuracy [%] for training dataset and k-Nearest Neighbor classifier

Ranking Size & Sequential

SelectionVariant

Feature selection variant

R = 32 R = 64 R = 128 R = 256

SFS SBS SFS SBS SFS SBS SFS SBS

BFS 42.25 22.50 37.50 32.50 55.25 22.25 56.00 61.25TPFS 33.00 19.50 46.75 39.75 43.50 26.75 53.75 39.50

PARFS 49.75 53.00 56.25 60.75 61.00 69.25 63.00 70.50GFS 51.00 50.75 51.25 62.00 61.50 69.75 62.75 75.00

NGFS 38.25 20.25 32.75 22.50 25.00 20.25 38.50 24.25

T a b l e 3

Classification accuracy [%] for training dataset and Naïve Bayes classifier

Ranking Size & Sequential

SelectionVariant

Feature selection variant

R = 32 R = 64 R = 128 R = 256

SFS SBS SFS SBS SFS SBS SFS SBS

BFS 37.50 24.00 27.25 22.50 40.50 18.75 39.75 58.75TPFS 27.00 20.00 40.00 29.75 30.00 23.00 41.00 56.25

PARFS 42.25 42.25 45.75 59.00 48.75 62.25 50.50 55.75GFS 38.00 44.00 41.75 57.75 43.75 62.25 45.50 56.25

NGFS 38.00 17.00 26.75 19.50 20.75 14.50 36.25 21.50

T a b l e 4

Classification accuracy for training dataset and Neural Network classifier

Ranking Size & Sequential

SelectionVariant

Feature selection variant

R = 32 R = 64 R = 128 R = 256

SFS SBS SFS SBS SFS SBS SFS SBS

BFS 41.75 29.25 47.25 46.25 56.75 42.50 45.00 60.50TPFS 27.75 26.25 48.25 41.25 50.00 33.25 48.75 60.50

PARFS 53.75 42.25 45.00 46.75 50.00 62.75 63.50 60.25GFS 35.00 43.25 41.00 51.00 52.25 56.50 56.60 61.25

NGFS 38.50 17.00 26.25 11.75 24.25 17.25 35.00 22.25

13Ta b l e 5

Classification accuracy for training dataset and Random Forests classifier

Ranking Size & Sequential

SelectionVariant

Feature selection variant

R = 32 R = 64 R = 128 R = 256

SFS SBS SFS SBS SFS SBS SFS SBS

BFS 66.00 33.50 57.25 51.50 69.25 58.75 68.00 82.00TPFS 37.75 28.25 61.50 44.25 57.50 47.25 62.25 83.50

PARFS 62.75 64.75 60.25 69.00 69.25 80.50 76.00 79.25GFS 61.75 63.75 53.75 70.75 71.00 80.50 68.00 80.75

NGFS 44.25 27.75 36.75 24.25 34.50 23.25 46.25 32.00

It canbe seen that employingabroad rangeofgrammatical features is crucial forobtaining high classification accuracy.RandomForests classifierwas found to be thebestperformingone.Amongvariousschemesof featureselection themost successfulwereTwoPhase Feature Selection,Basic Feature Selection andGrammatical FeatureSelection.

Finally selected best-performing classifiers based on Random Forests and k-NearestNeighborweremorethoroughlyevaluatedforbothdatasets.WeusedsetofcharacteristicswithR=256 andSequentialBackwardSearchalongwithTwoPhaseFeatureSelectionorGrammaticalFeatureSelection(fork-NearestNeighbor).Resultsobtainedfor10runsandbothtrainingandtestingdatasets,with5-foldcross-validationareshownonFig.2.

Fig.2.Classificationaccuracyfortrainingandtestingdatasets

Rys.2.Trafnośćklasyfikacjidlazbiorówuczącegoitestującego

14

4. Conclusion

This contribution examined the possibility of employing various characteristics ofdocuments in computer-written form for authorship attribution.The suitability of severalfeatureswasconsideredforparsedinstancesofnewsreports,takingintoaccountapossibilityofusingfewdiscriminationalgorithmsforconclusiveclassification.Itwasestablishedhere,that for ensuring proper classifier performance the most important ones are those basedonoccurrenceofgrammatical structures.Wealso identified tree-basedclassifiersasmostpromisingintermsofaccuracy–forbothtrainingandtestingdatasets.Ingeneral,authors’classificationforselectedfeaturesandbothinstancesprovedtobereasonablyaccurate.Itcanbenotedthatpresentedapproachcanbeusedfordifferentlanguages–providedthatsuitableparsingprocedurecouldbeconductedbeforehand.

Furtherwork in the research area of this contributionwill involve employinggeneticfeature set selection. Supplementary experimental studies on effectiveness of variousclassifiersareplannedaswell.Thepossibilityofusingauthorshipattributiontechniquesforidentifyingsourcecodecreatorconstituteslikewiseapromisingareaofupcomingresearch.

The study is co-funded by the European Union from resources of the European Social Fund. Project PO KL “Information technologies: Research and their interdisciplinary applications”, Agreement UDA-POKL.04.01.01-00-051/10-00.

R e f e r e n c e s

[1] ArgamonS.,Levitan S., Measuring the Usefulness of Function Words for Authorship Attribution,Proc.JointConferenceoftheAssociationforComputersandtheHumanitiesandtheAssociationforLiteraryandLinguisticComputing,paperno162,2005.

[2] BrocciasC.,Cognitive linguistic theories of grammar and grammar teaching,[in:]DeKnopS.,DeRyckerT.(eds.),Cognitive Approaches to Pedagogical Grammar,WalterdeGruyter,Berlin2008,67-90.

[3] ChengN.ChandramouliR.,SubbalakshmiK.P.,Author gender identification from text,DigitalInvestigation,vol.8,2011,78-88.

[4] EderM.,RybickiJ.,Do birds of a feather really flock together, or how to choose training samples for authorship attribution,LiteraryandLinguisticComputing/toappear.

[5] Grieve J.,Quantitative authorship attribution: An evaluation of techniques,LiteraryandLinguisticComputing,vol.22,2007,251-270.

[6] Hastie T., Tibshirani R., Friedman, J., The Elements of Statistical Learning: Data Mining, Inference, and Prediction,Springer,NewYork2009.

[7] Hoover D.L., Another Perspective on Vocabulary Richness, Computers and theHumanities,vol.37,2003,151-178.

[8] JagadevA.K,DeviS.,MallR.,Soft Computing for Feature Selection,[in:]DehuriS.,Cho,S.-B.(eds.),Knowledge Mining using Intelligent Agents,ImperialCollegePress,London2011,217-258.

[9] JockersM.L.,Witten D.M.,A comparative study of machine learning methods for authorship attribution,LiteraryandLinguisticComputing,vol.25,2010,215-223.

15

[10] JurafskyD.,MartinJ.H.,Speech And Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition,PrenticeHall,NewJersey2009.

[11] Karlgren J., Eriksson G., Authors, genre, and linguistic convention, Proc. SIGIRWorkshop on Plagiarism Analysis, Author ship Attribution, and Near-DuplicateDetection,2007,23-28.

[12] Klein D.,Manning C.D.,Accurate Unlexicalized Parsing, Proceedings of the 41stMeetingoftheAssociationforComputationalLinguistics,2003,423-430.

[13] Koppel M. Schler J., Argamon S., Authorship attribution in the wild, LanguageResourcesandEvaluation,vol.45,2011,83-94.

[14] KowalskiP.A.,Procedure of feature extraction from face image for biometrical system (inPolish),TechnicalTransactions,vol.1-AC/2012,CracowUniversityofTechnologyPress,55-79.

[15] Layton R., Authorship Attribution for Twitter in 140 Characters or Less, CracowUniversityofTechnologyPress,Proc.2ndCybercrimeandTrustworthyComputingWorkshop,2010,1-8.

[16] ŁukasikS.,HaręzaM.,KaczorM.,Grammatical structures ranking (supplementary material), http://www.pk.edu.pl/~szymonl/nauka/Author_suppl.pdf (date of access:9.10.2013).

[17] ŁukasikS.,KulczyckiP.,Using topology preservation measures for high-dimensional data analysis in a reduced feature space (in Polish), Technical Transactions, vol.1-AC/2012,CracowUniversityofTechnologyPress,5-15.

[18] PanichevaP.,CardiffJ.,RossoP.,Personal Sense and Idiolect: Combining Authorship Attribution and Opinion Analysis, [in:]Proc. InternationalConferenceonLanguageResourcesandEvaluation,Valletta,paperno.10.491,2010.

[19] PennTreebankProject,http://www.cis.upenn.edu/~treebank/[20] Punctuation,[in:]Merriam-Webster’sCollegiateDictionary:EleventhEdition,1009,

Merriam-Webster,Springfield2004.[21] Radford A., Minimalist Syntax: Exploring the structure of English, Cambridge

UniversityPress,Cambridge2004.[22] Stamatatos E., A survey of modern authorship attribution methods, Journal of the

AmericanSocietyforInformationScienceandTechnology,vol.60,2009,538-556.[23] Wang L.P, FuX.J., Data Mining with Computational Intelligence, Springer, Berlin

2005.[24] ZhengR.,LiJ.,ChenH.,HuangZ.,A framework for authorship identification of online

messages: Writing style features and classification techniques,JournaloftheAmericanSocietyofInformationScienceandTechnology,vol.57,2006,378-393.

* MałgorzataCharytanowicz,Ph.D.,InstituteofMathematicsandComputerScience,TheJohnPaulIICatholicUniversityofLublin;SystemResearchInstitute,PolishAcademyofSciences.

**HenrykCzachor,D.Sc.,Ph.D.,BohdanDobrzański, InstituteofAgrophysics,PolishAcademyofSciences.

***JerzyNiewczas,D.Sc.,Ph.D., InstituteofMathematics andComputerScience,The JohnPaul IICatholicUniversityofLublin.

MAŁGORZATACHARYTANOWICZ*,HENRYKCZACHOR**,JERZYNIEWCZAS***

NONPARAMETRICREGRESSIONAPPROACH:APPLICATIONSINAGRICULTURALSCIENCE

ZASTOSOWANIEREGRESJINIEPARAMETRYCZNEJWNAUKACHROLNICZYCH

A b s t r a c t

In this paper, a method for determining the soil pore size distribution, constituting the subject of thepresentedinvestigations,isproposed.Aresearchstudywasconductedusingimageanalysisalgorithms,andinturn,nonparametricstatisticaltechniques.Theresultsandfurtherworkwillbediscussedinsectionfour.Thepurposeofthisinvestigationistodiscovertherelationshipbetweentheporesizeandvolumeofthecorrespondingpores.Thealgorithmpresentedhereisbasedonthetheoryofstatisticalkernelestimators.Thisfreesitofassumptionsinregardtotheformofregressionfunction.Theapproachisuniversal,andcanbesuccessfullyappliedformanytasksindatamining,wherearbitraryassumptionsconcerningtheformofregressionfunctionarenotrecommended.

Keywords: nonparametric regression, kernel estimators, morphological image processing, closing procedure, pore size distribution, pore space, total porosity, soil structure, aggregation, soil compaction

S t r e s z c z e n i e

Celem niniejszego artykułu jest zaprezentowanie procedury wyznaczania rozkładu wielkości porówwagregatachglebowych.Doscharakteryzowaniazależnościpomiędzybadanymizmiennymiwykorzy-stanazostaniefunkcjaregresji.Wprzeprowadzonychbadaniachzastosowanoalgorytmyanalizyobrazówcyfrowychorazmetodykęstatystycznychestymatorówjądrowych.Przedstawionametodaumożliwiauzy-skaniewłaściwejcharakterystykirozkładuwielkościporówimożestanowićefektywnenarzędziestosowa-newwieluzagadnieniacheksploracjidanych.Jakomodelnieparametryczny,niewymagazałożeńdotyczą-cychkształtuzależnościfunkcyjnejmiędzyrozpatrywanymizmiennymi.

Słowa kluczowe: regresja nieparametryczna, estymatory jądrowe, przekształcenia morfologiczne, operacja zamknięcia, rozkład wielkości porów, porowatość ogólna gleby, struktura gleby, agrega-cja, gęstość gleby

TECHNICAL TRANSACTIONSAUTOMATIC CONTROL

1-AC/2013

CZASOPISMO TECHNICZNEAUTOMATYKA

18

1. Introduction

Poresizedistributionisoneofmanyphysicalmeasurementscharacterizingsoilstructureasfarasplantgrowthisconcerned.Anumberofscientistshavereportedstudiesofporespaceasageneralmethodfordefiningsoilproperties.Inthisrespect,acompleteanalysisofthesoilporesizedistributionisusedforpredictingthegasdiffusivity,waterinfiltrationrates,wateravailabilitytoplants,water-storagecapacityandmovementofwater.

Themostcommonmeasurecharacterizingthefractionoftheporespacewithinasolidisthetotalporosity,definedbytheratio:

ϕ =VV

P

T

(1)

where:Vp – thevolumeofvoid-space,VT – thetotalvolumeofsoilmaterial,includingthesolidandvoidcomponents.

Porosityisadimensionlessquantityandcanbereportedeitherasadecimalfractionorasapercentage.Beingsimplyafractionoftotalvolume,itcanrangebetween0and1,typicallyfallingbetween0.3and0.7formostsoils.Thisprovidesamoreusefulphysicaldescriptionofanparticular soil, suchasprovidinganestimateofcompactionand themaximumspaceavailableforwater.Moreover,anumberofscientistshavereportedthatstudiesofporesizedistributionareusefulasageneralmethodfordefiningthesoilstructure.Poresizesusuallyhavetraditionallybeendividedintomacroporesandmicropores,withthedivisionbetweenthetwobeingarbitrary.Becauseoftherelationshipbetweentheporepropertiesandthereactionofchemicalsinthesoil,amoredetailedpore-fractionanalysisseemswarranted[7,15].

Thepurposeof this investigation is to elaborate amethodofmeasuring thepore sizedistributioninthesoil.Internaldifferenceinporositywithinanaggregatecanbevisualizedby microtomography scanning of the air-dried samples. Once scanned, the computedtomographyinformationallowsthenon-destructivevisualizationofslices,arbitrarysectionalviewsandpseudo-colorrepresentations[14].

Fig.1.Crosssectionofatypicalsoilwithporespaceinblack

Rys.1.Przekrójagregatuglebowego,poryzostaływyróżnionekoloremczarnym

19

Forimageprocessing,theauthorsusedaprogramforcomputerimageanalysispackage.After preprocessing methods, morphological operation closing, consisting of a dilationfollowed by erosion,was used to fill in holes and small gapswithout changing the sizeandoriginalboundary shape [5,6,12,17].Subsequently,digital imageswere segmentedintoporespaceandsolid.Thedataderivedautomaticallyfromimageswasthenstatisticallyexamined,asnonparametricstatisticalregressionallowedforthedeterminationofthesoilporedistribution.

2. Material and Methods

ThesestudieswereconductedusingsoilaggregatesfromthecultivatedsoillayerexploredattheInstituteofAgrophysics,ofthePolishAcademyofSciencesinLublin.Thedirectandnondestructive analyses of internal soil aggregate structureswere detected using computedtomographyequipmentNanotom800,with thevoxel-resolutionof2.5micronspervolumepixel[4,8].Threetypesofaggregates,differentintermsoffertilization,denotedasaggregates0–without fertilization,NPK–mineral fertilization, andOB–pigmanure,were studied.TomographysectionswereprocessedusingmacroswritingintheAphelion4.0.1package.

Fig.2.Thesoilaggregatemicrotomographicimage

Rys.2.Obraztomograficznyagregatuglebowego

Theentireprocedurewascomposedofthefollowingsteps.Firstly,grayscaleimageswerepreprocessedbyremovingringartifactsusingtheROImethod.Afterselectingtheregionofinterest,theautomaticOtsubinarizationmethodwasthenemployed.

Subsequently, binarymorphological closingwith increasing sizeof square structuringelement(startingwithsize2),wasprocessed.Ineachstep,thesourceimagewassubtractedfrom the target image, and the result volumes were listed, giving soil aggregate poredistribution. The operation was repeated until all pores were filled. Subtraction of thetransformedimagefromtheoriginalimagegivesthetotalporevolumeinthesamplesoilpores.Finally,theporefractionsanalysiswasperformedusingtheregressionapproach[3].

20

Fig.3.ThesoilaggregateimageafterselectingROIandbinarizationthroughusingOtsumethod

Rys.3.ObrazagregatuglebowegopozastosowaniuselekcjiROIibinaryzacjimetodąOtsu

Theprocedurefordeterminingthesoilporesizedistribution

1.X-raymicrotomographicimageanalysisofaggregates.2.Ringartifactremovalusing–theROImethod.3.Imagebinarization–theOTSUmethod.4.Determiningtheporesizedistribution–binarymorphologicalclosing.5.Porefractionsanalysis–regressionapproach.

Innextsection,theregressionapproachisshortlydescribed.Asimplelinearregressionmodel is not appropriate for the data. Therefore nonlinear estimation and nonparametricmethodswereconcerned.

3. The regression analysis

Thevalidationoftheefficiencyandaccuracyofregressiontechniquewasexploredbycomparingresultsonavarietyofrealdatasets.Bothaclassicalparametricregressionmodelandseveralnonparametricmethodswereexaminedasfarastheporesizedistributionwasconcerned.

The best fit to the data in the family of nonlinear estimationwas revealed for twomodels:polynomialwiththepowerofthree;andlogarithmic,forwhichthedeterminationcoefficientswereinrange80–90%.Theproposedregressionfunctionsdidnotwelldiscoverthepropertiesof theporesizedistributions,especiallyon the left side,becauseof theirpositively skewed and unimodal character.Therefore, the nonparametric kernelmethodwasrecommended[1,9,10].

21

Fig.4.Theporesizedistribution:thedatapresentsthreetypesofaggregatesdifferentintermsoffertilization,denotedasaggregates:0–withoutfertilization,NPK–mineralfertilization,

andOB–pigmanure

Rys.4.Rozkładyporów,wykresyprzedstawiajątrzytypyagregatów:0–beznawożenia, NPK–nawózmineralny,OB–nawóznaturalny

22

Classical parametric methods of determining an appropriate functional relationshipbetweenthetwovariablesimposearbitraryassumptionsconcerningthefunctionalformoftheregressionfunction.Moreover,thechoiceofparametricmodeldependsverymuchonthesituation.Ifachosenparametricfamilyisnotofappropriateform,thenthereisadangerofreachingincorrectconclusionsintheregressionanalysis.Thisalsomakesitdifficulttotakeintoaccountthewholeoftheaccessibleinformation.However,therigidityofthisregressioncanbeovercomeby removing the restriction that themodel isparametric.Thisapproachleadstononparametricregression.Thisletsthedatadecidewhichfunctionfitsthembest.In this study,aclassofkernel–type regressionestimatorscalled ‘localpolynomialkernelestimators’ispresented.

Lettherefore,m elements(xi,yi)∈ R × R,i=1,2,...,mbegiven,wherevaluesximaydesignatesomenon-randomnumbersorrealizationsoftheone-dimensionalrandomvariableX,whereasyidesignaterealizationsoftheone-dimensionalrandomvariableY.Assumingtheexistenceofthefunctionf : R→Rhavingacontinuousfirstderivativethat:

y f xi i i= +( ) ε (2)

whereεiareindependentrandomvariableswithzeromeanandunitfinitevariance.Letthenp ∈ Nbethedegreeofthepolynomialbeingfit.Thekernelregressionestimator obtainedbyusingweightedleastsquareswithkernelweights,isgivenbytheformula:

(3)

where:

y y y ym= [ , , ... , ]1 2T (4)

isthevectorofresponses,

X

x x x x x xx x x x x x

x x x x

p

p

m m

=

− − −− − −

− −

11

1

1 12

1

2 22

2

( ) ( )( ) ( )

( )

��

� � � � �22 � ( )x xm

p−

(5)

isanm × (p+1)designmatrix,and:

Wh

Kx x

h hK

x xh h

Kx x

hm=

−

−

−

diag , , ... ,1 1 11 2

(6)

isanm × mdiagonalmatrixofkernelweights,while:

e = [ , , ... , ]1 0 0 T (7)

23

is the 1 × (p + 1) vector having 1 in the first entry and zero elsewhere.The coefficient h>0iscalled’abandwidth’,whilethemeasurablefunction K R: ,→ ∞[ )0 ofunitintegral,symmetricalwithrespecttozero,andhavingaweakglobalmaximuminthisplace,takesthenameofthekernel.

Animportantproblemisthechoiceoftheparameterp.Forsufficientlysmoothregressionfunctions,theasymptoticperformanceof improvesforhighervaluesofp.However,forhigher p,thevarianceoftheestimatorbecomeslarger,andinpractice,averylargesamplemayberequired.Ontheotherhand,theevendegreepolynomialkernelestimatorhasamorecomplicatedbiasexpressionwhichdoesnotlenditselftosimpleinterpretation.Thesefactssuggeststheuseofeitherp=1orp=3.Moreover,forp=1,theconvenientexplicitformulaeexists:

(8)

where:

forr=0,1,2 (9)

Therefore,exceptinmoreadvancedstatisticalapplications,p=1ispreferred.Thechoiceofthekernelformhasnopracticalmeaningandthankstothis,itispossibleto

takeintoaccounttheprimarilypropertiesoftheestimatorobtained.Mostoften,thestandardnormalkernelisexpressedbytheconvenientanalyticalformula:

K x ex

( ) =−1

22

π

2

(10)

isused.Thepracticalimplementationofthekernelregressionestimatorsrequiresagoodchoice

ofbandwidth.Ifhistoosmall,aspikyroughkernelestimateisobtained,andifhistoolarge,it results inaflatkernelestimate.A frequentlyusedbandwidthselection technique is the‘cross-validationmethod’introducedbyClark[2],whichchooseshtominimize:

(11)

where: – theleave-one-outkernelregressionestimatorbasedondata

x y x yi i1 1 1 1, , , ,( ) ( )− − , x y x yi i m m+ +( ) ( )1 1, , , , .Atypicalregressioncurveobtainedbythekernelregressionmethodform=200data

pointsgeneratedfromfunction f x x e x( ) , sin ( ) ( )= ⋅ − + − −0 1 5 2 1 42 asdefinedoninterval[0,3]

isdemonstratedonFig.5.AstandardnormalkernelKgivenbyrule(10)andbandwidthh calculatedbythecross-validationmethod(11)wereused.

24

Fig.5.Theestimatedregression:thedatapointsarerepresentedbyacross,thetruefunctioncurvebyadashedline,theregressioncurvebyasolidline,andkernelsforargumentsx=1andx=2by

adottedline

Rys.5.Jądrowyestymatorfunkcjiregresji:wartościpróbylosowejoznaczonokrzyżykami,estymo-wanąfunkcjęliniąprzerywaną,jądrowyestymatorfunkcjiregresjiliniąciągłą,jądradlaargumentów

x=1ix=2liniąkropkowaną

Thetasksconcerningthechoiceofthekernelform,thebandwidth,aswellasadditionalproceduresimprovingthequalityoftheestimatorobtained,arefoundin[13,16].Theutilityof local linearkernelestimatorshasbeen investigated in thecontextofsometypicaldataderivedfromthesoilaggregateimages.

4. Results and discussion

Themainaimofthisresearchis todiscovertherelationshipbetweentheporesizeand volume of the corresponding pores in soil aggregates. The data obtained by theproposedmethod,basedontheimageprocessingalgorithms,allowstheuseofregressionapproach.

Thekernelregressionbuiltuponaweightedlocallinearregressionwasusedtoanalyzethesoilporesizedistribution.Foreaseofcomputation,thestandardnormalkernel(10)wasused.Thebandwidthwasdeterminedusingthecross–validationmethod(11).

Thus,particle-sizedistributionwastranslatedintoanequivalentregressionmodel,whichinturn,isrelatedtocharacterizewaterretention,crucialinanymodelingstudyonwaterflowandsolute transport insoil.Thiscanbeused forcomparingandpredictingsomespecificpointsofinterestofthewaterretentioncharacteristics.

25

Fig.6.Theestimatedregression:thedatapresentsthreetypesofaggregatesdifferentintermsoffertilization,denotedasaggregates0–withoutfertilization(asolidline),NPK–mineralfertilization

(adashedline),andOB–pigmanure(adottedline)

Rys.6.Jądrowyestymatorfunkcjiregresjidlatrzechtypówagregatów:0–beznawożenia(liniaciągła),NPK–nawózmineralny(liniaprzerywana),OB–nawóznaturalny(liniakropkowana)

Fractionof total aggregatevolumeoccupiedbyporeswas significantlygreater inOBaggregates.TherewasalsogreaterpercentageoflargeporesinOBaggregatesthanin0orNPKaggregates.Inthesmallestrange,however,porosityof0aggregatesexceededthatofNPKandOBaggregates.TotalporositieswerehigherforbothNPKandOBaggregates(24%and32%,correspondingly)thanthe14%for0aggregates.

AcombinationofhighermicroporosityandhigherpercentoflargeporesinOBaggregatesmaygeneratemorefavorableconditionsformicrobialactivitythroughacombinationofhighwater-holdingcapacities,increasedaerationandgastransport.Ourcurrentaimiscomparingandrelatingspecificsofinternalporestructuresintheaggregatesfromtheirwaterstability,composition and chemical properties.Water stable aggregates are much more porous inrelationtothenonstableaggregates.

Soilaggregationanditsmaintenanceisveryimportantforsustainableagriculturebecauseit impactsmajorityofbiological andphysical soilproperties andprocesses.Thisdirectlypromotesbettermovementof air andofwater in the soil, prevents runoff and soilwatererosion,andindirectlydeterminesplantgrowth.Howeversoilaggregatesareverysensitiveagainstwaterandmechanicalstresses.Discoveringthemainfactorsdeterminingtheirwaterstability is a great challenge forworldwide agriculturewhichwould help to prevent soildegradationandtoincreaseacarbonsequestrationinsoilsandtheirfertility.

The aim of future research is to discover the differences between the inter aggregatepore structureofwater stableandnonstableaggregates [8].Additionalphysico-chemicalparametersobtainedbychromatographyanalysisaregoingtobeused.

26

5. Conclusions

Recent advances in computed tomography and digital image processing algorithmsprovidetechnologicallyadvancedmeasurementtoolsforstudyingtheinternalstructuresofsoilaggregates.Thisseemsveryusefulincharacterizingtheporesizedistributionandinquantifyingthedifferencesinporestructuresoftheaggregatesfromthedifferenttypesofsoil.

Amoredetailedanalysiscanbeobtainedbyderivingvariousmethodstoquantifytheporestructureanddevelopingaporesize-distributioncurvetopredictretentionproperties.Theproposedalgorithm,basedonimageanalysisandkernelestimatormethodology,isexpectedtobeaneffectivetechniqueforvariousagriculturalstudies.

Nowadays,kernelregressionisacommontoolforempiricalstudies inmanyresearchareas.Thisisalsoaconsequenceofthefactthatkernelregressiontechniquesareprovidedbymanysoftwarepackages.

Thekernelregressionapproachisalsocommoninimageprocessingandreconstructionmethods.Quiterecently,kernelregressionhasexperiencedakindofrevivalinEarthsciencesonestimatingsomecharacteristicsofthedistributioninnonparametricmodels[11].

R e f e r e n c e s

[1] CharytanowiczM.,KulczyckiP.,Nonparametric Regression for Analyzing Correlation between Medical Parameters,[in:]PietkaE.,KawaJ.(eds),Advances in Soft Computing – Information Technologies in Biomedicine,Springer-Verlag,Berlin,Heidelberg2008.

[2] ClarkR.M.,Non-Parametric Estimation of a Smooth Regression Function,JournaloftheRoyalStatisticalSociety,SeriesB39,1977,107-113.

[3] DraperN.R.,SmithH.,Applied regression analysis,JohnWileyandSons,NewYork1981.

[4] GonetS.,CzachorH.,Organic carbon and humic substances fractions in soil aggregates 16thMeetingoftheInternationalHumicSubstancesSociety,FunctionsoftheNaturalOrganicMatterinChangingEnvironment,China,Hangzhou,9-14.09.2012,215-217,2012.

[5] GonzalezR.C.,WoodsR.E.,Digital Image Processing,Prentice-HallInc.,NewJersey2002.

[6] Kowalski P., Procedura ekstrakcji cech z obrazu twarzy dla potrzeb systemu biometrycznego, Technical Transactions, vol. 1-AC/2012, Cracow University ofTechnologyPress,2012,55-79.

[7] Kravchenko,A.,ChunH.C.,MazerM.,WangW.,RoseJ.B.,Smucker,A.,RiversM., Relationships between intra-aggregate pore structures and distributions of Escherichia coli within soil macro-aggregates,AppliedSoilEcology,vol.63,2013,134-142.

[8] KrólA.,NiewczasJ.,CharytanowiczM.,GonetS.,LichnerL.,CzachorH.,LamorskiK.,Water stable and non stable soil aggregates and their pore size distributions, 20thInternationalPosterDayand InstituteofHydrologyOpenDay“Transport ofwater,chemicalsandenergyinthesoil–plant–atmospheresystem”,2012,870-871.

[9] KulczyckiP.,Estymatory jądrowe w analizie systemowej,WNT,Warszawa2005.

27

[10] KulczyckiP.,Kernel estimators in industrial applications, [in:] PrasadB. (ed)SoftComputingApplicationsinIndustry,Springer-Verlag,Berlin2008.

[11] Perzanowski K.A., Wołoszyn-Gałęza A., Januszczak M., Indicative factors for European bison refuges in the Bieszczady Mountains,Ann.Zool.Fennici 45, 2008,347-352.

[12] PrattW.K.,Digital Image Processing,JohnWileyandSons,NewYork2001.[13] SilvermanB.W.,Density Estimation for Statistics and Data Analysis,Chapmanand

Hall,London1986.[14] Smith J.R., Chang S.F., VisualSeek: A fully automated content-based image query

system,Proc.ACMMultimediaConf.,1996,87-98.[15] VanderWeerdenT.,KleinC.,KelliherF.,Influence of pore size distribution and soil

water content on N2O response curves, 19thWorld Congress of Soil Science, SoilSolutionsforaChangingWorld,1–6August2010,Brisbane,Australia2010.

[16] WandM.P.,JonesM.C.,Kernel Smoothing,ChapmanandHall,London1994.[17] WayneL.W.C.,Mathematical Morphology and Its Applications on Image Segmentation,

MSthesis,Dept.ofComputerScienceandInformationEngineering,NationalTaiwanUniversity,Taiwan2000.

* DominikaGołuńska,M.Sc.,DepartmentofAutomaticControlandInformationTechnology,FacultyofElectricalandComputerEngineering,CracowUniversityofTechnology;PhDStudies,SystemsResearchInstitute,PolishAcademyofSciences.

**MałgorzataHołda,Ph.D.,DepartmentofAutomaticControlandInformationTechnology,FacultyofElectricalandComputerEngineering,CracowUniversityofTechnology.

DOMINIKAGOŁUŃSKA*,MAŁGORZATAHOŁDA**

THENEEDOFFAIRNESSINGROUPCONSENSUSREACHINGPROCESSINAFUZZYENVIRONMENT

KONCEPCJA„SPRAWIEDLIWOŚCI”WPROCESIEOSIĄGANIAKONSENSUSUWGRUPIE

WWARUNKACHROZMYTOŚCI

A b s t r a c t

Inthispaperweexplaintheneedof“fairness”inthehuman-consistentcomputationalsystemwhichsupportsagroupconsensusreachingprocess.Weproposethemodelwhichcombinesmathematical approach based on fuzzy environment, i.e. “soft” consensus developed byKacprzyk and Fedrizzi and socio-psychological approach concerning fairness component.Weviewfairnessfromtwopossibleperspectives:afairdistributionandafairfinaldecision.Finally,weconfirmourassumptionsbyobservationsintheanalyzedgroupsofstudents.

Keywords: group consensus reaching process, decision support systems, fairness, fair resource allocation, soft consensus

S t r e s z c z e n i e

W niniejszym artykule wyjaśniono potrzebę „sprawiedliwości” w systemie obliczeniowymwspomagającymprocesosiąganiakonsensusuwgrupiedecydentów.Autorkizaproponowałymodelłączącypodejściematematyczneopartenaśrodowiskurozmytymorazczynnikispołecz-no-psychologicznewyjaśniająceopisywanąkoncepcję.Pojęcie„sprawiedliwości”rozpatrywa-nejesttuwdwóchkategoriach:rozkładuzasobóworazdecyzjiostatecznej.Założeniapopartesąwnioskaminapodstawieobserwacjigrupstudentów.

Słowa kluczowe: proces osiągania konsensusu w grupie, systemy wspomagania decyzji, spra-wiedliwość, sprawiedliwy rozkład zasobów, miękki konsensus

TECHNICAL TRANSACTIONSAUTOMATIC CONTROL

1-AC/2013

CZASOPISMO TECHNICZNEAUTOMATYKA

30

1. Introduction

Decision theory is an interdisciplinary domain which combines research from manydisciplines, i.e. psychology, sociology, economics, philosophy, political science, etc. Theformaldirectioncannotbetheonlycourseofdecisionmakingproblemssincealltheclassicalmethodshaveaverylimitedcapacityforexplainingempiricalchoices.

Regardlessofitsorigin,theessenceofdecisionmakingprocessisalwaysthesame:therearesomeoptionstochoosebetweenandthechoiceismadeinagoal-directedway[6].Infact,manydifferentmodelsofdecisionmakingprocessoccurredandenrichedananalysisofhumanbehavior,socialinteractionsandothersocio-economicdescriptionsdependingonthe respectivepurpose.All of thesenovel agent-basedcomputationalmodels appeared inordertomaketheprocessmorehuman-consistentandbelievable.Thatisthereasonwhywedecidedtoapplypsychologicalandsociologicaltheoriestoinvestigateanddesignsystemsinthisresearchtopic.

Bythesametoken,theconceptoffairnessappearstobeamultifariousissuewhichdrawsuponideasfromawholepanoplyofscientificdisciplines.Theprevalentaspect,however,istheimpossibilitytoachieveanideallyfairdistributionofpowerinthedecisionmakingprocess. Deviations of behavior from the presumed results in decision making processsuggestandalsoconfirmthatfairnessinfluencesdecisionmakingprocess.

Groupsofindividualsareknowntobeeffectiveorgansindecisionmakingprocess.Inspiteofseveraldysfunctionsofgroupwork,therestillaremorecrucialbenefits(processgains).Namely,groupsarebetterthanindividualsatunderstandingproblems,atcatchingerrors,sotheyprovidelearning.Moreover,agrouphasmoreinformationthananymemberandcancombinethisknowledgetoderivebettersolutionsandstimulatethecreativityofparticipantsand theprocess itself.Hence, thegroup decision making processwillbe thegroundworkofourfurtherconsideration.Consideringdifferent typesofgroups[7] indecisionmakingproblem,wetookintoaccountinterpersonal orientation group.Itmeansthatoneinwhichthefinalsolutionoftheproblemisonlyaminorgoal.Here,thepriorityistoensureagoodrelationamonggroupmembersduringdecisionmakingprocessandtoachieveconsensusinthesenseofsomesatisfactoryagreementastothechosenoption.

We want to guarantee an equal participation of all decision making members in theconsensusreachingprocess.Inmostcases,thereisalsoasmallgroupofoutsiderswhoareisolatedintheiropinionsastotherestofthegroupandareomitted.Significantly,outsidersdonotfeelthesatisfactionofdiscussion,whataffectstheeffectivenessofanentiregroup.Ofcourse,itdoesnotexcludethefinaldecisionachievement,butdecreasestheopportunityofmanyfurtheractivities,i.e.practicalimplementationofthefinaldecision,survivalofthegroupinthelongtimeperiod,etc.

Allofthesocio-psychologicalaspectsforcedustoseekanovelapproachofconsensusdegreewhichwill consider the satisfaction of every individual throughout the consensusreachingprocess.InSectiontwowewillbrieflypresenttheoverallstructureofconsensusreachingprocess.Sectionthreeshowsanexplanationofgroupdecisionsupportsystemswithawiderdescriptionofamoderatorwhoplaysherethemainrole.Furthermore,weattempttoreducethecomplexityoftheproposedsystemwithadetaileddescriptionofrelevantaspectsonly.Inthepreviouswork[3],wemainlyfocusedonperformanceofthecoreofmultistagemodel of consensus reaching process under fuzziness. In the light of the fact that mosthumanbehaviorshavenotbeen formalizedmathematicallyyet,ourpurpose in thispaper

31

istogetabetterunderstandingofhowsocialmechanismingroupdecisionsupportsystemsworks.Therefore,inSectionfourwewillnamesomepsychologicalstudiesandsociologicalexplanationsofactualfairbehaviorandconfirmthemwithourobservationsintheanalyzedgroups of students. For comparison, we investigated the problematics of fairness in itsversatilityontheexampleofastudyoftwogroupsofstudents.Ananalysisoftheproblemoffairnessinthegroupsofstudentsundergoinganexaminationshowsthemanifoldfactorsinvolvedintheassessmentofwhatkindofbehaviorisfairandwhatkindisnot,andhowtoachievefairnessindecisionmakingprocess.Ourproposalconcerningdivisionoffairnessapproachintwopossibledirections:afairdistributionandafairfinaldecisionisdescribedinSectionfiveandsixrespectively.

2. Group consensus reaching process

Basically,weassumethefollowingsettings:thereisafinitesetofindividuals(experts,agents)andthefinitesetofalternatives(options).Expertsopenlyexpresstheirpreferencesbymeansofpairwisecomparisonastotheeverypairsofavailableoptions[13].Whatmattershere is that themaingoal of groupdecisionmakingprocess is tofind a solution that alldecisionmakersarewillingtosupport[2].

Decisionmakingproblemisaniterativeandinteractiveprocesswhichincludesseveraldifferent levels, i.e. aggregating all individual preferences into one common decision,elaborating on the agreement in the spirit of consensus reaching process etc. Reachingconsensus requires: time,activeparticipatingofallmembers,creative thinkingandbeingopen-minded,activelistening,consideringideas,feelingsandsituationsofeveryparticipant.The model of consensus reaching process is manageable only if individuals are able tonegotiateandchangetheirpreferences,thustheyarewillingtosupport.

Consensus is reachedwhen each expert in the group agrees to support the selectedfinal decision, though itmay not have been his or her first choice. It forces the groupto consider all aspects of a problem and voice objections to possible alternatives [2].Hence,themainpartofthisprocessisdiscussion,whichgivestheopportunitytoexchangeknowledge, clarify point of view, defend own preferences or to become convinced todifferentopinions.Anymembercanblockconsensus.Thatiswhythesekindsofdecisionsaremoredifficult andcomplex thanothers.Thus, toachieve themaingoal,weassumethatindividualsare“committedtoreachingconsensus”–theyareexpectedtoiterativelyupdate their testimonies, and as a result to finally attain a satisfactory agreement.Weassumethetopologicalapproachofwhereagreementismeasuredonthebasisofdistancebetweenindividualsduringeverystageoftheprocess.Initially,expertsdisagreeintheirpreferences,sotheyarefarawayfromconsensus.Theaimistominimizethisdistance,andconsequentlyleadthegroupclosertotheacceptableagreement[14].Whatmattershere,isthattheseinitialdifferencesofopinionsareastrengthofthegroupandakeytogatheradditionalinformation,clarifyissuesandforcethegrouptosearchforbettersolutionswithbiggerbenefitsforeveryone[2].

32

3. Group decision support systems for consensus reaching

Sincethedevelopmentofmoderntechnology,computerizedsupportinmakingdecisionhas enormouslyprogressed.Today’s tools areflexible, efficient, easy inuse andallow tocreateaninteractiveuser-friendlyinterfacetoviewdata,configuremodels,etc.Thisclassof computer-based information systems including knowledge based systems that supportdecisionmakingactivitiesisdefinedbyoneterm–decision support systems.Theycombinetheintellectualreservesofindividualswiththeproficiencyofcomputertoenhancethequalityof final decision [16, p. 13]. Similarly,group decision support systemsmean interactive,intelligent,computer-basedsystemsthatfacilitatesolutiontounstructuredproblemsbyasetofdecision-makersworkingtogetherasagroup.Unstructuredproblemsare“fuzzy,complexprocessestowhichtherearenocut-and-driedsolutionmethodsandwherehumanintuitionisoftenabasis fordecisionmaking[16,p.11].”Softwareproductsprovidecollaborativesupporttogroups,i.e.supplyamechanismforteamstoshareopinions,data,information,knowledge,andotherresources.Whatmattershereisthatgroupdecisionsupportsystemisanadjuncttodecisionmakerstofacilitatetheirdecisionmakingprocessbutnottoreplacetheir judgments.Moreover, it isadynamicsystemwhichisadaptiveovertime,therefore,decisionmakersshouldbereactiveandabletochangetheiropinionsquickly.Groupdecisionsupportsystemsattempttoimprovetheeffectivenessofdecisionmaking(accuracy,quality)ratherthanitsefficiency(thecostofmakingdecisions).

Thekey to success is tocreatemorehuman consistent andhuman centered toolsandtechniquestograspanddealwithdifficult(decisionmakingtype)problems.Thesesystemsshouldprovidecomputational tools,cognitiveaspectsandsocialdimension.In theGDSSconsideration it means that a computer asks a group to solve a problem, then collects,interpretsandintegratesthesolutionsobtainedbyhumans.

Themainroleofthiscomputer-basedsystemplaysmoderatororfacilitatorwhotakescareofrunningthewholediscussionprocess.Moderatorconstantlymeasuresdistancesbetweenindividualsandcheckswhetherconsensusisreachedornot.Moreover,hismostimportanttask is to support thediscussion, i.e.he stimulates theexchangeof information, suggestsarguments,convincesdecisionmakerstochangetheirpreferences,focusesthediscussionontheissueswhichmayresolvetheconflictofopinionsinthegroup.Thisisrepeateduntilthegroupgetssufficientlyclosetoconsensus,i.e.untiltheindividualfuzzypreferencerelationsbecomesimilarenough,oruntilwereachsometime limit [11].Doubtless, themoderatoraffectsthegeneralsenseofsatisfactionwithinthegroupandhasadirectinfluenceonthequalityoffinaldecision.Whatmattershereisthatheonlytriestopersuadeproperexpertstochangetheiropinionsandsuggestssomerationalarguments–hedoesnotforce,argueorpushindividualstochangetheirtestimonies.

Ourtaskistodevelopandenhancethediscussionpartandprovidethemoderatorwithaspecificknowledgeaboutgroupmembers.Brieflyspeaking,wewanttofacilitatetheworkofmoderator,providehimwithsomeusefulguidelinesandadditionalindicatorsand,asaresult,maketheconsensusreachingprocesseasier,faster,moreeffective.

33

4. Notion of fairness in the consensus reaching support systems – a socio-psychological explanations with observations in the analyzed groups of students

One of the definition of fairness says that “fairness means the satisfaction of justifiedexpectationsofagentsthatparticipateinthesystem,accordingtorulesthatapplyinaspecificcontextbasedonreasonandprecedent[19]”.Fairnessisanintricateideathatdependsonmanyfactors,e.g..culturalvaluesorthecontextoftheproblem.Itcombinesmanydifferentresearchareassuchasmathematics,philosophy,economicsandothersocialsciences,especiallysocialpsychology.Thelastresearchareaiscrucialbecauseitgivesananswertoaquestion:“howpeopleunderstandfairbehavior[18,p.15]?”Generally,fairnessisunderstoodasequalityandbecomesanessentialelementofthenewagent-basedcomputationalmodelswhichaimatexplainingactualbehavior.Thus,anotherquestionarises:whetheronecan talkabouta simpleunifiedprinciplewhichcouldpossiblysolvethequandaryofdecisionmakingprocess?Therangeoffactorsinfluencingtheequilibriuminagroupusuallyturnsouttobeversatileanddependingongroupdynamics,psychologicalandsituational,aswellaspersonalcharacteristicsofgroupmembers.Theoverallideaisthatsubjectsmaybesimplyprejudicedintheirunderstandingoffairness.Opponentsinthedecisionmakingprocessmaybeattunedtoadifferentperceptionoffairness,whichimpedesdecisionmakingprocess,orcausesthatconsensusbecomesimpossible.Groupdecisionmakingisusuallydependableupontheactualstateofemotionspervadingthecircleofpeoplewhosetthemselvesataskofreachingconsensus.

Ouranalysisofgroupbehaviorintwogroupsofstudentsconsistingof14and12membersrespectively (groupAandB– studentsof the thirdyear at theDepartmentofAutomaticControl and Information Technology, Faculty of Electrical and Computer Engineering,Cracow University of Technology) allowed to draw the following conclusions. It wasnoticeablethatifatleastsomepeopleinagroupcaredaboutequity,consensuswasreachedmorequicklyandthedecisionmakingprocessbecamesmoother.Thesecondoftheanalyzedgroups(B)showedaslightlymoreconsensuspronecharacter.Thedistinctionsbetweenthestandpointsof the individualagentsandsubgroupswerenot so sharpas ingroupA.ThemajorfactorinreachingconsensusinasmootherandquickerwayingroupBthaningroupAwasalowerstateofpsychologicalagitationcharacteristicofthegroup,andacalmer,lessaggressivelevelofcommunication.Initsmajorpart,thegroupconsistedofagentswithmuchless individualizedpersonalities, andabetterdeveloped senseof cooperation resulting inasubsequentfasterimplementationofthefinaldecision.

The other crucial and decisive factor in reaching consensus in this groupwas a veryefficientcooperationinsmallsubgroups.TheindividualsingroupBusuallyexertedamuchmoreopenattitudetothepreferencesofotherindividuals,andwerelesslikelytofallpreytoaself-servingbias.Theoverallmechanismofthefunctioningofthisgroupandreachingconsensusprocesswasbasedonaveryconspicuoussupportivesystemofthepresentationofideasandtheskillfulputtingforthofthecarefullyselectedarguments.Generallyspeaking,mostofthemembersofthegroupshowedarelativelyhighlevelofemotionalintelligenceallowingtoanalyzealmostemotionlesslytheargumentsstatedbytheiropponents.WhatwasalsowellobservablewasthefactthatsomemembersofgroupBexertedastrongpositiveinfluenceontheentiregroupbyexhibitinggooddiplomaticandnegotiatingskills.Twooftheagentsinthisgroupwereextremelyflexibleinswiftchangesofthecommunicativestrategiestheydeployedtoinvigoratetheothermembersofthegrouptosharethemostexposedandmostsuccessfullysupportedopinion.

34

Theexplanations as for thebehavioralpatterns in thisgroupcanbe supportedby thedefinitionofthe cooperative game theorywhichvirtuallyisagamewhereplayerscanenforcefairbehavior.Cooperativegametheoryisconnectedwiththedistributionofbenefitsthatagroupofagentsachievesfromcooperation.Themodelassumesthatthegroupofindividualswishes tosolveacommonproblemandbycooperating theycansolve theproblemmoreefficiently [18,p.56]. In fact, research inpsychologyhasshown that ingroupsituations,decisions of individuals are influencedbymotives such as “groupperformance, sense ofresponsibilityforothers,orsocialconcerns[18,p.61]”.Itisalsoworthnoticingthatagreaterleveloffairnessandmoresuccessfulachievementofgoalsindecisionmakingprocesswereattributedtosomecharacteristicsoftheobservedgroupssuchas:aneagernesstolearntheactualdifferencesbetweentheirownopinionsandthoseoftheopponents,andanabilitytoeliminatesuchopinionswhichwere impedingagreement themost.Theopennessand themarkedly interactive character of the group relations proved to be the core of success inmeetingthepointofbalance.

Bycomparison,thefirstgroupofstudents(A)whichblockedconsensusreachingprocesswascharacterizedveryoftenbyanoverallunwillingnesstocometoasatisfactorysolutionforallofthemembers,andaninsistenceonparticularinterestsofminigroups,orindividualswithinthegroup.Theminigroupswereveryassertiveinexpressingtheiropinionsandoftenignorantof theopinionsofopponents.Sticking to theproclaimedopinionsandnotbeingopentotheproposalsofothers,especiallythosewhichwereverydifferentendedupinanimpasseofdiscussionandan impossibilityof reachinganagreement.Quite frequentwasthesituationinwhichsomemembersofthegroupshowedatotaldisregardfortheopinionsoftheiropponents,closingthemselvesinaverylimitedmindframework.Thecreativityinreachingsatisfactorysolutionswasdecreased,whichwasnotanoptimisticforecast.Wemayassumethatthediminishedlevelofthecoherenceandcreativeunityofthegroupwillnotleadinthelongruntoanexpectedconsensus,aswellasaverymuchneededimplementationofthepossiblesolutions.

Itcannotgounnoticedthattheentirecontextofcomingtoanagreementisfundamentallyasituational,contextual,andpsychologicalphenomenon.Suchfactorsasculturalunificationorculturalclashes,andmuchinthesamemanner,economicdifferencesorsimilareconomicstatusmaybeofsignificanceingroupconsensusreachingprocess.Theseelementsmayhaveinfluenceonwhetherthemoreegoisticallyorientedindividualsorthemorefairproneagentsoverbalancetheprocessofreachinganagreement.Theeconomicelementofgroupconsensusreaching process was thoroughly examined, for instance, by Ernest Fehr and Klaus M.Schmidt[5].Inouranalysisweignoredthesefactorsfocusingonotherprevalentingredientsofthesituationalcontextsuchas,forinstance,persuasiveandmanipulativecapabilitiesofthegroupmembers.

Inourstudyweobservedthatagentspushedequityprincipleswhichwereadvantageousfor themmore than for other parties, in particular, thosewhichwere disadvantageous topartieswithgreatpersuasionpower.Thefirstofthetwoanalyzedgroupsofstudentsshowedacomplexityofreactionswithinthesmallgroupsorsubgroupsindecisionmakingprocess,highlydependableonpsychologicalfactors.Oneofthefirstthingswhichwerenoticedprovedthatgroupbehaviortosomeextentwasinlinewithindividualbehaviorofparticularagents.Basically,groupbehaviordependedonthedecisionrules thatagentsselectedandusedtoarriveatgroupdecisioninabiasedway.Anotherthingwhichwasverywellseenwasthatsubgroupsexpressingsimilarstandpointswithintheobservedgroupsignoredthedecision

35

ruleoftheirsocalled“opponents”–whichmeantthattheyfollowedtheself-serving,orself-interestpreferenceprinciple.What ismore,agivensubgroupsharingasimilarviewpointtypically disregarded other subgroups and thought of themselves as if they were singleagents.Aboveall,onemuststressthatexpectationsasfortheleveloffairnesswereoftennotconsistentwiththeoutcome;weexpectedtheleveloffairnesstobehigher.

These observations bear out the importance of the self-interest preference in groupbehavioral patterns. The choices of the respective members of the group underpin ourtheoreticalassumptionsasfortheself-centerednessoftheindividualsinfluencingthefinalgroupdecisionmakingprocess.

However,psychologicalstudieshaverevealedthatinreallife,decisionmakersarenotasselfishaswhat isshownin thesolutionsreceivedusingmechanismsofrationalchoiceapproaches,inthesenseofmaximizationofsomeutilityfunction[6].Experimentsshowedthatindividualstendtocooperateandgiveprioritytofairnessovergreedybehavior.Trust gamewilltransparentlyperformthisactivity.Inthetrust game,AhasaninitialamountofmoneyheorshecouldeitherkeeportransfertoB.IfAtransfersit,thesumistripled.Bcouldkeepthisamount,ortransferit(partiallyortotally)toA.TraditionalgametheorysuggeststhatAshouldkeepeverything,or ifAtransfersanyamount toB, thenBshouldkeepall.Experimentalstudieshaverevealedthatagentstendtotransferabout50%oftheirmoneyandthisfairnessandcooperationisrelatedtoallcultures,sexes,etc[1].

With reference to our assumption that fairnessmeans the satisfaction of expectationsof agents, groupdecision support system should provide the sense of satisfaction amongthegroupmembersduring thediscussion andafter theprocess completion.According topsychological research, satisfaction of decision makers has a direct influence on higherqualityoffinaldecisionandseveral furtheractivities, i.e.practical implementationof thefinaldecisionorsurvivalofthegroupinthelongtimeperiod[15].

5. Fair share of distributed resources

Inourresearchwemainlyreflectedononeoffairnessjudgmentsidentifiedbysocialpsychology,namelydistributive fairness[17].Itisusuallyrelatedtothedistributionofresources,goodsorcosts,thustofair resource allocation problems.Resourceallocationproblemsareconcernedwiththedistributionofconstrainedresourceswithincompetingactivitiessoastoachievethebestgeneralimplementationofthesystemwithrespecttofairmanagementofalltheparticipants.Brieflyspeaking,theaimistotakeafair share of the distributed goods, thus to find such distribution that is perceived as fair by allindividuals.

Theoverallresourceallocationproblemmightbestatedinthefollowingway.ThereisasetofI={1,2,...,m}ofmactivities.ThereisgivenasetPoflocationpatterns(locationdecisions).Foreachactivity i, i ∈ I,anonlinearfunction fi(x)of thelocationpatternx isdefined.Thisfunctionmeasurestheoutcomeyi = fi(x)ofthelocationpatternforeachactivityi.

Togettheindividualsclosertoeachother(toobtainanagreementbetweenthem),therehavetooccursomechangesintheirinitialpreferences.Theoverallamountofchangesintheindividuals’preferencesconstitutes theresource.Thus, foragivensetofdecisionmakersmoderatorwantstoallocatefairlytheresourcewiththeobjectiveofminimizingtheoutcome(i.e.thedistancesbetweenindividuals).

36

Themaingoalofour system is to take intoaccountpreferencesofevery individualandgettheentiregroupclosertotheconsensuswithfairtreatmentofalltheparticipants.Weneglectthesituationwhenthemoderatorgetsdecisionmakersclosertotheconsensusbyargumentationandpersuasiononlyaimingatthemostpromisingdirectionsoffurtherdiscussion (thosewho reach consensus quickly), while individuals who are isolated intheiropinionareomitted.Wefoundconfirmationofthisassumptioninourobservations.Asweobservedgroupbehaviorintwoselectedgroupsthereweremanysituationswhenconsensusreachingprocesswasblockedandthedomineeringmembersof thegroup,aswellasthedominantsubgroupsexertingapowerfulinfluenceontheentiregroupcausedthat some individuals found themselves in a situation of a total isolation. One of theanalyzed groups (groupA)was likely to undergo all the distortions resulting from theself-interestprejudice.The interpersonal relations in theobservedgroupwerenotgoodenoughtoovercomeallthedifficultiescausedbythedifferencesinopinions.Someofthemembersofthegroupwereleftonthemarginandcouldnotfeelthesatisfactionensuingfromdiscussion.This,inturn,broughtaboutasituationoflittleeffectiveness.Allinall,the image of the groupwas negative, thereweremany lost opportunities of achievingconsensus.Brieflyspeaking,moderatorcannotignoretheindividualswhoareisolatedintheiropinionsastotherestofthegroupmembers,quiteonthecontrary,hehastoconvincethemtochangetheirpreviouspreferences.Thisattitudeundoubtedlyconfirmsoneofourassumptions,namely,animportanceofactiveparticipationofeveryindividualduringtheentireconsensusreachingprocess.

Asweassumed,ourresearchshouldbedonewithrespecttofairdistribution.Thetheoryof distributive fairness can be applied whenever it is possible to precisely define a fairdistributionproblemandtofindasolutionthatisacceptedbyparticipants(orproposedbythemoderator).Ifweconsiderthedistancesoftheindividuals’opiniontothefinalopinion,naturally,thefinalopinionshouldbefairinthesensethatthedistancesoftheindividuals’preferencestothefinaldecisionshouldbefairlydistributed.

6. Fair final decision

Whileconsideringtheconceptoffairnesswithreferencetoconsensusreachingprocess,wedecidedtoviewthebasicideaofthisnotionfromtwopossibleperspectives.Thefirstone,presentedintheprevioussection,concernsafairdistributionofresources,whereasthesecond isdirectlyconnectedwith theoutcomeofdecisionmakingprocess,namelya fair final decision.

Wedefineafair final decision asapossibilitytoreachafinalconsensusduringaseriesofdiscussions.However,themajorityhererefersdirectlytotheoutcomeandcanbedefinedasthesoft consensus, aconceptualhuman-consistentframeworkproposedbyKacprzykandFedrizzi[9,10],andZadrożny[4].Thedevelopedideaismeantbasicallyasanagreementofaconsiderablemajorityofindividualswithregardtoaconsiderablemajorityofalternatives.Thisoperationaldefinitionofconsensuscanbe,forinstance,expressedbyalinguisticallyquantifiedpreposition:most of the individuals agree in their preferences to almost all of the options,andtheconsensusdegree(intherange[0,1]) iscomputed.Itmeansthat,exceptnoneortotalagreementbetweenagentsastothechosensolution,thisapproachallowssomepartial,acceptableconsistency.

37

Noticethattodefineafuzzymajorityformeasuringadegreeofconsensustheapplicationoffuzzy linguistic quantifiers (most,almostalletc.)hasbeenperformed.ThecomputationsofthisrelativetypeoflinguisticquantitycanbehandledviaZadeh’sclassiccalculus[20].Regardlessofthewayofimplementation,themainconditionofthisnovelapproachisthatitdefinitelyovercomestheconventionalconceptinwhichconsensuswasunderstoodasa“fulland unanimous agreement”,whichmeans that the preferences of all the decisionmakersshouldbeexactly thesame.Obviously, thisscenario isutopianandunrealistic inpracticebecause individuals usually expose relevant differences in their standpoints, flexibility,tendencytochangeopinions,etc.Allofthesefactorsgenerallyblockthegroupfromgainingafullandunanimousagreement.

7. Conclusions

Inthisarticleweproposedanewconceptofsupportinggroupconsensusreachingprocessenhanced by the concept of “fairness”.We showed that this notion should be definitelytakenintoaccountwhilecreatinghuman-consistentsupportsystemsbecauseit isstronglyconnectedwithpsychology,economics,gametheory,etc.and,asaresult,takescognizanceofsocio-psychologicalaspectsofgroupbehavior.Infact,ithelpsusunderstandthetypicalhumanbehaviorwithinagroupofindividualsandtodevelopmoreintelligent,human-centricandhuman-consistentsystemsforsupportingconsensusreachinginthefuturedevelopment.

InourresearchwecametoaconclusionthatadegreeofconsensusobtainedbyincludingaspectoffairnesswouldbehigherthanthepreviousapproachbasedsolelyonsoftconsensuswiththeuseoffuzzylogicproposedandsuccessfullyimplementedbyKacprzykanZadrożny[12].Weenrichedourconceptbythenovelfairnesscomponent.Hence,wetakelibertyofputtingforthahypothesisthattheconceptofnovelapproachaffectsdirectlytheeffectivenessofdecisionmakingprocess,satisfactionamonggroupmembersand,astheresult,thequalityofthefinaldecision,whichbecomeshighlyjustified.Theultimategoalofourfurtherresearchisthemathematicalformalizationofthefairgroupconsensusreachingprocess(buildingamodelwithregardtorealeventsandpsychologicalfactors)inordertoverify–confirmorreject–ourassumptions.

Thus,ourcomprehensiveapproachproposedforsupportingconsensusreachingprocessunderfuzzinessrefersparticularlytoadegreeofagreementinagroup,individualpreferencesof group members and the moderation of the discussion in order to gain satisfactorysolutionsinthemoreeffectiveandefficientway.Whatmattershere,isthatwerespectthefair distribution in the sense of contribution of all decisionmembers to choose the finalsolution.Hence,thesituationwhenminoritymustobeymajorityandchangetheiropinionsaccordinglyisintheproposedsystemignored.

Dominika Gołuńska’s contribution is partially supported by the Foundation for Polish Science under International PhD Projects in Intelligent Computing. Project financed from The European Union within the Innovative Economy Operational Programme 2007–2013 and European Regional Development Fund.

38

R e f e r e n c e s

[1]Camerer C.F., Psychology and economics. Strategizing in the brain, Science, 300,2003,1673-1675.

[2]CenterforExcellenceinGovernment,Facilitator’sToolbox,1996,http://www.employeesu.com/docs/EmployeeResources/Creating%20Consensus.pdf,[dateofaccess:23.03.13].

[3]FalkiewiczD.,KacprzykJ.,Different Aspects of Supporting Group Consensus Reaching Process Under Fuzzines,TechnicalTransactions,vol.1-AC/2012,CracowUniversityofTechnologyPress,17-27.

[4]Fedrizzi M., Kacprzyk J., Zadrożny S., An interactive multi-user decision support system for consensus reaching process using fuzzy logic with linguistic quantifiers,DecisionSupportSystems,vol.4,no.3,1988,313-327.

[5]FehrE.,SchmidtK.,A Theory of Fairness, Competition and Cooperation,TheQuarterlyJournalofEconomics,MITPress,no.3,vol.114,1999,817-868.

[6]HansonS.O.,Decision Theory: A Brief Introduction,Philosophy,vol.23,RoyalInstituteofTechnology,Stockholm,1994,1-94.

[7]Jennings H.H., Sociometric differentiation of the psychegroup and sociogroup,Sociometry,vol.10,1947,71-79.

[8]KacprzykJ.,Neuroeconomics: yet another field where rough sets can be useful?, ChacC.Cetal.(Eds.):RSCTC2008,LNCS,vol.5306,2008,1-12.

[9]Kacprzyk J., FedrizziM.,A ‘human-consistent’ degree of consensus based on fuzzy logic with linguistic quantifiers,MathematicalSocialSciences,vol.18,1989,275-290.

[10]KacprzykJ.,FedrizziM., A ‘soft’ measure of consensus in the setting of partial (fuzzy) preferences,EuropeanJournalofOperationalResearch,vol.34,1988,315-325.

[11] KacprzykJ.,ZadrożnyS.,On a concept of a consensus reaching process support system based on the use of soft computing and Web techniques,[in:]D.Ruan,J.Montero,J.Lu,L.Martínez,P.D’hondt,E.E.Kerre(Eds.):ComputationalIntelligenceinDecisionandControl.WorldScientific,2008,859-864.

[12]Kacprzyk J., Zadrożny S., Soft computing and Web intelligence for supporting consensus reaching,SoftComputing,vol.14,no.8,2010,833-846.

[13]Kacprzyk J., Zadrożny S., Supporting Consensus Reaching Processes under Fuzzy Preferences and a Fuzzy Majority via Linguistic Summaries,PreferencesandDecisions StudiesinFuzzinessandSoftComputing,Springer,vol.257,2010,261-279.

[14]Kacprzyk J., Zadrożny S., Raś Z., How to Support Consensus Reaching Using Action Rules: a Novel Approach, InternationalJournalofUncertainty,FuzzinessandKnowledge-BasedSystems,WorldScientific,vol.18,no.4,2010,451-470.

[15] PriemR.L.,HarrisonD.A., MuirN.K.,Structured Conflict and Consensus Outcomes in Group Decision Making, JournalofManagement,1995,vol.21,no.4,691-710.

[16]TurbanE.,AronsonJ.E.,LiangT.P.,Decision Support Systems and IntelligentSystems,6thEdition,PrenticeHall,11-19,2005,94-101.

[17]TylerT.R.,SmithH.J.,The handbook of social psychology,vol.2, Social justice and social movements,McGraw-Hill,Boston,1998,595-629.

[18]WierzbickiA.,Trust and Fairness in Open, Distributed Systems,Springer,2010,11-19,56-70.

[19]YoungH.P.,Equity: In Theory and Practice,PrincetonUniversityPress,1994.[20]Zadeh, L, A computational approach to fuzzy quantifiers in natural languages,

ComputersandMathematicswithApplications,no.9,1983,149-184.

* KrzysztofSchiff,Ph.D.,DepartmentofAutomaticControlandInformationTechnology,FacultyofElectricalandComputerEngineering,CracowUniversityofTechnology.

KRZYSZTOFSCHIFF*

ANTCOLONYOPTIMIZATIONALGORITHMFORTHESETCOVERINGPROBLEM

ALGORYTMMRÓWKOWYDLAZAGADNIENIAPOKRYCIAZBIORU

A b s t r a c t

This articledescribes anewhybrid ant colonyoptimizationalgorithms for the set coveringproblem.Theproblemismodeledbymeansofabipartitegraph.Newheuristicpatterns,whichare used in order to choose a vertex to a created covering set have been incorporated intomodifiedhybridalgorithms.Resultsoftestsoninvestigatedalgorithmsarediscussed.

Keywords: set covering problem, ant colony optimization, heuristic rule

S t r e s z c z e n i e

Wartykuleprzedstawiononowyhybrydowyalgorytmmrówkowydlaproblemuzagadnieniapokryciazbioruominimalnymkoszcie.Problemjestzamodelowanyzapomocągrafudwu-dzielnego.Wmodyfikowanymalgorytmiewprowadzononowąheurystykęwyboruwierzchoł-kówdopodzbioruwierzchołkówpokrywających.Opracowanyalgorytmprzetestowanoipo-równano,awynikitychbadańomówiono.

Słowa kluczowe: zagadnienie pokrycia zbioru, algorytm mrówkowy, heurystyka

TECHNICAL TRANSACTIONSAUTOMATIC CONTROL

1-AC/2013

CZASOPISMO TECHNICZNEAUTOMATYKA

40

1. Introduction

Many practical optimization problems such as for example facility location problem,airlinecrewscheduling,nursescheduling,vehicleroutingandresourceallocationproblemcanbedescribedandmodeledas theSetCoveringProblem(SCP) [1,2,10–12]andmanyothercombinatorial problems can bemodeled in such a way. There aremany kinds of computeralgorithmsthathavebeendesignedsofartosolveSCPsuchasexactalgorithms[13,14],heuristicalgorithms[15–18],meta-heuristicsalgorithms[19–21],alsoalgorithms,whicharebasedonantcolonyoptimizationstrategy[3,6,22,23]andhybridofAntColonyOptimizationAlgorithm(ACO)withsocalledConstraintProgramming(CP)[26,27].Antalgorithmsweredesignedtosolvemanycombinatorialproblems[5,8,24,25]. Thispaperpresentsnewalgorithms,whicharebasedonanantcolonyoptimizationstrategy,forthesetcoveringproblemwithaminimumcoveringcostandanewheuristicinformationpatterns,whichhavebeenusedinthesealgorithms.TransitionprobabilityruleandpheromoneupdateruleandalsoamechanismcheckingconstraintsconsistencyareusedalltogetherinordertosolvetheSCPandtominimizethetotalcoveringcost.Theremainderofthispaperisstructuredasfollows:insection2theSCPisintroduced,insection3thestructureofACOalgorithmisdescribed,insection4pseudo-codeofantalgorithmswithnewheuristicpatternsandtransitionprobabilityrules,whichisusedinnewelaboratedantalgorithms,arediscussedandinsection5resultsoftheconductedcomputationalexperimentsonaspecialkindofagraphwithanalmostequaldensityandinsection6conclusionsarepresented.

2. Set covering problem

ThesetcoveringproblemcanbemodeledasabipartitegraphnetworkG(V1 + V2, E, w)withweightswijassignedtoedgeseij,suchthateij = (vi ∈ V1, vj ∈ V2), eij ∈ EasitisshowninFig.1andinthesamewayasitispresentedin[9].Thedegreeofvertexiisthesumofedgesadjacenttothisvertexi.Allverticesv1i canbegroupedintosubsetsinsuchawaythatallverticesfromthesetV2arecoveredbyverticesfromthesomesubsetVsofverticesV1.Avertexv2i iscoveredbyavertexv1jifanedgeeij existsbetweenavertexi andavertexjinabipartitegraph,forexamplesubsetVs ⊂ V1whichconsistsofverticesv11,v13,v14andv16 coversallverticesfromthesetV2andtheanotherexampleofsetVsissubsetwhichconsistsofverticesv14andv17.IngeneralifthecardinalitynumberofsetVsislowerthanthetotalsetcoveringcostishigher.InthesetcoveringproblemwithminimumthecardinalitynumberofsetVsandthecostthetotalcostofsetcovering,thismeansthesumofweightsassignedtoallgraphedges,whichareparticipatinginthesetcovering,hastobeminimized.

TheobjectivefunctionFistofindacoversetwithaminimumcostandisdescribedin(1)and(2):

min , ,F x w i V j Vij ijjn

i

n= ( ) ∈ ∈== ∑∑ 11 1 2 (1)

andsubjecttoconstraintsuchthat:

x i V j Viji Vsn

∈∑ ≥ ∈ ∈1 1 2, , (2)

41

where:xij = 1,whenavertexjiscoveredbyavertexi,xij = 0,whenavertexjisnotcoveredbyavertexi,wij–thisisaweightassociatedtoedgeeij (acostofcoveringavertexjbyavertexi),Vs – isasubsetofverticesV1whichcoverallverticesV2.

Fig.1.Thesetcoveringproblemmodeledbyabipartitegraph

Rys.1.Problempokryciazbiorumodelowanygrafemdwudzielnym

3. Structure of ACO algorithm

Inantalgorithmsacolonyofartificialantsislookingforagoodqualitysolutionoftheinvestigatedproblem.Thepseudo-codeofACOprocedureispresentedasalgorithm1.Eachartificialantconstructsanentire solutionof theproblem insomenumberof steps,calledintermediate solutions.Anyof intermediate solutionsare referred toas solution states. Ineachstepmofthealgorithmeachantk goesfromaonestateitoananotherstatejandthusconstructsanew intermediatesolutioncalled laterapartial solutionof theproblemsincetheentiresolutionisreceivedinsomenumberofstepsandateachofthesestepsthereisanintermediatesolutioncalledapartialsolutionorasolutionunderconstruction.Ateachstepeachantkcomputesasetoffeasibleexpansionstoitscurrentstateandmovestooneoftheseinprobability.Thissetoffeasibleexpansionsiscalledaneighborhoodofcurrentstate.Inpresentedalgorithmsconcerningprocessingofbipartitegraph,thismeansworkingonagraphmodelofthesetcoveringproblemateachstateeachantchoosesavertexfromthesetV1andaddsittoapartialsolutioninordertoconstructfinallyattheendofalgorithmactiontheentiresolutiontotheSCPproblem.AttheendofalgorithmactionthesetofverticesVsconstituteasolutiontotheSCPproblem.Eachantk startswithanemptysetVsandsuccessivelyaddstothissetVsavertexchosenoneaftertheotherfromthesetV1withprobabilityp

kijmovingfrom

aonetoanotherstate.Ateachstatei therearesomeverticesinthesetVsandtheseverticesfromthesetVsconstituteapartialsolutionofaproblematstepm,thismeansatstatem.Eachantinordertoconstructasolutionusescommoninformationwhichisencodedinpheromonetrails,thismeansthetraillevelofthemove,indicatinghowproficientithasbeeninthepasttomakethatparticularmove.Eachantalsodepositsapheromoneonatrailwhenasolutionhasbeenfoundandaquantityofthepheromonedepositeddependonaqualityofthissolution.Themoveofeachantalsodependsonsocalledtheattractivenessofthemove,ascomputedbysomeheuristicindicatingtheaprioridesirabilityofthatmove.Inordertoavoidaveryfast

42

convergencetoalocallyoptimalsolutionanevaporationmechanismisused,thismeansthatoverthetimethepheromonetrailevaporates,thusreducingitsattractivestrength.

Algorithm1ACOprocedure

begin while(existcycle)do while(existanyant,whichhasnotworked)do while(asolutionhasnotbeencompleted)do chooseanextvertextoaconstructedsolutionwithaprobabilitypkij ; updateneighborhoodofcurrentstate; end updateabestsolutionifabettersolutionhasbeenfound; end updateaglobalbestsolutionifabettersolutionhasbeenfound; useanevaporationmechanism; updateapheromonetrailsτ(i) = τ(i) + ∆τ; end end.

Eachantkmovesfromonestateitoanotherstatejwithatransitionprobabilityrulepkij(t),whichisdescribedbytheformula:

pj N

j N

ij

ij ij

ij ijj N

i

i

=

( )( )

∈

∈

∈∑τ µ

τ µ

α β

α β, for

, for which not0 1

(3)

usingthepheromonetrailτijandtheattractivenessµijofthemove.Thepheromonetrailτij is theuseful information,which isdepositedbyothersants, for eachantduring itsworkon construction of solution, about the usage of vertex j in the past by others ants. TheattractivenessµijisadesireofchoosingavertexjfromtheneighborhoodNi ofcurrentstate when there isapartial solutionyetconstructed instate i and theattractivenessµij canbyexpressedbyasomeheuristicformula.Theattractivenessµijallowstobetterchooseasomevertexfromallvertices,fromtheneighborhoodNi ofcurrentstate,tobeaddedtoasolutionunderconstructiontakinganobjectivefunctionintoaconsideration.TheneighborhoodNi of state i is constituted by verticeswhich can be added to a constructed partial solution.At the start all vertices can be added to a partial solution of the problem, thismeans toasolutionofaproblemunderconstructionandthenumberoftheseverticesisreducednotonlybecauseoftheirinclusionintothesolution,whichisundertheconstruction,butalsobecausesomeoftheseverticescannotbeyetaddedtoasolution,whichisunderconstruction,since these vertices does not satisfied solution constraints and only these vertices can beadded to constructed partial solutionwhich still satisfied solution constraints.The partialsolutionoftheproblemisapartofsolutionandthepartialsolutionisasubsetofvertices,whichconstituteasolutionoftheproblem.Parametersαandβwhichisusedinthetransition

43

probabilityrulepkij(t)expressedby(3), indicateabout this,howimportant thepheromonetrailτijandtheattractivenessµijareduringtransitionfromonetoanotherstate.Valuesoftheseparametersαandβ shouldbesetbyexperimentandtunedtothesetcoveringproblemwithminimumcoveringcost.

Afterasolutionhasbeenfoundeachantdepositsapheromonewithaquantity∆τonallvertices,whichconstitutethesolutionVs,inaccordancewiththepattern:

τ τ τij ijt t( ) ( )= + ∆ (4)

Thus these vertices which were included into a solution have received an a