Optymalizacja-algorytmy i zastosowania (English)

Computational complexity – theory by examples

Mariusz Paradowski

General issues Problem Algorithm – “a way to solve the

problem” There can be many ways to solve the

same problem Can an optimal solution for the

problem be found? How to choose the best way to solve

the problem?

Algorithms

How to measure algorithm „speed”?

How to compare algorithms? How to pick the algorithm we

need?

What a computer can do? Computer is a kind of complex

calculator Computer has a set of actions

Add (A+B=C) Subtract (A-B=C) Multiply (A*B=C) Divide (A/B=C) Compare numbers (A>B) Many, many more

Algorithm as a set of computer actions

On comparing algorithms

121213+

?

253532+

?

1) A+B=C

452187+

?

1) A+B=C 1) A+B=C

1 action needed in each case.For adding 2 numbers we always need 1 action.

This is a rule:problem of adding 2 numbers can

be solved using 1 action.

On comparing algorithms253532132189211+

?

1) A+B=C2) C+D=E3) E+F=G4) G+H=I

253532132+

?

1) A+B=C2) C+D=E

For adding 3 numbers we need 2 actions.For adding 5 numbers we need 4 actions.

Is there some kind of a rule? YES

Problem of adding n numbers can be solved using n-1 actions.

On comparing algorithms

M[1]=253M[2]=532M[3]=132

1) R=M[1]2) M[2]>R => R=M[2]3) M[3]>R => R=M[3]

M[1]=253M[2]=532M[3]=132M[4]=44M[5]=543

1) R=M[1]2) M[2]>R => R=M[2]3) M[3]>R => R=M[3]4) M[4]>R => R=M[4]5) M[5]>R => R=M[5]

Problem of finding maximum

of n numbers can be solved

using n actions.

On comparing algorithms

M[1]=17M[2]=143M[3]=276

1) R=M[1]

M[1]=356M[2]=698M[3]=789M[4]=987M[5]=999

1) R=M[1]

Problem of finding maximum of

n numbers sorted can be solved using 1 action.

IDEA: Let’s first sort the numbers!NO! (answer on the next slides)

On comparing algorithmsM[1]=253M[2]=532M[3]=132M[4]=143M[5]=543

M[1]=253M[2]=532M[3]=132M[4]=143M[5]=543

M[1]=253M[2]=132M[3]=523M[4]=143M[5]=543

M[1]=132M[2]=253M[3]=523M[4]=143M[5]=543

M[1]=132M[2]=253M[3]=143M[4]=523M[5]=543

M[1]=132M[2]=143M[3]=253M[4]=523M[5]=543

M[1]=132M[2]=143M[3]=253M[4]=523M[5]=543

TOTAL:7 comparisons4 replacements

On comparing algorithmsM[1]=123M[2]=356M[3]=378M[4]=894M[5]=977

Best case total:5 comparisons0 replacements

M[1]=977M[2]=894M[3]=378M[4]=356M[5]=123

Worst case total:10 comparisons10 replacements

Will be no better than n actions

Will be no worse than n*n - n actions

On comparing algorithmsALGORITHM 1: Use first sorting, then maximum from sorted

ALGORITHM 2: Use maximum from unsorted

n*n – n actions from sorting1 action from maximum selection

Total worst case: n*n – n + 1Total best case: n + 1

n actions from the algorithmTotal: n actions, BETTER!

We have just compared 2 algorithms!

Big-O notation There is a formal way to write

down the number of operations n is often referred as „the size of

the problem” Three major cases

Optimistic (we just got lucky) Expected (usual case) Pessimistic (we had bad luck)

Big-O notation

A function f(n) is O(g(n))as n -> infinity if and only if

there exists numbersn0 and c such that

|f(n)| <= c|g(n)|for n > n0

Big-O notation O(f(n))O(g(n)) =

O(f(n)g(n)) O(f(n)) + O(g(n)) =

O(max{f(n),g(n)})

T(1) = O(1) T(log n) = O(log n) T(n) = O(n) T(5n) = O(n) T(2n2) = O(n2) T(n6+n5+n4)=O(n6) T(1/2n) = O(n) T(2n) = O(2n) T(2n + n100) = O(2n) T(3n + 2n) = O(3n)

Big-O notationO(1) constant

O(log n) logarithmic

O([log n]c) polylogarithmic

O(n) linear

O(n log n) loglinear

O(n2) quadratic

O(nc), c > 1 polynomial

O(cn) exponential

O(n!) factorial

O(nn) -

Big-O notation Find maximum from sorted

O(1) Find maximum from unsorted

O(n) Insert sort (similar to presented)

Pes: O(n2), Exp: O(n2), Opt: O(n) Find maximum by sorting and then get from sorted

Pes: O(n2), Exp: O(n2), Opt: O(n) Traveling salesman problem, exact solution

algorithm O(2n)

Genetic algorithm, not exact solution Will be determined soon…

A word about GA01) FOR EVERY GENOTYPE02) GENERATE RANDOM GENOTYPE03) NEXT GENOTYPE04) FOR EVERY GENERATION05) FOR EVERY GENOTYPE06) CONVERT GENOTYPE TO PHENOTYPE07) CALCULATE FITNESS08) NEXT GENOTYPE09) CHECK STOP CONDITION, IF YES - STOP10) MAKE SELECTION11) MAKE CROSSOVER12) MAKE MUTATION13) NEXT GENERATION

A word about GA

Population of size p Genotype of b bits Random function – O(1) T(p,b)=O(p)O(b)O(1)=O(pb)

01) FOR EVERY GENOTYPE02) GENERATE RANDOM GENOTYPE03) NEXT GENOTYPE

A word about GA

Population of size p Genotype of b bits Conversion of 1 bit to phenotype – O(1) Fitness calculation, directly value of phenotype

– O(1) T(p,b)=O(p)O(b)O(1)=O(pb)

05) FOR EVERY GENOTYPE06) CONVERT GENOTYPE TO PHENOTYPE07) CALCULATE FITNESS08) NEXT GENOTYPE

A word about GA

Selection – pick p random numbers Crossover – for every genotype (p), combine

with another, length of genotype is b Mutation – for every genotype (p), for every bit

in genotype (b), perform mutation Complexity

Selection – O(p) Crossover – O(pb) Mutation – O(pb)

10) MAKE SELECTION11) MAKE CROSSOVER12) MAKE MUTATION

A word about GA01) FOR EVERY GENOTYPE02) GENERATE RANDOM GENOTYPE03) NEXT GENOTYPE04) FOR EVERY GENERATION (n generations)05) FOR EVERY GENOTYPE06) CONVERT GENOTYPE TO PHENOTYPE07) CALCULATE FITNESS08) NEXT GENOTYPE09) CHECK STOP CONDITION, IF YES - STOP10) MAKE SELECTION11) MAKE CROSSOVER12) MAKE MUTATION13) NEXT GENERATION

T(p,b,n) = O(pb) + O(n)[O(pb) + O(p) + O(p) + O(pb) + O(pb)] = O(pbn)

P, NPC, NP

NP (non polynomial), O(2n), O(3n), O(n!),…

P (polynomial) O(1), O(log n), O(n),

O(n2), O(n3),…

NPC (non polynomialcomplete), we do not know if

NPC = P or NPC = NPOver 1.000.000$ reward

Genetic Algorithms and computational complexity Our n is really big (e.g. 1000, 5000,

100000) If we know a is in class P in most

cases we should not use GA If we know a solution in class P with

big exponent (e.g. 20) we should use GA

If we know our problem is NPC or NP we should use GA

Genetic Algorithms and computational complexity Our n is small

(e.g. 10, 20) We probably

should not use GA in any case

1. Determine O(f(n))2. Determine possible

values of n3. Determine if we

need exact solution4. Determine how

much time we have5. Pick an algorithm

Summary

Use GA only if you know that P class solution does not exists

Use GA only if your data is really large Using GA lead to non optimal solutions But… Using GA in some cases is the only

solution

Computational complexity

Thank you for your attention