Time used during callback

[rengel8]

I have made a larger test and found out that my callback function is not that ideal in aspects of performance.

def callback(ga_instance):
    print("Generation : {gen}".format(gen=ga_instance.generations_completed))
    print("Fitness    : {fitness}".format(fitness=ga_instance.best_solution()[1]))
    solution, solution_fitness, solution_idx = ga_instance.best_solution()       
    print("Solution   : {solution}".format(solution=solution))
    recheck = tsa_execution(solution)
    print("Return     : {sresult}".format(sresult=recheck['summary']))
    print('')

It seems that I have 3 times heavy multicore processing introduced. One redundancy of which is the second call of ga_instance.best_solution(), which can and should be avoided. In the figure below one can see three plateaus.

If I completely disable/uncomment on_generation=callback I get only one plateau. This results in a performance gain, which is quite significant again (1091.18 second(s) vs. 775.59 second(s)) .

I was also taking the best solution to run my time series analysis again (recheck) to retrieve the result, which is needed (the basis) for calculating the actual fitness funtion.

Findings
It seems that ga_instance.best_solution() is causing a full execution on its own again, which might be unimportant in certain cases, where the needed performance/time for that is not significant. In my case it does not appear to be wise.
It can also be seen, that the gene space of 8 genes with a total population (unique combinations) of 5 trillion 😵 is taking some time on its own each generation, but it is working.
Maybe it would be nice to have some sort of verbose-mode to see some printing of internal numbers to get a feeling about quantification/calculations under the hood. I'm not sure if these cascading processes can be speed enhanced, but it is interesting, that the multi-core part I do is not taking that much time compared to the overall cycle (in my destinctive example above).

[ahmedfgad]

You are right. There was a performance issue in calling the best_solution() method because it was recalculating the fitness of each solution to determine the best solution.

The issue was solved by:

1. Introducing some new instance attributes that hold information about the outcomes from each generation. The names of these attributes start by last_generation_. One of these attributes is called last_generation_fitness which saves the fitness values of the solutions in the last generation. It is supported starting from PyGAD 2.12.0. Check the release notes: https://pygad.readthedocs.io/en/latest/Footer.html#pygad-2-12-0
2. Introducing a new argument in the best_solution() method called pop_fitness. It accepts the fitness values of the population. So, if you would like to find the best solution in the population of the last generation, then pass the last_generation_fitness attribute to this argument. Otherwise, it will have the default value None which means then the fitness function must be called to calculate the fitness values of the solutions. This way causes delay.

Here is an example of using the the last_generation_fitness attribute:

def callback_generation(ga_instance):
    print("Fitness    = {fitness}".format(fitness=ga_instance.best_solution(pop_fitness=ga_instance.last_generation_fitness)[1]))

For the complete example, check the example.py script.

I hope this solves your issue. As usual, you are welcome to ask for any feature or report any issue that comes to your mind. I will be happy to make them supported.

Thanks, Rainer!

[rengel8]

I followed the advice and use this code now to also get the best/recent solution (with save_best_solutions=True).

def callback(ga_instance):
    print("Generation : {gen}".format(gen=ga_instance.generations_completed))
    print("Fitness    : {fitness}".format(fitness=ga_instance.best_solution(pop_fitness=ga_instance.last_generation_fitness)[1]))
    print("Solution   : {solution}".format(solution=ga_instance.best_solutions[-1]))

This does work for me and results in a 20% speed gain. In another, even larger test, I noticed beyond that, that I do not have the valleys betwenn plateaus anymore. I can not say more about that right now, but the overall performance seems to be better this way.

[ahmedfgad]

That is good!

The save_best_solutions parameter causes the best solution per each generation to be saved. If many generations are used, this would cause a memory problem.

You can still use the best_solution() method to get the best solution. This is by simply by returning the value at index 0 of the result of this method.

print("Solution    : {solution}".format(solution=ga_instance.best_solution(pop_fitness=ga_instance.last_generation_fitness)[0]))

[rengel8]

Now I know what is causing the longer valley part in the CPU-usage plot at the beginning of this topic. I tried mutation_type with random and adaptive in comparison and the characteristic valley above is related to the adaptive mode.

Identical GA (with):

adaptive
1 generation, 732secs
5 generations, 3182secs

random
1 generation, 227secs
5 generations, 675secs

So it seems very clear that in my case the adaptive mode isn't by far that performant as random. The adaptive mode is based on a nice idea and might be neat to use in some cases, but it might come at some speed cost to consider.

[ahmedfgad]

Thanks @rengel8.

There is a room for speeding-up adaptive mutation to avoid recalling the fitness function and make use of the last_generation_fitness attribute.