At Mozilla CI Tools we have a script that can sometimes be annoyingly slow. I used cProfile with gprof2dot to better understand how to improve it.

To get a pretty graph of the script’s behaviour I ran:

python -m cProfile -o timing myscript.py
gprof2dot -f pstats timing -o graph.dot
dot -Tsvg graph.dot -o graph_master.svg

This gave me a very useful graph.

Looking at the graph I was able to identify two bottlenecks that were low-hanging fruit: query_jobs and valid_revision. These two functions are called a lot of times in the script with the same arguments. This means that by adding some simple caching I could improve the script’s speed. Preliminary results show a 2x speed-up. There is still a lot of room for improvement, but it’s a nice start.

tags : Mozilla