At Mozilla CI Tools we have a script that can sometimes be annoyingly slow. I used cProfile with gprof2dot to better understand how to improve it.
To get a pretty graph of the script’s behaviour I ran:
python -m cProfile -o timing myscript.py gprof2dot -f pstats timing -o graph.dot dot -Tsvg graph.dot -o graph_master.svg
This gave me a very useful graph.
Looking at the graph I was able to identify two bottlenecks that were low-hanging fruit: query_jobs and valid_revision. These two functions are called a lot of times in the script with the same arguments. This means that by adding some simple caching I could improve the script’s speed. Preliminary results show a 2x speed-up. There is still a lot of room for improvement, but it’s a nice start.
tags : Mozilla