Linux system profiling with perf

Most probably you have heard of dtrace. And not so much about the Linux alternatives of it. Well today I decided to give the Linux perf tool a shot while trying to find performance problem on a heavily loaded LAMP server.

Installing perf

Actually installing perf on Ubuntu 12.04 is really straightforward :

Collecting performance events

After if finished installing you can run a system wide profiling with :

And stop with Ctrl-C, to finish collecting the data.

will show the what processes are using the most CPU:

perf report

Obviously most CPU goes¬† to apache(mod_php) and and php5 standalone processes. And looking at the names of the functions, on top is the garbage collection. If you don’t have the debug symbols¬† installed then you won’t see this breakdown by function but instead just total process percentage which is not so useful.

To install debug symbols install the php5-dbg package, and update the perf buildid database.

Of course just knowing which function is taking up a lot of CPU is very useful but even more useful is to have the backtrace of who called it. Luckily perf does that. Just add -g to the record command:

And after with :

you can expand the function to see the callers with E and C shortcuts. Or use :

To see it as ASCII tree in the console like this:

perf report call tree

The profiling can be done only for a single running process :

Or for a single execution of a command:

Also there is simple statistics mode :

This is the default set of collected performance events, but you can sample more with the -e switch. For a list of all supported events run:

You can find more details and examples on the project wiki .

Leave a comment