Promoted from the comments section:

Out of curiosity, what is the max % of CPU usage reported for a thread that’s hammering away at a compile? (In the screenshot the max is 2.8% but it’s sorted and I imagine some scrolled off the screen.)

For single-thread processes I’ve seen it hover around 2.8 to 3%. For multi-thread processes I’ve seen larger numbers of course. I’m not too sure that exact number is very valuable in the larger scheme of things. I’ve hammered apache on it yesterday, on a copy of my weblog (if you go to the machine URL, you’ll see this weblog with an older copy of the postings database), and experimented a bit with the number of threads requesting pages, and the number of preforked apache processes. Sun recommends using the prefork model, but I’m also going to try the mpm model we’re using on the other Suns since I didn’t really like the behavious apache was showing – it rapidly grew to using 4.5 Gb of memory, which might be a problem for us, since we’ll also be running a few java virtual machines on it with between 2 and 3 Gb of memory allocated to each of them. In the prefork configuration it served between 100 and 400 weblog pages per second without breaking a sweat depending on how I put a load on it.

Anyway, coming back to your question, the /server-status page in apache said apache was using between 400 and 950% CPU, which is a different number than you’d get by adding all the 2.8% and 3% processes and multiplying that by 32. In the screenshot you saw mostly 2.8%, and not the “theoretical” 3.1 % if you divide 100% by 32 CPU’s, and I think that’s because the cc processes were fairly short-lived, just compiling one file. If you run a single-thread task and let it live longer, you’ll see prstat reports of 3 and 3.1%.