UPDATE: This is a followup – please read the roogiional – and remember, these tests are NOT Microsoft official and NOT endoursed by my Employer.
Last week I posted some comparative performance result that I got from comparing the relative performance of PHP on PHP 5.2 to PHP 5.3, PHP on Linux to PHP on Windows and PHP to ASP.NET.
First, I want to thank the PHP community for their responses. With only a few exceptions the PHP community proved that they are not a bunch of teenage jealous (anymore ? ) but rather a group of professionals, confident in their work and ready to engage in respectful dialog.
In my tests I declined to install PHP op-code caching for my tests because its not installed on some of the hosts I use and is not built in to PHP (yet – though I understand it will be in a future version and is part of Zend’s free Community Edition Server)
Andi, Brandon, and other pointed out that in business application deployment, most PHP shops use op code caching, so I installed it on Linux and Windows and re-ran most of the tests.
Note that the PHP 5.3 with APC tests were run on a clean Debian 5 setup on the same hardware and running the same code. I moved to Debian because building on Ubutu was giving me dependency fits. You’ll note that the performance delta between Ubuntu and Debian was negligible.
You can click on the image below to see the whole updated times table.
[ Click here for the whole table. ]
Some items showed a small improvement in performance, some stuff up to 25% faster, but overall it was far less than I expected.
Some things ran slower with APC but I attribute that to simple machine variance. (Note, the numbers in the table are NOT the first page run. I loaded the page, refreshed twice to make sure to hit the cache and then took the numeric results.)
Some folks suggested that these weren’t “fair” tests.
An empty loop test or an empty function call, for example, is very important because the results indicate language or platform OVERHEAD. That overhead is addition latency in page delivery and this, if it exists, is an important consideration.
A few of my PHP friends concede the accuracy of the results but made very insightful comments.
I’ll paraphrase them here.
- The amount by which ASP.NET is faster than PHP doesn’t matter to me. PHP is simply my preference and it’s fast enough in my applications.
- I build on Drupal and I know PHP best so what I might need to spend on more hardware I make up for in developer productivity.
These are all excellent points !
Though I think the tests that I’ve run so faw are fair, they are incomplete and their incompleteness prevents us from seeing the whole story.
Contrary to the couple of inevitable allegations that I’m “just” a Microsoft shill. I’m doing this because I want to KNOW.
My last tests revealed things like PHP file access and MySQL access on Windows has real performance issues and this information was received by the various teams involved here at Microsoft. They are following up and hopefully the data will directly result in improvements.
So – HERE IS YOUR CALL TO ACTION.
Use the contact form http://misfitgeek.com/contact/ and tell me what tests you would like me to run in order to grow my test suite in ways that you think are fair and meaningful.
Here are some of what I’m planning.
- Real Page tests – loops, calls, and object instantiation are one kind of test, but full page rendering is another meaningful benchmark.
- Load tests – can one environment or another handle more simultaneous requests.
- During Simultaneous requests, does one stack degrade performance more than another.
- In the above context – how much does 64 bit matter on each platform
I hope you’ll send me your suggestions.