Update: Turns out I am the bronze medallist. Geish Hori is the silver medallist. A well-deserved win. Well done Geish!

Most of the blogs on the Ozblogistan network use BLC. Recently, Larvatus Prodeo came on board and expected it to work for them too. But it simply hung without giving much in the way of error message. I pottered around with it at the time without much success.

Today it occurred to me that the problem is the way BLC works. BLC aims at being compatible with MySQL 3.x. Among other things, this means giving up subqueries and having piss-poor SELECT DISTINCT.

So BLC issues this beauty of a query:
SELECT comment_post_ID, post_title
FROM (wp_comments LEFT JOIN wp_posts ON (comment_post_ID = ID))
WHERE comment_approved = '1'
AND wp_posts.post_status='publish'
AND comment_type<>'pingback'
AND comment_type<>'trackback'
ORDER BY comment_date DESC;

This query pulls out the post ID and post_title for every approved, published comment in the database. On a site like Larvatus Prodeo, the results run to ~165k lines. BLC pulls these results into PHP as objects, one for each line; then it merges down the results to determine which posts have comments.

As you can imagine, this is memory intensive. Very memory intensive. And it’s a bit brutal on the CPU too, especially when garbage collection occurs. The upshot is that on Larvatus Prodeo, BLC exceeds the memory limits for PHP, and that for other sites, it adds about between 600 and 1200 milliseconds of processing time. Not cool.

Naturally, I’m running MySQL 5.x on my server, so SELECT DISTINCT works ‘as advertised’. So I’ve implemented a super high tech optimisation on BLC:

SELECT DISTINCT comment_post_ID, post_title
FROM (wp_comments LEFT JOIN wp_posts ON (comment_post_ID = ID))
WHERE comment_approved = '1'
AND wp_posts.post_status='publish'
AND comment_type<>'pingback'
AND comment_type<>'trackback'
ORDER BY comment_date DESC;

While this query is slower than the original, it basically returns only the list of posts with comments on them. On Larvatus Prodeo, this is a much more manageable ~2.5k lines of results. The reduction in PHP runtime completely overshadows the increase in query time; and besides, the smaller result sets don’t clog up the MySQL query cache.

So there you have it: a one-word way to dramatically improve the performance of Brian’s Latest Comments on MySQL 5.x-backed WordPress.

Update: Two additional optimisations are worthy. First, add a LIMIT clause to your copy of BLC along the lines of:

$posts = $wpdb->get_results("SELECT DISTINCT
comment_post_ID, post_title
FROM ($wpdb->comments LEFT JOIN $wpdb->posts ON (comment_post_ID = ID))
WHERE comment_approved = '1'
AND $wpdb->posts.post_status='publish'
$ping
ORDER BY comment_date DESC
LIMIT $num_posts;");

Second, the SELECT DISTINCT hammers the tables much harder, so add indexes for the relevant fields:

CREATE INDEX post_status_index ON wp_posts(post_status);
CREATE INDEX comment_date_index ON wp_comments(comment_date);
CREATE INDEX comment_type_index ON wp_comments(comment_type);
CREATE INDEX comment_date_approved_index ON wp_comments(comment_date_gmt);
CREATE INDEX post_title_index ON wp_posts(post_title(50));

Update 2: The query above doesn’t order the results according to the time of the latest comments; it winds up ordering by posting date. Not what my users want. Try this instead:

$posts = $wpdb->get_results("SELECT comment_post_ID, post_title, max(comment_date) AS max_date
FROM ($wpdb->comments LEFT JOIN $wpdb->posts ON (comment_post_ID = ID))
WHERE comment_approved = '1'
AND $wpdb->posts.post_status='publish'
$ping
GROUP BY post_title
ORDER BY max_date DESC
LIMIT $num_posts;");

Update 3: Nope. Performance is still atrocious. I’ll revisit this in a few weeks when I’ve settled in Darwin.

I have HTTP compression enabled on the webserver (gzip, to be precise). I found out through the excellent YSlow plugin that not everything was being compressed. HTML was compressed, but not style sheets or scripts.

The nginx settings are now as follows:

gzip on;
gzip_buffers 128 8k;
gzip_types text/plain application/xml text/xml application/xml+rss text/css application/x-javascript text/javascript application/xhtml+xml;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";

This provides a large buffer to prevent unexpected hangs. I’ve added mimetypes for javascript, RSS feeds and CSS files.

On the MySQL front, I received some advice from MySQL expert Paul Moen of Pythian.

Paul’s main advice upon looking over the MySQL settings was to increase the key_buffer variable from 8Mb. It’s currently at 96Mb. This has sped up some queries. He also advised enabling and watching MySQL’s slow query log. This helped me spot queries where indexes might speed up things, so I added a few to relevant columns which also reduced the number of slow queries. Thanks Paul.

Another change was to notice that the advice for setting connection_max in MySQL is essentially Apache/mod_php-specific. It doesn’t make sense in a FastCGI context. In FastCGI you launch a fixed number of PHP instances. The number of connections from WordPress into MySQL is limited to the number of PHP instances; so there’s no requirement to set a higher connection_max setting. This is not an optimisation per se, but it tidies things up a bit.

I double checked my PHP opcode cacher XCache and found that it was not enabled. Annoyingly, aptitude will install but not activate this for you. To actually enable it, it’s necessary to edit the php.ini file and add a line:
extension=xcache.so
Somewhere in the file.

Previously I have added favicons. Watching the error log, I saw requests for apple-touch-icon.png. Apple, in their infinite wisdom, have decided to add Yet Another Assumed File to the list of things webmasters need to know about. I’ve gone through and added icons for each of the Ozblogistan sites. It was, I must tell you, tedious and time-consuming. Though Apple currently scale the images to 57×57, they will almost definitely upgrade this in future, so I’ve placed 128×128 icons for now.

Finally, I adjusted the ‘swappiness’ of the server. I configure PHP and MySQL to occupy less than the full RAM available; thus, I don’t want anything swapped out to disk. I’ve set swappiness to 0, which tells Linux to keep applications in memory at all times until it has no choice other than to swap them out.

A WXR splitter (there are several) is pretty much required. Why doesn’t WordPress perform this task itself?

It’d also be nice if it did even very basic duplicate checking.

Secondly, they’re not going to introduce software patents.

NZ: 2. Australia: Sux.

By ‘beef’ they mean that a cow managed to wander into the factory once. They had to herd it out past the lasagne department. “I have a great idea!”, says the marketing chief.

There are tofu cubes with more beef in them, I reckon.

When I sent my resume to CDU, I didn’t hear back from them for a few weeks. I figured they’d quietly discarded my resume, which I was a bit sad about. Then, while I was on holiday in Darwin, they rang to ask if I could do a phone interview. Luckily I was in a position to interview in person, which I think helped my chances.

Last week CDU offered me the spot, with a good base rate and excellent (17%) superannuation. The super is one of the things I found most attractive. Also attractive were the location (I’m a Darwin boy) and the opportunity to become familiar with Oracle technologies, which continue to be in wide demand.

I’ve verbally accepted the offer today and I expect to receive the contract paperwork in the next few days. I’m looking forward to it.