Damn Quick Drupal: How to Make Drupal Perform and Scale Like a Rockstar!

This is a live blog of the sessopm “Damn Quick Drupal: How to Make Drupal Perform and Scale Like a Rockstar!” by Michael Cooper at DrupalCon London.

This is one of the more anticipated sessions here at DrupalCon. One of the criticisms is that Drupal doesn’t scale and that it can get very slow. Looking forward to hearing how to avoid both of those. To give you an idea about how big this issue is – the hall is absolutely packed. There are actually more people in here than this morning’s keynote.

Why are many Drupal sites so slow?

• Full page renders.
• Dynamic content to anonymous users – anonymous users don’t need a fresh page. this should be cached.
• Excessive/slow/non-optimized database queries. – SQL needs to be optimized
• Naughty modules

The Answer? Cache!

The internet is one long line of caching… everything is cached because stuff is slow.

How does Drupal cache?

Drupal core and many modules. Successive versions of Drupal have cached more and more and done less on the fly.

How Drupal uses the database to cache:

• cache tables store data for various modules and core
• they share the same schema and use sac he_set() and cache_get() to store and retrieve data.
• caching is plugable

The Testbed:

• Cheap $10/month hosting
• 256MB Ram
• Apache 2.2
• PHP 5.2.17

Tools:

• Devel module
• HProf – integrates with devel module
• New Relic – 3rd party service
• Apache benchmark etc.
• Different test for different things…

Terms:

• Cold cache – an empty cache
• Warm cache – state of your Drupal site and/or server after data has been generated and stored in cache.
• Op-Code Caches/Compilers – APC should be running on your web server.

After loading the cache a page would be rendered with 155 queries in 250ms page rendered in 1.5 seconds. even that is slow. This is before all the elements are added to the basic HTML that a page needs to load.

After adding key-value stores that went down to 107 queries in 36ms

MySQL Caching

There is no one size fits all setting here: it depends on your server.

There are 7 settings you must set. The values here are based on his very cheap server. You may need to change these based on your config.

• key_buffer_size=12M
• query_cache_size=24M
• query_cache_limit=2M
• table_cache=96
• sort_buffer_size=12M
• myisam_sort_buffer_size=12M
• tmp_table_size=12M

(google MySQL performance for more info)

Now we’re down 17ms to run all the queries for the site.

APC

APC compiles your PHP code and makes everything much quicker. Every time a file is read by PHP it is compiled – APC does this once then puts the output into a cached file. After that, it just checks to make sure its the same, then runs. Way less time to get PHP to run.

The last part of this is the session table. it can be added to the APC cache as well.

With all of the above done we’re at 97 queries in 14.8MS page execution time – 180ms

To check this out – use the NET panel in Firebug. Open Firebug, click the NET panel and see how long it takes.

So…

• When does Drupal create cache? – All the time.
• When does Drupal clear cache? – All the time.

Easy wins

• Set and save the minimum cache page – give it at least 5 minutes – you can always manually clear the cache if you need to.
• Set the page cache.
• Always double check the headers – make sure there’s no header tags that prevent caching.

By setting these up you can dramatically reduce page load time. In Drupal 7, you need to set up a PHP file to set up an aggressive cache.

Apache & PHP Issues

This part is way beyond me – I’m not an Apache geek. This is something you’ll definitely want to talk to your ISP or developer about.

• use mod_php vs FastCGI(d) (prefers mod_php)
• Maximum simultaneous processes (server limit/maxprocesses)
• Apache modules
• Apache pre-fork vs. worker (prefers pre-fork)
• Min servers and spare servers
• Child lifetime

One way to help you understand what you’ll need is the Devel module which has performance logging.

Take the amount of memory it takes on average to load a page and divide by the amount of memory you have available for PHP. On his cheap test server:
120MB memory = 10 simultaneous

Max pages per second ended up at about 30, although that’s not a lot when you get slashdotted …

Formula: (P/M) x (e/60) = C
p = # of page views that hit drupal
M = minutes page views collected over
E = execution time per page
c = concurrent requests…

This formula will help you see what kind of performance you can expect.

In his test, setting the maxprocesses to 20 actually hurt performance. This is BIG issue for default Apache. Check this setting: if it’s at 256, your server isn’t configured well.

So in testing and thinking this through: Assume the Worst! Make sure caching is turned on.

Further site optimization

• Save the 404s
• Don’t let anonymous hit image cache generation URL’s
• Be careful with cookies
• Path alias cache (sets up a cache for all the paths on a page)
• Session data caching
• Examine the view queries and view pages
• Edge side includes
• Consider always caching your front page.

Front End Optimization

• Mod Pagespeed helps their browsers load your site faster.
• Reverse proxies save Apache/PHP from running when they don’t have to
• CDNs offer shorter round trips (see Amazon Cloud)
• Domain Sharding

Wow – this was a very technical session… but really great!