Caching PHP pages, Varnish, nginx, other?

Question

I have a web application running on a low end box (1gb RAM), serving a mixture of static and dynamic (php) pages. These PHP pages are querying a MYSQL database which does not change often at all - once a week?.

I am looking to do a fair amount of caching to keep everything nice and speedy. I have pages that although are php, the information rarely changes (Getting a list of items that might change once every few monthly?). A couple of pages may be listing up to 400 records.

I have Varnish, nginx, PHP-FPM, APC, MYSQL installed. I 'think' I have everything set up correctly. Pages are being served, and I have hits showing up in Varnish... Brill! However, because of the nature of the website, I am not sure it is as optimised as it could be.

A recent search has suggested a number of things that may help with my php pages:

• nginx FastCGI caching
• memcached
• MYSQL Query Caching

An example: A fresh PHP page, where it is listing quite a few (200+) records: 2 seconds After a refresh, 1.5(ish) seconds. edit: Am I being unrealistic to expect this page to be cached somewhere along the line and be supplied much quicker after it has been visited?

What would be my best option? One or all of the above?

Answer 1

• Memcached requires your code to actually use it. But if you wrote this code then this should be easy to do :)
• MySQL caching works to an extent. Ideally, upgrade to SSD drives if not already.
• I've never heard of FastCGI caching. Are you referring to caching dynamic pages as static?

Honestly, if you have tuned everything that best that you can, I would consider hardware upgrades. If you have time, look into Facebook's research like HipHop (https://github.com/facebook/hiphop-php/wiki). They have done some amazing research and development for creating fast loading dynamic pages.

Best of luck!

Answer 2

Be careful with MySQL's query cache - it uses a global lock, so when you start to get too much activity you'll see periodic deadlocks. The query cache is only useful for queries that are exactly the same, and if you're having a lot of those you aren't doing application-level caching right.

The first step in working on performance is always to profile; intuition-based optimization is a really good way to waste your time.

At work, we use Graphite as our data-store, with a modified version of StatsD and pipe-to-graphite as the primary data-sending methods. Both of these tools make it very easy to send data in, and then Graphite has a multitude of analysis tools.

For instance, using the pipe-to-graphite varnish script, we grab all the statistics from varnishstats. This allows us to easily make hit/miss graphs in Graphite, like this:

You can do the same thing with memcached, and add hooks into your own application to record... well, anything and everything!

You may also find single-page analysis tools like xhprof and YSlow useful.

Once you have metrics, then not only do you know what you should be working on, but you'll be able to measure the improvement when you're done. Everyone likes validation!

Answer 3

when you dont need special features from varnish, you could drop it and use nginx's fastcgi_cache. but beware, in opposite to proxy_cache your can have only 1 cache_zone in your whole setup.

maybe your setup is a little over-optimized ... i'd think about either use nginx or varnish only.

when it comes to memcache: depending on how many requests/second you have memc can be a real boost, but maybe not neccessary on your case. when you have a good frontside-cache you dont need to enable memcache, but YMMV. what i understand: you deliver pages that arent changed often, right?

i'd always try to optimize thw following way:

1. try to avoid requests being answered by your application-stack -> intercept requests by frontenend (nginx/varnish) cache and static server, where possible
2. if php, use an opcode-cache like APC
3. tune your db
4. monitor your system (load, mem, ram, netio, hdio, netconns, netconns:http, site_stats)
5. repeat tuning