- 2015-08-21: 50% less server load with MongoDB on the Drupal 7 site factory at France Télévisions
- 2015-07-15: Our first Drupal 8 production site at France Télévisions is live
- 2014-08-18: 400% speedup in 3 weeks for http://france3-regions.francetvinfo.fr/ : who said Drupal back-offices had to be slow ?
- 2014-02-07: Sotchi Olympics traffic not a problem for http://www.francetvsport.fr/ , which I rearchitected on Drupal 7 in 2013
- 2011-09-14: Completed migration of FranceInfo.FR from SPIP to Drupal
- 2011-07-13: The new social network features of Le Figaro are now powered by an OSInet-designed MongoDB implementation
- 2010-12-21: Madame Figaro brand new site by OSInet and others
- 2010-08-16: France.FR is back online with OSInet and Typhon
- 2010-06-15: the new France Culture, which OSInet helped reach its performance goals, is now online
What to do when your Varnish directory fills up
While browsing my servers Munin reports, I recently noticed how used disk space was constantly increasing on a filesystem which should not have been seeing such growth. After a bit of digging, it appeared the
/usr/local/var/varnish/(host) directory was filled with dozens of sparse files all named
varnish.??????. What could have been happening ?
These files are actually the file backing for the memory-mapped cache used by Varnish by default to hold cache data. Turns out the
varnishd was being run from
/etc/init.d/varnish with a minimal set of options, which did not include the
-s storage option.
In such cases, Varnish defaults to using file-backed memory mapping, and creates a partly-randomly-named file to back the memory map, and preallocates them to the size limit. And, without a storage specification, this means the maximum addressable limit: 2GB on a 32-bit system. Luckily, these are sparse files, not occupying the full 2 GB allocated space, but over time, with periodis server restarts, this can become a problem.
The solutionThe solution is simply to tell Varnish to use a file with a constant name and defined size: that way, whenever Varnish restarts, it will simply reuse the same file name and not waste space creating a new file and forgetting about the earlier one. A start line like this in your will typically fix the problem:
varnishd -a :(port1) -b (host):(port2) -T localhost:(port3) -s file,/usr/local/var/varnish/cache,2G
And, of course, in many cases you will want to add a
-f (/some/config/file.vcl) for your custom rules.
Thanks DocWilco on #varnish for his explanations.