- 2012-08-19: Working on the new Drupal 8 Entity system at Drupalcon Munich
- 2012-06-15: Working on the new Drupal 8 Entity system at DrupalDevDays Barcelona
- 2012-03-23: Working on the future Drupal Document Oriented Storage at DrupalCon Denver. D8 or later ? Bets are on.
- 2011-09-01: Building an Unfuddle to Drupal Casetracker import module using Migrate
- 2011-08-28: Back from DrupalCon London and its WSCCI code sprint. Wow.
- 2009-11-29: mongodb_watchdog module created by dereine, ported to D7 by me in about half an hour, and migrated in a larger MongoDB project by damz before the hour ended. Wow...
What to do when your Varnish directory fills up
While browsing my servers Munin reports, I recently noticed how used disk space was constantly increasing on a filesystem which should not have been seeing such growth. After a bit of digging, it appeared the
/usr/local/var/varnish/(host) directory was filled with dozens of sparse files all named
varnish.??????. What could have been happening ?
These files are actually the file backing for the memory-mapped cache used by Varnish by default to hold cache data. Turns out the
varnishd was being run from
/etc/init.d/varnish with a minimal set of options, which did not include the
-s storage option.
In such cases, Varnish defaults to using file-backed memory mapping, and creates a partly-randomly-named file to back the memory map, and preallocates them to the size limit. And, without a storage specification, this means the maximum addressable limit: 2GB on a 32-bit system. Luckily, these are sparse files, not occupying the full 2 GB allocated space, but over time, with periodis server restarts, this can become a problem.
The solutionThe solution is simply to tell Varnish to use a file with a constant name and defined size: that way, whenever Varnish restarts, it will simply reuse the same file name and not waste space creating a new file and forgetting about the earlier one. A start line like this in your will typically fix the problem:
varnishd -a :(port1) -b (host):(port2) -T localhost:(port3) -s file,/usr/local/var/varnish/cache,2G
And, of course, in many cases you will want to add a
-f (/some/config/file.vcl) for your custom rules.
Thanks DocWilco on #varnish for his explanations.