- 2012-08-19: Working on the new Drupal 8 Entity system at Drupalcon Munich
- 2012-06-15: Working on the new Drupal 8 Entity system at DrupalDevDays Barcelona
- 2012-03-23: Working on the future Drupal Document Oriented Storage at DrupalCon Denver. D8 or later ? Bets are on.
- 2011-09-01: Building an Unfuddle to Drupal Casetracker import module using Migrate
- 2011-08-28: Back from DrupalCon London and its WSCCI code sprint. Wow.
- 2009-11-29: mongodb_watchdog module created by dereine, ported to D7 by me in about half an hour, and migrated in a larger MongoDB project by damz before the hour ended. Wow...
When Dropbox ignores files to sync on Linux...
Much like many Drupal devs, I happen to make fairly intensive use of Dropbox, and even use a "pro50" account to sync my always increasing set of "current" source folders, including checkouts of all major Drupal versions and lots of contrib.
Which means that, beyond the number of gigabytes of data Dropbox has to sync, the number of the files making up these gigabytes has also been increasing, currently to around 100k files. After I started playing with checkouts of the Drupy project in preparation for the Drupyx experiment, I noticed that, when I created some new files under the Drupy directories, their creations and subsequent changes would not be tracked by Dropbox, but they would correctly sync if I renamed the Drupy directory itself or a directory above it. Something like this:
|Action||Result on Dropbox.com|
|mv ~/Dropbox/src/drupy/src/foo.py ~/Dropbox/src/drupy/src/bar.py||Ignored|
|mv ~/Dropbox/src/drupy ~/Dropbox/src/drupa||Full sync below ~/Dropbox/src/drupa|
|mv ~/Dropbox/src/drupa ~/Dropbox/src/drupy||Full sync below ~/Dropbox/src/drupy, including "foo.py"|
And all this while operations on a PC running Windows tied to the same account did not experience any similar problem. What could be going wrong ?
Having already met folder corruption during the early days of Dropbox on Linux, I went as far as unlinking this computer from the account, even uninstalling, and
rm ~/Dropbox ~/.dropbox ~/.dropbox-dist, removing the visible folder, its local work area/cache, and its code, then reinstalling and relinking the PC. Nada. Zilch.
Now, the Dropbox implementation for Linux rests heavily on the inotify subsystem to avoid having to do active background directory scans. And inotify has a per-user limit on the number of watches per real user id. As per http://www.kernel.org/doc/man-pages/online/pages/man7/inotify.7.html:
This specifies an upper limit on the number of watches that can be created per real user ID.
cat /proc/sys/fs/inotify/max_user_watches gives a clue: on Ubuntu Lucid, the default value is a paltry 8192, much too low for similar situations. And this helps us find a mention in the Dropbox help section about syncing:
There's an easy fix for this. Open a terminal and enter the following:
sudo sysctl fs.inotify.max_user_watches=100000
This does indeed fix the problem: after applying that command, the ignored folders are no longer ignored. But the explanation is still missing a key point: at this point, after the next reboot, the parameter will return to its default value, 8192. To make the change permanent, the value needs to be saved in one of the sysctl configuration files: typically
/etc/sysctl.conf for a one-off change or a specific
/etc/sysctl.d/60-dropbox.conf when using a special build of the Dropbox client. Just add a line like
fs.inotify.max_user_watches=100000 to that configuration file, then do a
sysctl -p to reread it.
Your value will then be reloaded at each boot by the
init process when it reads
/etc/init/procps.conf, and your Dropbox will continue syncing happily.