- 2015-08-21: 50% less server load with MongoDB on the Drupal 7 site factory at France Télévisions
- 2015-07-15: Our first Drupal 8 production site at France Télévisions is live
- 2014-08-18: 400% speedup in 3 weeks for http://france3-regions.francetvinfo.fr/ : who said Drupal back-offices had to be slow ?
- 2014-02-07: Sotchi Olympics traffic not a problem for http://www.francetvsport.fr/ , which I rearchitected on Drupal 7 in 2013
- 2011-09-14: Completed migration of FranceInfo.FR from SPIP to Drupal
- 2011-07-13: The new social network features of Le Figaro are now powered by an OSInet-designed MongoDB implementation
- 2010-12-21: Madame Figaro brand new site by OSInet and others
- 2010-08-16: France.FR is back online with OSInet and Typhon
- 2010-06-15: the new France Culture, which OSInet helped reach its performance goals, is now online
When Dropbox ignores files to sync on Linux...
Much like many Drupal devs, I happen to make fairly intensive use of Dropbox, and even use a "pro50" account to sync my always increasing set of "current" source folders, including checkouts of all major Drupal versions and lots of contrib.
Which means that, beyond the number of gigabytes of data Dropbox has to sync, the number of the files making up these gigabytes has also been increasing, currently to around 100k files. After I started playing with checkouts of the Drupy project in preparation for the Drupyx experiment, I noticed that, when I created some new files under the Drupy directories, their creations and subsequent changes would not be tracked by Dropbox, but they would correctly sync if I renamed the Drupy directory itself or a directory above it. Something like this:
|Action||Result on Dropbox.com|
|mv ~/Dropbox/src/drupy/src/foo.py ~/Dropbox/src/drupy/src/bar.py||Ignored|
|mv ~/Dropbox/src/drupy ~/Dropbox/src/drupa||Full sync below ~/Dropbox/src/drupa|
|mv ~/Dropbox/src/drupa ~/Dropbox/src/drupy||Full sync below ~/Dropbox/src/drupy, including "foo.py"|
And all this while operations on a PC running Windows tied to the same account did not experience any similar problem. What could be going wrong ?
Having already met folder corruption during the early days of Dropbox on Linux, I went as far as unlinking this computer from the account, even uninstalling, and
rm ~/Dropbox ~/.dropbox ~/.dropbox-dist, removing the visible folder, its local work area/cache, and its code, then reinstalling and relinking the PC. Nada. Zilch.
Now, the Dropbox implementation for Linux rests heavily on the inotify subsystem to avoid having to do active background directory scans. And inotify has a per-user limit on the number of watches per real user id. As per http://www.kernel.org/doc/man-pages/online/pages/man7/inotify.7.html:
This specifies an upper limit on the number of watches that can be created per real user ID.
cat /proc/sys/fs/inotify/max_user_watches gives a clue: on Ubuntu Lucid, the default value is a paltry 8192, much too low for similar situations. And this helps us find a mention in the Dropbox help section about syncing:
There's an easy fix for this. Open a terminal and enter the following:
sudo sysctl fs.inotify.max_user_watches=100000
This does indeed fix the problem: after applying that command, the ignored folders are no longer ignored. But the explanation is still missing a key point: at this point, after the next reboot, the parameter will return to its default value, 8192. To make the change permanent, the value needs to be saved in one of the sysctl configuration files: typically
/etc/sysctl.conf for a one-off change or a specific
/etc/sysctl.d/60-dropbox.conf when using a special build of the Dropbox client. Just add a line like
fs.inotify.max_user_watches=100000 to that configuration file, then do a
sysctl -p to reread it.
Your value will then be reloaded at each boot by the
init process when it reads
/etc/init/procps.conf, and your Dropbox will continue syncing happily.