- 2015-08-21: 50% less server load with MongoDB on the Drupal 7 site factory at France Télévisions
- 2015-07-15: Our first Drupal 8 production site at France Télévisions is live
- 2014-08-18: 400% speedup in 3 weeks for http://france3-regions.francetvinfo.fr/ : who said Drupal back-offices had to be slow ?
- 2014-02-07: Sotchi Olympics traffic not a problem for http://www.francetvsport.fr/ , which I rearchitected on Drupal 7 in 2013
- 2011-09-14: Completed migration of FranceInfo.FR from SPIP to Drupal
- 2011-07-13: The new social network features of Le Figaro are now powered by an OSInet-designed MongoDB implementation
- 2010-12-21: Madame Figaro brand new site by OSInet and others
- 2010-08-16: France.FR is back online with OSInet and Typhon
- 2010-06-15: the new France Culture, which OSInet helped reach its performance goals, is now online
From logging primitives to drivers, to servers, to logs analysis platforms.
Think Drupal watchdog(), FIG PSR-3 standard, Monolog vs log4php, syslog vs gelf vs logstash, Graylog, fluentd, Flume, Rollbar, Loggly, and coopetitors.
Graylog is, in the words of its creators, a tool to
Store, search & analyze log data from any source, and it puts a lot of power in our hands to slice, dice, and generally combine, gather, and parse content from various sources, notably syslog and Gelf sources, as well as many file-type sources thanks to the Graylog Collector. Which means it makes it a snap to build event-oriented dashboards like the left part of this example, and even some event volume graphs like the topmost one on the right.
The problem: logging non-event information
This covers only logged events, which is fine overall, since Graylog is a log analysis platform, not a graph-oriented monitoring system like Munin / Cati / Ganglia et alii. However, in many cases, especially when building dashboards instead of performing some specific research, one may want to keep an eye on average system load, or other non-event information, if only to know when to switch one's attention to the monitoring system.
So how can one add system load information, which is not event-based, to a log dashboard like the three bottom-right graphs on the previous dashboard ?
One nice thing during Drupal 7/8 development is the ability, thanks to the
devel module, to get a list of all SQL queries ran on a page. As I've been working quite a bit on MongoDB in PHP recently, I wondered how to obtain comparable results when using MongoDB in PHP projects. Looking at the D7 implementation, the magic happens in the
// Start logging on the default database.
// Get the log contents, typically in a shutdown handler.
$log = \Database::getLog(DB_CHANNEL);
With DBTNG, that's all it takes, and devel puts it to good use UI-wise. So is there be an equivalent mechanism in MongoDB ? Of course there is !
Continuing this exploration of logging solutions used in various projects, let's look at logging in Kohana 3.
While Monolog and log4php share a mostly common logging model of a frontal Logger object instantiated as many times as needed to supply different logging channels, in which log events are Processed/Filtered then written out by Handlers/Writers, Kohana builds upon a simpler model, which can be summarized by three patterns:
- Singleton: there is only one instance of the Kohana
Log_Writerinstances are attached (and detached) to(/from) the logger instance and handle events they are interested in based on their own configuration. Much like a Drupal hook, all writer instances receive each
- Delegation: the
write()to trigger the buffered writing, but does not implement it itself, but delegates to the
Log_Writerobjects to perform it. Buffered logging control is a
Logproperty, not a
It is based on the famous log4j package from the Java world, and from uses of this package I have seen on customer sites, I feel that it carries a lot of useless baggage, and is - in my opinion - significantly less of a good match than Monolog for Drupal 8.
Monolog vs log4php : equivalences
There is some degree of equivalence between the Monolog and log4php components:
|Log an event||Logger||Logger||Very similar|
|Store an event||Handler||Appender||both can be chained, group, control bubbling (Monolog) / filtering (log4php)|
|Format an event representation||Formatter||Layout||log4php layouts can format a group of events, Monolog formatters format an individual event|
|Massage event data||Processor||Renderer||Not so similar. Monolog processors will often add extra data, while log4php Renderers are typically used to format non-string events as strings.|
I've been discussing Monolog in Drupal events (DrupalCamp Lyon, DevDays Barcelona) as a possible alternative to the legacy Drupal
watchdog() service for quite some time, but never took the time to explain it in writing, and the feature freeze date is looming ahead, so since I'm taking part in th Gent code sprint, and code has been starting to take shape
here is an overview of the Monolog classes.
The diagram below is a simplified version of the Monolog architecture. It includes all classes and interfaces, but only the most significant methods, no constants, and none of the non-bundled classes and interfaces upon which some of the builtins depend.
Currently, various modules within core and contrib use various tables to store historical (i.e. "write once, read sometimes, never update") data. The most obvious is the
watchdog table, but others exist, including
zeitgeist's eponym table.