As this is the first in my update report series, it will contain a look at the past and try to explain things to people who are not so familiar with Drupal.
Tl;dr: for the first time, right now it seems the dream of running Drupal 8 on MongoDB only is totally attainable.
Previously, I was writing these in private email to my boss at 10gen, but I believe it's better in public. Most of this one was actually sent so the tone here and there might not actually be of a blog post. Recommended reading: Updated Drupal 8 release schedule from Dries.
Drupal always had modules extending its functionality (modules) and pluggable subsystems replacing some of the functionality. The method of pluggability was very simple and incredibly crude: the name of the file to be included, say, for session handling was read from a setting. Such settings are called 'variables' and they can be stored either in settings.php (where database connection details are) or in the database. So you added $conf['session_inc'] ='sites/all/modules/mongodb/mongodb_session/mongodb_session.inc'; and you were done. mongodb_session.inc implemented the same functions as session.inc did. If you consider that Drupal 7 was the first version to require PHP 5, this is not as bad as it soiunds: it was a fast, cheap, simple way to implement pluggability in a language (PHP 4) without interfaces. There was zero meta data, there was no discoverability and so it was absolutely impossible to provide an UI.
Drupal 8 attacks this from several fronts. One front is the plugin API itself. There is a very nice OOP-based plugin API http://drupal.org/node/1637614. Obviously, quite some of the things need to be overridden to make Drupal 8 run with MongoDB only will become plugins and so a good plugin API is necessary. The Drupal community have chosen annotations as the way to provide metadata. The interest on the community side in plugins were using them to provide a unified mechanism of output ( http://groups.drupal.org/scotch ) which means there will be a lot of plugins. That didn't lie well with the fact that the Doctrine annotation reader relied on ReflectionClass -- doing discovery might have meant consuming a lot of memory. I have
added the capability to use the PHP tokenizer to extract the doxygen from the source code which is slower and uses more peak memory but once the reading of one class is done, the tokens can be dropped and so it does not matter how many classes go through the annotation reader, the memory consumption does not skyrocket. Our annotation reader is based on this capability, it was well received and at the Midwest Drupal Developer Summit Dries accepted it and on August 11, this was committed. With this, the plugin system is ready.
Another front is using the Symfony dependency injection container (DIC). Now, this obviously won't work everywhere because Drupal core is a gigantic codebase and converting that to be fully injection-based is not feasible within one release cycle. To bridge this, Drupal 8 has a DIC statically stored and retrievable from any of the old code, the static function is called drupal_container(). There is a lot of work going on to make the DIC more widely used, speedier and so on. I am working on this too because fostering a more DIC-based codebase is clearly beneficial. It is now possible to write PHP files from Drupal and the DIC will be the first to use it.
Now, it is possible to change the services necessary for bootstrap from the same settings.php mentioned in paragraph 1, the rest can be changed from a module (this duality might or might not be resolved by release time). This is of course much nicer than the current pluggability and will work well for us.
That's about pluggability.
One of the biggest hindrances in getting Drupal to work with MongoDB only was how the configuration information is read from arbitrary, scattered SQL tables. This is rapidly coming to an end, there is a configuration management initative which provides a centralized way to manage configuration. The community is interested in this for deploy purposes, for MongoDB the interest lies in getting rid of those pesky SQL tables. The canonical storage for configuration are YAML files and normal caching is used to make sure it stays speedy. I am working hard on helping this initative -- its success is absolutely paramount. I got close to get rid of the "system" table (storing the modules data) but it was decided it's not fully configuration but also needs the new key-value API. I have made the latter ready and converting the modules to it right now. I am worried a little about the brittleness of this whole system install time.
The other thing I will work on CMI-wise is the date format storage. I already cleaned up the dependencies for that by finishing the basic API for multilingual configuration. While multilingual capabilities have nothing to do with MongoDB, getting rid of the three date formats table is absolutely necessary, it's three tables joined together on bootstrap if the cache is cold, so they must be gone.
Meanwhile a lot of simpler settings forms are getting converted to the config system and as Dries posted today, this conversion can be done before April 1, not December 1, making it very likely the conversion finishes.
The biggest problem for CMI is the lack of funding for the initative leader http://heyrocker.com/funding-cmi .
A thorny problem for the Drupal community is the path alias storage. It does not fit anywhere neatly. However, this is pluggable already in the old fashion (with a notion to convert it to the new fashion) so solving this is not a priority, writing a MongoDB specific path lookup implementation is already possible.
Drupal provides flexible data storage by the means of field API. This is progressing well without my help, other companies are financing progress here on several fronts. And , it already worked in Drupal 7 mostly, so this doesn't require much of my attention just a little review to make sure the pluggability is kept. I am not worried: the ability to handle remote data with this API is considered a priority so pluggability will not regress.
Drupal has a listing API which was not part of core before but it likely will be in Drupal 8. An update just has been posted
http://drupal.org/node/1772494 here. This seems to progress nicely aswell, some companies are providing funding but this is a tremendous undertaking also needing funding http://www.angrydonuts.com/help-fund-views-in-core Whether EFQ Views will become part of core is not yet clear altough Dries' mentioned it in the updated schedule. I had recent conversations with some of the Views on retooling EntityFieldQuery to disallow queries spreading across multiple entity type and so drop the trinity (entity, property, field) of the conditions and sorts and have a single condition and a single sort. Because once we know the entity type, the entity-level conditions can be specified as properties and fields can be derived by looking at which fields are attached to said entity type. Once this unification is done, we can unify conditions used in the database layer, in EFQ and in Views into one system. This will add AND / OR to EFQ and make efq_views hopefully smaller making the integration a lot easier. We have some use cases for cross entity type queries posted to an issue in the efq_views, I will think on those and provide recipes avoiding cross entity type queries (involving some sort of references for sure) -- they already need help for Drupal 7 and for D8, even the capability for these sort of queries will be gone. Also, I already looked into this and while the actual refactor is not a lot of work, the tests simply are not salvageable, they need to be rewritten.
Lastly, as Dries clarified what is possible between December 1 and April 1, I have some hopes on converting a lot of listing pages (aggregator, forum etc) to Views and further perhaps, perhaps to efq_views. Previously I had no hope of these modules having a chance working without some serious override work but now even these look hopeful.
Commenting on this Story is closed.