Almost 3 years of Drupal 7 development are behind us and we are finally about to release the beast. It is the first Drupal core release cycle that I actively participated in and I have to say that I learned a lot, from writing proper and core-worthy code, over phrasing usable user interface texts, to internals of almost all subsystems and modules in Drupal core.

Now that we're officially "closing" Drupal 7, it's time to slowly start to think about the future. Of course, while all Drupal core contributors have been busy with getting Drupal 7 out of the door in the past year, it's not like we wouldn't have identified countless of follow-up tasks and challenges for Drupal 8 already.

In fact, the amount of ideas and change proposals are truly amazing - a good sign that the Drupal community is very healthy and thinking beyond boundaries and horizons. Suggestions are ranging from technical things, like entirely revamping existing core subsystems, to changing crucial parts affecting each and everyone out there, like completely switching Drupal's output to HTML5.

As always, it's close to impossible to track all of these little ideas, especially if you happen to be a code gardener. Seeing myself in that (totally unofficial) position, I've mainly learned to distinguish between "good" and "bad" ideas, whereas I'm not only evaluating direct and indirect consequences of a certain change, but also trying to predict the overall impact of it on the entire Drupal eco-system. I've tried hard to make sure that webchick and Dries only have to deal with patches and changes that passed my initial "sanity check" - which very often required me to play the bad cop - intentionally offloading the "f*** you" and similar responses/feelings on my account for the good sake of freeing up and using Drupal core maintainer resources more effectively.

That said, I still vote for webchick to become a permanent Drupal core maintainer. Angie has done an overwhelming and awesome job in every possible way I can think of. Without her, Drupal 7 wouldn't be nearly as awesome as it is. And while I could certainly imagine that her talent could be leveraged in other fields, too, she's going to have the greatest impact on 42 as permanent Drupal core maintainer. Would you agree?

Now, Drupal 7 will ship with plenty new built-in core features that are replacing lots of contributed modules, and in addition to that, Drupal core's APIs have been unified and streamlined in a way that led to a new running joke in the past months:

If your D7 module is larger than 40 lines, you did something wrong.

Thanks to the #D7CX pledge movement, many contributed modules have been brought up to speed, which in turn allowed us to identify a lot of deeply hidden bugs and limitations in Drupal 7 that we were able to fix prior to release.

While the above quote might sound funny, it's actually true for most modules, if you take away the extra bells and whistles. The main functionality of most Drupal 7 modules is... already provided by Drupal core. Which means two essential things: 1) We've successfully established and provided rock solid APIs that can be leveraged by modules, and 2) Many contributed modules will (or should be) so small that they could hi-jack the very definition of Features in the long run.

As awesome as this may sound, there are some real, high-level challenges in front of us:

First of all, there are some contributed projects that really need our attention, such as Entity, Views, Translation, and Relation. In fact, we are facing an entirely new era of Drupal contributions that didn't really exist before — right now, everyone and the world already knows that these projects are going to play a key role for Drupal 7 and beyond. The difference is that the other 99.9% of Drupal contributions are considered "arbitrary", possibly fulfilling other use-cases; but those particular projects will move into Drupal core in some way at some point, no matter what.

Second, the infrastructure team is very close to replacing CVS with git. Likewise, this is going to impose an entire new reality upon us; one that many Drupal contributors are still unfamiliar with. Long story short: Forking entire projects and merging upstream/downstream changes into any branch/fork is going to be easier than ever before. In fact, every (working) copy of anything is an exact copy of an project, including its full history of changes. Technically, full-blown Drupal distributions and custom tailored (hacked?) Drupal code-bases are just a couple of clicks away.

With both topics we are entering unknown territory, and how we are going to handle them depends on the good will of the Drupal community. Good will, because many of the current and upcoming decisions will rely on the community's ability to accept and adopt a culture of compromises. Overall, that's just a natural process, which always happens when any kind of system standardizes.

All of that being said, and after working intensively on Drupal core and getting back to work on contributed modules, I finally understand some other core developers, who keep on telling that they try to stay out of contributed projects and rather work on Drupal core instead. Once you're into Drupal core development, it sounds, smells and feels "senseless" to work on contrib instead of core. Of what I can say, that's mainly because everyone else needs to separately download and install your fine but arbitrary module that extends Drupal core with some brilliant cutting-edge functionality. In other words:

Why should I contribute to a contributed module, if I already know that it's going to end up in core? So why don't I work on directly moving its functionality into Drupal core instead?

Seriously. The only thing that literally breaks this logic is the fact that you cannot use and actively build sites off Drupal HEAD. Thus, if I'd invest the first quarter of 2011 to move Entity module into Drupal (8) core, then I'd still have to sit and wait until 2013 in order to actually use it. Sounds like a silly idea and waste of time and resources.

Don't get me wrong — I'm totally not arguing for shorter or super-small release cycles; quite the contrary. Drupal core has reached a level of quality, robustness, and stability, at which point it no longer makes sense to schedule 3 months of heavy traction (so called "code thaw") and tell everyone to just be happy with anything that comes out of that. Site owners, builders, developers, and users need good and strong reasons for upgrading to a new version of Drupal core (and to potentially argue for an upgrade with someone else). People need and want a clear and concise set of new features; something they can sell to others. And developing new features, getting them into core, and also converting all of Drupal core to use them simply takes time.

At the same time, many people are arguing that the Drupal 7 release cycle of almost 3 years has been too long. Even though they do agree that it has been perfect to allow contributed projects to catch up with Drupal core (after the well-known drama of Drupal 6). But how do we prevent this core/contrib disconnect from happening again?

Even before the first alpha of Drupal 7 was released on January 15, 2010 (i.e., more than one year ago), various core developers started the HEAD2HEAD project in order to be able to actively use Drupal 7 for production sites already. Since that time, a few large-scale and high-profile sites as well as actually thousands of simpler Drupal 7 sites went live. Starting with the first beta on October 7, 2010, Drupal core itself re-started to provide a fully-fledged update path for existing user data in Drupal 7 sites. And even though some of the post-beta updates were quite substantial, everyone and all data survived. Additionally, most of Drupal core's functionality is covered by automated unit tests.

Various people confirmed to me that, some time ago, not only was running on latest Drupal HEAD, but also, Drupal core always contained fully-fledged updates for user data.

So if you'd ask me today what the most important task for Drupal core is, then I'd reply with this slogan:

Production sites have to be able to run on Drupal HEAD again.

Whether that is achieved by issuing occasional "edge" releases for people living on the cutting edge or some other way, doesn't really matter, but to me it looks and feels like the key for mastering the challenges ahead of us.

To continue the above example, we can start to move Entity module into core, and right away start to use it, hammering out its API with the first Drupal-8.x-EDGE1 release. More of core gets converted to use it and its API might be revised in subsequent edge releases, whereas those would also contain other major changes, of course. Only at some later point in time, the final Drupal 8 release would contain plenty of new functionality that has been thoroughly tested and already worked with, ready for prime-time and general mass-consumption.

What do you think?


Very good post, thanks for insights.

Your conclusion (slogan) seems like steroids for the D8 life-cycle! I hope this gains real traction, it would speed the 8 release, certainly boost core contribution, testing and generally sounds like alot of fun :)

i don't actually think there will be such edge-love right now, with drupal 7 just coming out the oven, some modules and themes still needing the port and proper testing. but i guess 6 months from now we will al be looking forward drupal 8, and if there is a d8-edge release that we can run, even if it is like playing Jenga on a cocaine addled elephant riding a skateboard being jabbed in the ass with a hot poker.

hope there are EDGE releases soon ;)

Given that a significant number of Drupal 7's core contributors in effect did what Daniel recommends, it's not possible to argue against his point.

Still, sites running on HEAD is quite a goal! At the least, we can adopt a watered-down approach: supporting data migrations for alpha, and an earlier alpha cycle, is more achievable? Or is the point to ensure that every database change is going to require an upgrade path anyway? (And maybe with Git forks fewer 'never-mind' changes will be committed?)

Another idea that's been proposed and i support, with the same goal of reducing the contributing to core / using the fruits gap, is to allow non-API changing fixes and API additions to be backported to the current stable version of Drupal. The two approaches are not mutually exclusive, but do you consider this idea to be backward?

benjamin, Agaric

Great post and lots of interesting information you share with us Daniel.

However, on thing I have to comment on:

Why should I contribute to a contributed module, if I already know that it's going to end up in core? So why don't I work on directly moving its functionality into Drupal core instead?

I think this mindset is not good at all and those who have it need to realise exactly how important, and vital, contributed modules are for the evolution of Drupal.

I was starting to write a longer comment, but it became so long so instead I ended up with Drupal Core Needs Contrib to Evolve where I try to describe from my user perspective why I believe contrib is so important.

@Thomas Svenson: [quoting from your post]

It is therefore I got surprised reading the above quote, for me it almost sounds like "once in core you have reached elite status and can't go back to the lower echelons of contribs". Oki, that sounds a bit harsh, but that was my initial impression when reading it.

wow, thanks for bringing this up. Your interpretation isn't my intention.

I wholeheartedly value contributed modules, themes, and installation profiles. They are what really drives Drupal. Drupal would not exist without them.

Here, however, I'm talking about de-facto "sub-core" stuff like Entity, Views, Edge, Translation, Relation, Title, and possibly others. While the future of Views is not entirely clear to me regarding core, the other aforementioned modules only exist, because time was over and no more core API changes were allowed.

Those modules will end up in core at some point, because they merely complete the already built-in but incomplete core functionality, which should be in core already, but wasn't allowed to get in. When they will be in core just depends on how early and aggressively we are going to work on moving them into core.

Thus, the question boils down to: At which point should and can we stop contributing to a contrib project, and focus our time, energy, and efforts on moving stuff into core instead?

And the next best question is: What exactly prevents us from improving Drupal core all over the time?

The only thing that prevents me from improving Drupal core every day is that sites like this cannot run on Drupal HEAD. It takes an entire armada of Drupal developers to figure out the required updates for existing user data. And that's what needs to be changed.

PS: Also, there's no such thing like an elite status of core developers, and that's exactly what makes Drupal awesome. :)

I understand, which I pointed out in my post, that certain features always will be needed to be developed directly for core, such as some of those you mention.

However, I do believe the core rules about API and feature changes needs to be kept. Developers of contributed modules needs the security of knowing that this will not change, unless extreme reason exists, between major releases. Then they can focus on their unique features instead of playing catch up with core changes.

Yes, they will will have to find ways around shortcomings in core. But in the long perspective I see that as a good thing. Then all these shortcomings and other needed core improvement can be looked at as a whole. Logically this should result in a much better implementation in the next major release.

Plus, as I also pointed out in my post, most off this has also then been tested in live conditions by a large part of the sites using Drupal. Thus the feature specification is almost writing itself.

In a way there already exists many contrib modules that in reality can be seen as being part of core. These modules are those that provide API's for other modules to leverage from. Dave Reid's new Redirect is a great example of such module, and I would be very surprised if it is not included in Drupal 8.

The difference though is that by being in contrib they don't have the restrictions core has. They can, and will change, as they are being improved upon and everyone knows that.

This is why I believe it would be good to change the mindset about contrib and actively advocate for the benefits, as well as the advantages for ensuring new API's and features are as complete and ready as possible for future major releases.

Oh, also want to say that I never thought you intentionally meant what I initial interpreted your it as.