Sorry for two things :
- Not being nice
- Not being a native english speaker.
Today, I was
enjoying passing throught a boring and alone day, after a incomplete night spent in a very light sleeping state. When I started my day, as always, I turned on my screen and started looking up at Drupal core issue queues, int order to see if some answers magically spawned during the night : as always, U.S. people are the most active while we europeans are deeply asleep.
Today, the main concern for me was a bunch of issues about core handling transactions and some others module related issues. Along those, an old one was stalling where I took the habit to argue louder, entitled Add a persistent key/value store for non-configuration state. I will name this issue The One.
This issue has some history, it was created one year ago. It bumped a lot, during this year, community initiatives happened in Drupal community. For those who don't know the Drupal development world, each one of those initiative has a leader, and a specific problem to solve, in order to clean up deeply the Drupal code base. This issue was originally issued by catch, someone that I like for his pragmatism and tact (something that I'm really not skilled with myself, this is not important right now, but it will be later). This issue was originally about creating a data store API for various module information, which is not really configuration, but is more site state related and so heavily mutant; This data which should probably not be stored along real site configuration, which in a ideal world would be readonly during the normal site production lifetime.
This issue had something special, it was created by someone who had a what I'd call a hunch ; I might be wrong here, but the original issue description was a bit messy, and mixed different matters altogether. A bit later, catch posted another issue, entitled Add a key/value store API, which I'll name The Two.This one was simple, build a high-level key/value oriented storage interface for allowing modules to store abitrary data that doesn't fit either in content, either in configuration. Something quite nice in theory. I hope you noticed now how close are those those two were at the time they were originally issued. I started working on The Two, almost one eight month ago, and ended up with relatively clean code, decoupled from core and working as a Drupal 7 module. The idea was definitely sexy, but the more I worked the code (this one was relatively easy considering I never finished it) the more I found out that trying to do too abstract APIs you always end up by writing code so much far away from any specific business, that nothing will fit right with it. When you try and think this kind of API, you have to anticipate every use that could be made of it, and the more I worked out of it, the more I started to see the complexity of making such API both complete and performance-wize.
Just as a small footnote, when you develop some application today, you need to choose wisely your storage backend : some are fast, some are complete, some will ensure you data consistency. Drupal has choosen to go with SQL relational databases, which was obviously the only mature path in the days ; But today, the community start to look at what's coming from the NoSQL world. Most of those things are really appealing, but there is still none that ensure full ACID compliancy (there is some theory about that, it's almost proven you can't be fast, scalable, error proof, and ACID at the same time), and worst of all, most of them don't support aggregate operations on multiple joined collections (or tables in relation databases). Even if today MongoDB is getting power, and is higly scalable thanks to the map reduce magic. Key/value stores are often a really good solution for speed and basic needs, mostly caching or storing statistics, but they are not when it comes to build an application such as CMS, where a document-oriented storage or a good old relation model becomes really handy.
Let's go back to today. Today, like yesterday and some days before, those issues came back to life, both of them. Both issues, with almost the same title, and worst, the same people acting in the threads. I started arguing in the The Two about the fact that this kind of generilization is something I'd call a leaky abstraction (we have to thanks Joel on Software excellent blog entries for that). Meanwhile, typing like a madman yelling those convictions, I started to leak myself on the The One, both opened on my screen. Those two issues in those description were so similar that I couldn't get myself out of my screen and started to say out loud how much I thought this wasn't a good idea.
This was an honest error, but a terrible, terrible judgement I made. The One's topic was totally different from the The Two, but quite wrongly entitled and described. I'm not looking for excuses here, I made a mistake. The One has derived, during this year, to a mere replacement of the variable system, but specialized for mutant data, such as various timer, checkpoint information, and stuff that doesn't belong neither to configuration, neither to data, neither to business mater. It derived slowly to the point of creating a minimalistic state database table for systems that needs to carry mutant and temporary variables being stored, stateful data, that doesn't imply site configuration to change nor extra data behavior. In order to clarify a bit more, this data store is supposed to keep information such as the last time the update manager checked for updates and features alike, no less, no more.
I got a bit far, in my judgment and posts, until someone posted the one that made me realize really what the issue was about : stateful information. It took someone to quote an wrong example from the The Two, which was a match for the The One, which itself didn't have any good example in the whole thread, for me to understand what it was all about. I yelled, said to people they were wrong, being almost unrespectful, and in the end I was the one to be wrong. My sincere apologies for this.
@sun : I'm really sorry for not understanding your point, you were right.
@catch : I'm really sorry for polluting your issues, you have better to do than read angry posts.
@all the people that contributed to the The Two : this is a bad idea, don't do this.
I have two conclusions to this :
- Correct wording is important, always, even when you issue a task over software you cherish as your child.
- When you speak, give the right examples. Don't make them fade away in false positives.
- I'm a bitch, I should be more prudent when dealing with the entire world over the internet. People in front of me, and you, are real people, and misunderstanding exists. I'm sorry for that folks, really.