Scaling Twitter

Interesting post by al3x at Twitter on how the hit scaling problems and had to re-thing their architecture.

From personal observation, there seems to be (at least) two related issues that affect how a startup progresses. The first is interest and the second is scale.

As was pointed out by Evan Williams, he believes he has reached his threashold on LinkedIn and Linked-In is a pretty content rich environment and so probably takes longer than most applications to reach that peak. Most startups likely go through a explosive phase where there is a lot of interest (everyone Twittering) and the question is that once that peaks off, is it maintained. On Twitter it seems to be (EBay was another i have read a lot about), but i've heard of many companies who find it hard to maintain that peak. Yahoo seems to have that problem right now.

The other, more personally familiar peak, is the application architecture. The reality, especially (but certainly not exclusively) in Web 2.0 is the time it takes to produce. It's typically very short and it's unlikely your initial "beta" (==live, but with some clauses) is designed to scale. Writing scalable architectures is pretty tricky and typically a lot more queuing and asychronous services are brought into play - and that stuff throws up a host of questions (timing, caching, performace, intelligent processing, background work and so on). In short you need to re-architecture your entire application... it's not changing a few methods or adding some hardware.

Twitter took off so very quickly, but al3x talks about the fact they had to re-architect. The real trick with this IMHO is that it's a game with the users. You need to look as though nothing is really changing, not piss off you majority user base (and the twitter IM stuff certainly tried hard in that respect), yet evolve into a platform that can grow. THEN you have to hope that your users never lost momentum and continue to use the system - and not only that - you want them to really TEST your new architecture. You built it for exactly this reason.

After all, anyone who has architected software systems and also likes to get into the code knows how brilliantly cool small things you do can have a huge impact on the application. You know, *the* code or architectural change you make which almost single-handedly takes the application to another level.

There must be some stories of companies who peak (either usage or performance) and never quite regained that momentum (this is where coming second often pays dividends!).

- http://steven.openid.org/