Tag: infrastructure

Bittorrent used to deploy updates

I just watched a video from Larry Gadea working at Twitter: Twitter – Murder Bittorrent Deploy System (speaking at CUSEC 2010).

Briefly, the problem Twitter was facing was the deployment of updates to thousands of servers in a short amount of time and dealing with errors (broken servers, e.g.). A nice, simple, cool and free way of solving this issue was to use the Bittorrent protocol (via Python and a stack of other free software) to actually deploy updates. In summary, you go from a unique repository facing thousands requests approximately at the same time:

And you end up with a nice “distribution chain”:

The beautiful thing is that they now go 75 times faster than before!

And now, the video:


The Murder software is hosted on Github (Apache 2 license).

Why do I blog this? First, I like to see simple ideas no one had before implemented like this. I also wonder how other companies facing the same problems are doing (status.net for example ; I don’t think it could be useful for Forban). Finally, you see, Bittorrent is sometimes about good stuff too!