Skip to content

Bittorrent used to deploy updates

July 20, 2010

I just watched a video from Larry Gadea working at Twitter: Twitter – Murder Bittorrent Deploy System (speaking at CUSEC 2010).

Briefly, the problem Twitter was facing was the deployment of updates to thousands of servers in a short amount of time and dealing with errors (broken servers, e.g.). A nice, simple, cool and free way of solving this issue was to use the Bittorrent protocol (via Python and a stack of other free software) to actually deploy updates. In summary, you go from a unique repository facing thousands requests approximately at the same time:

And you end up with a nice “distribution chain”:

The beautiful thing is that they now go 75 times faster than before!

And now, the video:

http://vimeo.com/moogaloop.swf?clip_id=11280885&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=00ADEF&fullscreen=1

The Murder software is hosted on Github (Apache 2 license).

Why do I blog this? First, I like to see simple ideas no one had before implemented like this. I also wonder how other companies facing the same problems are doing (status.net for example ; I don’t think it could be useful for Forban). Finally, you see, Bittorrent is sometimes about good stuff too!

Comments are closed.

%d bloggers like this: