David Greenberg on Sparrow: Distributed, Low Latency Scheduling

Meetup: http://www.meetup.com/papers-we-love/events/174731732/
Audio: http://www.mixcloud.com/paperswelove/dgreenberg_4_28_2014_sparrow_distributed_low_latency_scheduling/

When you need to execute code on a cluster of machines, deciding which machine should run that code becomes a complex problem, known as scheduling. We're all familiar with routing problems, such as the recent RapGenius incident. It turns out that simple improvements to randomized routing can dramatically improve the performance! Sparrow is a distributed scheduling algorithm for low latency, high throughput workloads. We'll review the Sparrow algorithm, and learn the tricks that they used. Then, we'll discuss other applications of Sparrow, besides the big-data map-reduce application it was created for.