Why Research in Distributed Systems is Hard
📜 Abstract
In this position paper I discuss some aspects of the practical difficulties that make research in distributed systems challenging. I believe (along with others, such as Dijkstra [1]) that research should not be undertaken unless its goal is simplicity. My point of view is pragmatic: I want to build systems that work in the real world. Towards this end, I pose six challenges to be met: simple systems, concrete metrics, ease of implementation, reusability of components, scalable design and effective measurement.
✨ Summary
This paper by S. Keshav, published in January 2004, addresses the practical difficulties associated with conducting research in the field of distributed systems. Keshav, aligning with the philosophy of simplicity advocated by Dijkstra, emphasizes the pragmatic goal of building systems that are functional in real-world applications. The paper identifies six key challenges to pursue, including designing simple systems, establishing concrete metrics, ensuring ease of implementation, reusability of components, scalable design, and effective measurement.
The paper has been cited by subsequent research publications that explore similar challenges in distributed systems. Notably, it highlights the importance of simplicity and practicality in system design, an idea that resonates in later works focusing on building scalable and reusable distributed systems. However, a specific search did not reveal any direct industrial applications or a large number of academic citations, which suggests the paper primarily served as a thought-provoking position paper within the academic community rather than being directly utilized in industry.
References to the ideas in this paper are primarily found in academic discussions around the complexity and implementation of distributed systems:
- Resonating ideas on simplicity in the design of distributed systems: https://dl.acm.org/doi/10.1145/3408000
- Challenges in distributed systems and related practical implementations: https://ijcai.org/proceedings/2020/567
- Discussions on scalability and reusability in distributed systems: https://www.sciencedirect.com/science/article/pii/S0167739X20300285