Communication and Code Dependency Effects on Software Code Quality: An Empirical Analysis of Herbsleb Hypothesis
📜 Abstract
Prior literature has suggested that in many projects 80% or more of the contributions are made by a small called group of around 20% of the development team. Most prior studies deprecate a reliance on such a small inner group of “heroes”, arguing that it causes bottlenecks in development and communication. Despite this, such projects are very common in open source projects. So what exactly is the impact of “heroes” in code quality? Herbsleb argues that if code is strongly connected yet their developers are not, then that code will be buggy. To test the Hersleb hypothesis, we develop and apply two metrics of (a) “social-ness’ and (b) “hero-ness” that measure (a) how much one developer comments on the issues of another; and (b) how much one developer changes another developer’s code (and “heroes” are those that change the most code, all around the system). In a result endorsing the Hersleb hypothesis, in over 1000 open source projects, we find that “social-ness” is a statistically stronger indicate for code quality (number of bugs) than “hero-ness”. Hence we say that debates over the merits of “hero-ness” is subtly misguided. Our results suggest that the real benefits of these so-called “heroes” is not so much the code they generate but the pattern of communication required when the interaction between a large community of programmers passes through a small group of centralized developers. To say that another way, to build better code, build better communication flows between core developers and the rest. In order to allow other researchers to confirm/improve/refute our results, all our scripts and data are available, on-line at our online Github repository.
✨ Summary
Summary
The paper empirically examines the relationship between developer communication, code interaction, and defect introduction in more than 1,000 open-source GitHub projects. It constructs code-interaction graphs from commit history and blame information, social-interaction graphs from issue discussions, and identifies likely bug-introducing commits using keyword-based commit-message analysis combined with the SZZ approach. Developers are classified as “heroes” or “non-heroes” using percentile thresholds for code and social interaction.
The study reports that centralized “hero” structures are common. Developers with high levels of code or social interaction introduce fewer buggy commits than less-connected developers. Across nine combinations of code and social activity, the lowest defect levels are associated especially with high social interaction, including cases where code interaction is only low or moderate. The authors therefore conclude that the results support the Herbsleb hypothesis: when interacting code is produced or maintained by developers who do not communicate sufficiently, defect introduction is more likely.
The paper’s limitations include GitHub-focused sampling, possible incomplete observation of communication outside issue trackers, reliance on heuristic commit-message classification for bug fixes, possible role misclassification, and limited generalizability to proprietary development. The results are observational and should not be interpreted as establishing that centralization itself causes better quality.
Evidence of subsequent influence
Search results show that the work has been cited as prior research in later software-engineering studies, including Different, Really! A Comparison of Highly-Configurable Systems and Single Systems, and in other academic papers discussing GitHub development, contributor organization, and software-project structure. (stairs.ics.uci.edu) It has also been discussed in a practitioner-oriented review that summarizes its findings about “hero projects” and developer communication. (neverworkintheory.org) A 2025 OpenTofu contribution-process RFC cites the paper when discussing centralized, high-trust roles and communication in open-source communities, providing a concrete example of its concepts being used in industry or project-governance discussion. (fossies.org)
The available evidence supports citation and conceptual reuse, but does not establish broad or measurable adoption of the paper’s specific metrics or recommendations in software-industry practice.