Out of the Tar Pit
đź“ś Abstract
Complexity is the single major difficulty in the successful development of large-scale software systems. Following Brooks we distinguish accidental from essential difficulty, but disagree with his premise that most complexity remaining in contemporary systems is essential. We identify common causes of complexity and discuss general approaches which can be taken to eliminate them where they are accidental in nature. To make things more concrete we then give an outline for a potential complexity-minimizing approach based on functional programming and Codd’s relational model of data.
✨ Summary
Summary
The paper argues that complexity is the principal difficulty in developing and maintaining large-scale software systems. The authors identify mutable state, explicit control flow, and code volume as major sources of complexity because they make systems difficult to understand, test, and reason about. They distinguish essential complexity, which is inherent in the users’ problem, from accidental complexity, which arises from implementation choices, performance concerns, infrastructure, and other nonessential mechanisms.
The central recommendation is to avoid accidental complexity wherever possible and separate the complexity that must remain. The proposed architecture divides a system into essential state, essential logic, and accidental state and control. Essential state represents information that must be retained; essential logic expresses domain rules and derived information declaratively; and accidental state and control contain isolated implementation mechanisms such as caching, physical storage decisions, and evaluation strategies. The authors argue that these components should be expressed using restricted, specialized languages so that each can be reasoned about independently.
The paper then sketches Functional Relational Programming (FRP), a hypothetical architecture combining pure functional programming with Codd’s relational model. In FRP, essential state is represented as relations, essential logic is expressed through relational algebra, pure functions, and integrity constraints, and performance optimizations are specified separately as declarative hints. External interaction is handled through feeders, which update relational state, and observers, which produce outputs from changes in derived relations. The paper illustrates the approach with an estate-agency example involving properties, offers, sales, and commissions.
The paper explicitly presented FRP as hypothetical rather than as a validated production architecture. Subsequent work nevertheless includes several explicit experimental implementations or extensions. The Yurt project states that it aims to implement FRP as described in the paper, using relations, relational-algebra queries, integrity constraints, feeders, and observers. (yurt-persistence.com) Jon Eisen’s Clojure project fyra describes itself as an experimental FRP implementation and reproduces the paper’s relational-algebra style and estate-agency example. (joneisen.me) The Cell programming language also explicitly attributes the functional/relational-programming concept to the paper and adopts related principles, including relational state representation, separation of pure computation from state updates, and declarative integrity constraints. (cell-lang.net)
The available evidence indicates conceptual and experimental influence on programming-language and architecture projects, but the search did not identify evidence of widespread industrial adoption or a large empirical evaluation validating the complete FRP architecture.