At the Moore School of Electrical Engineering at the University of Pennsylvania, where I am now Adjunct Faculty, they have been celebrating the 80-year anniversary of ENIAC. The Moore School is considered the birthplace of the computer industry, and ENIAC is regarded as the world’s first programmable, Turing-complete, electronic computer. The New History of Modern Computing also argues that ENIAC, more than Babbage’s difference engine, or the work of Alan Turing, had the largest influence on today’s computing landscape.
But digging a bit deeper, the way ENIAC worked would be unrecognizable to a current student of computer engineering. In contrast, ENIAC’s much less known successor, EDVAC, would actually be remarkably familiar. ENIAC and EDVAC represent two very different ideas of what “programming” is, with implications that are becoming extremely relevant in today’s AI age.
In this article, we’ll take a look at how ENIAC and EDVAC were programmed, why in some ways it is more appropriate to think of EDVAC as being the ancestor of modern computing than ENIAC, and how we might be about to do a U-turn back toward the ENIAC paradigm today.
This post is co-written with Bharath Suresh, who writes Chip Insights. If you like this kind of post, make sure to subscribe for new ones!
ENIAC’s programmers
Other than the computer itself, ENIAC also gave us the first known “programmers,” whose story was widely popularized only in the last 10 years. Quoting the Penn Today article:
Programming this flexibility required what historians have described as a “physical hack” of the hardware, and the work fell to six pioneering women: Frances Bilas Spence, Jean Jennings Bartik, Ruth Lichterman Teitelbaum, Betty Snyder Holberton, Kay McNulty Mauchly Antonelli, and Marlyn Wescoff Meltzer. As the first digital-age programmers, they translated logic into electronic signals for ENIAC to interpret.
But what exactly did the programmers’ work (this “physical hacking”) look like?
ENIAC’s logic was distributed across 40 individual modular panels that acted as distinct arithmetic, control, and memory units. To solve a particular problem, the programmers drafted and followed “configuration diagrams” to reconfigure the wires and switches when changing between problems.
Columbia University’s Computing History archive has an example of one of these configuration diagrams:
The programmers’ task was not easy, as suggested by the diagram and this passage:
One of the peculiarities that distinguished ENIAC from all later computers was the way in which instructions were set up on the machine. It was similar to the plugboards of small punched-card machines, but here we had about 40 plugboards, each several feet in size. A number of wires had to be plugged for each single instruction of a problem, thousands of them each time a problem was to begin a run; and this took several days to do and many more days to check out. When that was finally accomplished, we would run the problem as long as possible, i.e. as long as we had input data, before changing over to another problem. Typically, changeovers occurred only once every few weeks.
Because of the expertise the programmers developed, they helped scientists and engineers formulate their problems as sequences of ENIAC operations and converted the sequences to diagrams.
This process, going from wiring diagrams → problem decomposition → physical rewiring, is ENIAC’s version of “programming.”
A modern C compiler would be no help in programming ENIAC!
In this paradigm, it would have made intuitive sense to minimize wiring, because it saves wire and reduces the possibility of misconfiguration. We’ll come back to this.
EDVAC’s lasting legacy
Even in the early years of ENIAC, its creators Eckert and Mauchly were working on a draft for a new machine called EDVAC. At the same, a certain John von Neumann took an interest in and joined this project. The written result of this collaboration, the First Draft of a Report on the EDVAC (1945) is, without exaggeration, a blueprint for modern computer organization and architecture. It was publicly circulated by Herman Goldstine before it could be patented by Eckert and Mauchly, and therefore slipped into public domain, to the benefit of the field of computer engineering as we know it today. In fact, this is the reason we call it the “von Neumann architecture” (the name on the First Draft) rather than the “Eckert-Mauchly architecture.”
The First Draft articulated three key concepts that underpinned several EDVAC-like machines.
The first was the existence of a large high-speed memory which enabled this paradigm. Fortunately for the EDVAC project, Williams tubes first demonstrated working random-access memory in 1946. ENIAC had no addressable memory at all — data just resided in the accumulators or function tables and had to be precisely clocked between them.
The second and most important was the “stored-program concept,” where instructions and data are stored in the same memory. “Programming” meant loading instructions onto this memory.
The third concept is to do with instruction codes (read from instruction memory) that could be used to direct the control circuitry on how to process data.
The First Draft’s lasting legacy is that this architecture has been followed by CPUs from EDVAC all the way to now, and will be for the foreseeable future.
A modern C compiler could likely be made to generate an instruction sequence that could run on EDVAC!
A resurgence of ENIAC’s concept?
The physically demanding and highly specialized task of reconfiguring plugboards, as well as the increasing availability of memory, played a part in ushering in the stored-program era. It helped us create tools like high-level languages, compilers, and made programming accessible to a very broad audience.
However, it did have a cost.
ENIAC’s primary assignment was to speed up ballistic computations, not to be flexible. In a general purpose stored-program computer like EDVAC, the physical movement of data was abstracted away so that the same hardware could run different tasks. As a tradeoff for this programmability, executing a single instruction could take hundreds of fetches of data from registers, caches, or main memory.
ENIAC’s wiring diagrams, in contrast, bring the flow of data to the forefront. Defining a computation meant literally plugging in wires on a plugboard. Data could (and likely would) be made to take the most efficient path to reach the accumulators, multipliers and function tables used to perform the arithmetic operations
Fast forward to today, and with “Computing’s energy problem,” the inefficient movement of data is driving a lot of innovation in computer architecture. Even the picojoules required to move data to the adders and multipliers, when done trillions of times, slows down computations and can consume a lot of energy!
Taalas, the Toronto-based startup now being acquired by AMD, accelerates deep neural networks by etching model weights into the ROM of the chip and paying very close attention to each transistor and connection for the least expensive multiply-accumulate operations possible:
We actually designed all this stuff from scratch internally. We didn’t use off the shelf anything, we did lots of transistor level design, hand layout – basically our whole effort ended up being a throwback to the 1970s.
In this article, their VP of products, Paresh Kharya, specifically calls back to ENIAC while talking about Taalas’s technology.
A less exotic direction several other efforts are pointing toward is a dataflow architecture. I’ve previously mentioned the SambaNova RDU where data moves directly between functional units instead of through a central memory bottleneck:
Physically connecting the plugboards seems like a nice proxy reminder of the fact that the cost of memory movement can’t be ignored. With the most innovative breakthroughs of today, it seems like we are going back in time!
In dataflow architectures, the compiler needs to do a lot more to lay out the computation physically on to the chip. But when the task is predictable and energetic efficiency is paramount, that could easily be a worthwhile tradeoff. It just needs expert “programmers” — like ENIAC had.
Thanks for reading! If you enjoyed this post, please like (❤️) and restack — it helps others find my writing. Subscribe to receive new posts.
Further Reading
Thomas Haigh & Paul Ceruzzi, A New History of Modern Computing (MIT Press, 2021)
John von Neumann, First Draft of a Report on the EDVAC (1945)
In this post, we didn’t go into the rich story of ENIAC’s development, for which I’ll point you to some great writing on substack:
A fascinating reminder that “progress” in computing is not always a straight line. As AI makes data movement and energy efficiency increasingly important, some of ENIAC’s old ideas about physically mapping computation start to look surprisingly modern again.
Great article. I think CGRAs might be able to remove the abstraction of data movement while still keeping themselves sufficiently programmable.