Although the foundation for the physical design of computers was laid in the Golden Age of computer development (the years between the World Wars, during World War II, and the years shortly following that War) there's one important machine that preceded this era.
Charles Babbage and His Attempt to Build the World's First Computer
Most histories of computing begin with the English inventor, Charles Babbage. Babbage conceived and designed what we, today, think of as a modern computer, a century before the technology existed to actually build one.
The Difference Engine
In 1822 he wrote a paper describing a machine, called the Difference Engine, that could compute and print complex scientific tables. Later that year, he constructed a crank-turned prototype composed of shafts and cogwheels. This earned him a £1,500 grant from the Royal Society to build a complete full-scale working model. Babbage would spend the next 10 years (and another £15,000+) attempting to complete his machine, but due to the increasing complexity of his designs, health issue, and money troubles, he was forced to abandon the project.
The Analytical Engine
Surprisingly, considering the difficulties he had endured, shortly after halting his efforts on the Difference Engine, Babbage began, in 1833, to work on his project, the Analytical Engine. This would be far more ambitious -- a device that could not only solve one kind of problem, but could be programmed by an operator to carry-out a variety of tasks. He wrote that it was to be "a machine of the most general nature," in other words a programmable, general-purpose computer.
The machine was designed with a memory system, called a "store," and a processor, called a "mill." Like the Difference Engine, it also would be made of shafts and cogwheels. It would be configured to operate on numbers that could be moved back-and-forth between the store and the mill. This machine also would include the first "software:" punched cards that could be "programmed" to give the machine its instructions.
The Fate of the World's First Computer
Mostly because the components (or even a means to build them) didn't exist, the Analytical Engine was never completed. However, it was something of a salve to Babbage in his later years that Swedish inventor, Pehr Georg Scheutz, with considerable help from Babbage, was able to display a modified version of the Difference Engine in London in 1854. Less than two decades after Babbage's death, another of Babbage's concepts, punched cards, would be used in a working machine, a statistical tabulator built by the American inventor, Herman Hollerith.