Did big projects used to get finished on time and on budget?

Question

Big projects (software, engineering, architecture etc.) are plagued by cost and time overruns, or so it is widely thought.

Nicholas Taleb argues in Antifragile that this is a modern phenomenon:

But the puzzle was that such underestimation did not seem to exist in the past century or so, though we were dealing with the very same humans, endowed with the same biases. Many large-scale projects a century and a half ago were completed on time; many of the tall buildings and monuments we see today are not just more elegant than modernistic structures but were completed within, and often ahead of, schedule. These include not just the Empire State Building (still standing in New York), but the London Crystal Palace, erected for the Great Exhibition of 1851, the hallmark of the Victorian reign, based on the inventive ideas of a gardener.

Is this just because we remember the successful ones are is his generalization right? Did we used to do a better job of delivering big projects on time and on budget?

Answer 1

National monument of Scotland

This is not a new phenomenon and there is standing to this day a quarter built monument on Calton hill in Edinburgh that is testament to this fact. It is the National monument of Scotland and the project began in 1826 but by 1830 it had ran out of funds, and so the National monument of Scotland became better known as Edinburgh's disgrace.

Britannica encyclopedia, Calton hill

Behind this rise 12 columns of an intended replica of the Parthenon that was designed by Playfair in 1822 as a memorial to the Scots who died in the Napoleonic Wars. Construction of the memorial was abandoned when funds fell short in 1830.

National monument of Scotland, by User:Colin / Wikimedia Commons

Cologne Cathedral

Another example is Cologne Cathedral which began being erected in 1248 but the project was abandoned in 1473 leaving the incomplete building standing for approximately 400 years before work finally recommenced.

Britannica encyclopedia, Cologne Cathedral

An older cathedral was destroyed by fire in 1248, and immediately thereafter work began on the present cathedral, which was designed in the Gothic style in emulation of French church architecture. The choir was consecrated in 1322, but construction continued until 1560 (or only until 1520, according to some authorities). The project then stalled for centuries, with a large wooden crane left standing some 184 feet (56 metres) above the ground, at the top of the south tower. During the 1790s, troops of the French Revolution occupied Cologne and used the cathedral as a stable and a hay barn. Restoration work began in the 1820s, spurred on by Sulpiz Boisserée, a German proponent of the Gothic Revival movement. In 1842 a new cornerstone was laid by King Frederick William IV of Prussia, and work to complete the cathedral resumed in earnest. The architects Ernst Friedrich Zwirner and Richard Voigtel carried out the enterprise, guided by architectural drawings made in about 1300. Construction finally ended in 1880.

Cologne Cathedral, by Thomas Wolf, Wikipedia

Connecticut Route 11

To this day a half built highway exists in Connecticut in which began construction in 1966 but was abandoned only half complete in 1971.

Kurumi

Nicknamed "Route 5½," Route 11 is a half-completed freeway following the busy Route 85 corridor toward New London. The missing link, continuing to I-95 in Waterford, would be about 8.5 miles long.

Funding and environmental difficulties caused the state to give up on Route 11 in the early 1990s,

Route 11, by Polaron at English Wikipedia

Cincinnati Subway

Cincinnati has a series of underground subway tunnels which were never completed and the project was abandoned in 1928 due to escalating costs.

Cincinnati Subway, Wikipedia

The Cincinnati Subway is a set of incomplete, derelict tunnels and stations for a rapid transit system beneath the streets of Cincinnati, Ohio. Although it is only a little over 2 miles in length, it is the largest abandoned subway tunnel system in the United States. Construction began in the early 1900s as an upgrade to the Cincinnati streetcar system, but was abandoned due to escalating costs, the collapse of funding amidst political bickering, and the Great Depression during the 1920s and 1930s.

In 1928, the construction of the subway system in Cincinnati was indefinitely canceled. There are no plans to revive the project.

Cincinnati Subway, by Jonathan Warren, Wikipedia

Deutsches Stadion

In Germany they have the remains of a sports stadium which began construction in 1937, but was left abandoned incomplete due to world war 2.

Deutsches Stadion, Wikipedia

The Deutsches Stadion ("German Stadium") was a monumental stadium designed by Albert Speer for the Nazi party rally grounds in Nuremberg, southern Germany. Its construction began in September 1937, and was slated for completion in 1943. Like most other Nazi monumental structures, however, its construction was interrupted by the outbreak of World War II and was never finished.

Deutches Stadion, by Mikmaq, Wikipedia

Marble Hill Nuclear Power Plant

Indiana has an incomplete power plant which began construction in 1977 but was left abandoned in 1984 after $2.5 billion had already been spent on the incomplete complex.

History, Wikipedia

Construction at Marble Hill began in 1977 and ended in 1984, when the Public Service Company of Indiana (PSI), now Duke Energy, abandoned the half-finished nuclear power plant. With $2.5 billion spent and, as the most expensive nuclear construction project ever abandoned

Marble Hill Nuclear Power Plant, by Bryan Napier at English Wikipedia

The Great Tower of London

The Great Tower of London was supposed to be taller than the Eiffel Tower when it began construction, but the Tower ended up becoming abandoned and demolished in 1907 before it ever reached a height anywhere near the Eiffel Tower.

The Great Tower of London is now referred to as Watkins Tower.

Watkins Tower, Wikipedia

Watkin's Tower was a partially completed iron lattice tower in Wembley Park, London, England (then in Middlesex). Its construction was an ambitious project to create a 358-metre (1,175 ft)-high visitor attraction in Wembley Park to the north of the city, led by the railway entrepreneur Sir Edward Watkin. Marketed as the "Great Tower of London", it was designed to surpass the height of the Eiffel Tower in Paris, and it was part of Wembley Park's emergence as a recreational place. The tower was never completed and it was demolished in 1907. The site of the tower is now occupied by the English national football ground, Wembley Stadium.

Watkins Tower, by Milkomède, Wikipedia

Did big projects used to get finished on time and on budget?

The list above was just a small list from the overwhelmingly vast amount of historical examples of construction projects that remained incomplete after construction had began. Whilst it is debatable that there may be more arguments over finances and time lapses today, some of the arguments arguably appear trivial in contrast to some of the historic construction travesties that we have. And whilst Nicholas Taleb may show a great amount of knowledge regarding current affairs it could certainly be argued that he is not doing history justice.

Answer 2

In fact almost all projects will fail to meet some of the criteria they originally started with. Sometimes they don't make it on time or others fall short on budget. That is the idea most engineers have, and many times they do plan knowing that at some point they won't be able to deliver what was originally intended. Now the reason that they do fail is due to unrealistic expectations from the executive management, or in general people who make the calls without actually having any expertise on the subject, unrealistic planing (sometimes due to pressure), human errors (accidents etc), and a a rather huge reason according to myself is the lack of proper communication (the requirements are not stated clearly so we end up with many time consuming errors). In fact only a third of all projects were successfully completed on time and on budget over the past year according to CHAOS. Please read this for a better analysis.

Also have a look at some notable project failures here.

Continuing on you question on if things are getting worst or not we can safely assume that in fact project success rates are rising rather than falling, according to surveys at least (Have a look here). Personally i do believe this is the case if we exclude IT project. All the factors stated before do apply here as well, and there are just so many project with unrealistic expectations that are deemed to fail in some way (Have a look at this article too).

Summarizing i have to say that project failure rates were dropping over the period from 1994 until 2011, with a high rise last year. Have a look at the tables below for more info.

Standish Findings By Year. Updated for 2009 report.

(Project outcome)   1994    1996    1998    2000    2002    2004    2009

Succeeded           16%     27%     26%     28%     34%     29%     32%

Challenged          53%     33%     46%     49%     51%     53%     44%

Failed              31%     40%     28%     23%     15%     18%     24% 

IT Project Success Factors

1. User Involvement    (15.9%)

2. Executive Management Support    (13.9%)

3. Clear Statement of Requirements    (13.0%)

4. Proper Planning    (9.6%)

5. Realistic Expectations    (8.2%)

6. Smaller Project Milestones    (7.7%)

7. Competent Staff    (7.2%)

8. Ownership    (5.3%)

9. Clear Vision & Objectives    (2.9%)

10. Hard-Working, Focused Staff    (2.4%)

Project Challenged Factors

1. Lack of User Input    (12.8%)

2. Incomplete Requirements & Specifications    (12.3%)

3. Changing Requirements & Specifications    (11.8%)

4. Lack of Executive Support    (7.5%)

5. Technology Incompetence    (7.0%)

6. Lack of Resources    (6.4%)

7. Unrealistic Expectations    (5.9%)

8. Unclear Objectives    (5.3%)

9. Unrealistic Time Frames    (4.3%)

10. New Technology    (3.7%)

Project Impaired Factors

1. Incomplete Requirements    (13.1%)

2. Lack of User Involvement    (12.4%)

3. Lack of Resources    (10.6%)

4. Unrealistic Expectations    (9.9%)

5. Lack of Executive Support    (9.3%)

6. Changing Requirements & Specifications    (8.7%)

7. Lack of Planning    (8.1%)

8. Didn't Need It Any Longer    (7.5%)

9. Lack of IT Management    (6.2%)

10. Technology Illiteracy    (4.3%)

Source