As we make our way into the new year, many analysts will be making predictions about energy markets in 2017. Chief among them is the U.S. Energy Information Administration (EIA), which just released its latest Annual Energy Outlook. The Outlook offers predictions about the future of energy prices, production and consumption in the United States. As with stock prices, interest rates and consumer spending, making predictions about the future of energy markets is fundamentally hard. In recent years, EIA has found that its own predictions of crude oil and natural gas prices differ from realized prices by 30 to 35%. Their forecast errors for renewables are sometimes even larger.
The EIA is not alone in making bad predictions. Professional oil price forecasters and futures market participants make bets about future oil and gas prices that routinely turn out to be completely wrong. But contrary to what you might think, these forecasting errors should not be viewed as evidence that the EIA or any of these forecasters are doing a bad job, or even as mistakes at all. Instead, they point to the key role that changing technology—and specifically supply-side technology—has played in the energy landscape in recent years. The history of forecasting errors in the U.S. natural gas market is a perfect example of this phenomenon.
First, a little background. In making predictions about both the price of natural gas and the quantity that will be produced, the EIA is telling us where they believe the supply and demand curves will cross. By comparing EIA’s predictions about future prices and quantities to their realized levels, and interpreting those differences through the lens of microeconomics, we can partially determine whether forecasting errors are caused by unexpected shocks to supply or to demand.
Consider EIA’s 2000 forecast of natural gas markets in 2015: 25 trillion cubic feet (TCF) of production at an average price of about $5 per thousand cubic feet (MCF). Actual production in 2015 was about 27 TCF at an average price of $3.37 per MCF—we got more gas at a lower price than expected. Simple economics tells us that this can only be explained by a positive supply shock. If the increased quantities of gas were purely demand-driven, prices would have been higher, not lower.
This logic applies more generally. When prices and quantities are both higher than forecasted, actual demand must be larger than expected. When they are both lower, actual demand must be smaller. In contrast, if price and quantity forecast errors have opposite signs, supply forecast errors must be the cause. Lower prices and higher quantities (as in EIA’s 2000 prediction of 2015 conditions) indicate that supply must be larger than the forecast. Finally, when prices are higher and quantities lower than forecasts, supply must be smaller.
We can take this logic to data, and, fortunately, the EIA has made this exercise easy to do. Since 1996, they have published retrospective analyses of their predictions. In this useful and under-appreciated effort, the EIA compiles 20 years of forecasts into a single report and compares those forecasts to what actually happened. The last of these analyses was published in May 2015, comparing forecasts as far back as 1994 to realized outcomes through 2013. For U.S. natural gas markets, I have extended this data (both forecasted and realized) through 2015 and characterized the forecasting error of each report year, prediction year combination as driven by unexpected shocks to supply or demand. I have plotted this data in the figure below, separating forecasts made before the start of the recent shale boom in 2005 and after.
The figure shows that 77% of EIA’s forecasting errors are best interpreted as forecasting errors about the supply curve for natural gas, as opposed to the demand curve. This is true for forecasts made before the shale gas boom (80%) as well as for more recent forecasts (72%). The figure also shows that EIA’s forecasts made before the start of the shale gas boom tended to overestimate supply (66% of the time), while forecasts made after tended to underestimate supply (63% of the time).
Why has supply been harder to predict than demand? Over the last 20 years, there have been two large shocks to the “technology” of natural gas production: an unexpected decrease in natural gas discoveries in the Gulf of Mexico starting in the early 2000s, followed by the unexpected boom in shale gas development more recently.
Until the mid-2000s, the EIA forecasted about 5 TCF of offshore gas production per year. In reality, offshore gas production decreased in nearly every year since 1997, and now stands at just 1.3 TCF. An important cause of this decline in production has been a 70% drop in the rate of new offshore gas field discoveries since 2000. Because discoveries had been stable at about 2 TCF per year for the 15 years leading up to 2000, it is fair to say that this absence of technological progress, and therefore negative shock to supply, was rather unexpected.
The opposite is true for onshore gas fields, where rapid and unexpected improvements in hydraulic fracturing technology helped gas production far outpace forecasts in places like Texas, Louisiana, Pennsylvania and West Virginia. Although EIA correctly anticipated as early as 2000 that shale gas resources would grow in importance, they underestimated the speed and magnitude of this change. As recently as 2005, EIA still forecasted less than 10 TCF of shale gas production per year for the then foreseeable future. In reality, U.S. shale gas production is now more than 15 TCF per year.
Unlike the unexpected changes to supply, changes to factors of natural gas demand, like regulation and consumption technology, have been relatively expected over the last two decades. Environmental regulations that encourage the substitution of coal-fired production for gas have, so far, been determined many years before those rules go into place. For example, recent changes to air quality standards were decided in 1997, but states had many years to decide how power plants would comply. On the technology side, though natural gas-fired power plants have become more efficient over time, the gains are small and somewhat predictable due to thermodynamic limits on how efficient those plants can actually be.
While analysts like EIA might not correctly predict prices or quantities, their forecasts and associated forecast errors teach us valuable lessons about why the supply side of energy markets can be so uncertain. Recently, this uncertainty has been driven by the emergence of new ways of getting more gas inexpensively. However, it is important to remember that this uncertainty can also be caused by technological “misses,” like declining exploration success in the Gulf of Mexico. Though I have argued before that our world is awash in yet-to-be-discovered fossil fuel resources, the work in EIA’s retrospective analyses reminds us that our forecasts of future supply can be too optimistic just as often as they are too pessimistic.