USDA WASDE: How accurate are the estimates?
As the end of the crop season approaches each year, the market eagerly awaits the August USDA World Agricultural Supply and Demand Estimate (WASDE) report, released by Cornell University’s Mann Library. CropProphet is a web-based solution providing users with weather-based corn yield forecasts of the end-of-season USDA national corn and soybean yield and production estimates. As such, we monitor and compare our predictions relative to the USDA WASDE corn yield forecasts during the season.
On Monday, August 12th, 2019 the USDA WASDE report increased its August monthly report updating the national estimate of the end-of-season corn yield (to be reported in January 2020) from 166.0 bpa to 169.5 bpa, an increase of 3.5 bpa.
While the USDA WASDE monthly report focuses on many other agricultural commodities, this analysis focuses on the corn yield forecast. The USDA WASDE is an important indicator of commodity markets of agricultural supply and demand. We recently published an analysis of using CropProphet to forecast the USDA NASS corn and soybean yield forecast changes, which are a component of the WASDE reports.
The USDA WASDE estimate of corn planted acreage was revised downwards by less than 2% to 90.0 million acres, versus an industry expectation of 88.0 million acres. Estimated corn production was almost unchanged at 13.90 billion bushels. The corn market was surprised by this information and Dec’19 corn futures prices immediately dropped limit down (25 cents) per contract and have yet to recover.
As of August 12, 2019, the difference between the USDA WASDE end-of-season corn yield forecast and the CropProphet end-of-season corn yield forecast is historically quite large, although it is not the largest ever. Such a historical comparison is possible because the CropProphet models have been developed using a 33-year cross-validated backtest. We cannot publicly release our current yield forecast as that information is reserved for customers, but an analysis of CropProphet backtests provides useful context for the current situation.
This analysis is summarized by the graph below showing the accuracy of CropProphet and the August USDA U.S. corn yield forecasts as a function of the difference between the two forecasts in August. You can read how we developed this analysis or you can jump right to the bottom of the post and read our conclusions.
Corn Yield Forecast History – August
It’s worthwhile to compare the historical accuracy of CropProphet corn yield forecasts relative to the August USDA WASDE corn yield forecast. To do this, we use the CropProphet Modeler data set, which provides as much as 33 years of daily corn (and soybean) yield forecast data based on a cross-validated backtest of the crop models. This data allows us to assess the performance of CropProphet in different scenarios and anticipate how it will likely perform in the future.
To start, we extract the CropProphet corn yield forecast from the same day that the USDA August WASDE corn yield estimates were released over a 33-year period from 1986 to 2018. We also extract the USDA end-of-season final corn yield estimates for each of these years, and all of this yield information is shown in the graph below.
The graph illustrates the significant deviations from the technology trend that occur in favorable and unfavorable weather years, both of which are generally anticipated by USDA and CropProphet, but with substantial differences and errors in each forecast. We’ll take a closer look at these differences.
USDA WASDE Corn Yield Forecast Errors – August
The graph below shows the yield forecast errors represented using two different metrics:
1) The left side graph shows the error of each August United States corn yield forecast from USDA and CropProphet by year.
2) The right side graph shows the same data but in terms of the absolute value of the forecast error.
It is quite evident from the right-hand graph that the red error bars (USDA forecast) are higher than the green error bars (CropProphet), which of course implies that the CropProphet forecasts are more skillful in August.
USDA WASDE vs. CropProphet
We already mentioned that on August 12, 2019, USDA U.S. corn yield forecast was significantly different from CropProphet suggesting substantially different supply estimates. The chart below shows the history of the differences between August USDA and CropProphet forecasts on the day of the August WASDE release.
The largest difference occurs in 1993 when USDA was 13.8 bpa higher than CropProphet. The graph also reveals that the August WASDE corn yield forecast and the CropProphet corn yield forecast are usually different by at least a few bushels per acre.
How did CropProphet perform in 1993? The chart below indicates CropProphet performed quite well with an error of 1.5 bpa compared to the USDA error of 15.3 bpa.
It makes sense, however, to analyze how well CropProphet has performed in all 33 Augusts, including those when the USDA forecast was significantly different from the CropProphet This was done by using the following method to analyze the above information:
1) Sort, from smallest to largest, the absolute difference between the August USDA WASDE corn yield estimate and the CropProphet corn yield forecast on the same day.
2) Compute the absolute yield error for USDA and CropProphet for each year in the historical 33-year period.
3) Plot the end of season yield forecast error as a function of the absolute difference between the August USDA and CropProphet corn yield forecasts.
The Conclusion – Corn Yield Forecast Comparison
We have arrived at a graph that reveals some valuable insights. The chart below shows the accuracy of the August USDA and CropProphet U.S. corn yield forecasts as a function of their difference.
X-axis: The absolute difference between the USDA and CropProphet forecasts on the day of the release.
Y-axis: The absolute error of both forecasts based on the USDA end-of-season final estimate released in January.
What does this result mean:
1) The August USDA forecast accuracy decreases as the USDA and CropProphet forecast differences increase.
2) CropProphet forecast accuracy remains relatively consistent even when it diverges from the USDA forecast.
3) If this result remains valid in 2019, it would suggest that the current CropProphet forecast will be considerably more accurate than the August USDA estimate.
There have been many questions in the market this year about the validity of USDA estimates and information, leading to market volatility. CropProphet provides customers an opportunity to assess yield and production using an objective corn yield forecast methodology that quantifies the impact of corn season weather conditions on yield, and – critically – is not subject to market hype.