CropProphet was developed in 2008/2009 and was first used in a beta test to predict U.S. corn and soybean yields in 2009.  We consider 2010 as the first year of “operational” CropProphet forecasts but have improved the model each year since then.

We predict the final USDA yield and production numbers released early in each new year after the crop is grown and harvested.  These “final” numbers are sometimes revised slightly in future years, and we update our historical data base accordingly.  We consider the USDA as the “ground truth”, because the long historical record allows for statistical modeling.

No. See above.

At the county level, we use weather and crop data since 1981.  For the state and national level, we add additional predictors such as the USDA crop condition reports that are available starting in 1986.

No, we no longer use satellite NDVI as a direct input to our model to predict yields and production.  Satellite data is a lagging indicator of the impact weather has on crops.  We do, however, provide a visualization of NDVI during the crop season.

The inputs are:

– Historical (USDA) end-of-season yield, production, and acreage data (county, state, national).

– Daily historical and recent gridded weather analysis data, derived from both weather observations and National Weather Service weather models.  We convert the gridded values into daily area-average values for each county.

– Weekly state-level USDA crop progress and crop condition data.

– USDA state-level acreage estimates, obtained from the prospective plantings report (prior to June 30) and then from the acreage report (June 30 and later).

– For the weather outlook component of CropProphet, we use gridded ensemble weather model forecast data from the National Weather Service and the European Centre for Medium-Range Weather Forecasts (ECMWF).  The gridded values are converted into daily area-average values for each county.

During the crop season all forecasts are made available at about 8 AM Eastern Time in the United States.  The Point-in-Time data were all produced to align with this daily release schedule.  The cross validated and rolling forecast data made available in Modeler are designed to emulate this release schedule such that customers can assume a consistency in time when creating models using the Modeler data but trading based on the daily, in-season crop forecasts.

We create models and forecasts for all counties with no more than 10 years of data missing from the past 40 years.  However, the aggregation of the county forecasts to the state and national forecasts takes account of missing counties by appropriately weighting the acreage.

We use a forward-looking approach to generate a linear technology trend for each crop and each geography (county, state, national).  Specifically, we combine linear trend estimates based on the past 20, 30, and 40 years (when available) to obtain a single best estimate of the trend.  We also use a median regression approach, rather than least squares regression, so that the trend estimates are not overly affected by outlier years.

The state and national level forecasts are generated using acreage weighted aggregation of all the county level forecasts.  However, not all counties have CropProphet forecasts, and so the acreage weights are computed after excluding the acreage contribution of the missing counties to the state and national totals.  We also perform a separate aggregation on the county technology trends and compare the results to the state and national technology trends, to ensure that the aggregated forecasts show consistent departures from trend.

We use the USDA state-level acreage estimates provided in the March prospective plantings report and the June 30 acreage report.  To obtain county estimates, we downscale the state values by using the historical ratio of county to state acres in the past 5 years.

The impacts of soil type are implicitly captured in the county-level regression models, which are developed for each county separately.  Also, as part of the modeling process, we use a different technology trend for each county, so the local baseline for the CropProphet forecasts is tuned to each county’s historical yield normal.

Forecasts for winter wheat are updated daily from early April through mid-June, and the corn and soybean forecasts are updated daily from early May through early November.

During the 2020 model update process, USDA state level planting delays were integrated into CropProphet.  In this case, a combination of sophisticated used of satellite NDVI data was used to estimate planting dates over the 1986 to 2019 history were integrated into the model and used as a predictor.

The effect of acreage abandonment is included implicitly in the county models.  This is easy to explain for the yield models, which predict yield per harvested acre.  If acres are abandoned, the yield does not change except insofar as the worst acreage may be abandoned preferentially, leading to a slight upward shift in yield; but this effect will be captured in the historical yield regression modeling.

Acreage abandonment is captured in the production forecasts by using regression models that predict production per planted acre rather than production per harvested acre.  The resulting forecasts are then scaled by estimated planted acres to obtain production.  This avoids the need to make an explicit estimate of harvested acres, which can change dramatically through the season depending on abandonment.  Instead, the effects of abandonment are captured within the production per planted acre regression; for example, when poor weather favors high abandonment, the production per planted acre will decrease more than the production per harvested acre, and the production forecasts will reflect this change.

In principle, yes, and we plan to add yield and production forecasts for Brazil and Argentina in time for the southern hemisphere summer of 2020/2021.  We have excellent weather and satellite data for international yield modeling, but it is challenging to obtain long-term local yield data for building our statistical models.  We expect to bridge this gap in data resources by applying CropProphet to downscale existing yield data, with additional constraints based on satellite crop analysis.

CropProphet uses USDA prospective planting acreage as the initial acreage estimate each season. That data is not updated again until the late June acreage report.  CropProphet is not forecasting planted acreage.

It is important to recognize that CropProphet models production and yield independently, so it may not be valid to back out acreage from the two numbers.  Independent modeling is necessary because production is modeled using yield per planted acre, which implicitly accounts for field abandonment.  The yield forecasts are yield per harvested acre, the same yield definition that USDA uses.  So not planting or abandoning a field does not affect yield, but it obviously impacts production.

CropProphet does not forecast planted acreage for each crop.   The CropProphet models have been designed to be systematic and fully objective.  Inputs into the system are not manually adjusted during the season.  The primary reason for this design is that each year we generate and provide a long-term history of model forecasts and performance statistics in order to demonstrate the skill and value of the CP forecasts.  Consequently, for our production forecasts, we have always used the USDA acreage estimates, and the cross validated verification statistics show that the production model skill is exceptional.

CropProphet, since 2009, quantifies the impact of weather on grain crops. We do this by:

1. Providing grain yield forecasts for the United States, Argentina, and Brazil
2. Providing crop weather risk analytics for major global grain exporting regions and select softs crops/regions.

There are multiple products within the CropProphet family of products. They include:

1. CropProphet Enterprise – Website that provides grain crop yield forecasts, crop weather observation and forecast maps, and crop weather charts to help quantify the impact of weather on grain crops. Countries available include the United States, Brazil, Argentina, Canada, Australia, Europe, Ukraine, and Russia.
2. CropProphet Enterprise SFTP – In addition to access to CropProphet Enterprise, this service provides SFTP-based access to CSV files containing all data available in the Enterprise interface for the current year.
3. CropProphet Weather SFTP — This service provides SFTP-based access to CSV files containing all historical observed and daily updated weather forecast data for all countries, as well as historical and ongoing USDA and other countries agricultural agency crop data updates. In addition to access to CropProphet Enterprise, the service includes access to CropProphet Weather SFTP.
4. CropProphet Modeler SFTP – This service provides access to CropProphet Enterprise and SFTP-based access to CSV files containing all data mentioned above, and as much as 38 years of historical daily-updated crop yield forecast data. The crop yield forecast history enables crop model performance and return back testing for discretionary and systematic grain traders.
5. PiT Forecast API For Agriculture – This service provides access to seven years of ECMWF, GEFS, and GFS forecasts and 43 years of observed crop production weighted temperature and precipitation data for Brazil, Europe, and the United States. to support discretionary and systematic agriculture traders to understand the market impact of changes in the weather forecast and the forecasts themselves to improve trade profitability.

This answer can be found using the which can be provided upon request.

Cross-Validated: April 4^th, every year at 2134 UTC

Expanding: April 4^th, every year at 2134 UTC

There are no price inputs to our yield/production models. The only input to our models is weather information. We aim to isolate weather’s impact on crop supply via an objective, systematic, and repeatable model.

Yes, the CropProphet model is updated every year. The updates include:

1. A new year of data, and
2. New methodologies. We strive to minimize the mean absolute error of our yield forecasts. Subsequently, as we learn about weaknesses in the model, new methodologies are extensively tested.

The models that we release include:

1. Year-to-date model – yield forecasts based on weather conditions that have occurred up to the prior day.
2. Weather model forecast impacted yield forecasts – this combines the YTD observed weather and combines it with available weather forecast models to forecast yield again. The forecasts are updated with the 00Z and 12Z forecast cycles. The models include the ECMWF IFS, ECMWF Ext, GEFS, and GFS.

The ECMWF IFS, GEFS, and GFS are each 15-day forecasts.  The ECMWF Ext is a 46-day forecast.

Weather impacts grain markets daily and there are a multitude number of ways it impacts the grain market. A list and explanation of the impact of weather on grain markets can be requested.

Crop production weighting simplifies a complicated county level map to a single crop weather related index. The weighting is a county’s percentage of a specific crops’ national crop production. That weighting is multiplied by the specific weather condition at a county. This is repeated for all counties that grow the specific crop and then summed over all counties. The result is a crop production weighted weather statistic, which tells us, on average, what weather the US crop experienced. It simplifies weather-based map analysis.

CropProphet provides yield and production forecasts for the county level in the United States, the department level in Argentina, and the micro-region level for the three southernmost provinces in Brazil.

A data offering that provides a daily historical weather archive along with daily real‐time updates of weather data that can be used for custom analytics and models and is available for many of the main growing regions around the world.

Cross-Validated: Uses a 38-year dataset with a “leave one out” method for training and prediction, providing highly accurate county-level hindcast yields for weather-influenced outcomes, excluding hail events.

Expanding: A 15-year “out-of-sample” information-based forecast of yield/production. This simulates the Point-in-Time data but for the current operational forecast model. This is the most constrained version of our forecasts and facilitates a quantitative understanding of the performance of the current model.

In-Sample: A 15-year yield/production data set created using the same input data used to train the operational forecast model. This model helps to confirm the proper behavior of the cross-validated and out-of-sample Validates the behavior of the model based on prior years.

Point-in-Time: The actual forecasts released by CropProphet from 2014 to 2023. It is important to recognize the CropProphet model input data and/or modeling methodology is updated each year. Accuracy estimates of CropProphet based on our Point-in-Time data should be treated carefully.

Real-time: The real-time, daily updated yield/production forecasts are produced from May 1 to October 15 of each year. This data is used during the current crop season.

• The state and national-level corn, soybean, winter wheat, and spring wheat yield forecasts are created from a county-level weather model and a state-level crop condition-based model. The state-level crop condition reports are used to forecast the end-of-season yield and production. The 4 pm data file is updated on Mondays after the crop condition report is released. It shows that when the yield forecast model changes in the morning what the component change will be because the crop condition report changed as well. It is what is the change induced in the yield forecast due to the crop condition reports changing.
• Since the first USDA crop condition report is released on June 3^rd, the first pmupdate file us on June 3^rd. Therefore, the appearance of the pmupdate file is dependent on the release date of the first crop conditions report. Although, for winter wheat, the crop condition report starts on October 30^th.

• We do not use a single precipitation data source for all the countries we provide precipitation for. As a result, the documentation provides the worst-case scenario for data availability.
• The data source for all US counties is PRISM (https://prism.oregonstate.edu/). We receive the previous days precipitation and temperature data early in the US East coast time morning. It’s processed and made available to customers in the SFTP as soon as possible.
• Other country’s precipitation data is a combination of different satellite and satellite/observed precipitation estimates. The data from those sources have a longer delay, which is why we say, “available within 2 days of real-time.” One of the sources publishes a preliminary estimate but then updates the estimates as additional data and post-processing on the original data occurs. This is why there may be a revision as long as 7 days after the original publication of the first estimate.

The time stamps are all in UTC time.

During the crop season, all forecasts are made available at about 8 AM Eastern Time in the United States. The Point-in-Time data were all produced to align with this daily release schedule. The cross-validated and rolling forecast data made available in Modeler are designed to emulate this release schedule such that customers can assume consistency in time when creating models using the Modeler data, but trading based on the daily, in-season crop forecasts.

• For US data, the “day” is considered to be from 12 UTC the day before to 12 UTC on the current day. For “current day” data, i.e., December 3, we retrieve precip, temperature, and dew point data from NOAAs Multi-Radar/Multi-Sensor system (https://www.nssl.noaa.gov/projects/mrms/).
• This provides a first estimate of the data from 12 UTC to 12 UTC (Dec 3).   This data is then replaced when the more reliable PRISM data becomes available (https://prism.oregonstate.edu/).
• So, when you download at ~8:30 AM ET its 13:30 UTC and we’ve retrieved, processed, and written the MRMS data.

• The PiT Forecast API for Agriculture is designed to help both discretionary and systematic agriculture traders assess the impact of weather forecasts and forecast changes on the market, enabling profitable trading. The API provides historical crop production-weighted weather indices, both observed and forecasted, which assist in grain trading. By using the API, traders can measure the influence of weather forecasts, forecast adjustments, and observed weather data on price movements in weather-sensitive markets, such as corn and soybean futures.
• If interested, one can request information and schedule a meeting.