Nuances in the Calculations of Daily Temperature Market Values

Why the National Weather Service Time Series Viewer, National Weather Service Official Daily Climatological Report, and Hourly METAR Values Can Differ.

Participants in the ForecastEx daily temperature markets (as well as analogous markets on other exchanges) will notice a common apparent discrepancy between data sources purporting to represent the same data. 

In particular, the temperature shown on the National Weather Service Time Series Viewer will not necessarily match the eventual high (or low) reported in the official National Weather Service Daily Climatological Report (CLI), nor the highest or lowest hourly temperature reported by WeatherUnderground, which are derived from METAR reports at airport stations. This can cause frustration because the National Weather Service time-series viewer is the most commonly used data stream for observing how temperatures are evolving during a trading day, whereas the latter two sources (National Weather Service Daily Climatological Report and Hourly METAR Reports) are used for definitive contract resolution at ForecastEx and on other exchanges. 

All three data sources correspond to the same weather stations, typically ASOS stations at airports. Therefore, discrepancies do not arise from different data sources. Rather, they tend to arise from systematic differences in how data is processed between the three sources.

The figure below illustrates how the discrepancies come about using synthetic data. 

Figure created with Python Matplotlib

The grey line in the background represents raw hygrothermometer measurements from the ASOS stations, with a 10-second sampling interval and 0.1°F precision. 

However, for the purposes of determining a daily high or low temperature, the meteorological convention is that an extreme must persist for more than 10 seconds. Thus, these 10-second samples undergo two levels of temporal averaging before they are considered eligible for determining the official high or low reported by the National Weather Service Daily Climatological Report. 

First, for each minute, the 10-second samples within that minute (as long as at least 4 are reported) are averaged to produce a single value per minute. This is the black line above. 

Second, these 1-minute values are smoothed using a 5-minute boxcar moving average. Each minute is assigned one of these 5-minute averages, computed from the preceding minutes. This is shown by the red line above. 

Additionally, the National Weather Service reports daily high and low temperature extremes in rounded whole-degree °F values. This is shown with the blue line above. 

Thus, the blue line shows the high temperature that the official National Weather Service Daily Climatological Report will likely report, with the time of occurrence corresponding to the time when the 5-minute rolling mean reached its maximum, rounded to the nearest whole minute. This is shown with the red dot.

Then there are the hourly METAR (Meteorological Aerodrome Report) reports. These report temperature to the nearest tenth of a degree Celsius, corresponding to a precision of 0.18 degrees Fahrenheit, and are typically recorded near 52 minutes past the hour. These METAR values are converted back into Fahrenheit and rounded to the nearest whole °F value. Thus, for the purposes of these markets, the METAR reports are effectively sampling from the blue line. However, their sampling frequency is low: they are typically produced only once per hour, which can result in missing any extremes that occur within the hour. 

Got that so far? Because things get a little strange when it comes to the National Weather Service Time Series viewer (purple dots).

Because Celsius is the more scientifically conventional unit and is also more commonly used in the aviation industry, the default automation systems that transmit temperature data from ASOS stations convert whole °F values (blue line) to whole °C values before transmission. However, when these whole °C values arrive at the National Weather Service systems, they are convert them back to whole °F values because that’s the unit Americans and the agency prefer.

This creates ambiguity because one °C equals 1.8 °F, so most whole °C values could represent two possible whole °F values. 

For example, 87°F converts to 30.6°C, which rounds to 31°C. 

But, 88°F converts to 31.1°C, which also rounds to 31°C! 

31°C converts back to 87.8 °F which rounds to 88°F.

This 88°F would be what the National Weather Service Time Series viewer would show. However, critically, the displayed 88°F could have originated from either 87°F or 88°F in the original data stream. 

This ambiguity, arising from passing values through a whole-degree Celsius mapping, is indicated by the purple error ranges in the figure above. 

You’ll notice that there are no error ranges on the value 86°F. That’s because some Fahrenheit values convert exactly to an integer Celsius value. When that happens, rounding in Celsius does not create ambiguity, as no neighboring Fahrenheit values yield that same Celsius result. 86°F, for example, converts to exactly 30.0°C and no other whole °F value would (85°F converts to 29°C rounded and 87°F converts to 31°C rounded). 

These special ‘no-ambiguity’ Fahrenheit values are spaced regularly, at every 9°F: 68, 77, 86, 95, and so on. So, when a market participant sees one of these special values as the day’s minimum or maximum in the time series viewer, they should note that it is much more likely than one of the ambiguous values to end up being the official high or low as determined by the National Weather Service’s daily Climatological Report. 

So, in the figure above, there is a period of about half an hour during which the 5-minute rolling mean time series (red line) rounds to 87°F (blue line). However, after these values pass through Celsius conversion (30.6°C), rounding (31°C), Fahrenheit conversion (87.8°F), and another round of rounding (88°F), the final value is displayed as 88°F. Furthermore, time-series viewers typically display values at 5-minute intervals, so there are five 88°F values shown. Also, this peak in temperatures occurs between METAR reports, so an extra degree was missed at their hourly sampling resolution.

In this example, the three data sources would indicate a different high temperature. The NWS Time Series Viewer would indicate 88°F, the NWS Daily Climatological Report would indicate 87°F, and the maximum of the hourly METAR reports would indicate 86°F.

These discrepancies arise from systematic differences in how data are processed, and skilled traders should be aware of them to maximize the chances of success in the daily weather markets. 

Summary of three data reports:

• The NWS Daily Climatological Report (CLI) high is calculated as the maximum of the 5-minute running mean, rounded to the nearest degree Fahrenheit.
• The METAR apparent high is the highest value observed in the METAR series; it effectively samples the same 5-minute running mean and is rounded to the nearest degree Fahrenheit, but only has a sampling frequency of once per hour (typically reported around 52 minutes past the hour). 
• The National Weather Service (NWS) time-series viewer’s apparent high is the highest value observed in that series, typically at 5-minute sampling intervals. It too effectively samples the same underlying 5-minute running mean but displays temperatures in whole degrees Fahrenheit after they have been converted to Celsius and back (creating ambiguity for which Fahrenheit value they originated from).