A conspicuous absence in storm reporting

The NYT has a nice article about the challenges of predicting hurricane intensity. A researcher pointed out it's difficult to get inside a storm to measure wind speeds and so:

it is not enough data to plug into a numerical model and yield a forecast that has a high degree of certainty.

I had the TV on most of the weekend, and there was around-the-clock coverage of Irene. I did not hear a single instance in which the forecaster (or broadcaster) provided information on the level of uncertainty of any of the predictions they were giving out. This would typically involve talking about probabilities or margins of error.

In other words, the experts admit that their forecasts are highly uncertain but the news report them as if they are 100% certain. We keep hearing comments such as "Irene will remain a Category 1 hurricane when it arrives in New York around Sunday morning and the maximum wind speed is expected to be around 70 mph."

***

The experts tell us the wind speed number is highly uncertain, and they quantify this uncertainty. Look at this Friday evening release from the National Hurricane Center, for example.

In the section labeled "wind speed probability table for selected locations", I looked up New York City. There was only a 5 percent chance of "sustained wind speeds (1 minute average)" of over 74 mph while there was a 56 percent chance that the wind speeds would be below 39 mph!

The press release is difficult to read, requiring readers to translate time, speeds, etc. to normal scales. However, if you are on the hurricane beat, there is no excuse for not spending the time to understand these data and conveying this information to citizens.

***

As discussed in Chapter 1 of Numbers Rule Your World, how much things vary around the average value is very important information not to be missed.

 

[Update, Sep 2]

Larry Cahoon, a reader of the blog and the book, emailed me about this post, and he makes several good points. In my original post, I didn't make these clear. It is in the hurricane wind speeds that the reporting failed; when it comes to the projected track of the storm, the reports typically include information on uncertainty. In addition, I'm complaining about the reporting of the science, not the science itself. Here, in Larry's words:

I read you piece on the error reporting for Hurricane for Irene and would say you have missed much of the reporting or the quality for the forecasts for the storm.

Perhaps I see more of the quality stuff as my main source for hurricane watching is the National Weather Service through http://www.hwn.org/ .
The discussion piece put out with each forecast is particularly interesting and reveals the weather services concerns with their forecasts in considerable detail. The error in forecast track is plotted at http://hwn.org/stormpulse_atlantic.html and is easy to see. This is picked up by many in the media and is sometimes referred to has the "Hurricane Cone."

The other area where the error in the forecast track shows up in the plots of the "spaghetti models" which seem to have been pickup up by a number of media outlets and even the major networks. A Google search of the phrase will give you many hit and some go into greater detail and discuss the impact of some of the forecast tracks.

There were areas where the media here in MD failed - mainly on wind forecasts that seemed to assume that the winds were of a uniform speed at a given distance from the storm regardless of which direction from the center of the storm someone was located.  That was clearly contrary to what the weather service tell everyone.

What I have failed to see are good discussions of the accuracy of the wind speed forecasts.  Although I did see a piece in the last two days discussing the National Hurricane Center's concerns with the quality for forecasts they are able to give for wind speeds.

 

Comments

[Gary]

Have you read the forecaster discussions from the National Hurricane Center (these are released along with the public advisories)? Two things jump out: (1) as alluded to in the NYT article, there is much uncertainty around measuring intensity, let alone forecasting it - I get a sense that current intensity estimates are built from both intuition and models on top of quite sparse data. (2) The NHC clearly calibrates its intensity forecasts (and estimates for that matter) to a broader goal of communicating the "danger" of the storm.

In this case as I read the discussions, it was clear that Irene was never getting as strong (from a wind-speed perspective) as they expected even prior to landfall, and often was likely not to be even as strong as they were publicly saying. The predictions and advisories followed suit.

I'm not accusing them of hyping the storm - just that they know that the windspeed forecast is what will get the attention, so when they see an enormous storm and dangerous like Irene, they use the intensity forecast to communicate a broader sense of the potential danger of the storm. Archive is at: http://www.nhc.noaa.gov/archive/2011/refresh/IRENE+shtml/120913.shtml - the Friday and Saturday archived discussions are illuminating.

NHC has done a lot in the past couple of years on the track forecasts to emphasize the track cone rather than the line (going so far as taking the trackline out of the default maps). They've also added an advisory that the storm is bigger than the forecast cone. But these advisories expand the apparent risk of the storm. I'll be interested to see what they come up with on the intensity side that doesn't reduce the apparent risk of the storm, which is not what they're in the business to do. Smaller intensity/windspeed estimates with error bands around them may not serve the purpose.

[Gary]

Also, given the data below, does the table report that there was a forecasted 56% probability of winds not exceeding 34kts (39mph) during the time period in question (Sunday presumably), or does it report that there is a 44% chance of between 34kt and 50kt, with a 27% chance of 50-64kt and 5% of >=64kt? That would leave only a 24% forecasted chance of <34kt winds.

I would think for a hurricane the cumulative probability (from now to the period) is more useful as well, so that's a 68% + 29% = 97% forecasted chance of >34kt winds.

NEW YORK CITY 34 1( 1) 23(24) 44(68) 1(69) X(69)
NEW YORK CITY 50 X( X) 2( 2) 27(29) X(29) X(29)
NEW YORK CITY 64 X( X) X( X) 5( 5) X( 5) X( 5)

[Kaiser]

Gary: Above the table, it said:

CHANCES OF SUSTAINED (1-MINUTE AVERAGE) WIND SPEEDS OF AT LEAST
...34 KT (39 MPH... 63 KPH)...
...50 KT (58 MPH... 93 KPH)...
...64 KT (74 MPH...119 KPH)...
FOR LOCATIONS AND TIME PERIODS DURING THE NEXT 5 DAYS

so I interpret these as cumulative (right-tail) probabilities.

[Gary]

Down on the bottom there are a number of cases where the probability for 50kt is higher than the probability for 34kt, which doesn't make sense if it's cumulative. And none of the probabilities that I could find added to more than 100%, which you'd expect at least one of to do, if they were cumulative. So the data doesn't match the description :)

NORFOLK NAS 34 19 59(78) 13(91)
NORFOLK NAS 50 X 26(26) 28(54)
NORFOLK NAS 64 X 4( 4) 8(12)

NORFOLK VA 34 23 59(82) 11(93)
NORFOLK VA 50 X 32(32) 27(59)
NORFOLK VA 64 X 5( 5) 10(15)

CAPE HATTERAS 34 76 22(98) 1(99)
CAPE HATTERAS 50 8 76(84) 3(87)
CAPE HATTERAS 64 X 46(46) 2(48)

[Gary]

I'm not even sure these numbers are internally consistent - look at the first couple days at Morehead City or Cape Hatteras...

[Nell]

I've always thought that average climate forecasts for a particular time and location should include 95% confidence intervals. If I'm traveling, your average daily temp of 75 tells me little about what clothes I should bring. Maybe, or even likely, it was 90 degrees one year and 50 degrees another.