Friday, September 30, 2005

The 2005-2006 Philadelphia Winter Forecast Challenge

Since this is my first winter of covering weather in the Delaware Valley in the blogosphere, I felt that it would be fun to see just who is the most accurate when it comes to forecasting snowfalls.

Watching promotions for each of the TV stations in town, you hear claims that 3, 6, 10, or 29 is better than the competition for whatever reasons. Yet when you hear the ads, they don't tell you how accurate their forecasts are or how much more accurate their forecast is than their opponent.

That's where I will attempt to come in and solve this dilemma.

Since I'm an independent party to the weather winter wars I felt that it was about time to spend our winter grading these meteorologists on who's better and who's best in forecasting snowstorms.

Where did this bright idea come from? Well, with the recent war between Accuweather and the National Weather Service there have been some calls on the weather message boards for a grading system to be developed by the American Meteorological Society (AMS for short) that would judge the accuracy of forecasts on a set standard. I'm not going to go THAT far and judge every forecast, but when the important weather comes in, it's time to see whose forecast stands out.

Here's how the contest works:

Starting November 1st, each low pressure system that impacts our area with a snow potential will be graded for accuracy by me.

Since forecasts DO often change, I will use the forecast that is closest to but not within 12 hours before the event begins. It will either be the AM forecast or the late evening forecast.

I will grade Channels 3, 6, 10, 29, and the NWS forecasts for PHILADELPHIA only and snow and rain measurements will be based on those taken at Philadelphia International Airport. For point of reference for the out-of-towners, Channel 6 in Philadelphia uses Accuweather and, for the most part, follows the forecast lock, stock, and barrel straight outta State College.

Snow amounts will be rounded to the nearest inch (2.4 going to 2 and 3.6 going to 4, for example). Points are given based on inaccuracy of forecast (how much and when it occurs). The total score of each event will be added and tracked throughout the winter season through March 31st. The lowest score wins.

Since most meterologists use 'ranges' to predict snowfall (such as 3-6", etc.), this could easily become a bit complicated, but we're going to use the following point system to make it easy:

For every inch of snow over or under... # of points credited
a 2 INCH RANGE 1
a 3 INCH RANGE 2
a 4 INCH RANGE 4
a 5 INCH RANGE 6
a 6 INCH RANGE 10

If someone predicts RAIN and 2" of snow falls, the penalty would be 2 points per inch, or 4 points. The same works in opposite fashion, someone predicting 3-6" of snow and rain falls would result in a 6 point penalty.

The same above scale works for timing...(for example, predicting a snow start between 9 and 12 and it starts at 8 would equal a 2 point penalty). One note -- I will be grading event timing separately and apply it only to the TV stations because the NWS does not forecast in this manner all the time.


For example, Channel 6 forecasts rain for the city, snow for the burbs. It rains at the airport with no snow accumulation. Channel 6 speculates the rain event to start between 5-8 AM and it starts at 9. They would score a 0 on precip and would score a 2 on timing (for being one hour off) for a total of 2 points for the event.

Example #2, Channel 29 forecasts 2-4" of snow for the city and says it starts around 7-9 AM. The airport gets rain but it starts as predicted, at 9 AM. 29 scores a 2 on precip but gets a 0 on timing for a total of 2 points on the event.

The time that the event will be classified as started will be the first hourly observation where precip falls at the Airport.

FAQ:

(Q) Where is CN8?

(A) CN8 only produces newscasts Monday-Friday. If a snow event occurs on a weekend, their forecast would not be included.

(Q) What happens if the TV stations give a snowfall prediction for grass and one for road surfaces? Which one do you use?

(A) Since nearly all snow measurements are done on grass or a low heat-condusive surface, grass measurements count.

(Q) Will you grade the chief weather people (Glenn, Rob, Kathy, Cecily)?

(A) Not this year. I will divide the point totals by 'shift' so you can see which forecast fared better (the AM crew or the PM crew).

Back to the results