First, we need to distinguish Silver’s state-by-state prediction with his “win probability”. The former was pretty unremarkable in 2016, and I think we can agree that like everyone else he incorrectly predicted WI, MI, and PA.
However, his win probability is a different algorithm. It considers alternate scenarios, eg Trump wins Pennsylvania but loses Michigan. It somehow finds the probability of each scenario, and somehow calculates a total probability of winning. This does not correspond to one specific set of states that Silver thinks Trump will win. In 2016, it came up with a 28% probability of Trump winning.
You say that’s not “getting it wrong”. In that case, what would count as “getting it wrong”? Are we just supposed to have blind faith that Silver’s probability calculation, and all its underlying assumptions, are correct? Because when the candidate with a higher win probability wins, that validates Silver’s model. And when that candidate loses, that “is not evidence of an issue with the model”. Heads I win, tails don’t count.
If I built a model with different assumptions and came up with a 72% probability of Trump winning in 2016, that differs from Silver’s result. Does that mean that I “got it wrong”? If neither of us got it wrong, what does it mean that Trump’s probability of winning is simultaneously 28% and 72%?
And if there is no way for us to tell, even in retrospect, whether 28% is wrong or 72% is wrong or both are wrong, if both are equally compatible with the reality of Trump winning, then why pay any attention to those numbers at all?
I think you’re missing the point of predictive modeling. It’s probability of separate outcomes is built in. This isn’t fortune telling, there is no crystal ball. Two predictive models can have different predictions and they both may have value. Just like separate meteorologists can have different forecasts, but predict accurately the same amount over time, all be it at different intervals. IIRC, the average meteorologist correctly predicts rain over 80% of the time. They are far over predicting by chance. But if you look at the forecast in more than one place you often get slightly different forecasts. They have different models and yet arrive at similar conclusions usually getting it mostly accurate. It’s the same with political forecasts, they are only as valuable as your understanding of predictive modeling. If you think they are intended to mirror reality flawlessly, you will be sorely disappointed. That doesn’t make the models “wrong”, it doesn’t make them “right” either. They are just models that usually predict a probable outcome.
I don’t expect a model to be perfect. But it is certainly possible for one model to be better than another, for example one might think the Weather Channel forecast is less accurate than AccuWeather (at least for your region).
Which, in turn, means that it is possible to decide when a forecast is more “right” or “wrong” than another, because what other basis would you have for judging which is better?
First, we need to distinguish Silver’s state-by-state prediction with his “win probability”. The former was pretty unremarkable in 2016, and I think we can agree that like everyone else he incorrectly predicted WI, MI, and PA.
However, his win probability is a different algorithm. It considers alternate scenarios, eg Trump wins Pennsylvania but loses Michigan. It somehow finds the probability of each scenario, and somehow calculates a total probability of winning. This does not correspond to one specific set of states that Silver thinks Trump will win. In 2016, it came up with a 28% probability of Trump winning.
You say that’s not “getting it wrong”. In that case, what would count as “getting it wrong”? Are we just supposed to have blind faith that Silver’s probability calculation, and all its underlying assumptions, are correct? Because when the candidate with a higher win probability wins, that validates Silver’s model. And when that candidate loses, that “is not evidence of an issue with the model”. Heads I win, tails don’t count.
If I built a model with different assumptions and came up with a 72% probability of Trump winning in 2016, that differs from Silver’s result. Does that mean that I “got it wrong”? If neither of us got it wrong, what does it mean that Trump’s probability of winning is simultaneously 28% and 72%?
And if there is no way for us to tell, even in retrospect, whether 28% is wrong or 72% is wrong or both are wrong, if both are equally compatible with the reality of Trump winning, then why pay any attention to those numbers at all?
I think you’re missing the point of predictive modeling. It’s probability of separate outcomes is built in. This isn’t fortune telling, there is no crystal ball. Two predictive models can have different predictions and they both may have value. Just like separate meteorologists can have different forecasts, but predict accurately the same amount over time, all be it at different intervals. IIRC, the average meteorologist correctly predicts rain over 80% of the time. They are far over predicting by chance. But if you look at the forecast in more than one place you often get slightly different forecasts. They have different models and yet arrive at similar conclusions usually getting it mostly accurate. It’s the same with political forecasts, they are only as valuable as your understanding of predictive modeling. If you think they are intended to mirror reality flawlessly, you will be sorely disappointed. That doesn’t make the models “wrong”, it doesn’t make them “right” either. They are just models that usually predict a probable outcome.
I don’t expect a model to be perfect. But it is certainly possible for one model to be better than another, for example one might think the Weather Channel forecast is less accurate than AccuWeather (at least for your region).
Which, in turn, means that it is possible to decide when a forecast is more “right” or “wrong” than another, because what other basis would you have for judging which is better?