Science Policy

(Mathematically) Predicting the Future with Professional Forecaster Philipp Schoenegger

03.03.23 | 6分钟阅读 | Text byJonathan Wilson

目前,与美国科学家联合会一起工作,预测平台Metaculus主持了有史以来的第一个气候临界点锦标赛. The aim is to show just how powerful a tool forecasting can be for policymakers trying to effect change.

使美军陆军Metaculus预测的一部分nity exciting is that anyone with even a passing interest in data science can jump in and begin making predictions; the wisdom of the growing crowd of forecasters helps make the platform more and more accurate over time.

Metaculus has recruited about two dozen of the very best forecasters in its community to become Pro Forecasters. To become a Pro for Metaculus, a forecaster has to have scores in the top 2% of all Metaculus users. The元路径评分系统很复杂, but for the mathematically-disinclined, you earn points on accuracy, how your prediction compares to those of the rest of the Metaculus community, and how early you make your predictions (in relation to the resolution of the question). Pro Forecasters also have experience forecasting for at least a year, and have a history of providing commentary explaining their methodology.

Philipp Schoenegger became one of the first members of the pro-forecasting team last year – and says he’s been forecasting for about a year and a half in total.

他说:“目前,我在Metaculus平台上有大约6,000个预测。”“我认为,在时间方面,预测问题的原因可能会在专业预测和业余爱好者预测之间差异很大。”

肖恩格(Schoenegger)最近在圣安德鲁斯大学(University of St. Andrews)辩护了他在行为经济学和实验哲学上的博士学位论文,他说,在过去的几年中,他发现自己花了自己所谓的“过度”时间来预测 - 有时会花在损害他的学业或其他工作。幸运的是,Metaculus的人们意识到了他的才华,并邀请他加入Pro预测团队Forecasting Our World in Data项目。

Schoenegger说,他喜欢在职业团队中预测的部分内容是解释和共享方法论的动机。他说,在公共比赛中,与社区的合作与获胜之间总是有紧张的局势。

“专业预测锦标赛的很酷的事情是分享信息并写出理由是我们激励要做的事情的一部分。我认为这真的很有趣,”他说。他还喜欢阅读其他预报员的想法。“我发现这非常有用和有用,尤其是当他们做事与我大不相同时。Pro预测者在我们的背景和方法中差异很大。有些人超过50多,并且在专业上做这件事的历史悠久。其他人只是没有博士学位。有些人预测仅用于元数据,但还有另一项非常无关的工作。”

But how does he do things? How does he feel confident offering up predictions on everything from when Queen Elizabeth II’s reign would end, to whether a major nuclear power plant will be operational in Germany by this summer?

他说:“我这样做的方式仅仅是非常基本的简单模型 - 但是这些模型的看法不超过我实际预测的50%。”他说,其他50%是他可以找到有关给定主题的所有背景研究和额外信息 - 可能影响结果的其他因素。“然后,非常简单的模型将背景阅读倒退,并了解某些类型的风险。”

他回到伊丽莎白二世女王的榜样。如果您试图预测她的统治何时结束[女王于2022年9月去世,但Metaculus首次摆姿势它的问题在2020年1月], Schoenegger says you would start with base rates on basic questions. “How often do British monarchs abdicate? Or I think the most basic question would be, ‘What’s average life expectancy?’ And looking at her age you might say there’s a 20% chance she dies each year after 2018. But then you might look at her track record of very good health, and update your numbers. And then you might read some article or argument in the other direction, and update depending on how credible you think it is.”

对于气候临界点锦标赛,一些问题集中在某些资源用于电动汽车技术方面。“这与资源和稀缺性有关,何时可以缓解这些资源。我接近这一点的方式就是尝试阅读一堆不同的[文章和报告],这些[文章和报告]试图查看这些特定的子问题,例如贵金属或锂供应,或类似的东西。”ReportsReports

And while sharing his methodology is part of the job now – at the outset of a question or tournament, Schoenegger says he guards against his forecasts being influenced by others in the Metaculus forecasting community. “On this tournament, I made an effort to be done with all my work on day one. I wanted to make sure that I get all my numbers into the system first, so I’m not then swayed by other people.”

Though Schoenegger obviously has a good track record as a forecaster now – he’s also very open to the possibility that things could change.

“I think the honest answer – and there are other people and forecasters who might give different, more optimistic answers – but I just think we don’t know yet if my track record over the past 18 months is actually at all predictive of me actually being better [than the average forecaster] over the next 40 years. I honestly think we don’t know yet,” he says.

但是,这种不确定性并不意味着Schoenegger害羞地让更多的人以循证方式尝试预测未来。

He says he’s twisted the figurative arms of ‘half of my family and most of my friends’ into making at least one official forecast. He tells people new to forecasting to start with a question or topic that they already have some interest or expertise in. “Take a look and try to flesh out your thinking and feelings in actual numbers and distributions and dates, because that’s something that most people don’t do. ‘It is very likely.’ What exactly do you mean by that? And I think just that exercise of putting fuzzy feelings into actual numbers – I think that can be very illuminating for oneself.”

更广泛地预测的未来是什么?Schoenegger认为,尽管与一群研究单一问题领域的专家专家的专业研究实验室可能总是比Metaculus这样的众包平台具有优势,但仅在其特定的研究领域才是正确的。

他说:“如果您拥有专有的超级特殊模型,而十位研究科学家正在研究它 - 对于土耳其洪水或类似洪水的某些参数保险非常好。”“但是(对于元流苏)我认为这是您可以预测的速度和敏捷性。乌克兰的战争,气候,总统选举,一些国王或王后的事情 - 非常广泛。希望是,一旦您击中了一定数量的专门用户,他们不断预测这些事情,凭借人群的智慧,您实际上就可以对各种事情进行预测。而且我认为他们的特定内容可能表现不佳。但是[众包预测]将赢得几乎所有其他一切的胜利。”

Schoenegger says he’s not positive he wants professional forecasting to be his main focus forever, but he’s also loath to give it up. “I’m still undecided on going the route of forecasting as my main job, or academics as my main job,” he says. “I think what I end up doing is basically both – because I like forecasting way too much. It’s way more fun than writing papers, to be honest.”