报告题目：On Mixture of Experts and Local Model Averaging

报告时间：11月10日（周二）下午16:00—17:00

报告地点：嘟嘟成人网 106会议室

报告摘要：Mixture of experts (MOE) plays an essential role in LLMs. It is closely related to localized model averaging. In this talk, we will examine benefits of combining models in a way that allows the weights to depend on the input (i.e., localized model averaging), establish oracle inequalities for certain simplified MOE models, and discuss natures of several key features in MOE such as shared experts and top-k in routing. This talk is based on joint work with Ziwen Gao, Baihua He, Siyuan He, Jie Hu and Bokai Yang.

报告人简介：杨宇红，清华大学丘成桐数学科学中心教授，国际数理统计学会（IMS）会士，美国统计学会（ASA）会士，美国NSF CAREER Award获得者，连续入选斯坦福大学全球前2%被引科学家名单。他于2023年全职加入清华大学，此前长期在美国明尼苏达大学任教（full professor），是统计学与机器学习领域的资深学者，在国际统计学界有着重要的影响力。他的研究兼具理论深度与应用广度，核心方向包括模型选择与平均、高维数据分析、机器学习、因果推断、组合预测等。他在统计学、计量经济学、信息论等领域的顶级期刊上发表了大量有影响力的论文，包括 Annals of Statistics、Journal of the American Statistical Association (JASA)、Journal of the Royal Statistical Society: Series B (JRSSB)、Biometrika、Journal of Econometrics、IEEE Transactions on Information Theory 等。