Statistics & Optimization for Trustworthy AI

Our Research

We develop principled and empirically-impactful AI/ML methods

• mathematical foundations of transformers and attention
• trustworthy and efficient language models, LLM systems
• reinforcement learning, control, LLMs as interactive agents
• core optimization and statistical learning theory

News

• Transformers as SVMs and FedYolo will appear in NeurIPS workshops
• Two papers are accepted to NeurIPS 2023
  • Max-Margin Token Selection in Attention Mechanism, spotlight paper!
  • Dissecting Chain-of-Thought: A Study on Compositional In-Context Learning of MLPs
• Grateful for the Adobe Data Science Research award!
• Our new works develop the optimization foundations of Transformers via SVM connection 🔥
  • Transformers as Support Vector Machines => Twitter thread
  • Max-Margin Token Selection in Attention Mechanism, NeurIPS’23
• Two papers appeared at ICML 2023
  • Transformers as Algorithms: Generalization and Stability in In-context Learning
  • On the Role of Attention in Prompt-tuning
• Two papers appeared at AAAI 2023: Provable Pathways and Long Horizon Bandits
• Papers to appear at AutoML 2023 and ICASSP 2023.
• One paper is accepted to L4DC 2023 as oral presentation.