Multi-Armed Bandits
Pick an arm, observe a reward, learn the best choice.
Each pull gives only partial feedback, so the algorithm must learn from experience.
Explore first, regret later. Preferably in a theorem.
Bandits | Online Learning | Optimization | Teaching
Mudit Sharma
Currently: PhD Student in Computer Science and Engineering at IIT Ropar
I work on
I develop theoretically principled and computationally efficient algorithms for learning and decision-making under uncertainty.
Multi-Armed Bandits
Dueling Bandits
Online Learning
Nonconvex Optimization
Continuous Action Spaces
About Me
Decision-making before all the evidence is in.
I am a PhD student in Computer Science and Engineering at the Indian Institute of Technology Ropar, supervised by Dr. Shweta Jain. My work focuses on optimization and sequential decision-making, especially multi-armed bandits, dueling bandits, online learning, and nonconvex optimization.
My research goal is to design algorithms that are mathematically reliable, computationally efficient, and useful when feedback is partial, comparative, noisy, or expensive to obtain.
Area of Research
Learning through decisions and feedback.
My work studies decision-making systems that must learn from partial feedback: choosing actions, comparing alternatives, and improving policies while uncertainty remains, including continuous and nonconvex action spaces.
Exploration vs Exploitation
Should we test a new option or use what seems best?
Move the control to see the decision policy shift.
Nonconvex Action Spaces
Optimize over landscapes where the best action is not obvious.
Continuous choices can create many local peaks, so the learner needs principled exploration and efficient search.
Dueling Bandits
Learn from pairwise preferences instead of numeric scores.
Two actions are compared; the winner updates the learner's belief about preference.
Research Map
A connected view of my research area.
Bandits sit at the intersection of sequential decisions, preferences, online learning, optimization, and theory.
Publications
Research papers and current work.
2026
Accepted
Lipschitz Dueling Bandits over Continuous Action Spaces.
Mudit Sharma, Shweta Jain, Vaneet Aggarwal, and Ganesh Ghalme.
Advances in Neural Information Processing Systems (NeurIPS 2026). To appear. CORE Rank: A*.
2027
Under Review
Fairness-Aware Dueling Bandit.
Satvik Shrivastava, Ashish Gupta, Mudit Sharma, and Shweta Jain.
Submitted to the AAAI Conference on Artificial Intelligence (AAAI 2027).
2027
Under Review
Comparator-Path Dynamic Regret for Adversarial Lipschitz Bandits One-Point Learning and Two-Query Oracle Adaptation.
Mudit Sharma, Shweta Jain, Vaneet Aggarwal, and Ganesh Ghalme.
Submitted to the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2027).
WP 02
Working Paper
Adversarial Dueling Bandit under Preference Feedback.
WP 03
Working Paper
Improving Dueling Bandits.
WP 04
Working Paper
Envy-Free Cake cutting Only Using Preference Feedback.
Coursework
Academic training behind the research.
Doctoral Training
- Ph.D. CSEIndian Institute of Technology Ropar | 2025-present
- CGPA8.47 / 10
- AdvisorDr. Shweta Jain
Undergraduate Foundation
- B.Tech. CSEChandigarh University | 2020-2024
- CGPA7.81 / 10
- PreparationQualified GATE in 2024 and 2025
Research Preparation
- Online LearningSequential decisions, regret, and adaptive algorithms
- Bandit TheoryMulti-armed and dueling bandits with preference feedback
- OptimizationStochastic and nonconvex optimization for action spaces
Technical Toolkit
Theory, code, and experiments.
My current stack supports algorithm design, theoretical analysis, simulation-heavy evaluation, and clear research communication.
Active focus:
Bandits, nonconvex optimization, simulations, and regret analysis
Programming
01Research Expertise
02Systems and Writing
03Teaching Assistant Timeline
Teaching, learning, and mentorship.
My teaching assistant work at IIT Ropar has centered on programming, mathematical foundations for AI, and data structures.
GE 103: Introduction to Programming
Instructors: Dr. Shweta Jain and Dr. Shashi Shekhar Jha. Mentored students in C programming, problem solving, and debugging.
AI111: Mathematical Foundations of AI
Instructors: Dr. Puneet Kumar and Dr. Shweta Jain. Guided students through mathematical concepts underpinning artificial intelligence.
CS203: Data Structures
Instructors: Dr. Shweta Jain and Dr. Swapnil Dhalmal. Supported implementation, debugging, and complexity-aware problem solving.
Collaborators
People shaping the research.
Research is a shared practice. This section highlights current supervision and coauthored work from my resume.
Faculty Collaborators
Student Collaborators
Satvik Shrivastava
B.Tech student at IIT Ropar, 2022-2026
Ashish Gupta
B.Tech student at IIT Ropar, 2022-2026
Research Groups
Department of Computer Science and Engineering
Indian Institute of Technology Ropar. Home department for my doctoral research in algorithms, optimization, and sequential decision-making.
Visit CSE IIT RoparGame Theory and Machine Learning Lab
GTML Lab at IIT Ropar, led by Dr. Shweta Jain, works at the intersection of game theory, machine learning, and real-world decision-making.
Visit GTML LabGallery
Research life beyond the papers.
The gallery adds a more visual layer to the academic profile: workspace snapshots, travel photos, and moments beyond the desk.
Open Gallery
Contact
Collaborate on research.
I welcome conversations about bandits, online learning, nonconvex optimization, teaching, and future research collaborations.
Academic Email
Email me at IIT Ropar Personal emailDepartment of CSE
Indian Institute of Technology Ropar, Punjab, India
GitHub
Open GitHubScholar / ORCID
Research conversations
Bandits, online learning, preference feedback, nonconvex optimization, and collaboration ideas.