Dueling bandits over continuous action spaces
I investigate how a learner can identify strong actions using only noisy pairwise comparisons, particularly when the action set is too large or continuous to enumerate.
Research Details
Bandits, preferences, and nonconvex optimization.
I study sequential decision-making when feedback is limited to rewards or pairwise preferences. My work connects bandit algorithms, dynamic regret, fairness, and optimization over continuous and nonconvex action spaces.
Explore
Compare
Adapt
Optimize
Prove
Research Interest
Active research directions.
I design algorithms with provable guarantees for partial, comparative, and changing feedback.
Learning in changing and adversarial environments
I study decision rules that remain effective when the best action changes over time and feedback may be selected adversarially.
Optimization in nonconvex action spaces
I explore bandit problems whose feasible actions form continuous, nonconvex landscapes, where local structure must guide exploration under a limited query budget.
Fairness-aware preference learning
I examine how fairness constraints change exploration and selection when an algorithm learns from comparisons rather than absolute rewards.
Concepts
The ideas behind the research.
Multi-Armed Bandits
Imagine several options, each with an unknown payoff. The learner chooses one option, observes only that result, and must decide what to try next. The core tension is exploration versus exploitation.
Dueling Bandits
Sometimes users cannot give a reliable score, but they can compare two options. Dueling bandits learn from pairwise preferences such as "A is better than B" instead of numeric rewards.
Online Learning
Online learning studies systems that receive data sequentially and update decisions over time. The learner cannot wait for a perfect dataset; it must act, learn, and correct itself as evidence arrives.
Preference Learning
Preference learning asks how to infer what people or systems value from comparisons, rankings, clicks, selections, or other indirect signals. It connects naturally to recommender systems and human-AI feedback.
Nonconvex Optimization in Action Spaces
In continuous-action problems, the learner may search over a landscape with several local optima. Nonconvex optimization asks how to make useful progress without assuming the reward or preference surface is simple.
Learning Theory
Learning theory gives the mathematical language for asking when an algorithm should work, how much data it needs, and how much regret or error it may accumulate along the way.
Why They Fit Together
Bandits define the feedback structure, online learning defines the sequential setting, preference learning defines the signal, nonconvex optimization shapes the action search, and learning theory helps explain what can be guaranteed.
Decision Loop
The common pattern underneath the work.
Most projects on this page can be described as a loop: choose an action from a possibly nonconvex space, receive limited feedback, update a belief, and choose again.