Research Projects

ESCORT Framework

Completed

Efficient Stein-variational and Sliced Consistency-Optimized Temporal belief Representation for POMDPs. A particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces.

Key Contributions:

  • Correlation-aware projections modeling state dependencies
  • Temporal consistency constraints for stable updates
  • Superior performance on multi-modal distributions
Open Source

NS-Gym

Completed

First simulation toolkit designed explicitly for non-stationary MDPs, integrated with OpenAI Gymnasium. Provides standardized benchmarks and environments for testing adaptive algorithms.

Features:

  • Modular environment parameter evolution
  • 6+ benchmark environments
  • Compatible with standard RL algorithms
  • Comprehensive evaluation metrics

Shrinking POMCP

Completed

Real-time UAV search and rescue framework combining advanced simulation with novel POMDP planning. Addresses time constraints by guiding agents toward non-sparse belief regions.

Applications:

  • UAV search and rescue operations
  • 3D AirSim-ROS2 integration
  • Neuro-symbolic navigation

AIROAS

Completed

Annealed Importance Resampling for Observation Adaptation Search. Addresses particle degeneracy and sample impoverishment in POMDP belief updating.

Innovations:

  • Sigmoid-based tempering for tree search
  • Target inefficiency ratio mechanism
  • Superior performance in highly observable settings

Vehicle-to-Building Optimization

Completed

Online decision-making system for V2B energy management using Monte Carlo Tree Search. Deployed with Nissan Advanced Technology Center.

Impact:

  • 30% reduction in peak power demand
  • Real-world EV testbed validation
  • Handles heterogeneous charger configurations

I-TAP

Active

In-Context Latent Temporal Abstraction Planner combining in-context adaptation with online planning in learned temporal abstraction spaces.

Benchmarks:

  • MuJoCo locomotion tasks
  • High-dimensional Adroit manipulation
  • Effective under partial observability

PA-MCTS

Completed

Policy-Augmented Monte Carlo Tree Search for non-stationary environments. Combines offline learning with online search for robust decision-making.

Results:

  • Outperforms AlphaZero in non-stationary settings
  • Theoretical convergence guarantees
  • Validated on OpenAI Gym environments

Adaptive MCTS

Completed

Adaptive Monte Carlo Tree Search that learns updated dynamics while maintaining safe exploration through dual-phase sampling strategies.

Key Features:

  • Bayesian uncertainty quantification
  • Risk-averse exploration
  • Online adaptation to environment changes