Research

My research sits at the intersection of probabilistic inference, decision-making under uncertainty, and machine learning, with the goal of enabling autonomous systems to operate reliably in complex, partially observable environments. I am particularly interested in combining the principled structure of classical estimation and planning methods with the adaptability of modern learning approaches.

Publications

Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs

Illustration comparing particle-based and Rao-Blackwellized belief representations in a planning tree — Illustration of Rao-Blackwellized particle generative evaluation and analytical update during online planning. **Top:** Quadrature-based integration. Deterministic quadrature nodes are combined with the particle state to produce multiple weighted next states, rewards, and observations. The analytical state components are updated using the expected observation and expected next sampled component via their quadrature weights, yielding an updated analytical distribution stored at the corresponding tree node. **Bottom:** Monte Carlo evaluation. Analytically tractable components are randomly sampled to generate a single next-state realization, introducing additional sampling-induced variance into predicted rewards and requiring substantially more tree simulations.

Robots operating in the real world must make decisions based on incomplete and noisy observations — a setting formally captured by Partially Observable Markov Decision Processes (POMDPs). While sampling-based POMDP solvers are flexible, their performance degrades in high-dimensional state spaces due to the variance inherent in Monte Carlo estimation. This work extends the Rao-Blackwellized POMDP (RB-POMDP) framework to support arbitrary analytically tractable belief components through hybrid continuous-discrete state representations. By analytically propagating uncertainty for structured parts of the state during tree-based planning, the approach substantially reduces sampling variance — achieving higher cumulative rewards with far fewer particles than purely sampling-based methods. We validate the framework on a robotic search-and-rescue task by integrating it with FastSLAM 2.0, where the agent must simultaneously localize itself and locate victims under partial observability.

Rao-Blackwellized POMDP Planning

ICRA 2025

Jiho Lee, Nisar Ahmed, Kyle Hollins Wray, Zachary Sunberg

2025 IEEE International Conference on Robotics and Automation (ICRA), Atlanta, USA

POMCPOW and RB-POMCPOW tree structure comparison — POMCPOW (left) and RB-POMCPOW (right) Tree Structure Comparison. Each square and larger circle represents an action node and an observation node, respectively. In the POMCPOW tree, particles are shown as black dots while each particle in the RB-POMCPOW tree is associated with a Gaussian distribution. Due to the nature of POMCPOW, these particles form a weighted mixture of beliefs.

Partially Observable Markov Decision Processes (POMDPs) provide a structured framework for decision-making under uncertainty, but practical deployment depends on efficient belief updates and scalable online planning. This paper introduces Rao-Blackwellized POMDP (RB-POMDP) approximate solvers and a new RB-POMCPOW planner that combines analytical filtering with quadrature-based integration to reduce variance in both estimation and value computation. In a GPS-denied localization task, the proposed approach improves planning quality and computational efficiency compared to standard particle-filter-based POMCPOW under matched computational limits.

IEEE Xplore