Pranav Agarwal

I’m finishing my Ph.D. at Mila (advised by Sheldon Andrews and Samira Ebrahimi Kahou), with a focus on reinforcement learning for robotics and character animation. My research explores efficient reinforcement learning algorithms using improved prior modeling for robotic applications. Specifically, I have worked on:

• World models (Transformer based) for sample-efficient reinforcement learning
• Automating reward modeling from large offline trajectories
• Continual Learning
• Applications in complex robotic systems, such as autonomous driving
• Interpreting decision-making of LLMs

Previously, I was a student researcher at Inria, collaborating with Natalia Díaz-Rodríguez and Raoul de Charette. I completed my Bachelor's in Electronics and Communication Engineering at IIIT Guwahati, where I was awarded the President's Gold Medal. During my undergraduate studies, I worked as a research intern at SUTD with Professor Gemma Roig.

Research Interests: Reinforcement Learning | Lifelong Learning | World Models | Video Models | Robotics | LLM Interpretability

Hobbies: Outside of research, I enjoy long walks, capturing nature (check out my photos), reading (see my book collection), and lifting weights.

[ Email  /  CV  /  Github  /  Twitter  /  Google Scholar  /  Linkedin /  Projects ]

In this work, we interpret the personality traits of Large Language Models (LLMs) using our proposed Supernova Event Dataset, which includes Wikipedia articles consisting of historical events, biographies, news events, and scientific discoveries. We benchmark models based on their identification and ranking of key life or discovery events, a complex task requiring causal reasoning. A second LLM acts as a judge to infer each model’s personality based on its event selection and interpretation. Our analysis show distinct traits, like emotional reasoning in Orca 2 and analytical framing in Qwen 2.5, enhancing interpretability and trust.

This survey explores the impact of transformers in reinforcement learning, addressing common RL challenges, while examining their applications in representation learning, policy optimization, and interpretability.

This work introduces Discrete Abstract Representations for Transformer-based Learning (DART), a sample-efficient method that utilizes discrete representations to improve world modeling and learning behavior in reinforcement learning, achieving superior performance on the Atari 100k benchmark compared to existing methods.

The Medical Decision Transformer (MeDT) leverages the transformer architecture to enhance offline reinforcement learning for sepsis treatment recommendations, utilizing a goal-conditioned RL paradigm that improves interpretability and clinician interactivity, while achieving competitive results on the MIMIC-III dataset.

This paper introduces TPTO, a Deep Reinforcement Learning approach that utilizes Transformer and Proximal Policy Optimization to efficiently offload dependent IoT tasks to edge servers, significantly reducing latency for IoT applications compared to state-of-the-art methods.

A novel automatic evaluation strategy for excavator operators is proposed, utilizing machine dynamics and safety criteria, which is then validated through reinforcement learning in a simulation, resulting in safer and more realistic excavator maneuvering policies.

A curriculum-based deep reinforcement learning approach for end-to-end driving is proposed, using sparse rewards and navigation view maps to achieve generalization on unseen roads and longer distances.

A new image captioning dataset, Egoshots, is introduced alongside a novel evaluation metric, Semantic Fidelity, to address biases in existing models and enable caption assessment without annotations.

A framework for cross-modal biometric matching is proposed, generating faces from voice clips using various generative networks, with RC-GAN achieving the best identity accuracy of 84.52% and VAE producing the highest quality images.