Research

• Learning Generalized Temporal Abstractions [Reasoning and Learning Lab,  Mila-McGill]

  • Supervisor:  Doina Precup

Learning temporal abstractions which are partial solutions to a task and could be reused for other similar or even more complicated tasks is intuitively an ingredient which can help agents to plan, learn and reason efficiently at multiple resolutions of perceptions and time. Just like humans acquire skills and build on top of already existing skills to solve more complicated tasks, AI agents should be able to learn and develop skills continually, hierarchically and incrementally over time. In my research, I aim to answer the following question: How should an agent efficiently represent, learn and use knowledge of the world in continual tasks? My work builds on the options framework, but provides novel extensions driven by this question.

• Attend Before you Act: Leveraging human visual attention for continual learning [Reasoning and Learning Lab,  Mila-McGill]

  • Supervisor:  Doina Precup

When humans perform a task, such as playing a game, they selectively pay attention to certain parts of the visual input, gathering relevant information and sequentially combining it to build a representation from the sensory data. In this work, we explore leveraging where humans look in an image as an implicit indication of what is salient for decision making. We build on top of the UNREAL architecture (Jaderberg et al., 2016) in DeepMind Lab’s 3D navigation maze environment. We train the agent both with original images and foveated images, which were generated by overlaying the original images with saliency maps generated using a real-time spectral residual technique. We investigate the effectiveness of this approach in transfer learning by measuring performance in the context of noise in the environment.

• Safe Hierarchical RL  [Reasoning and Learning Lab,  Mila-McGill]

  • Supervisor:  Doina Precup

Currently exploring the literature and studying various approaches to make learning in an artificial agent more understandable by humans. While optimizing for returns alone could lead to an optimal behaviour, it does not always guarantee the desired behaviour. We explore a constrained optimization approach and introduce a notion of safety in learning agents. In the past, such methods have been explored for primitive actions. In this direction of research, we aim to introduce notions of safety such as reduction in variance of return in temporal abstractions as opposed of primitive actions.

• Saliency-based visually attended locations  [Human Centered Computing Lab,  UF]

  • Supervisor:  Eakta Jain

This work comprised investigating the use of computational saliency-based models for computation of salient areas – where humans would look in a comic panel. Followed by understanding and evaluating the performance of different saliency models on comic art. We employed Winner-Take-All (WTA) neural networks to generate eye fixations.

This work was carried in collaboration with Ishwarya Iyengar (Siemens), Dr. Olivier Le Meur (IRISA),  Dr. Sanjeev Koppal (UF) and Dr. John Shea (UF).

• Predicting visual attention – in context of comic art [Human Centered Computing Lab,  UF]

  • Supervisor:  Eakta Jain

This research aimed at laying down a preliminary benchmark of saliency models for predicting eye fixations in an image. This work was done under the supervision of Dr. Jain in the Human-Centered Computing(HCC) Lab. The primary aim of this research encompassed understanding the concepts of saliency, reviewing the current state-of-art in saliency detection in images, with evaluation of these models on various datasets.

Why do we care about Saliency? – Saliency in the context of an image is important to tell us significant or more “informative” pixels apart from the not so important ones. However, it is apparent that salient regions would differ widely for different applications. Our focus here is to find saliency in images in the context of humans. What are the hot-spots in a picture which drive visual attention?

Understanding visual attention in comic art aids several design decisions for artists and filmmakers. Furthermore, this knowledge plays a vital role in the organization of objects, text, color patterns, etc.  In general, saliency in a visual finds several applications, such as automatic cropping in an image, design, graphics, video editing, automatic estimation of salient object regions enhancing object detection, motion recognition, etc.

Performance evaluation of different models for one of the benchmark datasets from MIT Saliency Benchmark, comprised of using standard metrics such as Area under the curve(AUC) with ROC curve, for instance (an interactive visualization of ROC-AUC; Understanding ROC curves ), and Normalized Scan-path Saliency (NSS) were employed.

• Towards path-planning and vision based algorithms  [Machine Intelligence Laboratory, UF]

  • Supervisor:  Eric Schwartz,  Antonio Arroyo

This work comprised exploring potential approaches working towards a robust path-planning controller for Propagator, UF’s autonomous surface vehicle designed at MIL. I was primarily involved in studying different graph traversal algorithms such as D*, A*, and variants of these. I spent a lot of time understanding the complete system architecture of the boat based on ROS middleware. Writing nodes for thrust controls, listening to sub-system ROS messages and integrating nodes with the existing system were basic tools of the work.

Working with the boat, testing the boat on the pool is the most adventurous. Here is my blog post about the Propagator testing at the Graham Pool at UF.  While testing the boat is fun, it gave me an exposure to see the complete system architecture flow in a real-time setup. Have a peek at one of these sessions everything you need to know about Propagator here. 

• Autonomous Mobile Robot Navigation  [Intelligent Systems Laboratory, IITK]

  • Supervisor:  Laxmidhar Behera

I  worked towards a comprehensive study of different learning algorithms for mobile robot navigation in unstructured environments. Autonomous robot navigation is a challenging problem in the context of unknown circumstances. I envision applications such as assisting humans in daily life tasks, intelligent navigation and decision making on rough terrains, assisting people in disaster-hit locations, and rescue and search operations. This research dealt with neural and fuzzy controllers for mobile robot navigation within the single objective and multi-objective evolutionary optimization framework. This study involved evolving robot behavior that simultaneously achieves obstacle avoidance and target seeking using a single controller.

With interest in developing algorithms that focus on robotics perception and control,  a reactive control strategy was adopted for robot learning. The robot has been trained in an unknown, unmapped environment. The offline trained controller was then tested in various new environments with the same task involving navigational behaviors.

Player/Stage

Player is a language/platform independent device server which provides a simple yet powerful interface to a diverse range of robot’s sensors and actuators. It allows robot control program to be written in any programming language that supports TCP sockets. Player aids distributed sensing, along with monitoring of experiments. Stage is a robot simulator, wherein mobile robots navigating in a two-dimensional bitmapped environment, are controlled through Player. Stage works with Player as a driver, similar to a combination of Player and a real robot.

The snippet here shows an example of Player/Stage training environment where the robot(in red) is navigating to a stationary, known target while avoiding obstacles(marked as black shapes and boundaries). A reactive control strategy is followed, wherein no prior information of the environment is given to the robot. 

Emergence of multiple robot behaviors using a single controller:

Evolutionary optimization framework employed to accomplish various behaviors using a single controller. Genetic Algorithm(NSGA-II)  finds it use in parametric optimization of the weights of the neural/fuzzy controller. Training is performed in a single environment, and the trained controller is validated across several environments entirely new to the robot. Using a single objective genetic algorithm, few interesting results are shown below.  This work focuses on the implicit learning of robot behavior in unstructured environments. Implicit learning allows various solutions to evolve, in the lines of the human-like system.

Behavior Switching:

A very common approach to mobile robot navigation is behavior-based robotics, wherein, we have well-defined behaviors, and a hierarchical approach is followed to switch between different actions. However, hard switching between different controllers leads to chattering.  My initial work dealt with the same, and the following video demonstrates the principal problem faced in behavior switching.