Funded Researchers: Dr. Jonas Henske & Dr. Robin Schiewer
Dr. Robin Schiewer
Faculty of Computer Science
Reinforcement Learning (RL) is an area of artificial intelligence focused on solving sequential decision-making problems across a broad variety of domains, such as finance, robotics, and healthcare. Despite its empirical success, RL is notoriously data-hungry, requiring large amounts of data to achieve optimal performance. Consequently, there is growing interest in methods to reduce RL’s data-intensive nature. One source of inspiration is natural intelligence, where constructing predictive models of the surrounding world is instrumental in solving complex and long-term problems with relatively little data. These models are adaptable, flexible, and robust against generating highly unrealistic predictions.
Model-based Reinforcement Learning (MBRL) is a subfield that harnesses the concept of world models within the RL domain. In my research, I investigate the capabilities and interrelationships of various RL world model architectures, aiming to equip future RL methods with more powerful and robust world models. In this context, the Gateway Fellowship allows me to join Royal Holloway, University of London (RHUL), and collaborate with Dr. Anand Subramoney in developing novel world model architectures using the recently popularized deep state space model formalism. These models combine predictive power on par with the well-known transformer architecture, while offering superior flexibility and efficiency, making them promising candidates for a new generation of world models. During my stay at RHUL, I will benefit from the expertise and strong theoretical background of Dr. Subramoney’s group, which specializes in state space model research. To complement this expertise with extensive theoretical and practical RL experience, a potential collaboration partner for our project is Professor Chris Watkins, the inventor of the well-known Q-learning algorithm.
By bringing together my research focus in MBRL with Dr. Subramoney’s expertise in state space models and Professor Watkins’s profound knowledge of RL algorithms, we aim to create an ideal research environment for constructing this novel class of world model architectures. Consequently, this collaboration has the potential to yield substantial contributions to the field.
Dr. Jonas Henske
Faculty of Biology and Biotechnology
Male orchid bees collect volatiles from various sources, including orchids, to concoct complex perfumes. Females select males based on these perfumes, which act as intersexual signals that convey the sender’s availability as a mate. However, what additional information is encoded in the signal and which chemical components are behaviorally relevant remains unknown. Furthermore, the origin of major, possibly essential perfume compounds remains elusive. Orchid bee perfumes are species-specific, yet intraspecific variation exists, suggesting that perfumes are fitness indicators shaped by female choice.
My overall project intends to further clarify the function and origin of male perfumes by analyzing perfume phenotypes, establishing perfume DNA metabarcoding, and testing perfume effects on male fitness in island populations of the orchid bee Euglossa dilemma. Neurophysiological studies indicated that E. dilemma’s major perfume compound, HNDB, is a key signal feature, either for species-recognition or as a fitness indicator, or both. E. dilemma was recently introduced to the Bahamas, Hispaniola and the Florida Keys. Island bees were shown to possess less complex perfumes due to reduced odor availability and therefore, small island populations represent a unique opportunity to study perfume making under constrained conditions. This could highlight essential compounds and reveal their source.
The main project goal is to determine the contribution of HNDB to E. dilemma’s perfume by comparing the composition of perfumes collected by bees sampled from several small island populations, hypothesizing that HNDB is present in all populations due to its central role in signaling, whereas other compounds are variable. The project will be conducted at the Ramírez Lab at the University of California, Davis.