DOI | Resolve DOI: https://doi.org/10.1007/978-3-030-47358-7_6 |
---|
Author | Search for: Bellinger, Colin1; Search for: Coles, Rory; Search for: Crowley, Mark; Search for: Tamblyn, Isaac2 |
---|
Affiliation | - National Research Council of Canada. Digital Technologies
- National Research Council of Canada. Security and Disruptive Technologies
|
---|
Format | Text, Article |
---|
Conference | Canadian Conference on Artificial Intelligence, Canadian AI 2020: Advances in Artificial Intelligence, May 13–15, 2020, Ottawa, Ontario, Canada |
---|
Subject | materials science; reinforcement learning; semi-Markov decision processes |
---|
Abstract | Reinforcement learning (RL) has been demonstrated to have great potential in many applications of scientific discovery and design. Recent work includes, for example, the design of new structures and compositions of molecules for therapeutic drugs. Much of the existing work related to the application of RL to scientific domains, however, assumes that the available state representation obeys the Markov property. For reasons associated with time, cost, sensor accuracy, and gaps in scientific knowledge, many scientific design and discovery problems do not satisfy the Markov property. Thus, something other than a Markov decision process (MDP) should be used to plan/find the optimal policy. In this paper, we present a physics-inspired semi-Markov RL environment, namely the phase change environment. In addition, we evaluate the performance of value-based RL algorithms for both MDPs and partially observable MDPs (POMDPs) on the proposed environment. Our results demonstrate deep recurrent Q-networks (DRQN) significantly outperform deep Q-networks (DQN), and that DRQNs benefit from training with hindsight experience replay. Implications for the use of semi-Markovian RL and POMDPs for scientific laboratories are also discussed. |
---|
Publication date | 2020-05-06 |
---|
Publisher | Springer Nature Switzerland AG |
---|
In | |
---|
Series | |
---|
Language | English |
---|
Peer reviewed | Yes |
---|
Export citation | Export as RIS |
---|
Report a correction | Report a correction (opens in a new tab) |
---|
Record identifier | 3f553680-18d6-429c-8fc0-4c3ae973c0aa |
---|
Record created | 2020-07-24 |
---|
Record modified | 2022-01-14 |
---|