Artificial Intelligence and Science

Artificial Intelligence & Science

In our group, we investigate the fundamental limits and potential of artificial intelligence (AI), and the positive influence AI can play in future research. We have developed a learning model, projective simulation [1], that highlights this potential, and we combined it with state-of-the-art tools in machine learning, which range from reinforcement learning to deep neural networks and representation learning. We are enthusiastic about the methods developed in modern computer science, and want to find their place in modern-day science, with a focus on applications in quantum information.

fig_RL

Projective simulation

Projective simulation (PS) combines notions of simulation, memory, and random walks into a new physical framework for learning [1,2]. The key concept is the episodic and compositional memory, a stochastic network of episodic memories called clips. When the agent encounters a new percept clip, it triggers a stochastic random walk through the clip network that results in an action. The hopping probabilities evolve through environmental feedback, while new clips can be created by variation and composition of existing clips.

We developed notions of quantum and classical learning agents that operate in unknown or partially known environments, where both the environment and the agent may involve quantum degrees of freedom [3,4]. All these settings have been investigated in combination with other well-established approaches, such as neural networks, pattern recognition techniques and representation learning [5].

Classical agent

– Classical environment

bees_compact

PS can be used as a model for reinforcement learning (RL). We have investigated how well PS agents perform compared to other approaches, e.g. in standard RL tasks [2, 6] and for skill learning in robotics [7]. We are studying new ideas to develop PS and take advantage of its features, e.g. in combination with neural networks [5], as well as its applications to fundamental questions in behavioral biology [8, 9, 10] and philosophy [10,11].

– Quantum environment

fig_CQ A related direction we are exploring is the application of classical PS agents to problems in quantum information. Here, relevant results we achieved are the autonomous design of state-of-the-art quantum experiments [13] and the protection of quantum information in where, for example, PS finds surface codes that are near-optimal solutions in the number of qubits [14].

Quantum agent

– Classical environment

fig_qc Here, the agent operates in a classical environment but it can use quantum mechanics to learn and process its experience. In a quantum PS, the deliberation process of the agent corresponds to quantum random walks, allowing the agent to call its episodic memory in superposition. This notion of a quantized agent achieves a quadratic speed-up in decision-making and, consequently, an improved learning behavior in active learning settings [3,15].

– Quantum environment

fig_qq Beyond the improvement in the classical case, understanding its extensions and full potential in quantum environments is one of the cutting-edge lines of research we are excited about. A full quantum mechanical treatment of learning agents offers many conceptual challenges that are currently under investigation (in collaboration with Vedran Dunjko in Leiden), such as the understanding of the very meaning of learning in the general case when the agent and environment become entangled [4, 16].

Towards artificial research assistants in basic science

Artificial intelligence promises to have a dramatic impact in our everyday life, for instance in robotics, logistics, and self-driving vehicles. In the future, these and other areas are expected to increasingly benefit from the availability of virtual assistant systems which take decisions in a fast, reliable and autonomous way. In a similar, but even more dramatic way, basic science may be changed by the development of artificial research assistant systems.

The development of autonomous AI assistant systems for basic science requires us to go beyond application-specific algorithms and problem solving. Arguably, one key ingredient of such an AI research assistant is the ability to develop abstract models of nature, which are based on meaningful and operationally useful representations of physical systems. We believe that this aspect can in fact make the difference in real world applications, as well as in basic research.

In our group, we take on this challenge by developing various complementary approaches. For example, in Ref. [5], we define operationally meaningful representations and develop an architecture that can produce such representations in various experimental settings. Similarly, in Ref. [12], we propose and numerically investigate an operational definition of the ability to form abstract concepts and present a minimal learning agent with this capacity. Besides these results, we are also working on other approaches towards an autonomous research assistant: for instance, we are investigating how an AI agent can distill meaningful and useful sets of actions (`skills’) in the form of patterns hidden in structured data of large actions spaces. All these directions appear to be very promising, and we are excited to see where they will lead us in the future!

fig_assistant_KR

fig_assistant_HPN

[1] H. J. Briegel and G. De las Cuevas, Projective simulation for artificial intelligence, Sci. Rep. 2, 400 (2012)[arXiv:1104.3787].
[2] J. Mautner, A. Makmal, D. Manzano, M. Tiersch, H. J. Briegel, Projective simulation for classical learning agents: a comprehensive investigation, New Gener. Comput. 33, 69 (2015) [arXiv:1305.1578].
[3] G. D. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J. Briegel, Quantum speed-up for active learning agents, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].
[4] V. Dunjko, J. M. Taylor, and H. J. Briegel, Quantum-enhanced machine learning, Phys. Rev. Lett. 117, 130501 (2016) [arXiv:1610.08251].
[5] H. Poulsen Nautrup, T. Metger, R. Iten, S. Jerbi, L. M. Trenkwalder, H. Wilming, H. J. Briegel, R. Renner, Operationally meaningful representations of physical systems in neural networks, e-print arXiv:2001.00593 [quant-ph] (2020).
[6] A. A. Melnikov, A. Makmal, H. J. Briegel, Benchmarking projective simulation in navigation problems, IEEE Access 6, 64639 (2018) [arXiv:1804.08607].
[7] S. Hangl, V. Dunjko, H. J. Briegel, J. Piater, Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition, Frontiers in Robotics and AI 7 (2020) [arXiv:1706.08560].
[8] K. Ried, T. Müller, H. J. Briegel, Modelling collective motion based on the principle of agency: General framework and the case of marching locusts, PloS one 14 (2), e0212044 (2019) [arXiv:1712.01334].
[9] A. López-Incera, K. Ried, T. Müller, H. J. Briegel, Development of swarm behavior in artificial learning agents that adapt to different foraging environments, PLoS ONE 15(12): e0243628 [arXiv:2004.00552].
[10] A. López-Incera, M. Nouvian, K. Ried, T. Müller, H. J. Briegel, Collective defense of honeybee colonies: experimental results and theoretical modeling, e-print arXiv: 2010.07326 [quant-ph] (2020).
[11] T. Müller, H. J. Briegel, A stochastic process model for free agency under indeterminism. Dialectica, 72: 219–252 (2018).
[12] K. Ried, B. Eva, T. Müller, H. J. Briegel, How a minimal learning agent can infer the existence of unobserved variables in a complex environment, e-print arXiv:1910.06985 [cs] (2019).
[13] A. A. Melnikov, H. Poulsen Nautrup, M. Krenn, V. Dunjko, M. Tiersch, A. Zeilinger, H. J. Briegel, Active machine learning for quantum experiments, PNAS 115 (6), 1221-1226 (2018) [arXiv:1706.00868v3]
[14] H. Poulsen Nautrup, N. Delfosse, V. Dunjko, H. J. Briegel, N. Friis, Optimizing quantum error correction codes with reinforcement learning, Quantum 3, 215 (2019) [arXiv:1812.08451].
[15] S. Jerbi, L. M. Trenkwalder, H. Poulsen Nautrup, H. J. Briegel, and V. Dunjko, Quantum enhancements for deep reinforcement learning in large spaces, PRX Quantum 2, 010328 (2021), [arXiv:1910.12760].
[16] V. Dunjko, H. J. Briegel, Machine learning and artificial intelligence in the quantum domain: A review of recent progress, Rep. Prog. Phys. 81, 074001 (2018) [arXiv:1709.02779].