I don’t get to play video games much (does so-called reality count?) but here’s one I came across which is pretty special: Baba is You (also described here). Here‘s a tiny sample that gives a flavor of how it works:
What’s cool about this puzzle game (besides the fact that it’s challenging and fun) is that it gets you to break a number of mental categories and think more continuously and fluidly about topics relevant to the understanding of life and mind. Among other things, it dissolves barriers between data and algorithm, between a cognitive system and its contents, and gets you to think differently. People often ask me what kind of preparation they need to join our lab; I think I’ll put this game on that list. It’s a good introduction to the relevant mental flexibility, especially given our latest directions. Hofstadter’s incredible, classic GEB is another such.
The first, most basic thing here is that it erases the distinction between objects and rules, between things and informational messages. Some of the objects in this world have meanings, besides physical properties like location and impenetrability. Moreover, some of those meanings determine the physics of the world and how things will act once you change the positioning of the words relative to each other. Move the objects to make up new sentences, and you change how the world works and what’s possible to do. This of course brings up fascinating issues of semiotics, and forces you to think about the definitions and mechanics of: messages, observers, interpretation, rules, and many other metaphysical topics. How do you know if an object in the “real” world is just a thing or information? Is there a binary distinction? Can shapes and patterns be agents themselves?
How might this play out in the physical world and what is the status of the laws of physics? Can anything you do in the Universe change the laws of physics themselves? Well, where do those laws of physics come from? If they come from some aspect of the Universe itself, then they should be changeable from within, at least in principle. If they do not, what happened to the concept of Universe with a capital U – where else is there for them to originate from? How about the laws of biology – some of those are definitely changeable from within, because biological information (molecular, bioelectrical, etc.) can influence how molecular pathways, cells, organs, and organisms interpret biological information and how subsequent evolution will play out – an essentially self-referential dynamic.
The dichotomy between object and message is seen for example in classic cannibalism studies by McConnell (download the archive of hard-to-find papers here), in which trained planaria were eaten by naive hosts and conferred behavioral information. In that case, the chemical engrams of the donor were food, but also memories to be interpreted by the tissues of the host and used to guide behavior. More broadly, memory engrams composed of mRNA, proteins, or whatever substrate, are both physical objects to some levels of organization (e.g., the molecular) and a cognitive medium to others (the organism).
What about the distinction between software and the machine – between data and hardware? The game breaks that binary framing, which is good, because the notion of a active data (like our self-sorting arrays), and indeed agential memories, are powerful areas for development in the emerging field of Diverse Intelligence. The notion of a continuum between passive information patterns (thoughts) and active cognitive agents (thinkers) is hard for many to grasp, and playing a game like this may provide an intuitive understanding of the framework – in a way that reading scientific papers may not.
Another remarkable thing about this game is that one of the tokens in this world which the player can manipulate is “YOU”, which refers to the player themselves; consistently with the rest of the game mechanics, it enables a kind of meta-plasticity: composing new sentences containing this token changes your abilities and your relationship to the rest of the universe. This is a powerful way to begin to think about biological systems in which your actions (via development, metamorphosis, etc.) radically change your form and function, as well as the space within which you act, the goals you pursue, and the preferences you hold. This has many implications not only for cognition and the notion of Selfhood but also for an extension of game theory in which the actions of a player change the number of players and the preferences of the player (enabling a kind of dynamic, morphing payoff matrix).
This is not just a game; this is a virtual reality mental prosthesis – an active, dynamic, engaging set of visceral intuition pumps. This will facilitate entry into an important aspect of an emerging multi-discipliary field of science at the intersection between developmental biology, computer science, and cognitive science.
I can imagine at least three ways to move forward and extend, based on my TAME framework:
- Implement our polycomputing framework – the notion of multiple observers who interpret (and hack) physical features of the world in their own way, This could be a multi-player dynamic or done via biologically-inspired NPCs who also have ways of reading the objects in the environment for their own purposes. This could get the player used to thinking from an observer-focused, perspective-centered view and become accustomed to shifting meaning frames as needed.
- Lean harder into autonomous agency and implement a spectrum of intelligence where the text messages have their own goals related to information passing, interpretability, adoption by larger cognitive systems, etc.
- Implement multi-scale dynamics, enabling nested agents within agents and the ability of objects and messages to virtualize and generate others along the hierarchy:
There’s also plenty of opportunity to involve AI in this kind of virtual world. Thank you to Victor Schetinger for pointing me to these links:
- This paper on evolutionary optimization of Baba Is You agents
- This paper showing that Baba Is You is undecidable
- This Baba is You code for a simulator in C++ with reinforcement learning
Featured image by Midjourney.
Leave a Reply