I like thought experiments, especially the ones that have a philosophical angle. In my last post I described a thought experiment about temporal continuity and learning invariance. I have a similar thought experiment for the role of actions in perception.
Consider an organism named couchy. Couchy lives anchored to a rock — it does not move. It has a visual sensor to sense the environment, but it has no self-generated motion. Nor does it have any facility to make actions that are directed in space (for example, shooting a barb to the left or right). Couchy senses the environment through its visual sensor. Couchy has formed invariant representations for objects in the world through the temporal proximity mechanism we discussed earlier. When dangerous objects are recognized, couchy emits a sharp sound that propagates in an isotropic manner.
Couchy is like a couch potato, watching the world through the television screen, never making a move and not even moving his eyes. Hence the name.
Now the question: If all couchy does is observe videos of objects moving in the world without ever making a self-generated spatially directed action, will it develop a sense of space? That is, will couchy have an idea of left and right, up and down? Will couchy develop a sense of geometry in its lifetime?
We can replace couchy’s brain with any learning algorithm and ask the same question. Will the learning algorithm develop a sense of geometry if it is observing the world passively through videos? Couchy has only a lifetime to learn this. So the question really is whether algorithms can learn a sense of geometry in this setting in a reasonable amount of time. Anything that is exponential is clearly of no interest.
The role of actions in perception/cognition/language is a very important topic and I will be writing more about it. This thought experiment is a good start.