Subconscious Kindling
In the first chapter of The Beginning of Infinity, David Deutsch writes:
The role of experiment and observation is to choose between existing theories, not to be the source of new ones.
I have the most fun attacking questions with no satisfying theories in sight. So: how can we generate novel theories for a problem? It’s critical to recognize that the creation of a hypothesis demands a wholly distinct approach from its evaluation.
Sometimes I uncover a critical idea by consciously exploring possible avenues, but more often, the realization flashes unbidden, with my attention elsewhere—like a lightbulb out of a classic cartoon. In these cases, I suspect my mind has been churning ceaselessly at some tree of possibilities beneath the surface of consciousness, projecting potential futures, searching my memory for relevant assocations, and evaluating the relevance of its findings before bubbling any useful notions up to my awareness.
We can propel this part of our psyche along by seeding and predisposing it for this pursuit. By providing kindling. I like to imagine waking up little subconscious drone ants and sending them in every direction, hoping that one will return with something useful later.
I find that doodles, in particular, provide especially effective kindling—whatever that means in your field: instinctive improvisation at the piano, whiteboard sketches of systems in engineering, lists of assertions in writing.
This process sounds much like brainstorming or mind mapping, but I don’t think it is, exactly: those are aids to conscious invention and organization. I’m suggesting something more like rubber duck debugging, a technique in which the simple act of explaining a problem in great detail induces cognitive dissonance during the weakest parts of the explanation.
As a highly creative act, theory generation benefits from many of the same techniques that bolster artistic pursuits. Besides doodling, I’ve developed theories by exaggerating, by celebrating contrast, and by applying techniques from one field to another unrelated one. Whatever I can do to throw more wood on the fire.