I recall a conversation 7 years ago with a group of friends where we talked about where creativity sprung from. I stubbornly clung to the notion that creativity was due to randomness. "Try writing a program that produces unique non-repeating unpredictable output every time", I said, "Can you do that without a random number generator?". Seemed reasonable, but an incomplete explanation.
Today I realized, that in the universe, there isn't such a thing as creation. Something does come from practically nothing, under the right conditions.
Life, and the cosmos itself have come into being by the fundamental process of evolution (Read "Life of the cosmos" by Lee Smolin). Why should something like ideas also not?
My hypothesis is that the brain is a simulated arena of evolution - There exists a mechanism to choose a set of memes related to the task in hand, churn them in a maelstrom and provide selection pressures based on desired results, in order to make a thread of thought that dominates all the others.
People have this today with software, evolving code by mutation and "reproduction" to generate algorithms automatically.
Perhaps the range of ideas one can generate is limited by the breath of the "meme pool" from which one evolves thoughts. Which is why, maybe adding external input to the "working set" to stir things up a bit causes sudden flashes of "inspiration". And so too, this is an unconscious automated process, that happens without awareness or logical trains of thought, which are the methods of the conscious brain regions.
It might even be that all thinking is a continuous process of evolution, as thoughts float to the top of the ever varying meme ecosystem in the brain. This is somewhat related to Dennet's "multiple demons" model of consciousness where multiple independent processes within the brain wrest control of behavior in a sort of co-operative multitasking model.
Remember, you read it here first!