Where Do Ideas Come From?

By Geordie Williamson for Ockham's Razor

Mathematician Geordie Williamson had a dramatic "aha moment" in a hotel shower — and it got him thinking about where brilliant ideas come from.

In 2013 something amazing happened to me.

I had recently arrived in Beijing when, in the hotel shower, I had an experience that was as sudden and dramatic as unexpected thunder. I was exhilarated and scared at the same time.

I have had 20 or so such sudden illuminations in my life. Sadly, 90 per cent of them were wrong. The thunder stroke of illumination, though dramatic, is often misguided, at least for mortals like me.

Thankfully, the idea in the shower in Beijing proved to be correct. It concerned a problem that I had devoted seven years of my work as a mathematician to. Other mathematicians had been working on it for 30 years.

The idea concerned a highly specialized problem, and I won't be able to explain it here. What I'm interested in is where these ideas come from, and how they fit into the way we solve problems in mathematics.

I approach this through the lens of mathematics. I am not a psychologist, or a neurologist. I have no idea how brains work. 

Instead, I am basing my conclusions on my observations of mathematicians and the mathematical community.

Ideas are a beautiful human experience

Remarkable ideas are what much of our modern world is built on: the wheel, the combustion engine, the post-it note, GPS. Someone had each idea.

The story I told you is an instance of the "eureka moment" — Archimedes was in the bath, I was in the shower.

Most of us have had a similar experience — deep frustration while grappling with a problem or a puzzle such as a crossword or a sudoku, followed by that sudden feeling of surprise when the solution occurs to you.

About 50 per cent of mathematicians that I have consulted report having had this aha moment experience. Whenever I talk to colleagues concerning eureka moments they report a similar pattern.

Firstly, one has to work weeks, months or years without apparently progressing.

Then, if one is lucky, the idea comes during a period of relaxation, or while thinking of something else.

A very interesting feature (to which a number of mathematicians attest) is a feeling a few days or hours prior to the breakthrough that something is going on. It is almost as though one can hear the rumblings of the unconscious prior to the appearance of the idea.

The slow process of sudden ideas

The picture of the eureka moment is overly simplistic, I suspect.

By way of explanation, the background details to my experience are a little more complicated than I made out.

Firstly, prior to coming to Beijing I had been visiting a collaborator in Hong Kong. For eight days straight, he asked me questions on this subject. He thinks about things in a very different way to me.

However, in hindsight I have no doubt that he was pushing me to think differently. This effort to think differently and justify everything I believed was a catalyst to its solution.

Secondly, I was later surprised to realize that the idea had been staring me in the face for over a year, quite literally.

When I returned to my office in Bonn, Germany I was shocked to see that a calculation I had done over a year earlier with a good friend, Ben Elias, contained the germ of my idea — I had stuck it up on the wall of my office. 

Sometimes the big idea is staring right at us!

Eureka moments are solitary, but math is collaborative

Modern mathematics is intensely collaborative. The breakthrough which is considered 'mine' would have been impossible without the years of collaborative work with Ben Elias, as well as the time spent with Xuhua He in Hong Kong.

Newton famously said "If I have seen further it is by standing on the shoulders of giants".

This is truer today in mathematics than it ever has been. Consciously or unconsciously, mathematicians have mapped out our landscape, with each mathematician responsible for investigations in a particular corner of the landscape.

This collaborative work is probably most evident in so-called polymath projects.

This idea began in 2009 when a well-known mathematician, Tim Gowers, asked 'Is massively collaborative mathematics possible?' on his blog.

He followed up with a problem, asking his readers to suggest possible attacks, and document partial progress.

The project was a success, leading to two new solutions of a difficult problem. There have been 16 polymath projects to date, four of which have led to a solution of the problem they set out to solve.

Usually mathematicians go to great effort to hide the scaffolding of their ideas. However, in this collaborative technique everything is laid bare. One can see all the false starts, the promising ideas, and the dead ends of real research.

The story of the eighth polymath project is particularly interesting. 

Here the problem concerned gaps between prime numbers. We think, but can't prove, that there are infinitely many "twin primes". These are pairs of prime numbers that differ by 2 —like 3 and 5; 17 and 19; 41 and 43.

Computers searches easily find millions of such twin primes, however we still don't know that there are infinitely many. 

One can ask an apparently easier question: are there infinitely many prime numbers that differ by at most 1,000, or 1 million, or 1 billion? Even this question evaded mathematicians for centuries.

In 2013, Yitang Zhang, working alone, had a crucial breakthrough, showing that there are infinitely many prime numbers separated by at most 70 million. 

What followed was an explosion of work attempting to understand and extend his solution.

A polymath project was formed, led by Australian Terry Tao, one of the best mathematicians in the world.

You can still read the many false starts and attempts to understand in the comments on Terry Tao's blog. 

Every day the number got smaller. After several months, and hundreds of comments by amateurs and experts alike, 70 million was down to 4680. At this point progress slowed.

Out of the blue, came an idea from James Maynard, working alone at the Universities of Oxford and Montreal. With rather different ideas, he was able to lower 70 million down to 600.

Finally, Professor Maynard joined Professor Tao's collaborative polymath project, and the gap was down to 246, were progress has since stopped.

I think this story nicely sums up how mathematicians work nowadays. We are social beings, and much of our work is collaborative. 

Yet still there is room for an individual working alone to have their eureka moment.

Geordie Williamson is a Professor of Mathematics at the University of Sydney and director of the Sydney Mathematical Research Institute.