The first subtle hints of cognitive decline may reveal themselves in an artist’s brush strokes many years before dementia is diagnosed, researchers believe, The Guardian reports.

The controversial claim is made by psychologists who studied renowned artists, from the founder of French impressionism, Claude Monet, to the abstract expressionist Willem de Kooning.

While Monet aged without obvious mental decline, de Kooning was diagnosed with Alzheimer’s disease more than a decade before his death in 1997.

Alex Forsythe at the University of Liverpool analysed more than 2,000 paintings from seven famous artists and found what she believes are progressive changes in the works of those who went on to develop Alzheimer’s. The changes became noticeable when the artists were in their 40s.

Though intriguing, the small number of artists involved in the study means the findings are highly tentative. While Forsythe said the work does not point to an early test for dementia, she hopes it may open up fresh avenues for investigating the disease.

“I don’t believe this will be a tool for diagnosis, but I do think it will trigger people to consider new directions for research into dementia,” she said.

The research provoked mixed reactions from other scientists. Richard Taylor, a physicist at the University of Oregon, described the work as a “magnificent demonstration of art and science coming together”. But Kate Brown, a physicist at Hamilton College in New York, was less enthusiastic and dismissed the research as “complete and utter nonsense”.

Forsythe and her colleagues used digital imaging software to calculate how a mathematical feature called fractal density varied in artists’ paintings over their careers. The seven artists included Monet, Pablo Picasso and Marc Chagall, who all aged without obvious brain disease; Salvador Dali and Norval Morrisseau, who developed Parkinson’s; and de Kooning and James Brooks, another abstract expressionist who was diagnosed with Alzheimer’s in 1985, seven years before his death.

Fractals are geometric patterns that repeat themselves at different size scales. They are seen in nature in the branching of trees and rivers, and in the craggy contours of coastlines. In paintings, fractals appear when patterns made by the tiniest brush strokes repeat on larger scales. The fractal dimension is a measure of fractal complexity, where an artwork with a large fractal dimension has a high ratio of fine to coarse fractal patterns.

Forsythe found that paintings varied in their fractal dimensions over an artist’s career, but in the case of de Kooning and Brooks, the measure changed dramatically and fell sharply as the artists aged. “The information seems to be like a footprint that artists leave in their art,” Forsythe said. “They paint within a normal range, but when something is happening the brain, it starts to change quite radically.”

Writing in the journal Neuropsychology, the scientists claim that the fractal dimensions of paintings by Monet, Picasso and Chagall tended to rise as they aged. For Dali and Morrisseau’s work, the fractal dimension followed an upside-down U-shape over time, at first rising and then falling. The most stark result was seen in the works of de Kooning and Brooks, where the fractal dimension started high and dropped rapidly from the age of 40.

The work has echoes of previous studies that revealed early signs of dementia in the language used by the former US president Ronald Reagan, and the novelist and philosopher Iris Murdoch. Telltale hints of future dementia have also been spotted in autobiographical essays written by nuns in their 20s.

Taylor pioneered the use of fractals to study and even authenticate drip paintings by the late US artist Jackson Pollock. He believes Forsythe’s research could do the same for other artists and save museums from being conned into buying fake artworks. But he also saw more important applications. “This work could hopefully be used to learn more about conditions such as dementia,” he said.

“To me, the most inspiring message to come out of this work is that beautiful artworks can result from pathological conditions,” he said. When de Kooning was diagnosed with Alzheimer’s, some critics argued that he should stop painting, but as he slipped into dementia, his artwork changed and became more simple, Taylor said.

“To me, these more simple works conveyed a peacefulness that wasn’t present in his nurture-dominated earlier work. It all goes to show that sometimes you can think too much about art. Sometimes you just need to tune into your inner self, the ‘nature’ part,” he said.

But Brown disagreed. In 2006, she co-authored a paper in Nature that disputed Taylor’s research. She said that sketches dashed out on her computer had the same fractal dimensions as a Pollock drip painting and might be authenticated as the real thing.

“The whole premise of ‘fractal expressionism’ is completely false,” Brown said. “Since our work came out, claims of fractals in Pollock’s work have largely disappeared from peer-reviewed physics journals. But it seems that the fractal zealots have managed to exert some influence in psychology.”