Data Science,Climate and satellites Consultant John J Bates, who blew the whistle to the Mail on Sunday
The Mail on Sunday today reveals astonishing evidence that the organisation that is the world’s leading source of climate data rushed to publish a landmark paper that exaggerated global warming and was timed to influence the historic Paris Agreement on climate change.
A high-level whistleblower has told this newspaper that America’s National Oceanic and Atmospheric Administration (NOAA) breached its own rules on scientific integrity when it published the sensational but flawed report, aimed at making the maximum possible impact on world leaders including Barack Obama and David Cameron at the UN climate conference in Paris in 2015.
The report claimed that the ‘pause’ or ‘slowdown’ in global warming in the period since 1998 – revealed by UN scientists in 2013 – never existed, and that world temperatures had been rising faster than scientists expected. Launched by NOAA with a public relations fanfare, it was splashed across the world’s media, and cited repeatedly by politicians and policy makers.
But the whistleblower, Dr John Bates, a top NOAA scientist with an impeccable reputation, has shown The Mail on Sunday irrefutable evidence that the paper was based on misleading, ‘unverified’ data.
It was never subjected to NOAA’s rigorous internal evaluation process – which Dr Bates devised.
His vehement objections to the publication of the faulty data were overridden by his NOAA superiors in what he describes as a ‘blatant attempt to intensify the impact’ of what became known as the Pausebuster paper.
His disclosures are likely to stiffen President Trump’s determination to enact his pledges to reverse his predecessor’s ‘green’ policies, and to withdraw from the Paris deal – so triggering an intense political row.
Canadian Prime Minister Justin Trudeau, Microsoft CEO Bill Gates, US President Barack Obama, French President Francois Hollande and Indian Prime Minister Narendra Modi at the world climate change conference
The PM, the Prince and 'the pause': David Cameron and Prince Charles attended the historic 2015 Paris climate change conference with 150 world leaders. Cameron committed Britain to an EU-Wide emission cut as a result. And Charles, writing in this paper last month, stated there was no pause in global warming, influenced by the flawed NOAA paper that made this claim
In an exclusive interview, Dr Bates accused the lead author of the paper, Thomas Karl, who was until last year director of the NOAA section that produces climate data – the National Centers for Environmental Information (NCEI) – of ‘insisting on decisions and scientific choices that maximised warming and minimised documentation… in an effort to discredit the notion of a global warming pause, rushed so that he could time publication to influence national and international deliberations on climate policy’.
Dr Bates was one of two Principal Scientists at NCEI, based in Asheville, North Carolina.
A blatant attempt to intensify paper's impact
Official delegations from America, Britain and the EU were strongly influenced by the flawed NOAA study as they hammered out the Paris Agreement – and committed advanced nations to sweeping reductions in their use of fossil fuel and to spending £80 billion every year on new, climate-related aid projects.
The scandal has disturbing echoes of the ‘Climategate’ affair which broke shortly before the UN climate summit in 2009, when the leak of thousands of emails between climate scientists suggested they had manipulated and hidden data. Some were British experts at the influential Climatic Research Unit at the University of East Anglia.
Data published by NOAA, the world’s top climate data agency, claimed global warming was worse than previously thought. The information was published to coincide with the Paris climate change conference in 2015, where world leaders agreed that...
$100bn be given every year in extra ‘climate-related’ aid to the developing world by rich nations
2 degrees C be set as the limit for maximum temperature rise above pre-industrial times
40% of CO2 emissions would be cut across the EU by 2030
£320bn… what the UK’s pledges will cost our economy by 2030
NOAA’s 2015 ‘Pausebuster’ paper was based on two new temperature sets of data – one containing measurements of temperatures at the planet’s surface on land, the other at the surface of the seas.
Both datasets were flawed. This newspaper has learnt that NOAA has now decided that the sea dataset will have to be replaced and substantially revised just 18 months after it was issued, because it used unreliable methods which overstated the speed of warming. The revised data will show both lower temperatures and a slower rate in the recent warming trend.
The land temperature dataset used by the study was afflicted by devastating bugs in its software that rendered its findings ‘unstable’.
The paper relied on a preliminary, ‘alpha’ version of the data which was never approved or verified.
A final, approved version has still not been issued. None of the data on which the paper was based was properly ‘archived’ – a mandatory requirement meant to ensure that raw data and the software used to process it is accessible to other scientists, so they can verify NOAA results.
Dr Bates retired from NOAA at the end of last year after a 40-year career in meteorology and climate science. As recently as 2014, the Obama administration awarded him a special gold medal for his work in setting new, supposedly binding standards ‘to produce and preserve climate data records’.
Yet when it came to the paper timed to influence the Paris conference, Dr Bates said, these standards were flagrantly ignored.
The paper was published in June 2015 by the journal Science. Entitled ‘Possible artifacts of data biases in the recent global surface warming hiatus’, the document said the widely reported ‘pause’ or ‘slowdown’ was a myth.
Less than two years earlier, a blockbuster report from the UN Intergovernmental Panel on Climate Change (IPCC), which drew on the work of hundreds of scientists around the world, had found ‘a much smaller increasing trend over the past 15 years 1998-2012 than over the past 30 to 60 years’. Explaining the pause became a key issue for climate science. It was seized on by global warming sceptics, because the level of CO2 in the atmosphere had continued to rise.
NOAA’s climate boss Thomas Karl, below left, had a hotline to the White House, through his long association with President Obama’s science adviser, John Holdren.
Karl’s ‘Pausebuster’ paper was hugely influential in dictating the world agreement in Paris and sweeping US emissions cuts. President Trump, above right, has pledged to scrap both policies – triggering furious claims by Democrats he is a climate ‘denier’ and ‘anti-science’.
Thanks to today’s MoS story, NOAA is set to face an inquiry by the Republican-led House science committee.
Some scientists argued that the existence of the pause meant the world’s climate is less sensitive to greenhouse gases than previously thought, so that future warming would be slower. One of them, Professor Judith Curry, then head of climate science at the Georgia Institute of Technology, said it suggested that computer models used to project future warming were ‘running too hot’.
However, the Pausebuster paper said while the rate of global warming from 1950 to 1999 was 0.113C per decade, the rate from 2000 to 2014 was actually higher, at 0.116C per decade. The IPCC’s claim about the pause, it concluded, ‘was no longer valid’.
The impact was huge and lasting. On publication day, the BBC said the pause in global warming was ‘an illusion caused by inaccurate data’.
One American magazine described the paper as a ‘science bomb’ dropped on sceptics.
Its impact could be seen in this newspaper last month when, writing to launch his Ladybird book about climate change, Prince Charles stated baldly: ‘There isn’t a pause… it is hard to reject the facts on the basis of the evidence.’
Data changed to make the sea appear warmer
The sea dataset used by Thomas Karl and his colleagues – known as Extended Reconstructed Sea Surface Temperatures version 4, or ERSSTv4, tripled the warming trend over the sea during the years 2000 to 2014 from just 0.036C per decade – as stated in version 3 – to 0.099C per decade. Individual measurements in some parts of the globe had increased by about 0.1C and this resulted in the dramatic increase of the overall global trend published by the Pausebuster paper. But Dr Bates said this increase in temperatures was achieved by dubious means. Its key error was an upwards ‘adjustment’ of readings from fixed and floating buoys, which are generally reliable, to bring them into line with readings from a much more doubtful source – water taken in by ships. This, Dr Bates explained, has long been known to be questionable: ships are themselves sources of heat, readings will vary from ship to ship, and the depth of water intake will vary according to how heavily a ship is laden – so affecting temperature readings.
Dr Bates said: ‘They had good data from buoys. And they threw it out and “corrected” it by using the bad data from ships. You never change good data to agree with bad, but that’s what they did – so as to make it look as if the sea was warmer.’
ERSSTv4 ‘adjusted’ buoy readings up by 0.12C. It also ignored data from satellites that measure the temperature of the lower atmosphere, which are also considered reliable. Dr Bates said he gave the paper’s co-authors ‘a hard time’ about this, ‘and they never really justified what they were doing.’
Now, some of those same authors have produced the pending, revised new version of the sea dataset – ERSSTv5. A draft of a document that explains the methods used to generate version 5, and which has been seen by this newspaper, indicates the new version will reverse the flaws in version 4, changing the buoy adjustments and including some satellite data and measurements from a special high-tech floating buoy network known as Argo. As a result, it is certain to show reductions in both absolute temperatures and recent global warming.
The second dataset used by the Pausebuster paper was a new version of NOAA’s land records, known as the Global Historical Climatology Network (GHCN), an analysis over time of temperature readings from about 4,000 weather stations spread across the globe.
The unstable land readings: Scientists at NOAA used land temperature data from 4,000 weather stations (pictured, one in Montana, USA). But the software used to process the figures was bug-ridden and unstable. NOAA also used 'unverified' data that was not tested or approved. This data as merged with unreliable sea surface temperatures
The 'adjusted' sea readings: Average sea surface temperatures are calculated using data from weather buoys (pictured). But NOAA ‘adjusted’ these figures upwards to fit with data taken from ships – which is notoriously unreliable. This exaggerated the warming rate, allowing NOAA to claim in the paper dubbed the ‘Pausebuster’ that there was no ‘pause’
This new version found past temperatures had been cooler than previously thought, and recent ones higher – so that the warming trend looked steeper. For the period 2000 to 2014, the paper increased the rate of warming on land from 0.15C to 0.164C per decade.
In the weeks after the Pausebuster paper was published, Dr Bates conducted a one-man investigation into this. His findings were extraordinary. Not only had Mr Karl and his colleagues failed to follow any of the formal procedures required to approve and archive their data, they had used a ‘highly experimental early run’ of a programme that tried to combine two previously separate sets of records.
This had undergone the critical process known as ‘pairwise homogeneity adjustment’, a method of spotting ‘rogue’ readings from individual weather stations by comparing them with others nearby.
However, this process requires extensive, careful checking which was only just beginning, so that the data was not ready for operational use. Now, more than two years after the Pausebuster paper was submitted to Science, the new version of GHCN is still undergoing testing.
Moreover, the GHCN software was afflicted by serious bugs. They caused it to become so ‘unstable’ that every time the raw temperature readings were run through the computer, it gave different results. The new, bug-free version of GHCN has still not been approved and issued. It is, Dr Bates said, ‘significantly different’ from that used by Mr Karl and his co-authors.
Dr Bates revealed that the failure to archive and make available fully documented data not only violated NOAA rules, but also those set down by Science. Before he retired last year, he continued to raise the issue internally. Then came the final bombshell. Dr Bates said: ‘I learned that the computer used to process the software had suffered a complete failure.’
The reason for the failure is unknown, but it means the Pausebuster paper can never be replicated or verified by other scientists.
The flawed conclusions of the Pausebuster paper were widely discussed by delegates at the Paris climate change conference. Mr Karl had a longstanding relationship with President Obama’s chief science adviser, John Holdren, giving him a hotline to the White House.
The misleading 'pausebuster chart': The red line shows the current NOAA world temperature graph – which relies on the ‘adjusted’ and unreliable sea data cited in the flawed ‘Pausebuster’ paper. The blue line is the UK Met Office’s independently tested and verified ‘HadCRUT4’ record – showing lower monthly readings and a shallower recent warming trend
They were forced to correct it: 18 months after the ‘Pausebuster’ paper was published in time for the 2015 Paris climate change conference, NOAA’s flawed sea temperature dataset is to be replaced. The new version will remedy its failings, and use data from both buoys and satellites (pictured) – which some say is the best data of all. The new version will show both lower temperatures and a lower warming trend since 2000
Mr Holdren was also a strong advocate of robust measures to curb emissions. Britain’s then Prime Minister David Cameron claimed at the conference that ‘97 per cent of scientists say climate change is urgent and man-made and must be addressed’ and called for ‘a binding legal mechanism’ to ensure the world got no more than 2C warmer than in pre-industrial times.
President Obama stressed his Clean Power Plan at the conference, which mandates American power stations to make big emissions cuts.
President Trump has since pledged he will scrap it, and to withdraw from the Paris Agreement.
Whatever takes its place, said Dr Bates, ‘there needs to be a fundamental change to the way NOAA deals with data so that people can check and validate scientific results. I’m hoping that this will be a wake-up call to the climate science community – a signal that we have to put in place processes to make sure this kind of crap doesn’t happen again.
‘I want to address the systemic problems. I don’t care whether modifications to the datasets make temperatures go up or down. But I want the observations to speak for themselves, and for that, there needs to be a new emphasis that ethical standards must be maintained.’
He said he decided to speak out after seeing reports in papers including the Washington Post and Forbes magazine claiming that scientists feared the Trump administration would fail to maintain and preserve NOAA’s climate records.
Dr Bates said: ‘How ironic it is that there is now this idea that Trump is going to trash climate data, when key decisions were earlier taken by someone whose responsibility it was to maintain its integrity – and failed.’
NOAA not only failed, but it effectively mounted a cover-up when challenged over its data. After the paper was published, the US House of Representatives Science Committee launched an inquiry into its Pausebuster claims. NOAA refused to comply with subpoenas demanding internal emails from the committee chairman, the Texas Republican Lamar Smith, and falsely claimed that no one had raised concerns about the paper internally.
Last night Mr Smith thanked Dr Bates ‘for courageously stepping forward to tell the truth about NOAA’s senior officials playing fast and loose with the data in order to meet a politically predetermined conclusion’. He added: ‘The Karl study used flawed data, was rushed to publication in an effort to support the President’s climate change agenda, and ignored NOAA’s own standards for scientific study.’
Professor Curry, now the president of the Climate Forecast Applications Network, said last night: ‘Large adjustments to the raw data, and substantial changes in successive dataset versions, imply substantial uncertainties.’
It was time, she said, that politicians and policymakers took these uncertainties on board.
Last night Mr Karl admitted the data had not been archived when the paper was published. Asked why he had not waited, he said: ‘John Bates is talking about a formal process that takes a long time.’ He denied he was rushing to get the paper out in time for Paris, saying: ‘There was no discussion about Paris.’
They played fast and loose with the figures
He also admitted that the final, approved and ‘operational’ edition of the GHCN land data would be ‘different’ from that used in the paper’.
As for the ERSSTv4 sea dataset, he claimed it was other records – such as the UK Met Office’s – which were wrong, because they understated global warming and were ‘biased too low’. Jeremy Berg, Science’s editor-in-chief, said: ‘Dr Bates raises some serious concerns. After the results of any appropriate investigations… we will consider our options.’ He said that ‘could include retracting that paper’.NOAA declined to comment.
Helena Christensen addresses demonstrators in the center of Copenhagen on climate change
Dr John Bates’s disclosures about the manipulation of data behind the ‘Pausebuster’ paper is the biggest scientific scandal since ‘Climategate’ in 2009 when, as this paper reported, thousands of leaked emails revealed scientists were trying to block access to data, and using a ‘trick’ to conceal embarrassing flaws in their claims about global warming.
Both scandals suggest a lack of transparency and, according to Dr Bates, a failure to observe proper ethical standards.
Because of NOAA ’s failure to ‘archive’ data used in the paper, its results can never be verified.
Like Climategate, this scandal is likely to reverberate around the world, and reignite some of science’s most hotly contested debates.
Left, blowing up the graph show is disappears in 1961 artfully hidden behind the other colours. Right, the reason? Because this is what it shows after 1961, a dramatic decline in global temperatures
Has there been an unexpected pause in global warming? If so, is the world less sensitive to carbon dioxide than climate computer models suggest?
And does this mean that truly dangerous global warming is less imminent, and that politicians’ repeated calls for immediate ‘urgent action’ to curb emissions are exaggerated?