Phil Jones: ‘Contact NMS’s for raw data’

Over at Climate Audit there is renewed interest in data availability with McIntyre asking whether journals that don’t guarantee data archiving (The Holocene in this case) should be cited in IPCC reports.

It happened that yesterday Phil Jones of CRU gave a talk at KNMI in De Bilt, The Netherlands, where he also talked about availability of data, in this case the data behind the recently published Crutem4 and Hadcrut4 graphs. The talk itself was pretty neutral, just explaining what had been done to produce these two datasets. However at the end Jones made a statement that is relevant to the long lasting discussions about data availability:

For raw temperature data you have to contact the NMS’s.

NMS’s stands for National Meteorological Services, like the Met Office in the UK or KNMI in The Netherlands. The good news is, as we can also read in his latest Crutem4 paper, that CRU will make all data available. However the bad news is that these data have already been homogenized by the NMS’s and the original data are not available at the Crutem4 webpage.

Jones explained that he and Moberg concluded already in 2003 that the homogenisation of temperature data is best done at the NMS’s. So for Crutem4 whenever possible they used these homogenized data of the NMS’s directly, as can be seen at their webpage.

Now in itself this is a fair approach. If the NMS’s cannot figure out what happened with their stations in the past and how best to control for station moves and instrument changes who else can?

However this approach also generates new issues. The global temperature data play a crucial role in the IPCC reports and are used for detection and attribution studies and even for international policy targets (the two degree target for example). You would expect that a dataset that is so important is fully traceable: from raw data, adjustments to homogenised and finally gridded data.

We know that in some countries adjustments to raw data determine a large part of the trend. In New Zealand sceptics fight (see also here) with NIWA (the NMS of New Zealand) over the adjustments made to the raw data. The temperature trend in the raw data is only 0,3 degrees per century while the adjusted data show a trend of 1 degree per century. Jones uses the adjusted NIWA data in Crutem4.  Later this year the High Court in New Zealand will consider this case.

Jones seemed satisfied with the new situation. Anyone asking him for the raw data in the future will be referred to the NMS’s.

In his future answers to sceptics asking for data he can almost copy this paragraph of Joelle Gergis blowing off McIntyre when he requested some tree ring data from her:

This list allows any researcher who wants to access non publically available records to follow the appropriate protocol of contacting the original authors to obtain the necessary permission to use the record, take the time needed to process the data into a format suitable for data analysis etc, just as we have done. This is commonly referred to as ‘research’.

In the case of Crutem4 the raw data in many cases is also not publicly available and anyone interested has to contact each of the NMS’s trying to get these data. There is no guarantee at all that they will release the data.

Now although Jones was very obstructive to data requests from sceptics in the past, I don’t say that Jones is to blame for the current situation. At least he tried to get permission from the NMS’s to release the data as he promised in this Nature article in 2009, which was also covered in several Climate Audit posts (see here for example). In the Nature article Jones said:

“We’re trying to make them all available,” says Jones. “We’re consulting with all the meteorological services — about 150 members [of the World Meteorological Organization] — and will ask them if they are happy to release the data.” A spokesperson for the Met Office confirmed this, saying “we are happy for CRU to take the lead on this, as they are their data”.
But getting the all-clear from other nations won’t be without its challenges, says Jones, who estimates that it could take several months. In addition, some nations may object if they make money by selling their wind, sunshine and precipitation data.

In his new paper on Crutem4 he reports back on this attempt:

In November 2009, the UK Met Office wrote on our behalf to all NMSs to determine if we could release the versions of their monthly temperature series that we held. Of the about 180 letters, we received 62 positive replies, 5 negative replies, and the remainder did not reply.

These results are worrisome in itself. Almost two-thirds of the NMS’s didn’t even bother to answer to a request concerning one of the most important climate graphs in the world. For these countries Crutem4 uses the GHCN data.

From a policy perspective the current situation remains highly undesirable. Suppose a farmaceutical company uses processed data from different hospitals that participated in a clinical trial. Based on these processed data they conclude their drug is beneficial. However anyone who wants to see the original data should contact all the different hospitals himself. For most people this would be an unacceptable situation. It’s the responsibility of the farmaceutical company to collect and archive all the relevant data whenever they make a claim that a medicine is working or not.

In the case of Crutem4, I would say it’s CRU’s responsibility to make also the raw data available and the adjustments made to these data with a clear explanation for these adjustments. If the journals, the climate science community or IPCC doesn’t think so, whose responsibility it it? I am afraid their answer at this moment is that it is no one’s responsibility.

McIntyre sent a short note to IPCC lead author Valerie Masson-Delmote when he learned that Holocene is not demanding scientists to archive data:

As Dr Matthews says in his letter, the journal Holocene does not have an adequate or indeed any policy of requiring authors to archive data. Unless it establishes such a policy, I suggest that IPCC not permit the citation of articles from Holocene in the forthcoming assessment report.

Now to make the analogy, should IPCC rely on Crutem4 and Hadcrut4 in AR5? We all know they will. However the lack of availability of raw data and insight in adjustments by NMS’s remains an unsatisfactory issue.