One of the basic principles of blogging is to give credit when credit is due. I always try to mention the source of my information and most blogs do this using an acronym like h/t (hat tip). This week one of my own blog articles was not referred to as the source for another blog article on The Hockey Schtick. This blog then alerted WUWT? who brought the same news quite loudly and then also Steve McIntyre picked it up and Bishop Hill.
Now this is all fine, being given the credit for my blog post is not the most important thing in the world. However, a lot of noise in the blogosphere could have been saved if WUWT? had seen my original post. Hockey Schtick namely is writing about a “paper” presented at the EGU, which then became a “peer reviewed paper” on WUWT? (later corrected) which then irritated fellow Dutchman Victor Venema who happens to work on the same topic as the topic of my blog post, the homogenisation of temperature data.
A short history. In April I attended the EGU conference for the first time. I had lunch with Demetris Koutsoyiannis whom I spoke extensively in 2008 during the research phase of my Dutch book De Staat van het Klimaat. His work on long-term persistence is very interesting and also his analyses of climate models. I later also interviewed him for the Dutch magazine De Ingenieur. He invited me to attend the session in which his student Eva Steirou gave the now well-known presentation. He sent a link to the presentation shortly after the EGU but only this week I came about writing something about it.
It’s a pity that the posts on WUWT? and Climate Audit generated so much irritation, part of it caused by the misrepresentation of the status of the work (a presentation during a conference and not yet a peer reviewed paper). The topic itself is pretty important as the recent work of Venema shows.
Venema reacted on his blog. Some excerpts:
I have never seen an abstract that was rejected at EGU; rejection rates are in the order of a few percent and these are typically empty or double abstracts and are due to technical problems during submission. It would have been better if this abstract was send to the homogenisation session at EGU. This would have fitted much better to the topic and would have allowed for a more objective appraisal of this work. Had I been EGU convener of the homogenization session, I would probably have accepted the abstract, but given it a poster because the errors signal inexperience with the topic and I would have talked to them at the poster.
Now this reaction is a bit arrogant. I agree though that this session was the better place for the Steirou presentation. Steirou presented in the session that was organised by Koutsoyiannis himself. I will ask Koutsoyiannis to comment on this.
Now what are the major errors that Venema is talking about?
The first statement cited by Anthony Watts is from the slides:
of 67% of the weather stations examined, questionable adjustments were made to raw data that resulted in: “Increased positive trends, decreased negative trends, or changed negative trends to positive,” whereas “the expected proportions would be 1/2 (50%).”This is plainly wrong. You would not expect the proportions to be 1/2, inhomogeneities can be have a typical sign, e.g. when an entire network changes from North wall measurements (typical in the 19th century) to fully closed double-Louvre Stevenson screens in the gardens or from a screen that is open to the North or bottom (Wild, Pagoda, Montsouri) to a Stevenson screen, or from a Stevenson screen to an automatic weather stations as currently happens to save labor. The UHI produces a bias in the series, thus if you remove the UHI the homogenization adjustments would have a bias. There was a move from stations in cities to typically cooler airports that produces a bias and again this would make that you do not expect that the proportions are 1/2. Etc. See e.g. the papers by Böhm et al. (2001) Menne et al., 2010; Brunetti et al., 2006; Begert et al., 2005 or my recent posts on homogenization. Also the change from roof precipitation measurements to near ground precipitation measurements cause a bias (Auer et al., 2005).
Now these are some interesting arguments. Anthony Watts has frequently suggested that part of the warming could be due to the growth of airports. Now Venema is turning this argument around. The relocation of stations from cities to airports would actually lead to a cooling bias. I would be interested to see some examples.
Now the second problem that Venema has with the talk of Steirou is in this paragraph:
“homogenization practices used until today are mainly statistical, not well justified by experiments, and are rarely supported by metadata. It can be argued that they often lead to false results: natural features of hydroclimatic times series are regarded as errors and are adjusted.”
He defends homogenisation by writing:
The WMO recommendation is to first homogenize climate data using parallel measurements, but also to perform statistical homogenization as one is never sure that all inhomogeneities are recorded in the meta data of the station.
I was involved in the COST Action HOME, which just finished a study with a blind numerical experiment, which justified statistical homogenization and clearly showed that statistical homogenization improves the quality of temperature data (Venema et al., 2012). Many validation studies of homogenization algorithms have been published before (see references in Venema et al., 2012).
In a different approach, the statistical homogenization methods were also validated using breaks known in meta data in the Swiss (Kuglitsch, 2012). The size of the biased inhomogeneities is also in accordance with numerous experiments with parallel measurements; see Böhm et al. (2010) and Brunet et al. (2010) and references therein.
Definitely, it would be good to be able to homogenize data using parallel measurements more often. Unfortunately, it is often simply not possible to perform parallel measurements because the need for the change is not known several years in advance. Thus statistical homogenization will always be needed as well and as the validation studies show produces good results and makes the trends in temperature series more reliable.
I think based on my conversations with Koutsoyiannis that he and Venema more or less agree on this. As I said, Koutsoyiannis is very interested in long term persistence and he has found this kind of ‘behaviour’ in most if not all climatic time series. In the presentation they show that SNHT (the homogenisation protocol that they investigate) has the tendency to correct time series with long term persistence when no correction is needed. That is, the method detects an inhomogeneity that isn’t there, that just is part of the natural behaviour of a time series with long term persistence.
Now this is maybe the most interesting observation in the whole presentation and one that Venema should find very interesting as well. Much more work needs to be done as Koutsoyiannis this week already wrote me in an email.
Koutsoyiannis is always in favor of transparancy. Steven Mosher complained on one of the blogs that there is no station list available yet. I am convinced that Koutsoyiannis is willing to make this available and would have done so if a peer reviewed paper had been published.
The input of Venema who seems to be very active in the homogenisation community is very welcome of course. Hopefully this can lead to a more constructive exchange than so far is the case on the different blogs.
Taken out of context, the citation of my post may sound arrogant. All I wanted to argue was that there is no real review for EGU abstracts and certainly none for the slides presented at the conference. Anthony Watts giving this analysis the status of a “peer reviewed paper” misleads his readers, most of whom will not look at the slides and many of which do not have the background to judge the quality of the work. Even a single peer reviewed paper should be taken with a grain of salt and it is a pity that the media mainly… Lees verder »
Victor, thanks for commenting here. Koutsoyiannis is planning to write a guest post in a few days.
The relocation of stations from cities to airports would actually lead to a cooling bias. I would be interested to see some examples.
Didn’t Menne et al. show that poorly sited sites had a cool bias (see for example this article)?
Oh, and BTW, what exactly was arrogant in that first quote by Venema? Which part?
Neven, yes Fall et al :) showed a cool bias in Tmax at the poorly sited sites and a warm bias at the same sites in Tmin. The biases cancelled out and therefore the effect on Tmean was minimal. As far as I know both the Fall et al and the Menne et al paper do not explain why there is this cool bias.
Neven, the presentation was given by a student but you can be sure that Koutsoyiannis has checked the results. He is a very senior and experienced scientist. Of course you can disagree about the conclusions but to say based on the abstract that this is not interesting enough for a 15 minute talk is a bit arrogant. I think Victor was so annoyed about all the claim at WUWT? that it was “peer reviewed” that he was a bit too harsh for Steirou/Koutsoyiannis. Their approach to select 163 long and relatively complete series is interesting and deserves some serious discussion… Lees verder »
He is a very senior and experienced scientist. Experienced when it comes to homogenisation of temperature data? I thought he was an oceanographer or something. Their approach to select 163 long and relatively complete series is interesting and deserves some serious discussion as hopefully will take place here and elsewhere in the coming weeks. I’m not sure what this is about exactly, but my first impression is that we have some people who imply something is wrong with temperature data, but chances are that their lack of experience/knowledge is at the root of the perceived problem. This happens a lot… Lees verder »
You can see an example of cool bias from station relocation to airport on my site. http://stevemosher.wordpress.com/2012/03/22/metadata-dubuque-and-uhi/ It’s part of a longer study i’ve been doing on cooling stations in the US. using unadjusted data from berkeley earth I’m isolating those stations will long complete records that have statistically meaningful negative trends. many are in the US. looking further at them, a good number look to have discontinuties that correspond exactly with a move from a city to a airport. In the example I present you can see why the airport is cooler. I look at the “built area”, the… Lees verder »
Marcel. Menne has explained the bias in Diurnal trend found by fall et al. Simply listen to the presentation given at AMS ( I think christy or spenser did it) menne is in the audience and explains. As I recall it has to do with how the series are adjusted for MMTS discontinuity. I could ask matt, I dont think he will be writing anything up. Hmm, sorry I dont have a link. In any case the orginal work done by Leroy on mircosite bias only suggested a small bias (.1c) in mean. the principle effect is an increase in… Lees verder »
The bias in the US diurnal range arises mainly from the transition to MMTS from stevenson screens. This occurred over approximately 5 years in the late 1980s. It was confirmed by several side-by-side field studies. Maximum temperatures recorded by MMTS sensors are systematically cooler and minimum temperatures systematically warmer. It is also discussed in a recent paper (available outside a paywall) linked from http://surfacetemperatures.blogspot.com/2012/01/benchmarking-and-assessment-applied-to.html. This goes into some detail about the known and suspected issues with the US record and undertakes a network-wide benchmarking of the pairwise homogenization algorithm NCDC use. I find the discussion that has accrued across WUWT,… Lees verder »
That’s one cool website and effort, Peter! I didn’t even know it existed. BTW, I’ve been thinking about it some more. It is not arrogant at all for an expert to state that someone is making rookie errors. What is arrogant, is for a rookie to barge into an area that he/she has no expertise in and come up with all kinds of far-fetched conclusions, which then get propagated by people who should know better For instance, because they tried to do that themselves and then got burned, such as Watts and McKitrick. Acknowledge the errors due to inexperience, make… Lees verder »
Marcel, thanks for posting this and I look forward to reading Dr. Koutsoyiannis’ response. I’m slightly taken aback by Peter Thorne’s comment, though. We’d like to see more people create their own products tackling the issue of homogenization starting from the same raw data and running against consistent benchmarks. That way we may start to lift the fog of uncertainty. Really? The best way to lift the fog of uncertainty is to have lots of people using different ad hoc, sui generis statistical methods which are poorly understood instead of having a small number of people using well understood methods?… Lees verder »
“Experienced when it comes to homogenisation of temperature data? I thought he was an oceanographer or something.”
And your credentials in homogenisation of temperature data is exactly what?
“I’m not sure what this is about exactly, but my first impression is that we have some people who imply something is wrong with temperature data”
“Some people imply something” is hardly scientific, so what’s your expertise on this, really?
Spence_UK, 1. Define small number. My definition of small number is at least ten distinct methods so we can weed out the wheat from the chaff; run on the same raw data and consistent benchmarks. And why do expert statisticians have a monopoly on this? Why is your insight any less valuable a priori? I don’t get this argument. If we provide the robust testing environment of a single raw source and a set of consistent benchmarks you can post facto sort wheat from chaff and doing it twenty or thirty different ways you’d learn something about the strengths and… Lees verder »
Dear mr Spence, Spence: “I’m slightly taken aback by Peter Thorne’s comment, though.” … “Really? The best way to lift the fog of uncertainty is to have lots of people using different ad hoc, sui generis statistical methods which are poorly understood instead of having a small number of people using well understood methods?” Did you read anywhere that the methods were required to be ad hoc? Currently there is a limited number of people computing global mean temperature series. This is often wrongly used as sign of a conspiracy by people who do not realize that the trends found… Lees verder »
Peter, your response is absurd and completely departs from your original point. You claimed that you wanted to clear “fog and uncertainty”. And you wanted many different approaches. You now say you want to sort the “wheat from the chaff”. Lots of different people doing different things is just more chaff. Period. If I thought the instrumental record were important, I would want to look at the wheat. In detail. I’m surprised you have so much difficulty in understanding that. Sorting the wheat from the chaff is where you engage statisticians. And no, the code was not on-line when McIntyre… Lees verder »
Mr Spence, thank you for your nice words on my openness, but not even I am going to waste any more of my precious life time on answering you. Clear up your act.
And your credentials in homogenisation of temperature data is exactly what?
None, which is why I don’t do presentations at EGU, claiming that adjustments are responsible for half of the temperature rise of the past century or so, to be then happily quoted by Marcel, Hockey Schtick and Anthony Watts.
And even if I did, I wouldn’t immediately play the victim card if people told me I made some basic errors due to my lack of expertise.
Peter Throne writes “The bias in the US diurnal range arises mainly from the transition to MMTS from stevenson screens. This occurred over approximately 5 years in the late 1980s.” This is ridiculous, and indicates Mr. Thorne has no clue about what is actually going on with the US COOP Network. The conversion to MMTS is ongoing, well through the 90’s into the 2000’s and has actually accelerated after 2007 due to my work showing shoddy siting issues with Stevenson screens, as NOAA/NCDC reacted to it. In some cases, stations were closed because the MMTS placement would have been no… Lees verder »
Folks, Matt menne’s code was online from around the day he first published his paper, as I recall. It’s been a while since I downloaded it. I suppose if you look around you will find me pointing other folks to it. For folks who havent been following dr. Thorne’s project I highly recommend it to people. Many of us raised a stink ( and said some nasty things) about the state of climate data and it’s accessibility. Guess what? some folks saw past the nastiness and are actually working to improve the situation. A thank you would be in order.… Lees verder »
Menne’s code online? Where?
“Menne’s code”, the pair-wise homogenization algorithm, can be found under:
Many more homogenization algorithms can be found under:
Thank you. But that is not the currently used version, is it?
Also that is USHCN, which is different (I think?) from the algorithm for GHCN.
There is also a folder /pub/data/ghcn/v2/source/inhomog/
with GHCN v2 codes. Again this is not the version they currently use but it might be useful.
I had a look at the article describing GHCNv3. According to Lawrimore et al. (2011), the homogenization algorithm for GHCNv3 is the same as for USHCNv2: “The quality control algorithms are a combination of algorithms applied in version 2 with others adapted from those used to QC the GHCN-Daily data set [Durre et al., 2010] and to produce the USHCN-Monthly version 2 data [Menne et al., 2009].” Thus the link in my previous comment is the homogenization code you were looking for, which was also tested in the HOME blind benchmarking study. Sorry, for the confusion. Jay H. Lawrimore, Matthew… Lees verder »
The way I interpret that sentence is that GHCNv3 is a combination of some things from GHCNv2, some things from GHCN-Daily, and some things from USHCNv2.
What is different is the quality control (QC) (outliers). If you want to be sure just send Menne a polite e-mail.