Blog discussions, conference presentations and peer review

A blog post earlier this week about an EGU presentation of Eva Steirou (a researcher in the group of Demetris Koutsoyiannis) on temperature data homogenisation created some stir in the blogosphere after Watts Up With That? and Climate Audit paid attention to it. Koutsoyiannis has now written a guest blog to give some first reactions.

Guest post by Demetris Koutsoyiannis

I believe that science blogs have offered a very powerful means in scientific dialogue, which is a prerequisite of scientific progress. I have very positive personal experiences. In 2008, a poster paper in EGU, “Assessment of the reliability of climate predictions based on comparisons with historical time series”,  was widely discussed at blogs and this was very useful to improve it and produce a peer-reviewed paper, “On the credibility of climate predictions” , which again was widely discussed at blogs. In the follow up paper, “A comparison of local and aggregated climate model outputs with observed data” we incorporated replies to the critiques we have seen in lots of blog comments.

In comparison, the formal peer reviewed system, while in principle encourages post-publication discussion through formal Commentaries and Replies, was able to offer us a single Commentary for the second paper (none for the former), which also gave us the opportunity to clarify our methodology (and feel safer about it) in our reply, “Scientific dialogue on climate: is it giving black eyes or opening closed eyes? Reply to “A black eye for the Hydrological Sciences Journal” by D. Huard”.

In another example, my 2012 paper “Clausius-Clapeyron equation and saturation vapour pressure: simple theory reconciled with practice” received one formal Commentary and was marginally (and off-topic) discussed at a blog. This discussion annoyed me for its eristic tone, but also prompted me to revisit my calculations and spot an error, for which I published a Corrigendum (acknowledging the blog discussion).

Both these examples show that blog discussions can be complementary and synergistic to the more traditional scientific transactions, conference presentations and journal papers, and also extremely useful. Although we do not call such discussions peer reviews, they can provide very useful review comments by peers. Of course, blog discussions are not replacements of formal peer reviews and have their own weaknesses (as all human transactions). A first one is that they are too selective. You must be very lucky and your paper very topical to trigger discussions at blogs.

Eva Steirou’s presentation in the last EGU conference, coauthored by me, entitled “Investigation of methods for hydroclimatic data homogenization” was one of the lucky ones as it became the subject of several blog posts and extensive discussions in the last couple of days. Its fortune was clearly made by Marcel Crok, through his post of 16 July. I must note that the presentation was online since April and no one paid particular attention—and I do not remember to have alerted anyone about it. Throughout June it received 10 visits while in the last few days it received 2500 visits, about half of which come from WUWT. I am grateful to all who discussed it and particularly to Marcel who brought it into light, and I regret that subsequent bloggers did not give him the due credit.

In his second post, Marcel gave the full story of the presentation in an accurate manner. His first post was equally accurate. He knew the details because he attended the EGU session where it was presented.  Evidently, it was not a peer-reviewed paper but a conference presentation (although it is not infrequent to call “paper” a conference presentation or a poster, we should not use the term “peer-reviewed” for conferences that select papers reviewing only the abstracts).

But we plan to produce a peer-reviewed paper (unless we have made a fatal error, which we hope not) and we keep studying the topic more thoroughly. That is why we think that we are lucky to have received all these comments from the blogs. I did not have the time to read them all, let alone to assimilate them, so I will not provide replies here. From first glance I find most of them very useful, whether they are positive or negative.

In his second post, Marcel mentions two of the commenters, Victor Venema and Steven Mosher, and summarizes some points of their critiques. About Steven’s complaint that there is no station list, I think there is no need to complain because the list is contained in the last pages of Eva’s diploma thesis. Although the thesis is in Greek, the list and the accompanying graphs are in English. It’s a pity not to have clarified that, as we, all members of the team Itia, are fans of open information.

About Victor’s critique I had the impression that we had resolved several issues because he emailed me and we had a long exchange (which BTW I authorized him to publicize if he wished). Therefore, I was somewhat surprised to see a rather harsh post in his blog. I will not reply to the essence of his points. We will study them in detail—and I have already told this to him. But I note, when he says that our (prior) estimates of the expected proportions of data corrected upward and corrected downward should be 1/2 is “plainly wrong”, he would be more convincing if he gave his own estimate, rather than telling there are biases. Also, I am surprised to see that he criticizes our statement that homogenization practices often lead to false results. Is he so sure that they always give correct results?

But of course these are scientific disagreements and it is fine if scientists disagree. Some arguments, though, fall into other categories, such as arguments from authority or ad hominem. Well, I am familiar with such arguments within scientific transactions, formal (paper reviews) or informal (in blogs), but they are always saddening and also make it necessary to refer to personal information in order to reply.

Victor has written in several blogs, as well as in his own original blog post, that Eva is a student and that I am new to homogenization, and emphasized that I was the convener of the session, implying that the presentation was kind of approved by myself. I asked him to correct his downgrading of Eva who is a graduate, professional engineer, and write the names of the co-conveners of the session, because I was not alone. He did not exactly do what I asked, but he was kind to change the phrasing in his post.

The following phrase from Victor’s blog post, also quoted by Marcel, is really interesting:

Had I been EGU convener of the homogenization session, I would probably have accepted the abstract, but given it a poster because the errors signal inexperience with the topic and I would have talked to them at the poster.

I do not wish to dispute that Victor is an expert on the topic. I really think he would be an excellent convener of the homogenization session. I would be happy if he chose my abstract to be presented as a poster. Most of my papers, including the one in the top of this post, have been presented as posters. I would fully respect his right to “probably accept” and of course “probably reject” my abstract. Even, I recognize his right to say or imply (in the above phrase as well as in his first email inviting me to “discuss these matters with the experts”) that I am not an expert.  Actually, I take this as a compliment as I prefer to view myself more like a student than an expert. I have no more respect to experts than I have to students and often collaboration is more constructive and fascinating with those who feel students than those who declare themselves to be experts. I do not care (rather I enjoy) being “new to homogenization” or to everything else—except “new to thinking” (to “skeptesthai”, if you allow me to use the Greek word for “thinking”).

But it may not be fair for him to charge me with inexperience. I think I have some experience with hydrometeorological data management, from acquisition to processing:

1. As an engineering hydrologist I have participated in several professional studies of the design or management of several projects and the production of engineering reports. In hydrological engineering reports, data homogenization and further processing to extract design quantities are routine procedures.
2. Twenty years ago, I have been the inspirer and project director of a Greek national-level project “Hydroscope: Creation of a National Databank for Hydrological and Meteorological Information”. Among other things, this produced several reports (in Greek; accessible via the above link) and software including methodologies for data homogenization.
3. Almost 20 years ago, Itia installed and operated, for the first time in Greece, a new-generation meteorological station, which gave free data on the Internet in real time, even before the shaping up of the World Wide Web (in the beginning using the Unix “finger” utility). We continue to provide this information, both in real time and historical.
4. Itia has produced several software products, again open and free, among which some are for hydrometeorological data processing. Our latest one, Hydrognomon, has been downloaded and used by colleagues internationally, despite its manual being only in Greek (we are now translating it to English).
5. Itia (with my colleague Nikos Mamassis as project director) has produced and maintains the software used by most (if not all) hydrological and meteorological organizations of Greece to store and process their data.
6. Recently, Itia (in particular, my colleagues Antonis Christofides and Stefanos Kozanis) has created the openmeteo.org, a public database with free hydro-meteorological data which offers anyone the possibility to upload and store his own data.

Thus, I think I have some experience with data and I have also some experience with stochastics. Given these two, I often have doubts whether what we are doing is correct or not. That is why I directed Eva to make this research, supervised by myself. I understand doubts may be unwelcome for some who prefer dogma over science, but for me it is a way of living—and I have no problem to tell my doubts publicly. Even if they will eventually prove to be groundless, they may be constructive in strengthening their subject.

Clearly, our research is not finished, and what we presented in EGU is our first communication in public. Despite being criticized by Victor for the opposite, I believe we chose an appropriate session to submit our work—and with this I do not of course mean that “his” homogenization session would be inappropriate. Victor may wish to see that Böhm himself, whom he heavily cites in his post, was invited and presented his work, related to a review of two decades of experience in the field of homogenizing instrumental climate data, in our same session two years ago (ignore the slightly changed session title this year). I hope the session will be organized again in next years and I will propose to the colleagues in the EGU HS Subdivision Precipitation and Climate who will be convening this session to also invite Victor to present his work, if he wishes of course. Our subdivision is very active, democratic, open and open-minded.