Center Home Science Policy Photos University of Colorado spacer
CIRES CU
Location: > Prometheus: A Colossal Mistake Archives

September 05, 2006

A Colossal Mistake


Posted to Author: Pielke Jr., R. | Climate Change

"A colossal mistake" is how Jerry Mahlman describes in an article in the Chronicle of Higher Education the IPCC's decision to feature the so-called "hockey stick" in its Summary for Policy Makers.

I am somewhat surprised that discussion of the hockey stick continues to be about the he said-he said conflcit between the camps in Real Climate and Climate Audit, as described by the Chronicle. As much fun as the personalities and politics are, at some point it is probably worth discussing the broader significance of the hockey stick debate for how we think about scientific assessments and their contributions to the needs of decision makers.

Along these lines, here are some questions worth asking about the hockey stick experience that are not receiving enough attention:

1. The Chronicle article notes that the author(s) of the hockey stick article were responsible for its inclusion in the IPCC report. What are the issues associated with having people (not just limited to the HS, of course) involved with assessing their own work? This would never fly for, say, journal peer review or the drug approval process. Other experience suggests that science can be misrepresented when people review their own work in assesments for policy makers. What are the alternatives? What are the general lessons for the emapnelment of science assessment bodies?

2. The Chronicle article noted of the IPCC's presentation of the HS, "caveats faded from view when leaders of the IPCC boiled down the 994-page scientific report into a 20-page synopsis." Representing complex science in a sound bite necessarily requires simplification, and arguably in this case, and many others, over-simplification of the science. Such over-simplifcations are amplified by the media and used in politically convenient ways by policy advocates on all sides of an issue (as described by the Chronicle, e.g., in the use of the HS by the U.S. National Asssessment and the WSJ). Does it make sense to "boil down" science in a manner that inevitably leads to a mischaracterization of that science? What are the alternatives? This isn't the only example where communication has suffered in the IPCC due to oversimplification of the relevant science.

In my view, both questions raised above might be addressed, at least in part, if the IPCC (or any assessment) were to ask and aswer "So what for action?" of science findings when bringing them to policy makers. The inclusion of any information in a "summary for policy makers" or a "policy relevant" document is based on an assumption that such information is in fact relevant to those policy makers. Scientists should be explcit about why, exactly, they are including some information and not others and what the criteria of relevance actually is. In the case of the IPCC, criteria of relevance are out of sight, and as far as I know completely arbitrary.

Posted on September 5, 2006 07:55 PM

Comments

Roger -

I cannot agree that the criteria seem "completely arbirary."

They in fact seem attuned to two requirements. One, the need for some to obtain future funding by invoking fear. The second seems to be an expansion of the wish to literally broker power, as in further exploitation of the Oil for Food program. It might be called synergy.

There may indeed be cause for concern. If so, would it not be reasonable and responsible to determine the actual reality of climate before advocating a reduction in atmospheric emission of the base of the food chain? To do otherwise seems barbaric...

Posted by: Steve Hemphill [TypeKey Profile Page] at September 5, 2006 09:05 PM


You've misquoted Mahlman. He doesn't say it was a mistake to use it in the SPM. He says:

"The panel's decision to emphasize the hockey stick so strongly "was a colossal mistake, just as it was a mistake for the climate-science-writing press to amplify it," says Mr. Mahlman, the scientist who named the curve. "In other words, was that the smoking gun for global warming? It's not the smoking gun. That's the data we've had for the past 150 years, which is quite consistent with the expectation that the climate is continuing to warm.""

Which is not the same thing at all...

Posted by: William Connolley at September 6, 2006 02:20 AM


Interesting article. "Boiling down" would include removing the error bars, I think?

---snippet----
"Advocates on both sides of the climate-change debate at various times have misrepresented the results for their own purposes," says Mr. Mann.

One common tactic used by both sides is to disrobe the curve, taking off the error bars that indicate how much uncertainty there is in the figure. The airbrushing makes the hockey stick look much more definitive than it really is. In 2001 the federal government's U.S. National Assessment of the Potential Consequences of Climate Variability and Change showed the stripped version of the hockey stick as part of a report on how climate change would affect the United States. (In a caption, the report described only part of the uncertainty in the graph.)

Those who doubt that humans are warming the climate substantially also often print the curve without its error bars, and then disingenuously attack it for being inaccurate. The editorial page of The Wall Street Journal recently published a naked copy of the hockey stick as part of a critique of it.

"That was very misleading, in fact downright dishonest," says Texas A&M's Mr. North.
---end snippet----

Do advocates have an ethical responsibility to keep the error bars in the charts shown to policymakers, and to be accurate in explaining them?

Posted by: hank at September 6, 2006 09:09 AM


William-

Here is how I characterized Mahlman's words:

""A colossal mistake" is how Jerry Mahlman describes in an article in the Chronicle of Higher Education the IPCC's decision to feature the so-called "hockey stick" in its Summary for Policy Makers."

I never said that Mahlman said it was a mistake for the HS to be used in the SPM. Any thoughts on the somewhat more important parts of the post? ;-)

Thanks.

Posted by: Roger Pielke, Jr. [TypeKey Profile Page] at September 6, 2006 10:42 AM


For a second time (after posting here) William Connelley of Real Climate on his own blog has accused me of misrepresenting Monastersky's article.

http://scienceblogs.com/stoat/2006/09/chronicle_article_climate_scie.php

I think that Mr. Connelley is either sloppy and didn't in fact read the article or is purposely out to slander me. Here is how I responded on his blog:

--------
William-

Read my post again. I've misrepresented absolutely nothing.

You ask, who says the IPCC boiled down the science?

Why Richard Monastersky in the article that you are describing. Here is what he writes in the article,

"Mr. Mann and other authors who wrote the section on temperature changes described the millennial reconstructions and the uncertainties inherent in them. But those caveats faded from view when leaders of the IPCC boiled down the 994-page scientific report into a 20-page synopsis. . . "

You may disagree with what Monastersky wrote, but this is quite different than falsely accusing someone of misrepresentation. Ironic, really.
----------

Posted by: Roger Pielke, Jr. [TypeKey Profile Page] at September 6, 2006 02:30 PM


Dr Pielke

I find word parsing to be annoying but you have used the word 'slander' with regard to another poster to describe his characterization of what you have written. In the first sentence of your post you have stated that Mahlman called it a collosal mistake to feature the hockey stick in the SPM. The actual wording in the Monastersky piece is "The panel's decision to emphasize the hockey stick so strongly was a colossal mistake".

This comes on the heels of the paragraph in the Monastersky piece that states: "For many in the news media and the general public, that graph appeared to be the star witness in the IPCC's case that humans were warming the globe, when in fact that argument actually rested on a mass of other evidence unrelated to the curve."

It was the emphasis on the stick of which Mahlman was critical, not its inclusion (featuring) in the SPM. There is a difference between the panels decision to emphasize the stick in promoting the TAR and SPM (e.g. using it as Houghton's backdrop at a press conference) and including it in the SPM.

Posted by: James Hamilton at September 6, 2006 02:56 PM


James-

Can this point really be the most important response to my post? My goodness ...

The word "feature" does not mean "include". From the Oxford Enlish Dictionary, here is the definition:

http://www.askoxford.com/concise_oed/feature?view=uk

"feature

noun 1 a distinctive attribute or aspect. 2 a part of the face, such as the mouth, making a significant contribution to its overall appearance. 3 a newspaper or magazine article or a broadcast programme devoted to a particular topic. 4 (also feature film) a full-length film intended as the main item in a cinema programme.

verb 1 have as a feature. 2 have as an important actor or participant. 3 be a feature of; take an important part in. "

Lets get back to what really matters here, and please buy a Thesaurus;-)

Posted by: Roger Pielke, Jr. [TypeKey Profile Page] at September 6, 2006 03:19 PM


Finally, my last words on this, from thesaurus.com:

"Main Entry: feature
Part of Speech: verb
Definition: stress
Synonyms: accentuate, advertize, blaze*, emphasize, give prominence, headline*, italicize, make conspicuous, mark, play up*, point up*, present, promote, set off*, spotlight*, star, underline, underscore"

Enough said? Thanks.

Anyone want to take a crack at the substance? ;-)

Posted by: Roger Pielke, Jr. [TypeKey Profile Page] at September 6, 2006 03:31 PM


Roger

The point is important because you have decided to place emphasis upon it by taking exception to how it has been characterized. I am well aware that "feature" is not equivalent to "include", but because of the wording in your piece a reader might conclude that Mahlman was critical of the inclusion of the stick in the SPM when in fact that is not what was reported.

Given your skill with language I find it odd that you would write in such a manner.

Posted by: James Hamilton at September 6, 2006 03:38 PM


James-

With due respect, please read what you just wrote:

"I am well aware that "feature" is not equivalent to "include", but because of the wording in your piece a reader might conclude that Mahlman was critical of the inclusion of the stick in the SPM . . . "

So you are criticizing me because I have properly used the word "feature" in its appropriate context, but your are concerned that others might not understand the term, and are thus blaming me? Right.

When I wrote feature, I meant feature. See definition and synonyms above.

Sometimes I think people parse words and change the focus of blog discussions to the author rather than the substance because they don't want to address the more important points raised. Maybe this is what you are doing, maybe not. In any case, thanks for your views, but (this time, really!) I am done discussing the meaning of the word "feature." ;-)

Thanks.

Any comments on actual substance?

Posted by: Roger Pielke, Jr. [TypeKey Profile Page] at September 6, 2006 03:46 PM


The _concise_ OED? ... sputter ....

Words chosen to mislead are often technically correct. This is known as lawyering.

You want definition 5 from the real full OED.
-----snippet------

5. intr. To be a feature (in); to participate or play an (important) part in.
...
1965 Listener 9 Sept. 371/1 At the University Physiological Laboratory in Cambridge cats feature in some of the fundamental research.
...
1976 Times 21 May 4/2 Libraries and the youth service feature prominently in many of the local authority cuts. ...
Hence ˈfeaturing vbl. n.; in quot. concr.
---end snippet-----

Posted by: Hank Roberts at September 6, 2006 05:36 PM


Actually Roger, the quote says the report was boiled down not the science. :) But seriously, James, finds "it odd that you would write in such a manner". In my experience, it is precisely those with good language skills that do write this way--they use one word in the precise dictionary definition without further exposition that leads to others miss-interpreting what is written. (Go back to the discussion about insurability for another example)

Furthermore, everyone would be a lot better off, admitting that language is unclear. If your first response to William would have been, "Point taken, I used the word feature to mean emphasize. Sorry if I was unclear." then perhaps the thread would not have been side tracked by semantics, and William would have been less hostile on his blog.

Which actually leads to something relevant:
The IPPC report, needs to be carefully edited with the explicit intension of looking for ways it will be miss-interpreted.

Personally, if well respected scientists (like Steven Weinberg), who understand how science works, looked into climate change and wrote a report on what was and was not know, I would take it very seriously. Clearly climate scientists need to have major input into the IPCC report, but I'm not sure they have to write and edit it.

Posted by: Nosmo [TypeKey Profile Page] at September 6, 2006 05:57 PM


Roger

Thank you for clarifying that when you wrote feature you meant feature. I would be delighted to discuss the substance of your post, and it seems to me that the post is about the importance of communication and clarity in the subject area of climate change science. What is unfortunate is that you have not written your post in that manner. In your first sentence we are told that Mahlman states that it was a colossal mistake to feature the hockey stick in the SPM. There are two problems with this, first that is not what the article that you reference says, and second the SPM did not feature the hockey stick. As you have clearly shown feature has a specific meaning. The hockey stick was one of many figures included in the SPM, no special treatment or meaning was given to it.

Posted by: James Hamilton at September 6, 2006 06:12 PM


The similarities and differences between the words "feature", "include" and "emphasis" don't bother me. What bothers me is that the first paragraph in this blog article puts words into Jerry Mahlman's mouth:

"A colossal mistake" is how Jerry Mahlman describes in an article in the Chronicle of Higher Education the IPCC's decision to feature the so-called "hockey stick" in its Summary for Policy Makers.

This suggests to me that JM wrote the article. Certainly nothing suggests otherwise: the actual author of the article is not mentioned
in the blog article.

JM didn't write the article and he doesn't *describe* anything in it. He is *reported* in the article as saying that something was a colossal mistake. The words "was a colossal mistake" appear in quotation marks, so we must assume he actually uttered them. The subject of the sentence is filled in for us by Monatersky as "The panel's decision to emphasize the hockey stick so strongly". From my reading of the article, the things that might have been a colossal mistake were the facts that the "[SPM] authors chose to highlight only one of the long-term temperature reconstructions" and/or the fact that John Houghton had a graphic of the hockey stick in the background at a television appearance. But we won't know exactly what JM thinks was a colossal mistake until he explains it to us.

It seems to me you've been doing a bit of boiling down of your own here, Roger.

Posted by: Mark Hadfield at September 6, 2006 08:10 PM


Okay, I'll take a crack at point 1. It seems that IPCC authors may have been selected based on their expertise in a given area, and the intentions were probably better than the outcome suggests, at least in terms of the conflict-of-interest messiness. I would imagine that there was some recognition that the process was going to be a long and arduous one, so the thinking was, "why not have those who have written do the writing?"

I can't think of too many good alternatives. If you have non-experts doing the writing, don't you run the risk of misunderstanding and then misrepresenting the actual science (to at least the same degree as in the conflict-of-interest case)? And if you merely have *different* experts, then you just get the same problem, in a new flavor.

One radical idea would be to have the IPCC funnel money into graduate programs to fund doctoral-level literature reviews on each of the relevant topics. It would be win-win. Everyone knows that graduate students get way too in-depth with literature reviews, and so the chance of them missing something would be small. Students would be happy to compete for something as prestigious as IPCC authorship, and, being so early in their careers, they are all but guaranteed to have no prior investment in whichever topics they end up reviewing and writing about. The IPCC reviewers could be drawn from a pool of largely reading-but-not-writing climate scientists; for example, the sorts of folks who show great promise but then get sidetracked by 15 years of administrative appointments just after getting tenure.

Just a thought. :)

Posted by: Kenneth Blumenfeld [TypeKey Profile Page] at September 6, 2006 11:52 PM


Hi Mr Peelke. Since responding on our own blogs in fashion, I've done that...

Posted by: William Connolley at September 7, 2006 03:20 AM


Hi Roger,

This is comment is offered in a constructive spirit, not as joining the dog-pile. One ambiguity you may not have thought of in the word "feature" comes from the context. The SPM material is based on the entire WG1, and therefore any content used in the SPM is de facto featured. And in fact when I read your post this is exactly how I took it, featuring the HS simply meant including it in the SPM.

One bit of good advice that may have slipped by due to the antagonistic tone of the thread was to simply offer clarification at the very first sign of misinterpretation.

Cheers.

Posted by: coby [TypeKey Profile Page] at September 7, 2006 03:43 PM




Sitemap | Contact | Find us | Email webmaster