Tepid support from BoMGuest author | January 9, 2011
The Australian Bureau of Meteorology (BoM), like the National Institute of Water and Atmospheric Research (NIWA) in New Zealand, is an advisory group of Government scientists responsible for the compilation and maintenance of official temperature records.
After NIWA scientists rewrote the official NZ temperature record — the Seven-Station Series — during 2010, their ‘Review Report’ included a letter of support from the BoM. This was seen as necessary, as NIWA’s credibility had been somewhat strained by its lengthy (and ultimately futile) defence of the old record.
Some are critical of the selection of the Bureau to review work by NIWA, as both groups have been widely criticised (especially in the blogosphere) for applying the same biases and questionable adjustment methods. See, for example, Australian Temperatures in cities adjusted up by 70%!? at Jo Nova’s blog.
As climate archivists, both agencies are extensively engaged in the work of the IPCC; and both are firmly of the school of thought led by Professor Phil Jones, head of the Climatic Research Unit at the University of East Anglia (UEA).
The NZ Climate Science Coalition wrote to Science Minister Wayne Mapp, suggesting the appointment of two genuinely independent reviewers, and putting forward names of highly regarded scientists and statisticians. The letter pointed out:
“Appointment of two additional reviewers need not create any delays or distractions. Irreconcilable differences are unlikely, once all the data and metadata are disclosed, but, if they do arise, it is far better that the differing views are transparent and defined. The reviewers’ role is only advisory, and NIWA will retain full control of both the process and outcomes, which must, however, include the publication in full of the reviewers’ report.”
This suggestion was not taken up.
Minister Mapp advised Parliament last February that the NZ Temperature Record was to be reviewed by Brett Mullan, and then peer-reviewed internally by David Wratt and James Renwick and externally by the Bureau. The work would then be submitted “as a paper to a scientific journal where it would be subject to the normal independent peer-review process.”
Terms of peer review not disclosed
A “normal independent peer-review” sought by a journal editor is reasonably well established by convention. In the best instances, the data and metadata are also made available so that the scientific or statistical work is reproducible by interested outside experts. But what are non-independent “internal and external peer-reviews”? They are not defined, and may mean whatever the user intends them to mean.
An analogous guide is provided by reviews of historical financial information, as laid down by International Standard ISAE 3000. Clearly, a peer-review does not reach the stringency of an “Audit” where the reviewer certifies that the record presents a “true and fair view”. ISAE 3000 requires the terms of any “Assurance Engagement” to be recorded in writing, defining such issues as evidence sources and materiality levels, and demands “an attitude of professional scepticism.”
The engagement letter between NIWA and the BoM remains confidential. But the role actually carried out by the BoM appears to be the ISAE 3000 equivalent of a “Limited Assurance” — the lowest standard that a member of the NZ Institute of Chartered Accountants is ethically permitted to undertake.
The BoM makes clear that its review is not a reanalysis of the record and is constrained to the “level of information supplied”. The Bureau’s formal assurance is stated in a single sentence:
“In general, the evidence provided by NIWA supports the homogeneity corrections that have been applied to the temperature record to create the ‘seven station’ series.”
Paraphrase: The material which NIWA chose to give us was supportive of their conclusions – or – NIWA offered us no evidence contradicting their own opinions.
No approval for methodology
But this needs to be read in the context of the many express caveats. NIWA’s data and methodology are assumed to be “an accurate representation of the actual analysis undertaken”. The Bureau was “not in a position to question all of the underlying analyses and data that have contributed to the final results.” Above all, there was no attempt to “independently determine the sensitivity of … the analysis methodology.”
The final sentence of the BoM letter has a less constrained assurance:
“It is also clear that a number of significant adjustments (as identified by NIWA in the reports) are clearly required for the raw/composite station series owing to inhomogeneities which would otherwise artificially bias results.”
But this is not contentious. Many inhomogeneities certainly exist, but the question is whether the NIWA adjustments have made the record better or worse? And with what degree of certainty?
BoM had limited access to data
The Bureau reveals that it did not have “full access to the raw and modified temperature data and metadata.” This is contrary both to assurances offered by Minister Mapp and the express injunction stated by the UK House of Commons in its recent report on “Climategate”:
“Climate science is a matter of great importance and the quality of the science should be irreproachable. We therefore consider that climate scientists should take steps to make available all the data that support their work (including raw data) and full methodological workings (including the computer [code]). Had both been available, many of the problems at UEA could have been avoided."
All in all, the Bureau's letter offers little or no assurance that NIWA scientists have got it right this time. At most, it offers only tepid support to NIWA’s Review Report.