A oldie but a goodie considering the recent adjustments to raw data they make. Some good valid points made from Jo Nova
BOM admits temperature adjustments are secret
The
BOM Technical Advisory Forum report is out. Finally there is the black and white admission that the BOM “adjusted” dataset cannot be replicated independently, has not been replicated by any other group, and even more so, that the BOM will not provide enough information for anyone who wants to try.
As we have said all along, the all new ACORN wonder-data was not created with the scientific method.
Adjustments to Australian temperature data were done with a black box mystery technique that only the sacred guild at the BOM are allowed to know. Far from being published and peer reviewed, the methods are secret, and rely on — in their own words — a “supervised process” of “expert judgment” and “operator intervention”. In other words, a BOM employee makes their best guess, ruling in or out the “optimal” choices, making assumptions that are not documented anywhere.
It’s a “trust us” approach. Would we let an ASX company audit their own books? Would you buy shares in such a company, or let it inform national policy on billion dollar schemes?
Here is the entire section on replication from page 9 and 10 (below). This is what any semi-skilled PR operative would write if they were trying to justify keeping their methods secret. My translations included.
Ability to reproduce findings
Only BOM staff are smart enough to understand “scientifically complex” thermometers (this is something that engineers, astrophysicists, aeronautics experts and physicists would not be able to do, is that what they are saying?):
The Forum considers that the algorithms and processes used for adjustment and homogenisation are scientifically complex and a reasonably high level of expertise is needed to attempt analysis of the ACORN-SAT data. For this reason the Forum had some queries about the ability to reproduce findings by both experts and members of the public.
Thinly veiled put-down coming:
It would be useful for the Bureau to provide advice about the necessary level of end-user expertise (notwithstanding a likely tendency for end-users to feel qualified to attempt such an analysis).
It might be more “useful” if the BOM staff provided their personal exam results in fluid dynamics, heat flow, mathematics and statistics. Or even just their resumes? We’ll find people who outscored them.
OK?
Here’s the statement that no one has replicated the Australian temperature set:
The Forum felt that reproducing the Bureau’s ACORN-SAT daily analyses would be a very onerous task, and advice was supplied at the Forum meeting day that, while international groups have provided independent data homogenized at the monthly time-scale, no groups other than the Bureau are known to have attempted to produce or analyse an homogenized daily data set for Australia. One option would the Bureau to work with local and international collaborators with the appropriate skill set to broadly assess the ACORN-SAT daily homogenisation methodologies.
Here is the statement that no one
can replicate them because only the BOM knows how it was done (my bolding):
The Forum noted that the extent to which the development of the ACORN-SAT dataset from the raw data could be automated was likely to be limited, and that the process might better be described as a
supervised process in which the roles of metadata and other information required some level of expertise and
operator intervention. The Forum investigated the nature of the
operator intervention required and the bases on which such
decisions are made and concluded that
very detailed instructions from the Bureau are likely to be necessary for an end-user who wishes to reproduce the ACORN-SAT findings. Some such details are provided in Centre for Australian Weather and Climate Research (CAWCR) technical reports (e.g. use of 40 best correlated sites for adjustments, thresholds for adjustment, and so on); however, the Forum concluded that it is likely to remain the case that several choices within the adjustment process remain a
matter of expert judgment and appropriate disciplinary knowledge.
The process can’t be “automated” — which means it can’t be described by a set of rules other people, or other computers could follow. It’s a bit of a red herring: skeptics have never demanded “automation”. We just want explanations. The crux of science is replication, not automation. If ad hoc judgements were part of the process, they need to be recorded and their impact on the numbers included in the processing from raw data to final product. Justifications can come afterwards; let’s first establish what happened.
These are weak and vague promises here for something that is not just a basic tenet of science, but should be obligatory for government funded work as well. (Bolding all mine):
The Forum recommends that the Bureau work
towards providing robust code that
supports a level of automation that allows sensitivity analyses to be
reasonably undertaken by independent parties.
What “independent re-analysis”? There is no independent analysis of all of ACORN.
This goal
could be pursued through a careful documentation of existing code and feedback from the independent re-analysis recommended in the preceding paragraph.
The Bureau would like to help but it costs too much, and skeptics will have to pay more for answers from these tax-funded workers:
While the Bureau
expressed willingness to support end-users who wished to reproduce findings or conduct independent analyses using the ACORN-SAT data, subsequent follow-up on such intentions may have significant resource implications. It is thus recommended that the Bureau limits the amount of assistance it provides end-users and includes a statement on the ACORN-SAT website that while reasonable assistance may be provided by the Bureau, extensive assistance could not be provided without an appropriate at-cost charge. Such limitations are likely to also limit the ability of end-users to replicate ACORN-SAT findings, but the resource implications of offering open-ended support to end-users may be substantial.
The
Bureau of Meteorology Budget was
344.2 million in 2014-15. The Australian climate is a national crisis, but the Bureau can’t employ one person to answer questions about its secret methods?
When will the BOM start to behave as though the climate is important? When will the Greens demand science be done properly for the sake of the environment?
