National Conference on Environmental Decision Making
To Bias or Not To Bias, That Is The Question; ...
by
Alfred A. Brooks Jr.
National Center for Environmental Decision Making Research
Knoxville, Tennessee - May
3 - 6, 1998
My Purpose: (after the Bard of Avon)
To
bias or not to bias, - that is the question;
Whether 'tis nobler in the mind to suffer
The slings and arrows of outrageous remediation,
Or to make argument against a sea of biased assumptions,
And by opposing end them. - Hamlet III.i
Somewhat later, Georges Bernanos said:
The worst, the most corrupting of lies, is a problem poorly stated.
So I shall ask:
How
well is the Superfund problem stated?
Congress, i.e., CERCLA, states the problem as:
Cost-effective remediation (4x) of substantive health risk (5x)
The above sounds straightforward.
BUT - How does EPA state the problem??
How are the intentions of Congress translated into regulations and remediation models?
When there is human data, use the conservative 90-th percentile value and, when in doubt, use the most sensitive test strain of the most sensitive species, and one or more conservative factors of ten. Use conservative linear extrapolations. Use Upper Limits.
"EPA must use data that give risk assessments that result in unquestionably safe remediation proposals."
"We try to make the always safe error."
"Conservatism in the face of uncertainty is logical."
"The public expects us to err on the side of safety."
This is called "Best Science" by Elliot Laws of EPA
But does Best Science have Deliberate Errors?
Covertly, all of these imply unlimited resources and are correct for that assumption.
Since any small fraction of unlimited is still unlimited, why do you not share?
Is the Basic Premise, unstated by EPA, wrong?
Are the risks described by such models "substantive health risks"?
Can such allocations of limited resources be cost effective?
I ask you, "Is this a well-stated problem?"
Is it even the same problem that Congress posed?
COST EFFECTIVE - What does it mean
if it does not mean the allocation of the available resources to get the
most risk reduction possible? Or, at least an approximation of this optimal
allocation.
OPTIMAL ALLOCATION - must mean the maximum of some response function from which all allowable directions and other allocations are downhill.
DEMO: Consider
an umbrella as
a response surface ....As a constrained surface. If the umbrella is positioned
erroneously, you get wetter than need be.
In the demo, the open, upright umbrella was used to illustrate that an optimal surface has a maximum from which all directions are down hill. By showing the intersection of the umbrella surface with a constraint surface, it was shown that a maximal point still existed. The umbrella, held far to one side, illustrated a badly biased responce surface, - hence
EPA'S COST and DATA MODEL CAN BE SAID TO BE ALL WET (pun intended).
THERE IS NO ALWAYS-SAFE ERROR, CONSERVATIVE OR OTHERWISE. EPA POSES A POORLY STATED PROBLEM.
INDEED, CONSERVATISM IN THE FACE OF UNCERTAINTY AND LIMITED RESOURCES IS ILLOGICAL and COUNTER PRODUCTIVE.
THIS IS NOT WHAT
CONGRESS REQUESTED. It leads to large exaggerations
of risk and remediations that are not cost-effective.
THE PRIMARY PREMISE OF ANY SUPERFUND MODEL SHOULD BE: LIMITED RESOURCES TO BE ALLOCATED TO SUPERFUND REMDIATION!
Cost-effective decisions require accurate data, without bias, deliberate or not.
No more safety factors, no more biasing assumptions.
This is a different data model.
It requires real data or unbiased estimates of real data.
It requires a different decision about data at every step: as accurate as possible, not biased toward some predetermined goal.
"One measurement is worth a thousand guesses."
It requires a new risk assessment database.
The data must not be biased in any way.
The real raw data are unbiased distribution functions expressing the frequency of measured values in the appropriate populations.
The real, computed values are the unbiased
values of the legitimate integrals of the appropriate distribution functions,
such as, the mean or the 95-th percentile value, etc.
They must approximate the values that would be measured if such measurements could be made.
What Replaces the Safety Factor?
In this model, the safety factor
is replaced by an agreed upon protected percentile of the desired population.
Specific generic regulations are replaced by approved methods of site-specific,
probabilistic risk assessment. A whole new generation of data needs to
be obtained: human-based distribution functions or the best, unbiased estimates
that can be made from which a meaningful risk distribution function can
be constructed.
The effect of bias on the accomplished
remediation of two identical sites:
| Bias Factor |
1 |
2 |
3 |
5 |
7 |
10 |
20 |
30 |
| Remediation | 1.0 | .983 | .948 | .913 | .871 | .816 | .627 | .624 |
| Minor Site | .5 | .439 | .393 | .350 | .310 | .254 | .005 | .000 |
Mercury distribution of LEFPC Superfund site soils.
First, it is site dependent, particularly upon the distribution of contaminant levels. It is dubious that any of risk assessments based on the EPA Guidelines with safety factors of 100 to 1000 and higher are suitable for even qualitative resource allocations, i.e., establishing priorities, much less quantitative allocations.
What Is A Substantive Health Risk? Who Should Set It?
Surely, a risk, a million times less than the common risks, is not a substantive risk in the minds of many. This is not a technical question but one in which the public must have a voice. The public is affected by many risk factors other than Superfund and is entitled to have input about which risks they would like to have reduced. Risks calculated with gross biases do not facilitate this.
Society's resources must be divided not only among CERCLA sites but also among a wide range of public and private goals. Therefore, it is of paramount importance that the risks or benefits be stated in comparable terms that the public can understand.
One example: Arbitrary levels of cancer risk of the order of 10E-6, when compared to total background cancer risks of about 0.3, exhibit several logical and practical problems. They can not be confirmed experimentally. If all causes were of similar risk, one would have to identify and remove 300,000 causes. If there are larger risks, are we devoting proportionate resources to the elimination of the large causes?
The EPA data and cost model do not allow us to address these questions.
There is much to be said for the
cost of a quality-adjusted, life-year saved as used by Tengs. In
this study. the range of cost per life-year saved is from money saved to
$99,000,000,000. How can one say this leads to reasonable cost allocations?
The public must contribute to the determination of the acceptable level
of risk and the associated level of costs. Public education is necessary.
The above is neither simple nor easy but I did not create the universe nor do I determine how it behaves and must be realistically described. However, neither did EPA or Congress nor can they determine its behavior. Nor can they proceed effectively with large deliberate errors.
However, it is
not as difficult as usually claimed. The methods date back one to
four hundred years. The methodology, but not the data, is old and well
developed. Monte Carlo methods will play a large role in the new applications.
Developing the data is not as difficult as EPA imagines as all current
data describes some distribution function, usually better than but no worse
than the data itself. The use of ranked-order, uniform probability distribution
functions is especially useful. This all adds up to:
Properly Applied, Site-Specific, Unbiased Probabilistic Analysis
The best we can do is to get on the
task required of us and do the best job we can. To do less is to fail as
professionals and to fail the American public. We also will continue to
yield to fear, misinformation, and substitute politics for science.