[House Hearing, 111 Congress]
[From the U.S. Government Publishing Office]
THE SCIENCE OF SECURITY,
PARTS I AND II
=======================================================================
HEARINGS
BEFORE THE
SUBCOMMITTEE ON INVESTIGATIONS AND
OVERSIGHT
COMMITTEE ON SCIENCE AND TECHNOLOGY
HOUSE OF REPRESENTATIVES
ONE HUNDRED ELEVENTH CONGRESS
FIRST SESSION
__________
JUNE 25, 2009
AND
NOVEMBER 17, 2009
__________
Serial No. 111-38
and
Serial No. 111-63
__________
Printed for the use of the Committee on Science and Technology
Available via the World Wide Web: http://www.science.house.gov
______
U.S. GOVERNMENT PRINTING OFFICE
50-324 PDF WASHINGTON: 2010
-----------------------------------------------------------------------
For Sale by the Superintendent of Documents, U.S. Government Printing Office
Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; (202) 512-1800
Fax: (202) 512-2104 Mail: Stop IDCC, Washington, DC 20402-0001
COMMITTEE ON SCIENCE AND TECHNOLOGY
HON. BART GORDON, Tennessee, Chair
JERRY F. COSTELLO, Illinois RALPH M. HALL, Texas
EDDIE BERNICE JOHNSON, Texas F. JAMES SENSENBRENNER JR.,
LYNN C. WOOLSEY, California Wisconsin
DAVID WU, Oregon LAMAR S. SMITH, Texas
BRIAN BAIRD, Washington DANA ROHRABACHER, California
BRAD MILLER, North Carolina ROSCOE G. BARTLETT, Maryland
DANIEL LIPINSKI, Illinois VERNON J. EHLERS, Michigan
GABRIELLE GIFFORDS, Arizona FRANK D. LUCAS, Oklahoma
DONNA F. EDWARDS, Maryland JUDY BIGGERT, Illinois
MARCIA L. FUDGE, Ohio W. TODD AKIN, Missouri
BEN R. LUJAN, New Mexico RANDY NEUGEBAUER, Texas
PAUL D. TONKO, New York BOB INGLIS, South Carolina
PARKER GRIFFITH, Alabama MICHAEL T. MCCAUL, Texas
STEVEN R. ROTHMAN, New Jersey MARIO DIAZ-BALART, Florida
JIM MATHESON, Utah BRIAN P. BILBRAY, California
LINCOLN DAVIS, Tennessee ADRIAN SMITH, Nebraska
BEN CHANDLER, Kentucky PAUL C. BROUN, Georgia
RUSS CARNAHAN, Missouri PETE OLSON, Texas
BARON P. HILL, Indiana
HARRY E. MITCHELL, Arizona
CHARLES A. WILSON, Ohio
KATHLEEN DAHLKEMPER, Pennsylvania
ALAN GRAYSON, Florida
SUZANNE M. KOSMAS, Florida
GARY C. PETERS, Michigan
VACANCY
------
Subcommittee on Investigations and Oversight
HON. BRAD MILLER, North Carolina, Chair
STEVEN R. ROTHMAN, New Jersey PAUL C. BROUN, Georgia
LINCOLN DAVIS, Tennessee BRIAN P. BILBRAY, California
CHARLES A. WILSON, Ohio VACANCY
KATHY DAHLKEMPER, Pennsylvania
ALAN GRAYSON, Florida
BART GORDON, Tennessee RALPH M. HALL, Texas
DAN PEARSON Subcommittee Staff Director
EDITH HOLLEMAN Subcommittee Counsel
JAMES PAUL Democratic Professional Staff Member
DOUGLAS S. PASTERNAK Democratic Professional Staff Member
KEN JACOBSON Democratic Professional Staff Member
ALEX MATTHEWS Research Assistant
TOM HAMMOND Republican Professional Staff Member
JANE WISE Research Assistant
C O N T E N T S
The Science of Security, Part I: Lessons Learned in Developing, Testing
and Operating Advanced Radiation Monitors
June 25, 2009
Page
Witness List..................................................... 2
Hearing Charter.................................................. 3
Opening Statements
Statement by Representative Brad Miller, Chairman, Subcommittee
on Investigations and Oversight, Committee on Science and
Technology, U.S. House of Representatives...................... 8
Written Statement............................................ 9
Statement by Representative Paul C. Broun, Ranking Minority
Member, Subcommittee on Investigations and Oversight, Committee
on Science and Technology, U.S. House of Representatives....... 10
Written Statement............................................ 11
Panel I:
Mr. Gene Aloise, Director, Natural Resources and Environment,
U.S. Government Accountability Office (GAO)
Oral Statement............................................... 12
Written Statement............................................ 14
Biography.................................................... 22
Dr. Micah D. Lowenthal, Director, Nuclear Security and Nuclear
Facility Safety Program, Nuclear and Radiation Studies Board,
National Research Council, The National Academies
Oral Statement............................................... 22
Written Statement............................................ 24
Biography.................................................... 28
Discussion
Prioritizing Security Needs.................................... 29
Decision-making: Processes and Timelines....................... 30
ASP in Context: The Global Nuclear Detection Architecture...... 31
Management of ASP Testing and Deployment....................... 32
Justifying the Cost of the Advanced Spectroscopic Portal
Program...................................................... 33
Panel II:
Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
Detection Office, Department of Homeland Security
Oral Statement............................................... 35
Written Statement............................................ 37
Biography.................................................... 40
Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of
Field Operations, U.S. Customs and Border Protection,
Department of Homeland Security
Oral Statement............................................... 41
Written Statement............................................ 42
Biography.................................................... 46
Discussion....................................................... 47
Appendix: Additional Material for the Record
Combating Nuclear Smuggling: DHS Improved Testing of Advanced
Radiation Detection Portal Monitors, but Preliminary Results
Show Limits of the New Technology, Report to Congressional
Requesters, U.S. Government Accountability Office, GAO-09-655,
May 2009....................................................... 54
Prepublication Copy, Evaluating Testing, Costs, and Benefits of
Advanced Spectroscopic Portals for Screening Cargo at Ports of
Entry, Interim Report (Abbreviated Version), Committee on
Advanced Spectroscopic Portals, Nuclear and Radiation Studies
Board, Division of Earth and Life Studies, National Research
Council of the National Academies.............................. 86
The Science of Security, Part II: Technical Problems Continue to Hinder
Advanced Radiation Monitors
November 17, 2009
Witness List..................................................... 178
Hearing Charter.................................................. 179
Opening Statements
Statement by Representative Brad Miller, Chairman, Subcommittee
on Investigations and Oversight, Committee on Science and
Technology, U.S. House of Representatives...................... 184
Written Statement............................................ 185
Statement by Representative Paul C. Broun, Ranking Minority
Member, Subcommittee on Investigations and Oversight, Committee
on Science and Technology, U.S. House of Representatives....... 186
Written Statement............................................ 187
Witnesses:
Mr. Gene Aloise, Director, Natural Resources and Environment,
U.S. Government Accountability Office (GAO)
Oral Statement............................................... 193
Written Statement............................................ 195
Biography.................................................... 199
Dr. Timothy M. Persons, Chief Scientist, U.S. Government
Accountability Office (GAO)
Biography.................................................... 200
Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of
Field Operations, U.S. Customs and Border Protection,
Department of Homeland Security
Oral Statement............................................... 200
Written Statement............................................ 202
Biography.................................................... 206
Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
Detection Office, Department of Homeland Security
Oral Statement............................................... 207
Written Statement............................................ 208
Biography.................................................... 211
Discussion
CBP Procedures After a Primary Alarm and the Effect of False
Positives.................................................... 212
Steps Taken to Reduce False Positives and Negatives............ 213
Mission Critical Failure....................................... 215
Energy Windowing to Improve PVT Performance.................... 216
The CBP Inspection System...................................... 217
The Helium-3 Shortage and Potential Alternative Materials...... 218
In What Circumstances Should ASPs Be Deployed?................. 219
Expectations of a Cost-benefit Analysis on ASPs................ 220
Metrics and Timelines for Making Decisions About ASPs.......... 222
Appendix: Answers to Post-Hearing Questions
Dr. William K. Hagan, Acting Deputy Director, Domestic Nuclear
Detection Office, Department of Homeland Security; and Mr. Todd
C. Owen, Acting Deputy Assistant Commissioner, Office of Field
Operations, U.S. Customs and Border Protection, Department of
Homeland Security.............................................. 230
Mr. Gene Aloise, Director, Natural Resources and Environment,
U.S. Government Accountability Office (GAO); Mr. Todd C. Owen,
Acting Deputy Assistant Commissioner, Office of Field
Operations, U.S. Customs and Border Protection, Department of
Homeland Security; and Dr. William K. Hagan, Acting Deputy
Director, Domestic Nuclear Detection Office, Department of
Homeland Security.............................................. 235
THE SCIENCE OF SECURITY, PART I: LESSONS LEARNED IN DEVELOPING,
TESTING, AND OPERATING ADVANCED RADIATION MONITORS
----------
THURSDAY, JUNE 25, 2009
House of Representatives,
Subcommittee on Investigations and Oversight,
Committee on Science and Technology,
Washington, DC.
The Subcommittee met, pursuant to call, at 10:00 a.m., in
Room 2318 of the Rayburn House Office Building, Hon. Brad
Miller [Chairman of the Subcommittee] presiding.
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
hearing charter
SUBCOMMITTEE ON INVESTIGATIONS AND OVERSIGHT
COMMITTEE ON SCIENCE AND TECHNOLOGY
U.S. HOUSE OF REPRESENTATIVES
The Science of Security, Part I: Lessons
Learned in Developing, Testing, and
Operating Advanced Radiation Monitors
Thursday, June 25, 2009
10:00 a.m.-12:00 p.m.
2318 Rayburn House Office Building
Purpose
The Subcommittee on Investigations and Oversight meets on June 25,
2009 to examine problems with the Department of Homeland Security's
(DHS) efforts to acquire its next generation radiation monitors known
as Advanced Spectroscopic Portals (ASPs). The ASP program has been
under scrutiny since 2006 for failing to have clear-cut program
requirements, an adequate test plan, sufficient timelines and
development milestones or a transparent and comprehensive cost benefit
analysis. Since the Domestic Nuclear Detection Office (DNDO), a DHS
component, was created in 2005, they have been responsible for
researching, developing, testing and managing the program.
The hearing will examine two new independent reports--one by the
Government Accountability Office (GAO) and the other by the National
Academy of Sciences--that identify ongoing and systematic problems in
the testing and development of the ASP program. With an estimated
program cost of $2-to-$3 billion the Subcommittee will evaluate the
rigor of the overall test program, the technical abilities of the ASPs
compared to existing radiation portal monitors and search for lessons
from the ASP program that can be applied to future DHS acquisitions.
Background
Since the terrorist attacks of September 11, 2001, protecting the
Nation from a nuclear or radiological attack has been a top national
security priority. In 2002, to help address this potential threat, the
U.S. Customs and Border Protection (CBP) agency began deploying
radiation monitors at U.S. border sites and ports of entry so its
officers could screen the more than 23 million containers of cargo that
enter the country every year for radiological and nuclear materials.
The equipment used to screen this cargo both then and now are
polyvinyl toluene (PVT) or ``plastic'' portal monitors able to detect
the presence of radioactive sources, but unable to identify the type of
radiation present. The PVT monitors, while relatively inexpensive,
robust and highly reliable, are unable to distinguish between
radioactive sources that might be used to construct a nuclear bomb,
such as Highly Enriched Uranium (HEU), and non-threatening naturally
occurring radiological materials (NORM) contained in ceramic tiles,
zirconium sand or kitty liter, for instance. As a result, any time a
PVT monitor detects a radioactive source the cargo is sent to
``secondary'' screening where CBP agents verify the detection of the
radioactive source with a second PVT monitor and use hand-held
Radioactive Isotope Identification Devices called RIIDs to help
identify the source of radiation.
This method of operation leads to many ``secondary'' inspections
for naturally occurring radioactive material or radioactive material
intended for benign purposes, such as radioactive medical isotopes. At
the Los Angeles/Long Beach port of entry, for instance, PVT monitors
routinely send up to 600 conveyances of cargo to secondary inspection
each day. The RIIDs, used in secondary inspections however, are limited
in their abilities to locate and identify potential threat material in
large cargo containers. As a result, CBP officers can consult with
scientists in CBP's Laboratories and Scientific Services (LSS) unit who
can often help them enhance the ability to correctly identify the
radioactive material of concern. As a last resort, CBP officers may
physically search a cargo container by emptying its contents and
closely scrutinizing it for potentially dangerous radioactive material.
If terrorists were to try to smuggle nuclear or radiological
materials in containerized cargo--and there are ample other pathways
for such smuggling--they would likely try to shield or ``mask'' those
materials in an attempt to make it more difficult to detect, identify
and locate the material of concern. Shielding requires that lead or
steal or other types of metal enclose the radioisotopes to hide its
radioactive signature. Potential terrorists may also attempt to
``mask'' threatening radioactive material by placing it together with
or alongside other non-threatening material that has a natural
radioactive signature, such as ceramic material, kitty litter or even
bananas. Most nuclear security experts believe smuggled radioactive or
nuclear material would be both shielded and masked in order to conceal
it from being located and properly identified. Obviously, these efforts
would make it harder to detect.
In order to help both improve the flow of commerce by eliminating
many of the false alarms that send cargo for secondary screening and to
more accurately identify radioactive or nuclear material, the
Department of Homeland Security (DHS) began developing Advanced
Spectroscopic Portals (ASPs) in 2004. The ASPs were intended to both
detect and identify radioactive material. In April 2005, the Domestic
Nuclear Detection Office was created by National Security Presidential
Directive-43/Homeland Security Presidential Directive-14 to, among
other things, research, develop, test and acquire radiation detection
equipment to be used by CBP and other federal agencies. The office was
not formally established until October 2006 under the SAFE Port Act.
From the very start of the ASP program, DNDO seemed to push for
acquisition decisions well before the technology had demonstrated that
it could live up to its promise. On July 14, 2006, Secretary of
Homeland Security Michael Chertoff and the Director of DNDO, Vayl
Oxford, announced contract awards to three companies worth an estimated
$1.2 billion to develop the ASPs, including the Raytheon Company, from
Massachusetts, the Thermo Electron Company from Santa Fe, New Mexico
and Canberra Industries from Connecticut. Both Chertoff and Oxford held
a press conference to announce the billion dollar contract awards just
a few months after highly critical reviews of the ASPs' abilities by
the GAO and the National Institute of Standards and Technology (NIST).
In March 2006, GAO said: ``it is not clear that the benefits of the
new portals would be worth any increased cost to the program.'' In June
2006, NIST submitted a report to DHS on results of side-by-side testing
the previous year at the Nevada Test Site of both ASP and PVT systems.
The DNDO had assumed that the ASPs would correctly identify HEU 95
percent of the time for both bare or unmasked HEU and HEU masked in a
container with more benign radiological material. Yet, NIST found that
the three best ASP systems tested identified HEU only 70 to 88 percent
of the time. Their ability to identify ``masked'' HEU was much worse.
The three ASP manufacturers did this only 53 percent of the time
(Raytheon), 45 percent of the time (Thermo) and 17 percent of the time
(Canberra). ``Despite these results,'' the GAO found, ``DNDO did not
use the information from these tests in its cost-benefit analysis.''
DNDO claimed that they assumed they would meet the 95 percent
performance level at some point in the future but provided no data on
why they reached this conclusion, said GAO.
At the Chertoff-Oxford press conference in July 2006, then
Secretary of Homeland Security, Michael Chertoff, said one of the key
reasons for developing the ASPs and replacing the existing radiation
monitors was to ``have fewer false positives.'' In September 2007, Vayl
Oxford, the Director of DNDO reiterated that point in testimony to
Congress where he emphasized that the ASPs would reduce the number of
false alarms from the nearly 600 experienced each day by the PVTs at
the port of Long Beach in California, for instance, to 20 to 25 per day
with the new ASP monitors.
That was the hope, anyway. One of the criteria for ASP primary
screening prior to certification of the new radiation monitors by the
Secretary of Homeland Security, which is required by the appropriations
committees, is that the ASPs must refer at least 80 percent fewer
conveyances for further inspection than the PVTs. But in ``field
validation tests'' earlier this year, by one of the two remaining
contractors, the ASPs being tested sent more innocent radioactive
shipments to secondary screening than the older PVT monitors. The cause
of the high false positives was apparently due to a software glitch.
This was a serious concern to the Customs and Border Protection
personnel who will have to operate and maintain the ASPs if and when
they are certified and deployed. The contractor has reportedly
corrected the software issue and intends to return the ASPs to field
validation testing next month.
Last fall, ``integration'' testing of the ASPs by the second
remaining contractor was halted because of different technical troubles
with its own software. The contractor corrected the problem and its ASP
machines re-entered integration testing late last year. The contractor
hopes to finish integration testing and begin field validation testing
in early August. Still, both contractors are now many months behind
schedule because technical issues have forced delays.
Virtually any high-technology research and development program
experiences bumps in the road, technical troubles and occasional set-
backs. However, well managed programs have clear technical requirements
and strategic goals. They ensure that the new technology being
developed is thoroughly tested and adequately integrated into the
operational plans and procedures of those who must operate them in the
field. When these vital components are short changed, when the test
plan is insufficient and the program's research, development and
testing methods are marred by scanty scientific rigor, the technical
tools being developed are bound to suffer as a result. Cutting critical
corners in the development process serves no one's interests. Yet, from
the start many of the leaders of the ASP program at DHS seemed more
interested in fielding this technology then in vigilantly validating
its performance and effectiveness. At the July 2006 press conference
unveiling the contractors on the ASP program, for instance, Vayl Oxford
said: ``the priority for the first year . . . is to get units out
immediately.'' Three years later, none of these units have yet cleared
field validation tests.
The policy governing the ASP program and the disproportionate focus
on getting the ASP units into the field quickly never matched the
multiple independent technical assessments of the technology being
developed and tested. Over the past three years the Government
Accountability Office has issued six reports on the ASP program and
testified before Congress multiple times on this matter. Last year the
Homeland Security Institute, a Federally Funded Research and
Development Center for DHS, issued a report on the ASPs that also
criticized the ASP test program, saying it provided insufficient data.
The National Academy's of Science, which will release an interim report
on the ASPs that they have just concluded this week, will provide
testimony at the Subcommittee hearing that echoes many of the concerns
raised by GAO over the years.
History of Problems
In 2006, the GAO issued a harsh critique of the DNDO's cost-
benefit-analysis (CBA) of the ASPs. The DNDO analysis omitted critical
test data that identified major technical problems with the ASPs and
they drastically increased the procurement costs of the PVTs. In short,
the GAO found DNDO's cost-benefit analysis was ``incomplete,'' based on
``unreliable'' data and used ``inflated cost estimates for PVT
equipment.''
In 2007, GAO concluded that tests of the ASPs conducted by DNDO
were ``biased'' and ``were not an objective and rigorous assessment of
the ASPs' capabilities.'' The tests, for instance, used insufficient
amounts of materials likely to mask or shield radioactive threat
sources that terrorists might attempt to smuggle into the country. The
tests, said GAO, did not attempt to test the limitations of the ASPs
and ``did not objectively test the performance'' of currently used
hand-held radiation detectors or RIIDs.
Last year, in their own independent cost estimate of the ASP
program, GAO found that the ASPs could cost about $3.1 billion, $1
billion more than the DNDO's estimate. The GAO also found that the DNDO
had often changed its deployment strategy, eliminating plans to develop
ASP portals for rail, airport and seaport cargo screening terminals,
for instance. As a result, GAO estimated the newest scaled back plan
reduced the potential costs of the program to about $2 billion from
2008 to 2017. The only documentation that DNDO provided to GAO for this
major change in the ASP program was a one-page spread-sheet and DNDO
has still not released an updated cost-benefit analysis of the program.
In addition, GAO criticized DNDO's decision not to complete
computerized simulations or ``injection studies'' of the ASPs prior to
certification by the Secretary of Homeland Security. The National
Academy of Sciences has also found that computer modeling is critically
important to the ASP program since running every potential radioactive
smuggling scenario in live tests is unrealistic. Computer simulations
would help provide a clearer assessment of the potential performance of
the ASPs in actual smuggling incidents and effectiveness at identifying
threatening radioactive material. DNDO, however, does not plan to
complete the studies prior to the Secretary of Homeland Security's
decision on certification, which DNDO expects to occur in October.
Problems Remain
While DNDO's past tests have been characterized as being unsound,
incomplete and limited in scope, the GAO's most recent work on the ASP
program does point to some improvements in the integrity of the latest
round of tests. However, they also pinpointed significant technical
limitations which have not yet been resolved.
The ASP portals did prove more effective than the PVTs in detecting
HEU materials concealed by ``light shielding.'' However, differences
between the ASPs and PVTs became less notable when shielding was
slightly increased or decreased. In past tests there was virtually no
difference in the performance of the two machines with regard to
detecting other kinds of radioactive isotopes, such as those used for
medical or industrial purposes, according to the GAO, except in one
case where the ASPs performed worse than the PVTs. Whether these other
forms of radioactive sources are sensed by a PVT or ASP machine they
all require secondary inspection to determine why a payload contains
radioactive material. In detecting HEU, the ASPs performed better only
in one narrowly defined scenario, which many experts see as an
unrealistic portrayal of a true attempted nuclear smuggling incident.
None of the tests run by DNDO, for instance, included scenarios that
utilized both ``shielding'' and ``masking'' as a means of attempting to
smuggle radioactive or nuclear material.
In addition, GAO and others have faulted DNDO for not focusing
enough on attempting to improve the current radiation portal monitor
program. Instead, DNDO has been nearly single-mindedly focused on
developing Advanced Spectroscopic Portals at the expense of other far
simpler alternatives. Surprisingly, for instance, DNDO has not
completed efforts to improve the performance of PVTs by a method called
Energy Windowing that could provide them with some limited, but
enhanced, performance. Energy Windowing efforts are controversial and
are believed to only provide modest enhancements to the performance of
PVTs. But both GAO and CBP has been pushing DNDO to do more on this
front for years. In addition, DNDO has not made efforts to upgrade the
software in the hand-held radiation detection units known as RIIDs that
could also provide a far less expensive alternative to enhancing the
operational effectiveness of radiation monitors.
Because both remaining ASP contractors suffered from serious
technical problems in their last round of testing, Customs and Border
Protection (CBP) agency personnel fear that if the ASPs are certified,
procured and deployed that they will encounter many problems in the
field that will negatively impact their day-to-day operations and
perhaps the technical effectiveness of the current radiation monitoring
program to actually detect illicit nuclear or radiological material
coming into the country. The GAO, National Academy of Sciences and
others have also criticized DNDO for not seeking input from CBP
officials on the ASP program from the start. The relationship has
improved and DNDO does attempt to include CBP in critical decisions
regarding the ASP program today. But many critics say perhaps one of
DNDO's biggest failures was the fact that they did not do this from the
beginning, seeking input from the operational users of the technology
that DNDO was tasked to research, test and develop.
As a result of all of these issues, the ASPs continue to suffer
from key questions about their ability to provide significant improved
performance over existing radiation detection equipment currently
fielded at U.S. ports. The Department of Homeland Security has already
spent more than $235 million on the ASP program. But if the Secretary
of Homeland Security certifies that the ASP monitors are worth
investing in this fall--just three to four months from now--then $2
billion more may be invested to procure ASP radiation monitors. Yet,
given the continued criticism of the narrowly focused and inadequate
ASP test program, the limited technical improvements they may offer
over current radiation monitors and significant increased costs to
maintain and operate the ASPs compared to the PVTs, the success of the
program remains in doubt.
Key Issues
[bullet] Go Slow. Uncovering and resolving technical problems
once newly developed radiation monitors are fielded may hinder
the ability to detect and identify radioactive or nuclear
material that poses a potential threat. It could disrupt
operations at U.S. borders and ports of entry curtailing the
flow of commerce and it will cost more to rectify these
problems in the field, rather than in the laboratory or at the
test range. Yet, rather than carefully testing and validating
the performance and effectiveness of the ASP monitors before a
major procurement decision is made DHS has continually sought
to get the ASPs into the field in spite of critical technical
flaws identified during testing.
[bullet] Cost Benefit Analysis. Even if the technical
abilities of the ASPs are proven, their relative technical
capabilities and increased costs must be carefully weighed in
comparison to the existing radiation monitoring system in place
today. Replacing a proven, less-costly system that has the
confidence of its operators, must be given careful
consideration. The DNDO has not yet provided an updated cost-
benefit-analysis that would validate a decision to procure the
multi-billion dollar ASP equipment.
[bullet] Judging Performance. As the House Committee on
Appropriations has said in the past, procurement of the
Advanced Spectroscopic Portal monitors should not proceed until
they are deemed to add a ``significant increase in operational
effectiveness'' over the current PVT system already in place.
Last July, CBP, DNDO and the DHS management directorate jointly
issued criteria for determining this increase in effectiveness
in both ``primary'' and ``secondary'' screening. In primary
screening the criteria requires ASPs to detect potential
threats as well as or better than PVTs, show improved detection
of Highly Enriched Uranium and reduce innocent alarms. In
secondary screening the criteria requires ASPs to reduce the
probability of misidentifying special nuclear material (HEU or
plutonium) and reduce the average time to conduct secondary
screenings. The Secretary of Homeland Security must certify to
Congress that the ASPs have met these criteria before funding
for full-scale procurement of the ASPs goes forward. However,
the criteria to measure this improvement are weak and rather
vague.
Lessons Learned. The Department of Homeland Security
must make greater efforts to avoid rushing to acquisition
decisions when the R&D is incomplete. With ASPs, the research
and development program itself has been hindered by a lack of
rigorous scientific evaluations, and undemanding testing
protocols. Moving to acquire systems plagued by such problems
may endanger security and significantly increase the costs of
the program. A review of DHS's major programs by GAO last
November found that 45 of 48 major programs did not adhere to
the agency's own investment review process that helps provide
appropriate oversight to address cost, schedule and performance
problems. In FY 2008, the review found, DHS spent $147.5
million on the ASP program despite the fact it did not have a
mission needs statement. The program also lacked operational
requirements documents and an acquisition program baseline.
Witnesses
Panel I:
Mr. Gene Aloise, Director, Natural Resources and Environment,
Government Accountability Office
Dr. Micah Lowenthal, Division on Earth and Life Studies, Nuclear and
Radiation Studies Board, National Research Council, The National
Academy of Sciences
Panel II:
Dr. William Hagan, Acting Deputy Director, Domestic Nuclear Detection
Office (DNDO), Department of Homeland Security (DHS)
Mr. Todd C. Owen, Acting Deputy Assistant Commissioner, Office of Field
Operations, U.S. Customs & Border Protection (CBP), Department of
Homeland Security (DHS).
Chairman Miller. The hearing will now come to order. Good
morning. Welcome to today's hearing, The Science of Security:
Lessons Learned in Developing, Testing, and Operating Advanced
Radiation Monitors.
In the wake of the terrorist attacks on September 11, 2001,
preventing the detonation of a nuclear or radiologic device--a
`dirty bomb'--in the United States has become a top national
security objective. We have invested billions of dollars since
9/11 to develop the means to prevent, detect and respond to any
attack by weapons of mass destruction. We developed radiation
monitors at our port and border crossings to screen millions of
cargo containers entering the United States every year, hunting
for radiological material that could be used for terrorist
purposes. Since 2004, the Department of Homeland Security has
spent more than $230 million on a program to develop a new
radiation detection system called an Advanced Spectroscopic
Portal, or ASP, that can both detect and identify nuclear
material.
Congress expects that the funding federal agencies receive
will be well spent. When it comes to scientifically challenging
or technically demanding programs, it pays to have well-
prepared program requirements, demanding testing protocols and
an independent and comprehensive cost-benefit analysis. Those
vital steps, those vital program components help managers make
informed decisions about whether to move forward with a
technology development program or to replace a proven
technology with a new technology. Unfortunately, despite recent
progress, the ASP program has suffered because it lacked all
the preparatory steps of a well-managed program. We will hear
about some of those problems today from the Government
Accountability Office and the National Academies of Science.
Over the years, the GAO has released six reports on the ASP
program. The GAO found that some of the Domestic Nuclear
Detection Office's tests were biased and did not provide a
rigorous assessment of ASP's capabilities. The Agency relied on
incomplete and unreliable data in their cost-benefit analysis,
omitting critical test data, inflating the cost of current
radiation detectors and underestimating the cost of ASP
monitors. The Department failed to produce a requirements
document or adequate documentation regarding major changes to
the planned ASP deployment strategy. DNDO never considered the
option of investigating improvements of the existing radiation
portal monitoring program, both the PVT monitors and the hand-
held detectors that Customs and Border Protection agents rely
upon.
The National Academy of Science's interim report on the ASP
program, released yesterday, reflects many of the same
concerns. The Academy calls for significant restructuring of
DHS testing procedures for the ASP program. They question the
criteria being used to judge the ASPs' performance and they
recommend that DHS not proceed with further ASP procurement
until they address all the findings and recommendations in
their report.
GAO's reports have provided a regular accounting of how the
ASP program was going wrong. The Academy report provides a
roadmap to how the program can be put back on track, assuming
that the Department determines that it is worth the cost and
effort.
Radioactive materials have become a normal part of
commerce. They are used for medical procedures and industrial
applications. Bananas have radiation. Technology can help us
detect and identify radioactive sources in cargo containers but
no technology can sort out good radioactive material intended
for legitimate purposes from the bad radioactive materials
intended to do us harm. As a result, human operators will still
need to make important decisions often informed by intelligence
efforts to keep the Nation secure. Well-trained, well-equipped
people, law enforcement officers and Customs and Border
Protection inspectors will always be critical to the equation.
This hearing addresses our responsibility to the
technological part of that equation. Before we move forward
with a $2 billion or $3 billion program, we must ensure that we
get our money's worth from the new technology. Put another way,
if we have $2 billion or $3 billion to spend to enhance our
security, is this technology really how we should spend it?
I look forward to hearing from all of our witnesses today.
And now I recognize Dr. Broun, the Ranking Member from
Georgia, for an opening statement.
[The prepared statement of Chairman Miller follows:]
Prepared Statement of Chairman Brad Miller
In the wake of the terrorist attacks of September 11, 2001,
preventing the detonation of a nuclear or radiological device in the
U.S. has become a top national security objective.
We have invested billions of dollars since 9/11 to develop the
means to prevent, detect and respond to any attack by weapons of mass
destruction. We have deployed radiation monitors at our ports and
border crossings to screen millions of cargo containers entering the
U.S. every year, hunting for radiological material that could be used
for terrorist purposes. Since 2004, the Department of Homeland Security
(DHS) has spent more than $230 million on a program to develop a new
radiation detection system called an Advanced Spectroscopic Portal or
ASP that can both detect and identify nuclear material.
Congress expects that the funding federal agencies receive will be
well spent. When it comes to scientifically challenging or technically
demanding programs, studies have shown that it pays to have well-
prepared program requirements, demanding testing protocols and an
independent and comprehensive cost benefit analysis. These vital
program components help managers make informed decisions about whether
to move forward with a technology development program, or to replace a
proven technology with a new technology. Unfortunately, despite some
recent progress, the ASP program has suffered because it lacked all the
preparatory steps of a well managed program. We will hear about some of
those problems today from both Government Accountability Office (GAO)
and the National Academies of Science.
Over the past three years the GAO has released six reports on the
ASP program. The GAO found that some of the Domestic Nuclear Detection
Office's (DNDO) tests were ``biased'' and did not provide a rigorous
assessment of the ASP's capabilities. The agency relied on
``incomplete'' and ``unreliable'' data in their cost-benefit analysis--
omitting critical test data, inflating the costs of the current
radiation detectors, and underestimating the costs of ASP monitors. The
Department failed to produce a requirements document or adequate
documentation regarding major changes to their planned ASP deployment
strategy. DNDO never considered the option of investing in improvements
to the existing radiation portal monitoring program--both the current
polyvinyl toluene (PVT) monitors and the hand-held detectors Customs
and Border Protection agents rely upon.
The National Academy of Sciences' interim report on the ASP
program, released yesterday, reflects many of the same concerns. The
Academy calls for a significant restructuring of DHS testing procedures
for the ASP program, they question the criteria being used to judge the
ASP's performance and they recommend that DHS not proceed with further
ASP procurement until they address all of the findings and
recommendations in their report.
GAO's reports have provided a regular accounting of how the ASP
program was going wrong; the Academy's report provides a roadmap to how
the program could be put back on track--assuming that the Department
determines that is worth the cost and effort.
Radioactive materials have become a normal part of commerce. They
are used for medical procedures and industrial applications. Technology
can help us detect and identify radioactive sources in cargo
containers, but no technology can sort out ``good'' radioactive
material intended for legitimate purposes from the ``bad'' radioactive
material intended to do us harm. As a result, human operators will need
to make important decisions, often informed by intelligence efforts, to
keep the Nation secure. Well trained, well equipped people--law
enforcement officers and customs and border protection inspectors--will
always be critical to the equation.
This hearing addresses our responsibility to the technological part
of that equation. Before we move forward with a two to three billion
dollar program we must ensure that we get our money's worth from the
new technology. Put another way, if we have two or three billion
dollars to spend to enhance our security, is this technology really how
we should spend it?
I look forward to hearing from all of our witnesses today.
Mr. Broun. Thank you, Mr. Chairman. I want to welcome our
witnesses here today and to thank you for participating in this
important hearing on the Department of Homeland Security's
Advanced Spectroscopic Portal program. I know it is hard to
say, particularly for the non-scientist, but I am a scientist--
I am a physician.
Yesterday the House took up consideration of the DHS
appropriations bill, and later today the Science Committee's
Technology and Innovation Subcommittee, which I am also a
Member of, will hold a hearing on cyber security. I also sit on
the Homeland Security Committee's Subcommittee on Emerging
Threats, Cyber Security, Science and Technology as well.
Needless to say, DHS has kept me busy this week.
This morning we will look into the status of the
Department's ongoing development of the next-generation
radiation portal monitors, get an update from GAO on their
continued work, and receive a report from the National Research
Council. It goes without saying that this program has been
followed closely for some time now and thankfully many of the
testing issues that GAO has brought up in previous reports seem
to have been mitigated, at least somewhat. However, this
program is far from out of the woods. In their most recent
analysis, GAO and the Academy raised new issues relating to the
rigor of the testing and the certification process and offer
paths forward for potential acquisition in the future. I hope
DHS takes these recommendations seriously, and I look forward
to ensuring that they are not summarily dismissed for the sake
of arbitrary timetables. We see that frequently here in
government.
Looking forward, DHS should conduct a rigorous cost-benefit
analysis that takes into account updated threat assessments, a
review of all variations of concepts of operations, potential
upgrades to existing technologies and independent cost
estimates. It also needs to weigh the pros and cons not just of
ASP versus Polyvinyl Toluene (PVT) and Radio-Isotope
Identification Devices (RIID), but also whether the additional
capability gained outweighs the needs of other aspects of the
global nuclear detection architecture. Unfortunately, this may
be hard to do at this point considering GAO indicated earlier
this year that the Domestic Nuclear Detection Office had, to
quote GAO, ``not developed an overarching strategic plan to
guide its development of a more comprehensive global strategy
for nuclear detection.''
All of these factors need to be taken into consideration as
DHS moves toward an acquisition. Even then I will remain
cautious, given the Department's track record with past
acquisitions. Many of the issues we are dealing with today
could have been prevented by engaging the end-users early on in
the process and also by clearly defining the requirements and
simply following existing Department acquisition processes.
Last week this subcommittee held a hearing on issues
plaguing NPOESS. The AST program exhibits eerie similarities to
that program in that it attempted to link research and
development activities with the acquisition of an operational
system and had unclear architectural priorities. Let us hope
other federal programs can learn from these lessons and protect
taxpayers from future inefficiencies and waste.
Mr. Chairman, I yield back the balance of my time and I
look forward to our witnesses' testimony. Thank you.
[The prepared statement of Mr. Broun follows:]
Prepared Statement of Representative Paul C. Broun
Thank you, Mr. Chairman. I want to welcome the witnesses here
today, and thank them for participating in this important hearing on
the Department of Homeland Security's (DHS) Advanced Spectroscopic
Portal (ASP) program.
Yesterday the House took up consideration of the DHS Appropriation
bill, and later today the Science Committee's Technology and Innovation
Subcommittee, which I also sit on, will hold a hearing on cyber
security. I also sit on the Homeland Security Committee's Subcommittee
on Emerging Threats, Cyber Security, Science and Technology, as well.
Needless to say, DHS has kept me busy this week.
This morning we will look into the status of the Department's
ongoing development of next-generation Radiation Portal Monitors, get
an update from General Accountability Office (GAO) on their continuing
work, and receive a report from the National Academy of Sciences. It
goes without saying that this program has been followed closely for
some time now, and thankfully many of the testing issues that GAO
brought up in previous reports seem to be mitigated. However, this
program is far from ``out of the woods.'' In their most recent
analysis, GAO and the Academy raise new issues relating to the rigor of
the testing and certification process, and offer paths forward for a
potential acquisition in the future. I hope DHS takes these
recommendations seriously, and I look forward to ensuring that they are
not summarily dismissed for the sake of arbitrary timetables.
Looking forward, DHS should conduct a rigorous Cost-Benefit
Analysis (CBA) that takes into account updated threat assessments, a
review of all variations of Concepts of Operations (CONOPS), potential
upgrades for existing technologies, and independent cost estimates. It
also needs to weigh the pros and cons of not just ASP versus Polyvinyl
Toluene (PVT) and Radio-isotope Identification Devices (RIID), but also
whether the additional capability gained outweighs the needs of other
aspects of the Global Nuclear Detection Architecture. Unfortunately,
this may be hard to do at this point considering GAO indicated earlier
this year that the Domestic Nuclear Detection Office (DNDO) had ``not
developed an overarching strategic plan to guide its development of a
more comprehensive global strategy for nuclear detection.''
All of these factors need to be taken into consideration as DHS
moves toward an acquisition. Even then, I will remain cautious given
the Department's track record with past acquisitions. Many of the
issues we are dealing with today could have been prevented by engaging
the end-users early in the process, clearly defining requirements, and
simply following existing Department acquisition processes.
Last week this subcommittee held a hearing on issues plaguing the
National Polar-Orbiting Environmental Satellite System (NPOESS). The
ASP program exhibits eerie similarities to that program in that it
attempted to link research and development activities with the
acquisition of an operational system, and had unclear architectural
priorities. Let's hope other federal programs can learn from these
lessons and protect taxpayers from future inefficiencies and waste.
With that, Mr. Chairman, I yield back my time.
Thank you.
Chairman Miller. Thank you, Dr. Broun.
Panel I:
Chairman Miller. I am now pleased to introduce our panel of
witnesses. Mr. Gene Aloise is the Director of Natural Resources
and Environment at the Government Accountability Office, GAO,
and Dr. Micah Lowenthal is the Director of the Nuclear Security
and Nuclear Facility Safety Program in the Nuclear and
Radiation Studies Board at the National Research Council of the
National Academy of Sciences. Is that how you describe your job
at a cocktail party?
As our witnesses should know, you each have five minutes
for your spoken testimony. Your written testimony will be
included in the record for the hearing. When you have all
completed your spoken testimony, we will begin with questions
and each Member will have five minutes to question the panel.
It is the practice of this subcommittee to receive testimony
under oath. Do either of you have any objection to taking an
oath? Okay. You also have the right to be represented by
counsel. Do either of you have counsel here? If you would
please stand and raise your right hand? Do you swear to tell
the truth and nothing but the truth?
The record will reflect that both Mr. Aloise and Dr.
Lowenthal took the oath. Mr. Aloise, you may begin.
STATEMENT OF MR. GENE ALOISE, DIRECTOR, NATURAL RESOURCES AND
ENVIRONMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE (GAO)
Mr. Aloise. Thank you, Mr. Chairman.
Mr. Chairman and Members of the Subcommittee, I am pleased
to be here today to discuss DHS's plans to develop and test
advanced portal monitors for use at the Nation's borders to
prevent nuclear materials from being smuggled into the United
States. According to DHS, the current system of radiation
detection equipment is effective and does not impede the flow
of commerce. However, DHS wants to improve the capabilities of
the existing equipment with new equipment.
One of the major drawbacks of the new equipment is the
substantially higher cost compared to the existing equipment.
We estimated in September 2008 that the life cycle cost of each
standard cargo version of the ASP to be about $823,000 compared
to about $308,000 for the PVT standard cargo portal and that
the total program cost for the Domestic Nuclear Detection
Office's (DNDO) latest deployment plan would be about $2
billion.
Since 2006, we have issued six reports, and including
today, four testimonies, and have made 19 recommendations for
improving DNDO's efforts to develop and test portal monitors.
Our concerns have focused on the need for realistic and
objective testing of ASPs, full disclosure and reporting of
testing limitations, development of cost estimates that
consider the full cost to deploy the new equipment, and
development of a sound cost-benefit analysis which determines
whether the marginal increase in security from the ASPs is
worth its very high cost.
My testimony today is based on our recent report, which
assessed the most recent round of ASP testing. I will also
discuss the lessons learned from ASP testing. Our work on the
latest round of ASP testing found that DHS increased the rigor
in comparison with previous tests, and thereby added
credibility to the test results. However, we still question
whether the benefits of the ASP justify the high cost. In
particular, DHS's criteria for significant increase in
operational effectiveness require only marginal improvement in
the detection of certain weapons-usable nuclear material. The
marginal improvement required of ASPs is particularly notable
given that DNDO has not completed efforts to enhance the
performance of the current generation of equipment. Customs and
Border Protection (CBP) officials have told us that they have
repeatedly urged DNDO to investigate improving the performance
of the equipment through what is known as energy windowing.
DNDO has collected the data necessary to do this, but has not
yet completed efforts to analyze the data and further improve
the technique.
Our analysis of the new test results shows that ASPs
detected certain nuclear materials better than PVTs when
shielding approximated DOE threat guidance, which is based on
light shielding. However, differences between the two systems
were hard to recognize when shielding was slightly increased or
decreased. Both systems had difficulty in detecting nuclear
materials when shielding was somewhat greater than threat
guidance, which is set to match the extent of the detection
limits of the PVTs. Importantly, the threat guidance is not a
realistic approximation of how a terrorist might shield nuclear
material to successfully smuggle it undetected through a border
crossing.
In addition, DNDO underestimated the time needed for ASP
testing, which was originally supposed to be finished by
September 2008. DHS's most recent schedule anticipated ASP
certification around May 2009, but testing has been delayed
even further, and DHS has not updated its schedule. As far as
we know, certification is scheduled for some time this fall.
Of concern to us is the fact that DNDO does not plan to
complete simulations that could provide additional insights
into ASP capabilities and limitations prior to certification.
On this point, DNDO does not seem to have learned from past
mistakes. A rush to deploy ASPs before all testing is complete
leads to shortcuts in testing and raises questions among
stakeholders about the equipment's effectiveness and
reliability. Furthermore, DNDO has not yet updated its cost-
benefit analysis, which might show that DNDO's plan to replace
existing equipment is not justified.
One of the primary lessons to be learned from our work is
to avoid the pitfalls in testing that stem from a rush to
procure new technologies. In the case of the ASP, a push to
rush to replace existing equipment led to a testing program
which lacked scientific rigor. Even for the new round of
testing, DNDO consistently underestimated the time necessary to
conduct tests and resolve problems, and testing is still not
completed.
In closing, we believe that given the importance of this
new equipment to our national security, that Congress and the
American taxpayer still need to know three things: does the
equipment work, how much will it cost; and does the marginal
increase in security justify its very high cost.
Mr. Chairman, that concludes my remarks and I would be
happy to respond to any questions you and the Ranking Member
may have.
[The prepared statement of Mr. Aloise follows:]
Prepared Statement of Gene Aloise
Combating Nuclear Smuggling: Lessons
Learned From DHS Testing of Advanced
Radiation Detection Portal Monitors
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today to discuss GAO's work on the
Department of Homeland Security's (DHS) testing of advanced
spectroscopic portal (ASP) radiation detection monitors. As you are
aware, the national security mission of U.S. Customs and Border
Protection (CBP), an agency within DHS, includes screening for smuggled
nuclear or radiological material that could be used in a nuclear weapon
or radiological dispersal device (a ``dirty bomb''). To screen cargo at
ports of entry, CBP conducts primary inspections with radiation
detection equipment called portal monitors--large stationary detectors
through which cargo containers and vehicles pass as they enter the
United States. When radiation is detected, CBP conducts secondary
inspections using a second portal monitor to confirm the original alarm
and a hand-held radioactive isotope identification device to identify
the radiation's source and determine whether it constitutes a threat.
The polyvinyl toluene (PVT) portal monitors currently in use can
detect radiation but cannot identify the type of material causing an
alarm. As a result, the monitors' radiation alarms can be set off even
by benign, naturally occurring radioactive material. One way to reduce
the rate of such innocent alarms--and thereby minimize unnecessary
secondary inspections and enhance the flow of commerce--is to adjust
the operational thresholds (i.e., operate the PVTs at a reduced level
of sensitivity). However, reducing the sensitivity may make it more
difficult to detect certain nuclear materials.
To address the limitations of current-generation portal monitors,
DHS's Domestic Nuclear Detection Office (DNDO) in 2005 began to develop
and test ASPs, which are designed to both detect radiation and identify
the source.\1\ DNDO hopes to use the new portal monitors to replace at
least some PVTs currently used for primary screening, as well as PVTs
and hand-held identification devices currently used for secondary
screening. However, in September 2008, we estimated the life cycle cost
of each standard cargo version of the ASP (including deployment costs)
to be about $822,000, compared with about $308,000 for the PVT standard
cargo portal, and the total program cost for DNDO's latest plan for
deploying radiation portal monitors--which relies on a combination of
ASPs and PVTs and does not deploy radiation portal monitors at all
border crossings--to be about $2 billion.\2\
---------------------------------------------------------------------------
\1\ DNDO was established within DHS in 2005; its mission includes
developing, testing, acquiring, and supporting the deployment of
radiation detection equipment at U.S. ports of entry. CBP began
deploying portal monitors in 2002, prior to DNDO's creation, under the
radiation portal monitor project.
\2\ GAO, Combating Nuclear Smuggling: DHS's Program to Procure and
Deploy Advanced Radiation Detection Portal Monitors Is Likely to Exceed
the Department's Previous Cost Estimates, GAO-08-1108R (Washington,
D.C.: Sept. 22, 2008).
---------------------------------------------------------------------------
Concerned about the performance and cost of the ASP monitors,
Congress required the Secretary of Homeland Security to certify that
the monitors will provide a ``significant increase in operational
effectiveness'' before DNDO obligates funds for full-scale ASP
procurement.\3\ The Secretary must submit separate certifications for
primary and secondary inspection. In response, CBP, DNDO, and the DHS
management directorate jointly issued criteria in July 2008 for
determining whether the new technology provides a significant increase
in operational effectiveness. The primary screening criteria require
that the new portal monitors detect potential threats as well as or
better than PVTs, show improved performance in detection of highly
enriched uranium (HEU), and reduce innocent alarms. To meet the
secondary screening criteria, the new portal monitors must reduce the
probability of misidentifying special nuclear material (e.g., HEU and
plutonium) and the average time to conduct secondary screenings.
---------------------------------------------------------------------------
\3\ Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121
Stat. 1844, 2069 (2007); Consolidated Security, Disaster Assistance,
and Continuing Appropriations Act, 2009, Pub. L. No. 110-329, 121 Stat.
3574, 3679 (2008).
---------------------------------------------------------------------------
DNDO designed and coordinated a new series of tests, originally
scheduled to run from April 2008 through September 2008, to determine
whether the new portal monitors meet the certification criteria for
primary and secondary screening and are ready for deployment. Key
phases of this testing program include concurrent testing led by DNDO
of the new and current equipment's ability to detect and identify
threats and of ASPs' readiness to be integrated into operations for
both primary and secondary screening at ports of entry; field
validation led by CBP at four northern and southern border crossings
and two seaports; and an independent evaluation, led by the DHS Science
and Technology Directorate at one of the seaports, of the new portal
monitors' effectiveness and suitability.
Since 2006, we have issued six reports and four testimonies on
development of radiation detection portal monitors, including today's
testimony, and have made 19 recommendations for improving DNDO's
efforts to develop and test portal monitors. Our concerns have focused
on key areas in which DNDO's efforts have lacked the necessary rigor
given ASPs' high cost and the importance of the radiation portal
monitor project to our national security. These areas include objective
and realistic testing of ASPs' performance in comparison with the
performance of current-generation equipment; full disclosure and
reporting of the limitations of tests used to support a decision by the
Secretary of Homeland Security on ASP certification; development of a
cost estimate that considers the full costs of the plan for deploying
radiation detection portal monitors; and development of a cost-benefit
analysis based on ASPs' demonstrated performance and a complete
accounting of the portal monitor project's costs. (App. I presents a
summary of our key findings and recommendations related to ASPs.) As I
will discuss today, DNDO has improved the rigor of testing but has not
yet updated the cost-benefit analysis that is critical to a decision on
whether to replace radiation detection equipment already deployed at
ports of entry with the significantly more expensive ASPs.
Specifically, my testimony discusses (1) our key findings on the
most recent round of ASP testing and (2) lessons from ASP testing that
can be applied to other DHS technology investments. These findings are
based on our report released this week and other related GAO
reports.\4\ We conducted this performance audit work in June 2009 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to produce a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our statement
today.
---------------------------------------------------------------------------
\4\ GAO, Combating Nuclear Smuggling: DHS Improved Testing of
Advanced Radiation Detection Portal Monitors, but Preliminary Results
Show Limits of the New Technology, GAO-09-655 (Washington, D.C.: May
21, 2009).
The Latest Round of Testing Highlights the Limitations of ASPs
Our report on the latest round of ASP testing found that DHS
increased the rigor of ASP testing in comparison with previous tests
and that a particular area of improvement was in the performance
testing at the Nevada Test Site, where DNDO compared the capability of
ASP and current-generation equipment to detect and identify nuclear and
radiological materials. For example, unlike in prior tests, the plan
for the 2008 performance test stipulated that there would be no system
contractor involvement in test execution. Such improvements addressed
concerns we previously raised about the potential for bias and provided
credibility to the results.
Nevertheless, based on the following factors, we continue to
question whether the benefits of the new portal monitors justify the
high cost:
[bullet] The DHS criteria for a significant increase in
operational effectiveness. Our chief concern with the criteria
is that they require a marginal improvement over current-
generation portal monitors in the detection of certain weapons-
usable nuclear materials when ASPs are deployed for primary
screening. DNDO considers detection of such materials to be a
key limitation of current-generation portal monitors. We are
particularly concerned about the marginal improvement required
of ASPs because the detection threshold for the current-
generation portal monitors does not specify a level of
radiation shielding that smugglers could realistically use. DOE
and national laboratory officials told us that DOE's threat
guidance used to set the current detection threshold is based
not on an analysis of the capabilities of potential smugglers
to take effective shielding measures but rather on the limited
sensitivity of PVTs to detect anything more than certain
lightly shielded nuclear materials. DNDO officials acknowledge
that both the new and current-generation portal monitors are
capable of detecting certain nuclear materials only when
unshielded or lightly shielded. The marginal improvement in
detection of such materials required of ASPs is particularly
notable given that DNDO has not completed efforts to fine-tune
PVTs' software and thereby improve sensitivity to nuclear
materials. DNDO officials expect they can achieve small
improvements in sensitivity, but DNDO has not yet funded
efforts to fine-tune PVTs' software. In contrast to the
marginal improvement required in detection of certain nuclear
materials, the primary screening requirement to reduce the rate
of innocent alarms could result in hundreds of fewer secondary
screenings per day, thereby reducing CBP's workload and delays
to commerce. In addition, the secondary screening criteria,
which require ASPs to reduce the probability of misidentifying
special nuclear material by one-half, address the inability of
relatively small hand-held devices to consistently locate and
identify potential threats in large cargo containers.
[bullet] Preliminary results of performance testing and field
validation. The preliminary results presented to us by DNDO are
mixed, particularly in the capability of ASPs used for primary
screening to detect certain shielded nuclear materials.
Preliminary results show that the new portal monitors detected
certain nuclear materials better than PVTs when shielding
approximated DOE threat guidance, which is based on light
shielding. In contrast, differences in system performance were
less notable when shielding was slightly increased or
decreased: Both the PVTs and ASPs were frequently able to
detect certain nuclear materials when shielding was below
threat guidance, and both systems had difficulty detecting such
materials when shielding was somewhat greater than threat
guidance. With regard to secondary screening, ASPs performed
better than hand-held devices in identification of threats when
masked by naturally occurring radioactive material. However,
differences in the ability to identify certain shielded nuclear
materials depended on the level of shielding, with increasing
levels appearing to reduce any ASP advantages over the hand-
held identification devices. Other phases of testing uncovered
multiple problems in meeting requirements for successfully
integrating the new technology into operations at ports of
entry. Of the two ASP vendors participating in the 2008 round
of testing, one has fallen behind due to severe problems
encountered during testing of ASPs' readiness to be integrated
into operations at ports of entry (``integration testing'');
the problems may require that the vendor redo previous test
phases to be considered for certification. The other vendor's
system completed integration testing, but CBP suspended field
validation after two weeks because of serious performance
problems resulting in an overall increase in the number of
referrals for secondary screening compared with existing
equipment.
[bullet] DNDO's plans for computer simulations. DNDO does not
plan to complete injection studies--computer simulations for
testing the response of ASPs and PVTs to simulated threat
objects concealed in cargo containers--prior to the Secretary
of Homeland Security's decision on certification even though
delays to the ASP test schedule have allowed more time to
conduct the studies. According to DNDO officials, injection
studies address the inability of performance testing to
replicate the wide variety of cargo coming into the United
States and the inability to place special nuclear material and
other threat objects in cargo during field validation. DNDO had
earlier indicated that injection studies could provide
information comparing the performance of the two systems as
part of the certification process for both primary and
secondary screening. However, DNDO subsequently decided that
performance testing would provide sufficient information to
support a decision on ASP certification. DNDO officials said
they would instead use injection studies to support effective
deployment of the new portal monitors.
[bullet] Lack of an updated cost-benefit analysis. DNDO has
not yet updated its cost-benefit analysis to take into account
the results of the latest round of ASP testing. An updated
analysis that takes into account the results from the latest
round of testing, including injection studies, might show that
DNDO's plan to replace existing equipment with ASPs is not
justified, particularly given the marginal improvement in
detection of certain nuclear materials required of ASPs and the
potential to improve the current-generation portal monitors'
sensitivity to nuclear materials, most likely at a lower cost.
DNDO officials said they are currently updating the ASP cost-
benefit analysis and plan to complete it prior to a decision on
certification by the Secretary of Homeland Security.
Our report recommended that the Secretary of Homeland Security
direct DNDO to (1) assess whether ASPs meet the criteria for a
significant increase in operational effectiveness based on a valid
comparison with PVTs' full performance potential and (2) revise the
schedule for ASP testing and certification to allow sufficient time for
review and analysis of results from the final phases of testing and
completion of all tests, including injection studies. We further
recommended that, if ASPs are certified, the Secretary direct DNDO to
develop an initial deployment plan that allows CBP to uncover and
resolve any additional problems not identified through testing before
proceeding to full-scale deployment. DHS agreed to a phased deployment
that should allow time to uncover ASP problems but disagreed with GAO's
other recommendations, which we continue to believe remain valid.
Procurement Decisions for New Technologies Require Rigorous Testing and
Thorough Analysis of Results
The challenges DNDO has faced in developing and testing ASPs
illustrate the importance of following existing DHS policies as well as
best practices for investments in complex homeland security
acquisitions and for testing of new technologies. The DHS investment
review process calls for executive decision-making at key points in an
investment's life cycle and includes many acquisition best practices
that, if applied consistently, could help increase the chances for
successful outcomes. However, we reported in November 2008 that, for
the period from fiscal year 2004 through the second quarter of fiscal
year 2008, DHS had not effectively implemented or adhered to its
investment review process due to a lack of senior management officials'
involvement as well as limited monitoring and resources.\5\ In
particular, of DHS's 48 major investments requiring milestone and
annual reviews under the Department's investment review policy, 45 were
not assessed in accordance with this policy. In addition, many major
investments, including DNDO's ASP program, had not met the Department's
requirements for basic acquisition documents necessary to inform the
investment review process. As a result, DHS had not consistently
provided the oversight needed to identify and address cost, schedule,
and performance problems in its major investments. Among other things,
our November 2008 report recommended that the Secretary of Homeland
Security direct component heads, such as the Director of DNDO, to
ensure that the components have established processes to manage major
investments consistent with departmental policies. DHS generally
concurred with our recommendations, and we noted that DHS had begun
several efforts to address shortcomings in the investment review
process identified in our report, including issuing an interim
directive requiring DHS components to align their internal policies and
procedures by the end of the third quarter of fiscal year 2009. In
January 2009, DHS issued a memorandum instructing component heads to
create acquisition executives in their organizations to be responsible
for management and oversight of component acquisition processes. If
fully implemented, these steps should help ensure that DHS components
have established processes to manage major investments.
---------------------------------------------------------------------------
\5\ GAO, Department of Homeland Security: Billions Invested in
Major Programs Lack Appropriate Oversight, GAO-09-29 (Washington, D.C.:
Nov. 18, 2008).
---------------------------------------------------------------------------
Based on our body of work on ASP testing, one of the primary
lessons to be learned is to avoid the pitfalls in testing that stem
from a rush to procure new technologies. GAO has previously reported on
the negative consequences of pressures imposed by closely linking
testing and development programs with decisions to procure and deploy
new technologies, including the creation of incentives to postpone
difficult tests and limit open communication about test results.\6\ We
found that testing programs designed to validate a product's
performance against increasing standards for different stages in
product development are a best practice for acquisition strategies for
new technologies. In the case of ASPs, the push to replace existing
equipment with the new portal monitors led to a testing program that
until recently lacked the necessary rigor. Even for the most recent
round of testing, DNDO's schedule consistently underestimated the time
required to conduct tests, resolve problems uncovered during testing,
and complete key documents, including final test reports. In addition,
DNDO's original working schedule did not anticipate the time required
to update its cost-benefit analysis to take into account the latest
test results. The schedule anticipated completion of testing in mid-
September 2008 and the DHS Secretary's decision on ASP certification
between September and November 2008. However, testing is still not
completed, and DNDO took months longer than anticipated to complete the
final report on performance testing.
---------------------------------------------------------------------------
\6\ GAO, Best Practices: A More Constructive Test Approach Is Key
to Better Weapon System Outcomes, GAO/NSIAD-00-199 (Washington, D.C.:
July 31, 2000).
As previously mentioned, a number of aspects of the latest round of
ASP testing increased the rigor in comparison with earlier rounds and,
if properly implemented, could improve the rigor in DHS's testing of
---------------------------------------------------------------------------
other advanced technologies. Key aspects included the following:
[bullet] Criteria for ensuring test requirements are met. The
test and evaluation master plan established criteria requiring
that the ASPs meet certain requirements before starting or
completing any test phase. For example, the plan required that
ASPs have no critical or severe issues rendering them
completely unusable or impairing their function. The criteria
provided a formal means to ensure that ASPs met certain basic
requirements prior to the start of each phase of testing. DNDO
and CBP adhered to the criteria even though doing so resulted
in integration testing taking longer than anticipated and
delaying the start of field validation.
[bullet] Participation of the technology end-user. The
participation of CBP (the end-user of the new portal monitors)
provided an independent check, within DHS, of DNDO's efforts to
develop and test the new portal monitors. For example, CBP
added a final requirement to integration testing before
proceeding to field validation to demonstrate ASPs' ability to
operate for 40 hours without additional problems and thereby
provide for a productive field validation. In addition, the
participation of CBP officers in the 2008 round of performance
testing allowed DNDO to adhere more closely than in previous
tests to CBP's standard operating procedure for conducting a
secondary inspection using the hand-held identification
devices, thereby providing for an objective test.
[bullet] Participation of an independent test authority. The
DHS Science and Technology Directorate, which is responsible
for developing and implementing the Department's test and
evaluation policies and standards, will have the lead role in
the final phase of ASP testing and thereby provide an
additional independent check on testing efforts. The Science
and Technology Directorate identified two critical questions,
related to ASPs' operational effectiveness (i.e., detection and
identification of threats) and suitability (e.g., reliability,
maintainability, and supportability), and drafted its own test
plan to address those questions.
Mr. Chairman, this completes my prepared statement. I would be
happy to respond to any questions that you or other Members of the
Subcommittee may have at this time.
GAO Staff Acknowledgments
Ned Woodward (Assistant Director), Joseph Cook, and Kevin Tarmann
made key contributions to this testimony. Dr. Timothy Persons (Chief
Scientist), James Ashley, Steve Caldwell, John Hutton, Omari Norman,
Alison O'Neill, Amelia Shachoy, and Rebecca Shea also made important
contributions.
Appendix I:
Key Findings and Recommendations from
Related GAO Products on Testing
and Development of ASPs
Combating Nuclear Smuggling: DHS Has Made Progress Deploying Radiation
Detection Equipment at U.S. Ports-of-Entry, But Concerns
Remain. GAO-06-389. Washington, D.C.: March 22, 2006.
Key findings. Prototypes of advanced spectroscopic
portals (ASP) were expected to be significantly more expensive
than current-generation portal monitors but had not been shown
to be more effective. For example, Domestic Nuclear Detection
Office (DNDO) officials' preliminary analysis of 10 ASPs tested
at the Nevada Test Site found that the new portal monitors
outperformed current-generation equipment in detecting numerous
small, medium-size, and threat-like radioactive objects and
were able to identify and dismiss most naturally occurring
radioactive material. However, the detection capabilities of
both types of portal monitors converged as the amount of source
material decreased.
Recommendations. We recommended that the Secretary of
Homeland Security work with the Director of DNDO to analyze the
benefits and costs of deploying ASPs before any of the new
equipment is purchased to determine whether any additional
detection capability is worth the additional cost. We also
recommended that the total program cost estimate for the
radiation portal monitor project be revised after completion of
the cost-benefit analysis.
Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support the
Purchase of New Radiation Detection Portal Monitors Was Not
Based on Available Performance Data and Did Not Fully Evaluate
All the Monitors' Costs and Benefits. GAO-07-133R. Washington,
D.C.: October 17, 2006.
Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy the
Next Generation of Radiation Detection Equipment Is Not
Supported by Its Cost-Benefit Analysis. GAO-07-581T.
Washington, D.C.: March 14, 2007.
[bullet] Key findings. DNDO's cost-benefit analysis issued in
response to our March 2006 recommendation did not provide a
sound analytical basis for DNDO's decision to purchase and
deploy ASPs. We identified a number of problems with the
analysis of both the performance of the new portal monitors and
the costs. With regard to performance, DNDO did not use the
results of its own tests and instead relied on assumptions of
the new technology's anticipated performance level. In
addition, the analysis focused on identifying highly enriched
uranium (HEU) and did not consider how well the new portal
monitors can correctly detect or identify other dangerous
radiological or nuclear materials. With regard to costs, DNDO
did not follow the DHS guidelines for performing cost-benefit
analyses and used questionable assumptions about the
procurement costs of portal monitor technology.
[bullet] Recommendations. We recommended that DHS and DNDO
conduct a new cost-benefit analysis using sound analytical
methods, including actual performance data and a complete
accounting of all major costs and benefits as required by DHS
guidelines, and that DNDO conduct realistic testing for both
ASPs and current-generation portal monitors.
Combating Nuclear Smuggling: DNDO Has Not Yet Collected Most of the
National Laboratories' Test Results on Radiation Portal
Monitors in Support of DNDO's Testing and Development Program.
GAO-07-347R. Washington, D.C.: March 9, 2007.
[bullet] Key findings. DNDO had not collected a comprehensive
inventory of testing information on current-generation portal
monitors. Such information, if collected and used, could
improve DNDO's understanding of how well portal monitors detect
different radiological and nuclear materials under varying
conditions. In turn, this understanding would assist DNDO's
future testing, development, deployment, and purchases of
portal monitors.
[bullet] Recommendations. We recommended that the Secretary of
Homeland Security, working with the Director of DNDO, collect
reports concerning all of the testing of current-generation
portal monitors and review the test reports in order to develop
an information database on how the portal monitors perform in
both laboratory and field tests on a variety of indicators,
such as their ability to detect specific radiological and
nuclear materials.
Combating Nuclear Smuggling: Additional Actions Needed to Ensure
Adequate Testing of Next Generation Radiation Detection
Equipment. GAO-07-1247T. Washington, D.C.: September 18, 2007.
[bullet] Key findings. We found that tests conducted by DNDO
in early 2007 were not an objective and rigorous assessment of
the ASPs' capabilities. Specifically, we raised concerns about
DNDO using biased test methods that enhanced the apparent
performance of ASPs; not testing the limitations of ASPs'
detection capabilities-for example, by not using a sufficient
amount of the type of materials that would mask or hide
dangerous sources and that ASPs would likely encounter at ports
of entry; and not using a critical Customs and Border
Protection (CBP) standard operating procedure that is
fundamental to the performance of hand-held radiation detectors
in the field.
[bullet] Recommendations. We recommended that the Secretary of
Homeland Security delay Secretarial certification and full-
scale production decisions on ASPs until all relevant tests and
studies had been completed and limitations to tests and studies
had been identified and addressed. We further recommended that
DHS determine the need for additional testing in cooperation
with CBP and other stakeholders and, if additional testing was
needed, that the Secretary of DHS appoint an independent group
within DHS to conduct objective, comprehensive, and transparent
testing that realistically demonstrates the capabilities and
limitations of ASPs.
Combating Nuclear Smuggling: DHS's Program to Procure and Deploy
Advanced Radiation Detection Portal Monitors Is Likely to
Exceed the Department's Previous Cost Estimates. GAO-08-1108R.
Washington, D.C.: September 22, 2008.
[bullet] Key findings. Our independent cost estimate suggested
that from 2007 through 2017 the total cost of DNDO's 2006
project execution plan (the most recent official documentation
of the program to equip U.S. ports of entry with radiation
detection equipment) would likely be about $3.1 billion but
could range from $2.6 billion to $3.8 billion. In contrast, we
found that DNDO's cost estimate of $2.1 billion was unreliable
because it omitted major project costs, such as elements of the
ASPs' life cycle, and relied on a flawed methodology. DNDO
officials told us that the agency was no longer following the
2006 project execution plan and that the scope of the agency's
ASP deployment strategy had been reduced to only the standard
cargo portal monitor. Our analysis of DNDO's summary
information outlining its scaled-back plan indicated the total
cost to deploy standard cargo portals over the period 2008
through 2017 would be about $2 billion but could range from
$1.7 billion to $2.3 billion. Agency officials acknowledged the
program requirements that would have been fulfilled by the
discontinued ASPs remained valid, including screening rail cars
and airport cargo, but the agency had no plans for how such
screening would be accomplished.
[bullet] Recommendations. We recommended that the Secretary of
Homeland Security direct the Director of DNDO to work with CBP
to update the projection execution plan to guide the entire
radiation detection program at U.S. ports of entry, revise the
estimate of the program's cost and ensure that the estimate
considers all of the costs associated with its project
execution plan, and communicate the revised estimate to
Congress so that it is fully apprised of the program's scope
and funding requirements.
Combating Nuclear Smuggling: DHS Needs to Consider the Full Costs and
Complete All Tests Prior to Making a Decision on Whether to
Purchase Advanced Portal Monitors. GAO-08-1178T. Washington,
D.C.: September 25, 2008.
[bullet] Key findings. In preliminary observations of the 2008
round of ASP testing, we found that DNDO had made progress in
addressing a number of problems we identified in previous
rounds of ASP testing. However, the DHS criteria for
significant increase in operational effectiveness appeared to
set a low bar for improvement--for example, by requiring ASPs
to perform at least as well as current-generation equipment
when nuclear material is present in cargo but not specifying an
actual improvement. In addition, the ASP certification schedule
did not allow for completion of computer simulations that could
provide useful data on ASP capabilities prior to the
Secretary's decision on certification. Finally, we questioned
the replacement of current-generation equipment with ASPs until
DNDO demonstrates that any additional increase in security
would be worth the ASPs' much higher cost.\1\
---------------------------------------------------------------------------
\1\ This testimony also summarized our September 2008 report on the
life cycle cost estimate to deploy ASPs (GAO-08-1108R).
Combating Nuclear Smuggling: DHS's Phase 3 Test Report on Advanced
Portal Monitors Does Not Fully Disclose the Limitations of the
---------------------------------------------------------------------------
Test Results. GAO-08-979. Washington, D.C.: September 30, 2008.
Key findings. DNDO's report on the second group of
ASP tests in 2007 (the Phase 3 tests) did not appropriately
state test limitations. As a result, the report did not
accurately depict the results and could potentially be
misleading. The purpose of the Phase 3 tests was to conduct a
limited number of test runs in order to identify areas in which
the ASP software needed improvement. While aspects of the Phase
3 report addressed this purpose, the preponderance of the
report went beyond the test's original purpose and made
comparisons of the performance of the ASPs with one another or
with currently deployed portal monitors. We found that it would
not be appropriate to use the Phase 3 test report in
determining whether the ASPs represent a significant
improvement over currently deployed radiation equipment because
the limited number of test runs did not support many of the
comparisons of ASP performance made in the report.
[bullet] Recommendations. We recommended that the Secretary of
DHS use the results of the Phase 3 tests solely for the
purposes for which they were intended--to identify areas
needing improvement--and not as a justification for certifying
whether the ASPs warrant full-scale production. If the
Secretary intends to consider the results of the Phase 3 tests
in making a certification decision regarding ASPs, we further
recommended that the Secretary direct the Director of DNDO to
revise and clarify the Phase 3 test report to more fully
disclose and articulate the limitations present in the Phase 3
tests and clearly state which insights from the Phase 3 report
are factored into any decision regarding the certification that
ASPs demonstrate a significant increase in operational
effectiveness. Finally, we recommended that the Secretary
direct the Director of DNDO to take steps to ensure that any
limitations associated with the 2008 round of testing are
properly disclosed when the results are reported.
Combating Nuclear Smuggling: DHS Improved Testing of Advanced Radiation
Detection Portal Monitors, but Preliminary Results Show Limits
of the New Technology. GAO-09-655. Washington, D.C.: May 21,
2009.
[bullet] Key findings. We reported that the DHS criteria for a
significant increase in operational effectiveness require a
large reduction in innocent alarms but a marginal improvement
in the detection of certain weapons--usable nuclear materials.
In addition, the criteria do not take the current-generation
portal monitors' full potential into account because DNDO has
not completed efforts to improve their performance. With regard
to ASP testing, we found that DHS increased the rigor in
comparison with previous tests, thereby adding credibility to
the test results, but that preliminary results were mixed. The
results showed that the new portal monitors performed better
than current-generation portal monitors in detection of certain
nuclear materials concealed by light shielding approximating
the threat guidance for setting detection thresholds, but
differences in sensitivity were less notable when shielding was
slightly below or above that level. Testing also uncovered
multiple problems in ASPs meeting the requirements for
successful integration into operations at ports of entry.
Finally, we found that DNDO did not plan to complete computer
simulations that could provide additional insight into ASP
capabilities and limitations prior to certification even though
delays to testing allowed more time to conduct the simulations.
Recommendations. We recommended that the Secretary of
Homeland Security direct the Director of DNDO to assess whether
ASPs meet the criteria for a significant increase in
operational effectiveness based on a valid comparison with
current-generation portal monitors' full performance potential
and revise the schedule for ASP testing and certification to
allow sufficient time for review and analysis of results from
the final phases of testing and completion of all tests,
including computer simulations. If ASPs are certified, we
further recommended that the Secretary of Homeland Security
direct the Director of DNDO to develop an initial deployment
plan that allows CBP to uncover and resolve any additional
problems not identified through testing before proceeding to
full-scale deployment.
Biography for Gene Aloise
Gene Aloise is a Director in the Natural Resources and Environment
team at GAO. He is GAO's recognized expert in international nuclear
nonproliferation and safety issues and completed training on these
subjects at the University of Virginia and Princeton University. His
work for GAO has taken him to some of Russia's closed nuclear cities
and the Chernobyl reactor in Ukraine as well as numerous nuclear
facilities around the world and in the United States. Mr. Aloise has
had years of experience developing, leading, and managing GAO domestic
and international engagements. His diverse experience includes
assignments with congressional committees as well as various offices
within GAO. He has received numerous awards for his leadership and
expertise including GAO's Meritorious Service Award. Mr. Aloise
received his Bachelor's degree in political science/economics from
Rowan University and holds a Master of Public Administration from
Temple University. Mr. Aloise is also a graduate of the Senior
Executive Fellows Program, John F. Kennedy School of Government,
Harvard University.
Chairman Miller. Thank you, Mr. Aloise.
Dr. Lowenthal.
STATEMENT OF DR. MICAH D. LOWENTHAL, DIRECTOR, NUCLEAR SECURITY
AND NUCLEAR FACILITY SAFETY PROGRAM, NUCLEAR AND RADIATION
STUDIES BOARD, NATIONAL RESEARCH COUNCIL, THE NATIONAL
ACADEMIES
Dr. Lowenthal. Good morning, and thank you, Chairman
Miller, Ranking Member Broun, Members of the Committee. My name
is Micah Lowenthal. As you said, I am the Director of the
Nuclear Security and Nuclear Facility Safety Program at the
National Research Council's Nuclear and Radiation Studies
Board. I am here to describe a recently issued interim report
from a Congressionally mandated National Research Council study
on Advanced Spectroscopic Portals.
I am the study director supporting the authoring committee
of that report, and I will begin by providing background on the
request for this study and then I will summarize the main
messages of the report.
The Department of Homeland Security, or DHS, wants to
deploy new radiation detector systems called Advanced
Spectroscopic Portals, or ASPs, to improve scrutiny of
containerized cargo for nuclear and radiological material. The
ASPs are intended to replace some or all of the radiation
portal monitors (RPMs) and hand-held radioisotope identifiers
(RIIDs) currently used at ports and border crossings across the
United States. Congress required that the Secretary of Homeland
Security certify that the ASPs provide a significant increase
in operational effectiveness over the old systems. Congress
also directed DHS to ask the National Research Council to
advise the Secretary on testing, analysis, costs and benefits
of the ASPs before the certification decision.
I want to point out that the appropriations committees who
made these requirements said they appreciate that certification
will be difficult. The study committee agrees with that, that
these are hard problems, both for the testing and the cost-
benefit analysis, so it is not surprising that DHS's work has
been challenging.
Here are the main findings and recommendations from the
interim report. The study committee found that the ASP
performance tests prior to 2008 had serious flaws. The Domestic
Nuclear Detection Office, DNDO, acknowledged problems with the
tests and addressed a number of those flaws in later testing.
The 2008 performance tests were indeed improved, but
shortcomings remain. DNDO's current approach to performance
testing involves physically testing detector performance
against a small number of configurations of threat objects and
cargo in one environment. The set of possible combinations of
threats, cargo and environments is so large and
multidimensional that DNDO needs an analytical basis for
understanding the performance of its detector systems, not just
an empirical basis. Our study committee recommended that DHS
use a more rigorous approach in which scientists use computer
models to simulate configurations and detector performance, use
physical tests to validate and refine the models and use the
models to select key new physical tests that advance our
understanding of the detector systems. This iterative modeling
and testing approach is standard in the development of some
high-technology equipment and is essential for building
scientific confidence in detector performance.
The idea of an iterative approach also extends to
deployment. The study committee recommends a process for
incremental deployment and continuous improvement of the ASPs
with experience in the field leading to refinement in both the
technologies and the operations for the next deployment. As a
first step in this process, DHS should deploy the unused ASPs
it already has, to assess the ASPs' performance in multiple
environments without investing in a much larger acquisition at
the outset.
The ASP cost-benefit analysis was not complete when our
interim report was written. Preliminary estimates by DNDO
indicate that the cost increases from replacing the currently
deployed systems with ASPs outweigh the cost reduction or
savings from operational efficiencies. Therefore, a careful
cost-benefit analysis will need to reveal the security
advantages, if any, of the ASPs over the current systems and
possible alternatives. Such a cost-benefit analysis should
include three key elements: a clear statement of the objectives
of the program, an assessment of meaningful alternatives; and a
comprehensive, credible and transparent analysis of benefits
and costs.
DHS should consider tradeoffs among different options for
allocating its efforts and funds, looking at the overall system
for ways to improve defense against nuclear smuggling. The
study committee recommended that DHS not proceed with further
procurement of ASPs until it has addressed the findings and
recommendations from the report, and the ASP has been shown to
be a favored option in the cost-benefit analysis.
Now, those are the main messages from the report. For this
hearing I was also asked to comment on any lessons learned
regarding processes by which the ASPs have been researched,
developed and tested to date. Although the study committee only
examined the testing of ASPs, not a broader portfolio, I think
there are three lessons to be learned based on the study.
First, the process of modeling and testing iteratively can be
applied more broadly to other complex technology development
programs. Second, incremental deployment with continuous
improvement is a good strategy for deployments of systems that
have not fully matured, especially if they are envisioned to
have an ongoing mission. And third, the systems-level approach
that is examining how to optimize choices within the overall
system, rather than focusing narrowly on one tradeoff decision,
is applicable to almost every use of equipment in security
applications.
Thank you for the opportunity to testify. I would be glad
to elaborate my testimony in response to questions.
[The prepared statement of Dr. Lowenthal follows:]
Prepared Statement of Micah D. Lowenthal
Good morning Chairman Miller, Ranking Member Broun, and Members of
the Committee. My name is Micah Lowenthal and I am the Director of the
program on Nuclear Security and Nuclear Facility Safety in the National
Research Council's Nuclear and Radiation Studies Board.\1\ I am here to
describe the recently issued interim report from a congressionally
mandated National Research Council study on advanced spectroscopic
portals (ASPs). I am the study director supporting the authoring
committee of that report.\2\ The full report is classified, but an
abbreviated version was also produced for unrestricted public
release.\3\ My testimony is based on the abbreviated version. I will
begin by providing background on the request for this study. I will
then summarize the main messages of the report and discuss some of the
points most relevant to this hearing.
---------------------------------------------------------------------------
\1\ The National Research Council is the operating arm of the
National Academy of Sciences, the National Academy of Engineering, and
the Institute of Medicine of the National Academies, chartered by
Congress in 1863 to advise the government on matters of science and
technology. The Nuclear and Radiation Studies Board is responsible for
oversight of National Research Council studies on safety and security
of nuclear materials and waste.
\2\ Dr. Robert Dynes, a physicist at the University of California,
member of the National Academy of Sciences, and former President of the
University of California, chaired this study.
\3\ The report is titled Evaluating Testing, Costs, and Benefits of
Advanced Spectroscopic Portals for Screening Cargo at Ports of Entry:
Interim Report. The abbreviated version of the report is available
online at http://www.nap.edu/catalog.php?record-id=XXXX
BACKGROUND ON THE REQUEST FOR THE STUDY
Containerized cargo entering the United States at sea ports and
land-border crossings for trucks is currently screened for radiation
using detectors, called radiation portal monitors (RPMs), in
conjunction with hand-held radioisotope identifiers (RIIDs). The
Department of Homeland Security (DHS) is seeking to deploy new
radiation detectors, called advanced spectroscopic portals (ASPs), to
replace the current RPM and RIID combination, which has known
deficiencies. The ASPs consist of new detector equipment and new
software, including algorithms for isotope identification.
Following some controversy over the testing and evaluation of the
new ASPs, Congress required in Title IV of Division E of the
Consolidated Appropriations Act, 2008 (Public Law 110-161) that the
Secretary of Homeland Security submit to Congress a report certifying
that ASPs would provide a ``significant increase in operational
effectiveness'' over continued use of existing screening devices. This
certification is a precondition for proceeding with full-scale
procurement of ASPs. Congress also directed DHS to request that the
National Academies advise the Secretary on the certification decision
by helping to validate testing completed to date, providing support for
future testing, assessing the costs and benefits of this technology,
and bringing robustness and scientific rigor to the procurement
process. Due to delays in the test and evaluation program, the
Academies and DHS agreed that the study committee would issue an
interim report that provides (1) the committee's evaluation of testing
plans and execution it has seen, and (2) advice on how the Domestic
Nuclear Detection Office (DNDO) can complete and make more rigorous its
ASP evaluation for the Secretary and the Nation.
This interim report is based on testing done before 2008 (referred
to as past tests), plans for and preliminary results from performance
tests carried out in 2008, and the agency's draft cost-benefit analysis
as of October 2008. The committee received briefings on the performance
test results and analysis and on the cost-benefit analysis but it did
not receive any written reports on those topics by February 2009, when
the interim report entered the Academies peer review process.
I will now discuss each element of the study task below.
SUMMARY OF THE MAIN MESSAGES OF THE REPORT
First, I want to note that the committee focused much of its
attention on performance testing. This is not because the other tests
are unimportant--the portals will be of little use if they are
incompatible with CBP's computer systems, for example--but the design,
execution, and evaluation of these tests are comparatively routine,
even if solutions to problems revealed by the tests are not. The
design, execution, and evaluation of performance tests for the ASPs is
more challenging and involves more of the science and engineering
principles on which the committee has advice to offer.
Past Performance Testing
Performance tests prior to 2008 had serious flaws that were
identified by the Government Accountability Office and the Secretary's
ASP Independent Review Team. All truck-conveyed containers at ports and
border crossings currently pass through primary screening, which is
conducted with a radiation portal monitor (RPM). Containers that
trigger an alarm are sent to secondary screening, which is conducted
with an RPM and a RIID. The tests prior to 2008 did not adequately
assess the capabilities of the ASP systems in primary and secondary
screening compared with the currently deployed RPM and RIID screening
systems, nor whether the ASP systems met performance criteria for
procurement.
There were serious flaws in the testing protocol. Notably, DNDO
utilized the same radiation sources in performance testing that were
used to set up and calibrate this testing. Device setup and any
calibration must use separate radiation sources from those used for
testing. Also, standard operating procedures for the use of RlIDs in
secondary screening were not followed in the performance tests, which
disadvantaged the RIIDs in comparisons with ASPs.
2008 Performance Testing
DNDO staff acknowledged several pre-2008 deficiencies and designed
its 2008 test plan to correct them. The study committee examined the
revised test plan, observed tests, and questioned test personnel, and
the committee concluded that DNDO did address those problems.
Because of the ASP configurations and the size of their detectors,
ASPs would be expected to improve isotope identification, provide
greater consistency and coverage in screening, , and increase speed of
screening compared to the current RPM-RIID combination when used in
secondary screening. Consequently, DNDO's 2008 performance tests of
ASPs in secondary screening focused on confirming and quantifying that
advantage for several threat objects, cargoes, and configurations.
When used for primary screening, an ASP system must be compared to
the existing RPM-RIID combination for primary and secondary screening.
This is because ASPs perform an isotope identification function in
primary screening. Isotope identification is only possible in secondary
screening with the current RPM-RIID system. DNDO's preliminary analysis
did account for this difference.
The study committee found that the 2008 performance tests were an
improvement over previous tests. They enabled DNDO to identify and
physically test some of the performance limits of ASP systems. However,
the committee identified several shortcomings of these tests: (1) The
selected test configurations were too limited to assess the performance
of ASP systems against the range of threat objects, cargoes, and
configurations that could be encountered during cargo screening
operations at ports without modeling to complement the physical
experiments; (2) the sample sizes (the number of test runs of each
case) are small and limit the confidence that can be placed in
comparing ASP and RPM-RIID performance; and (3) in its analysis, some
of the performance metrics are not the correct ones for comparing
operational performance of cargo screening systems. These shortcomings
are described in greater detail in our report. In the committee's
judgment, DHS cannot determine whether ASPs can consistently outperform
current RPM-RlID systems in routine practice until these shortcomings
are addressed. Better physical measurement and characterization of the
performance of the systems are a necessary first step but may not be
sufficient to enable DHS to conclude that the ASPs meet the criteria it
has defined for achieving a ``significant increase in operational
effectiveness.''
The committee recommends modifications to the testing procedures
that are being used by DHS. These modifications would influence
subsequent procurement steps, as described in the recommendations for
the procurement process.
Recommended Approach for Testing and Evaluation
To make the testing and evaluation more scientifically rigorous,
the committee recommends an iterative approach involving modeling and
physical testing. The threat space that is, the set of possible threat
objects, configurations, surrounding cargoes, and conditions of
transport--is so large and multi-dimensional that DNDO needs an
analytical basis for understanding the capabilities of ASP detectors
for screening cargo. DNDO's current approach is to physically test
small portions of the threat space and to use other experimental data
to interpolate and extrapolate to other important parts of the threat
space to test the identification algorithms in ASP systems.
The committee recommends that DNDO use computer models of threat
objects, radiation transport, and detector response to simulate ASP
performance. Then DNDO can use physical experiments to validate these
computer models, which would allow a critique of the models' fidelity
to reality and show where model refinements were needed. Physical
testing and model refinement would proceed iteratively until the model
provided an acceptably accurate depiction of reality. With validated
models, DNDO could evaluate the performance of ASP systems over a
larger, more meaningful range of the threat space than is feasible with
physical tests alone.
This kind of interaction between computer models and physical tests
is standard in the development and deployment of some high-technology
equipment and is essential for building scientific confidence in
technology performance. The performance tests conducted in 2008, and
even prior to 2008, can be used to help refine and validate models. The
committee also notes the skills required to proceed exist in the
National Laboratories.
Recommended Approach for the Procurement Process
The idea of an iterative approach also extends to deployment of ASP
systems at ports of entry. The committee noted that DHS' testing
philosophy is oriented toward a one-time certification decision in the
near future. However, the mandate for passive radiation screening of
cargo at ports of entry is expected to continue indefinitely. Rather
than focusing on a one-time decision about the deployment of ASPs, the
committee suggests that current testing be viewed as a first step in a
continuous process of system improvement and adaptation to changes in
the threat environment, composition of container cargo, technological
and analytical capabilities, and the nature of commerce at ports of
entry. These factors have changed significantly over the last decade
and can be expected to evolve--in both predictable and unpredictable
ways--in the future. The committee recommends that DHS should develop a
process for incremental deployment and continuous improvement, with
experience leading to refinements in both technologies and operations
over time, rather than a single product purchase to replace current
screening technologies. The ASP deployment process should be developed
to address and exploit changes. This would enable DNDO to adapt and
continually update its screening systems so that they do not become
outdated as they would after a one-time deployment.
As the first step in this process, the committee recommends that
DHS deploy its currently unused low-rate initial production ASPs for
primary and secondary inspection at various sites under a program of
extended operational testing. Such deployment, even on a limited scale,
would provide valuable data concerning ASP operation, reliability, and
performance, and would allow DHS to better assess ASP capabilities in
multiple environments without investing in a much larger acquisition at
the outset.
The development of the hardware for radiation detection and the
software for analyzing detector signals is separable. The current DHS
procurement process is a competition among vendors to provide the
combined systems, which has been useful. However, as DHS moves forward,
the committee recommends that it match the best hardware to the best
software (particularly the algorithms), drawing on tools developed by
the competing companies and others, such as the national laboratories.
The deployment of ASPs will not eliminate the need for hand-held
detectors with spectroscopic capabilities. Because some of the
improvement in isotope identification offered by the ASPs over the
RIIDs is a result of software improvements, the committee recommends
that these improvements be incorporated into hand-held detectors.
Improved software might significantly improve RIID performance and
expand the range of deployment options available to Customs and Border
Protection for cargo screening.
By separating the hardware and software elements of the system and
engaging the broader science and engineering community, DHS would have
increased confidence in its procurement of the best product available
with current technology, and simultaneously could advance the state-of-
the-art.
Recommended Approach for Cost-Benefit Analysis
The preliminary analysis presented to the committee suggests that
benefits of deploying the ASPs may not be clearly greater than the
costs. Because DNDO's preliminary estimates indicate that the cost
increases from replacing the RPM-RIID combination with ASPs exceed the
cost reductions from operational efficiencies, it is important to
consider carefully the conditions under which the benefits of deploying
ASPs justify the program costs. A cost-benefit analysis (CBA) can
provide a structure for evaluating whether a proposed program (such as
the ASP program) is reasonable and justified.
The Secretary's decision on ASP certification is based, at least in
part, on whether the ASPs meet the objectives in DHS' definition of
``significant increase in operational effectiveness'' (SlOE); however,
other factors relating to the costs and benefits of the proposed ASP
program will also need to be taken into account. DHS' definition of
SlOE is a modest set of goals: As noted above, the increases in
operational efficiency do not by themselves appear to outweigh the cost
increases from replacing the RPM/RIID combination with ASPs, based on
DNDO's preliminary estimates, and the criteria do not require a
significantly improved ability to detect special nuclear material (an
ingredient of a nuclear weapon) in primary screening. If the ASPs meet
the defined goals and are able to detect the minimum quantities of
nuclear threat material recommended by the ``DOE guidance,'' DHS still
will not know whether the benefits of the ASPs outweigh the additional
costs associated with them, or whether the funds are more effectively
spent on other elements of the Global Architecture.
A CBA can provide insight about alternative choices--for example,
whether the benefits of a given program exceed its costs, and which
choices are most cost-effective. To be effective, the CBA must include
three key elements: (1) a clear statement of the objectives of the
screening program; (2) an assessment of meaningful alternatives to
deploying ASPs; and (3) a comprehensive, credible and transparent
analysis of in-scope benefits and costs.
The CBA should begin with a clear statement of what operational
problem the ASPs are intended to address. This statement will define
the role that the system plays in providing a layer in the defense
against the importation of nuclear or radiological materials. The CBA
should also include a narrative that clarifies how improving detection
for containers at ports of entry to the United States fits into a
larger effort to improve detection capabilities, in recognition of the
many ways that materials could be brought into the United States
through ports of entry that are not already screened, or across
uncontrolled stretches of border. Furthermore, to be useful in a
procurement decision, a CBA must address whether funds are better spent
to replace the currently deployed equipment rather than to expand
coverage for other material pathways that currently have no radiation
screening. DHS should consider tradeoffs among different options for
allocating efforts and funds, looking at the overall system for ways to
improve defenses against nuclear smuggling. Such an analysis is needed
in the CBA for ASP systems because it is not evident that it has been
provided elsewhere.
The CBA also needs to account for meaningful alternatives
(including non-ASP systems) to reveal the scale of the benefits of ASPs
for radiation screening and determine whether these benefits outweigh
the additional costs for procurement and deployment. The complexity of
the container screening task suggests that there could be many
different options worthy of consideration. These options include
variations on ASP deployment configurations and operational processes,
and application of technologies beyond the current RPM-RIID and ASP
systems such as deploying hand-held passive detectors with state-of-
the-art software and advanced methods for detecting nuclear materials.
Considerations should include active interrogation, improved imaging
systems, and integration of these existing technologies. These
alternatives need to be compared to a baseline that reflects as
realistically as possible the screening capability that DHS currently
has in place. This baseline should reflect the number and placement of
current RPM and RIID detectors, sensitivity of these detectors based on
how they are operated at each port, and performance of existing hand-
held detectors in the manner they are used in the field. The CBA must
indicate what capability an investment in ASPs will provide beyond the
existing systems as they are currently deployed and operated, or beyond
alternative radiation detection technologies that could be developed
and deployed at ports of entry.
In comparing alternatives, it is important that the CBA treat
benefits and costs in a comprehensive, credible, and transparent
manner. The benefit assessment should show how the ASP system would
contribute to improving security with respect to prevention of the
detonation of a nuclear device or radiological weapon in the United
States. Because this is the primary objective of the ASP program, a CBA
that is silent on this subject would be incomplete. Such an assessment
is difficult and no assessment of such benefits will be definitive or
unassailable. The cost assessment should cover all phases of the
acquisition life cycle in a manner that is independent of contractor or
program office biases, and it should also assess the risk of cost
escalation associated with the estimate.
The committee recommends that DHS not proceed with further
procurement until it has addressed the findings and recommendations in
this report, and then only if the ASP is shown to be a favored option
in the CBA.
``Lessons Learned''
For this hearing I was also asked to comment on ``lessons learned''
regarding the processes by which the ASPs have been researched,
developed, and tested. Although the study committee only examined the
testing of ASPs, my personal view is that three lessons that could be
learned:
First, the process of iterative modeling and testing can be applied
more broadly to other complex technology development programs. Modeling
coupled to validating experiments is a necessity for some technology
applications because of the complex conditions in which these
technologies must operate.
Second, incremental deployment with continuous improvement is a
good strategy for deploying systems that have not fully matured,
especially if they are envisioned to have an ongoing mission.
Third, a systems-level approach--that is, examining how to optimize
choices within the overall system rather than narrowly focusing on one
tradeoff decision--is applicable to almost every use of equipment in
security applications.
This concludes my testimony to the Committee. Thank you for the
opportunity to testify on this important topic. I would be happy to
elaborate on any of my comments during the question and answer period.
Biography for Micah D. Lowenthal
Micah Lowenthal is the Director of the Nuclear Security and Nuclear
Facility Safety Program in the Nuclear and Radiation Studies Board at
the National Research Council of the National Academies. At the
Academies, he has directed domestic and international studies on
nuclear and radiological safety and security, nuclear nonproliferation,
nuclear fuel cycles, radioactive waste, and management of contaminated
environments. Before joining the National Academies' staff in 2001, Dr.
Lowenthal was a researcher and lecturer at the University of California
at Berkeley. In 1996 he was an American Association for the Advancement
of Science Environmental Science and Engineering Fellow. Dr. Lowenthal
received an A.B. degree in physics and a Ph.D. degree in nuclear
engineering, both from the U.C.-Berkeley.
Chairman Miller. Thank you, Dr. Lowenthal. Unfortunately,
you all probably know that our days here are somewhat
scattered, and I need to go to the Floor. We have tried to
arrange another Member of the Majority who is a Member of the
Subcommittee to preside in my absence. Ms. Dahlkemper has been
an especially conscientious Member and she may arrive between
now and 11:00, which is when I expect to come back, and if that
is the case, we will come out of recess and she will preside or
any other Member of the Majority who is a Member of the
Subcommittee.
Mr. Broun. Mr. Chairman, I would be glad to preside if you
would like me to.
Chairman Miller. Well, I thank Dr. Broun for that generous
offer but I think we will do it the way that every other
committee and subcommittee in Congress does it. So with that,
we will need to stand in recess. I apologize again for the
inconvenience. Believe me, it is more inconvenient for me than
it is for you, and we will be back in session but I appreciate
your being here and I thank you for your indulgence.
[Recess.]
Discussion
Chairman Miller. We are now out of recess. Floor colloquies
are undoubtedly one of the most peculiar stylized aspects of
Congressional service. I have heard of political campaigns that
80 percent of all the effort is wasted, the problem is, no one
knows what the 20 percent is that is not. I suspect that there
is at least that ratio of wasted effort in Congressional
service and I very much suspect Floor colloquies are in the
wasted effort category. But it was something I had to do.
Mr. Broun. Floor debate, isn't it?
Chairman Miller. I will resist temptation to point out
other aspects of Congressional service that may be wasted
efforts. If we may now begin our rounds of questioning, Mr.
Aloise, Dr. Lowenthal, Dr. Broun, I now recognize myself for
five minutes.
Prioritizing Security Needs
Dr. Broun raised the point that there are other priorities
that we have to decide between, priorities even within
security, so it is not security versus other things government
does. In October of 2007, this committee held a hearing on an
exercise on a scenario of a terrorist attack assuming that a
dirty bomb was detonated in two American cities, and what we
found was that we were probably better prepared, certainly
better prepared for Katrina than we were for dirty-bomb
attacks, despite the fact that if there is one punch that has
obviously been telegraphed by terrorists, by al-Qaeda in
particular, it is a dirty-bomb attack. We did not have the
capacity to conduct the test to see what the level of
contamination was, to let our people know that they can go back
into offices, office buildings--that we might shut down,
indefinitely, 10, 15, 20 square blocks of an American city, the
midtown of an American city or perhaps two--that would probably
take six years to complete the necessary environmental tests
for an attack in one city, nor testing on individuals to see if
they had been exposed to radiation. That might take a couple
years. We would have to tell parents that we could not test
their children to see if they had been exposed to radiation for
a couple years. But, you know, if the child's hair started
falling out, call us and maybe we can move them up in line. And
the failing there seemed to be simply one of funding. We are
not prepared to respond to a radiological attack, and here we
see we are spending $2 or $3 billion to develop a technology
that those who will use it really question whether it does much
if any good at all, certainly not enough to justify $2 or $3
billion. DNDO expects the Secretary of Homeland Security to
make a decision this October, four months from now, about
whether to certify that it is worth investing the $2 billion in
the ASPs. Do either of you believe that there will be enough
information available this fall for DHS to make that decision,
and how confident are you that the costs of the ASPs justify
the investment? Mr. Aloise.
Mr. Aloise. Well, that is the question. In our view, until
integration testing is done, field validation is done,
operational testing is done, the injection studies are
complete, an analysis of energy windowing versus the ASPs is
done, until all the testing is done and we know whether or not
this marginal increase in security is justified through a
sound, independent cost-benefit analysis, it is our belief that
we shouldn't spend any money on this program or go forward with
it.
Chairman Miller. And do you think there is any realistic
chance those preconditions may be satisfied between now and
October?
Mr. Aloise. At this point it does not look that way.
Chairman Miller. Dr. Lowenthal.
Dr. Lowenthal. We agree that these are the preconditions
for going ahead with certification, that there is a great deal
of information that is needed. I am not really in a position to
tell you how much time it will take them to get that
information together. There is some information that they
presumably have already put together, some analysis that they
have already done that we haven't seen. We will be happy to
look at it when they provide it. But in terms of timelines, we
would not be inclined to guess when they will be ready.
Decision-making: Processes and Timelines
Chairman Miller. Both of you appear to disagree with DHS's
view that they will be able to obtain the data they need from
field evaluation tests that are to begin next month. Why do you
not believe that those tests will be good enough for DHS to
base a decision on? Mr. Aloise.
Mr. Aloise. Well, that is only part of, as I just said,
what they need to base their decision on. You know, field
validation testing has not gone well so far. They suspended it
after two weeks because they were actually sending more--the
ASPs were actually sending more to secondary than the PVTs so
they had to work out a number of those problems and I am
assuming they have. It has been a while since the initial
testing. We just have to see how it goes, and we are going to
be looking to CBP for a lot of the answers because they are the
agency that has to live with this equipment, and I--we feel
fairly confident that they are not going to go forward until
they are satisfied that the testing has gone right at this
point.
Chairman Miller. Dr. Lowenthal.
Dr. Lowenthal. The testing that they are going to do for
field validation is going to tell them something that they need
to know but it won't tell them everything that they need to
know, and the committee--the study committee's recommendations
were to enhance what they are going to be able to learn from
these sorts of tests with modeling and simulations. The range
of environments that they are going to be operating in is very
limited. They are only doing field validation testing in four
different sites, and there are a lot of other conditions, a lot
of other kinds of cargo and so on that are worth learning
about. Some of that can be done with simulation and some of it
can be done with the deployment that the committee recommended.
Chairman Miller. Thank you. My time has expired.
Dr. Broun for five minutes.
Mr. Broun. Thank you, Chairman.
ASP in Context: The Global Nuclear Detection Architecture
How would y'all--``y'all'' is a southern word. How would
you all prioritize the upgrade from the PVT RIID system to the
ASP system within the entire global nuclear detection
architecture? Let us start with Dr. Lowenthal.
Dr. Lowenthal. I should point out that the committee really
only looked at the testing of the ASPs and talked about what
kinds of information a decision-maker would want to have; and
so the committee recommended that they (DHS) consider these
other components of the global architecture--and there are gaps
that are known and have been pointed out by others within that
architecture--but the committee did not study that question,
itself. What the committee did say is that if you look at this
from the perspective of an adversary who is trying to sneak
something into the United States, if you can--if you reinforce
the places where you already are screening but you leave other
avenues totally open, you may not be gaining much improvement
in security.
Mr. Broun. Mr. Aloise.
Mr. Aloise. We agree. As I mentioned to Dr. Lowenthal a
minute ago, we are spending a lot of money upgrading locks on
the front door but the windows and the back door are wide open.
Does that make sense to do with limited resources, so----
Mr. Broun. I agree wholeheartedly. Are there other areas
that would benefit greater from the same level of investment,
Mr. Aloise?
Mr. Aloise. Well, there are a lot of vulnerabilities that
need to be looked at that have not been properly addressed in
the architecture. The border crossing is really not the
greatest threat, so I would--we believe that we should be
looking equally hard at all the other vulnerabilities in the
architecture because as it was just said, that is probably not
where they are going to come through. If they have taken the
time and trouble to get this material, it is unlikely they are
going to stick it in a cargo container and drive it across
through a portal monitor.
Mr. Broun. And with open borders, they can just carry it in
wherever they want to carry it.
Dr. Lowenthal, particular--and let me ask you to answer
your question in view of comments you made in your testimony
about that we have ASPs sitting there now, if you would include
that discussion?
Dr. Lowenthal. Well, given the scope of our study, I
wouldn't be comfortable to try to identify what the most
promising area is. What the committee recommended is that DHS
do that analysis, look at the entire system. We have some
concerns that that hasn't been done, and--I will separate this
into two parts. One is that the committee recommended that DHS
look at that whole system in addition to just the tradeoff
between the ASPs and the PVTs, because as a decision-maker, the
people in the Administration and in Congress will want to know
whether the money is well spent. The analysis does not appear
to have been done elsewhere and so the committee recommends
that it be included in this ASP decision. Now, the other
component of that is the deployment of the ones that they
already own. That is more for a learning exercise. This is
something where they can learn more about the performance of
the systems in the field and improve them as they go forward. I
think these are directed at slightly different points.
Mr. Broun. Thank you. My time disappeared, so I guess I am
out of time. I am not sure where we stand. Thank you, Mr.
Chairman.
Chairman Miller. Well, you will have time for a second
round because I was planning to ask at least one more question.
I now recognize myself for a second round of questions.
Management of ASP Testing and Deployment
Your criticism, for both of you, the criticism appears to
be as much that the problems with the ASPs was not the
technical challenges but that the whole program was just poorly
managed. This is a recurring theme for this committee, you
know, how could management--how could programs be managed
better. But if you think the whole testing program has been
troubled from the start, and that the management more than
anything else is a problem, what lessons should we learn? How
should this program be managed differently? How can it be
managed differently in the future, this and other similar
programs?
Mr. Aloise. Well, I will take a shot at that. Back in 2006
when we issued our first report on the deployment of PVTs, we
learned about the ASPs and we knew then that there would only
be a marginal improvement in security. That is why we called
for a cost-benefit analysis before anything went further; is it
worth the marginal increase in security you are going to get
from the ASPs? Do that first, then go talk to the Congress and
the decision-makers can decide whether it is worth it or not,
and we know the story. It was poorly done, and it was based on
assumptions and not data, and that leads us to where we are
today. It was a rush to push technology through that was
immature, and that has got to be the true lesson of this, is
that without proper and scientifically rigorous testing,
without a technology that is mature, it should not be generally
rushed to deployment--especially when you have a system already
there that is working.
Chairman Miller. Right. Dr. Lowenthal.
Dr. Lowenthal. I should point out that our study committee
was silent on the matter of whether this was managed well or
not, but I think that the report highlights many of the points
that you made in your opening remarks about what should be done
in the future. I think you hit on all of those--a well-planned-
out testing program with criteria established in advance and an
idea as you go in for what you are going to learn from each
stage of this, making sure that it matches up with what the
customer is looking for, as Mr. Aloise mentioned. All of those
components lead to having a testing and evaluation program that
will lead to procurement that gives the customer what he needs
and the Nation what it needs.
Chairman Miller. Would it be a fair summary to say that the
problem is that we began with the conclusion and then found the
analysis to support it, rather than beginning with the
analysis, and then finding what conclusions would come from the
analysis?
Mr. Aloise. I think so, Mr. Chairman. I think we had a
solution in search of a problem.
Chairman Miller. There is a problem but there may have been
another solution. Our clock is not working, so each Member
needs to show self-restraint. I will now go to Dr. Broun for a
second round. Ms. Dahlkemper has arrived and I think is now
collecting herself.
Dr. Broun for five minutes, however that might be measured.
Five minutes based on the honor system.
Mr. Broun. I hope Ms. Dahlkemper can collect herself. I
trust that she can. She is very good at doing that.
I am very interested to hear Dr. Lowenthal's comments about
whether we can take the ASP units that are available now, put
them in a parallel screening process out there in the field, as
the appropriators evidently are preventing us from doing. Would
you recommend that we do so, and is that something that you
could put in your report to us, so that we can look at that
possibility and maybe even make that recommendation?
Dr. Lowenthal. Yes. Our study committee recommended that
they take the--they procured a number of these low rate initial
production [units] so that they can test and evaluate them.
They have a number of them that have been sitting moth-balled.
I understand that some of them have been cannibalized for parts
for other ones, and we don't have an exact number of how many
they have, but they have some number of them that they already
own that they could deploy in the field. Our study committee's
recommendation, explicitly in the report, is that these should
be deployed at selected sites, and the experience gained from
those will contribute to both their understanding of how they
perform, but also how they might be able to modify the
operations at the sites, and also the technology, what
improvements they might want to put in there. So that is
explicitly in the report.
Mr. Broun. Mr. Aloise, any comments?
Mr. Aloise. Just that we would agree with that. We think
that is a smart idea.
Mr. Broun. Thank you. I hope that the CBP people can be
consulted because I think the end-user has not been consulted
enough in this process, and with that, Mr. Chairman, I will
yield back.
Chairman Miller. Thank you. You may or may not have had
time to yield back. You probably did.
Ms. Dahlkemper for more or less five minutes.
Ms. Dahlkemper. Thank you very much, and I apologize for
being late. I had a markup in another committee, but I
appreciate the opportunity to ask a question of both of you if
you could just address this.
Justifying the Cost of the Advanced Spectroscopic Portal
Program
Can either of you provide some insight into whether
improving portal radiation detection is worth $2 billion to $3
billion? Assuming that ASPs prove to be robust enough to use,
and they deliver real improvement in detection in commercial
flow, would that still be enough to justify deploying them,
given all the other needs out there and our limited resources?
Mr. Aloise. I have to go back to, and I know I sound like a
broken record on this, a cost-benefit analysis that we have
been calling for a couple years, although those factors that
you mentioned need to be in there so that the decision-makers
can look at and see whether the marginal improvement is worth
it, considering all the vulnerabilities we have in the
architecture already that we need to address.
Ms. Dahlkemper. Dr. Lowenthal.
Dr. Lowenthal. We agree entirely that a cost-benefit
analysis will be needed to show that. We are not in a position
to judge whether they are going to be worth it, partly because
our study didn't examine the results of their tests for this
interim report. It looked at the approach. Once they have the
results and they can factor those into their cost-benefit
analysis, then a judgment can be made as to whether they are
worth it.
Ms. Dahlkemper. Mr. Aloise, the GAO has done a lot of work
on how agencies should do a cost-benefit analysis. Can you lay
out some principles that should be guiding DHS as they begin
their own cost-benefit analysis on whether to acquire ASPs or
not?
Mr. Aloise. Yes. In fact, we just developed this about a
year ago on how to develop a cost-benefit analysis. It was
based on working with 90 representatives from all the federal
agencies including DHS and it lays out exactly what the best
practices are. But some of the things must be based on fact,
must be fact based, based on data, not assumptions, must have
proper documentation all along the way, and those are the kinds
of things that we are missing from the ones we looked at for
DNDO.
Ms. Dahlkemper. Anything specific to this particular----
Mr. Aloise. In particular, test results, what do the test
results show.
Ms. Dahlkemper. All right. Dr. Lowenthal, do you want to
comment on that?
Dr. Lowenthal. If I can just add one point, we give some
advice on how to complete the cost-benefit analysis but one of
the things that is pointed out in our report is that no cost-
benefit analysis at the end that deals with security matters is
going to be totally unassailable. They are going to have to
produce something that they can defend, but you are always
going to be able to pick at some part of it. This is a
difficult problem. This is not an easy issue to work through.
Ms. Dahlkemper. Thank you. I yield back.
Chairman Miller. Thank you, Ms. Dahlkemper. You still had
five minutes.
That concludes the questioning of this first panel. Again,
we thank both Mr. Aloise and Dr. Lowenthal, and we will stand
at ease for a second while this panel can step down and the
next panel can come up.
Panel II:
At this time I would like to introduce our second panel.
Dr. William Hagan is the Acting Deputy Director of the Domestic
Nuclear Detection Office at the Department of Homeland
Security. That is also a mouthful of a title. Mr. Todd C. Owen
is the Acting Deputy Assistant Commissioner of the Office of
Field Operations at the U.S. Customs and Border Protection,
Department of Homeland Security. You each have five minutes for
your spoken testimony. Your written testimony will be included
in the record. When we have completed your spoken testimony, we
will begin with questions. Each Member will have five minutes
to question the panel. It is the practice of this subcommittee
to receive testimony under oath. Do either of you have any
objection to taking an oath? Both witnesses indicated that they
do not. If not, you also have the right to be represented by
counsel. Do either of you have counsel here? And both indicated
they do not have counsel present. Please stand and raise your
right hand. Do you swear to tell the truth and nothing but the
truth?
The record should reflect that both witnesses took the
oath. We will begin with Dr. Hagan.
STATEMENT OF DR. WILLIAM K. HAGAN, ACTING DEPUTY DIRECTOR,
DOMESTIC NUCLEAR DETECTION OFFICE, DEPARTMENT OF HOMELAND
SECURITY
Dr. Hagan. Good morning, Chairman Miller, Ranking Member
Broun and distinguished Members of the Subcommittee. As Acting
Deputy Director of the Domestic Nuclear Detection Office, DNDO,
at the Department of Homeland Security, I am honored to be here
with my colleague from U.S. Customs and Border Protection
(CBP), Mr. Todd Owen. I would like to thank the Committee for
the opportunity to share lessons learned and progress to date
on our Advanced Spectroscopic Portal Program. I would also like
to thank the Committee for its support of DNDO's mission to
reduce the risk of radiological and nuclear terrorism to the
Nation.
Over the past three years, we have made substantial
investments in the development of the next generation of
radiation portal monitor known as the Advanced Spectroscopic
Portal, or ASP. The goal of the ASP program is to advance
passive detection for cargo inspection. ASP is expected to
automatically discriminate threat from non-threat materials for
unshielded to lightly shielded threats in primary inspection
while improving identification capability in secondary
inspection. To ensure that ASP systems achieve the necessary
technical and operational performance, we are in the midst of
putting them through a rigorous test campaign. ASP has already
advanced through several rounds of performance testing, and
will face field validation at ports of entry, as well as
independent operational testing and evaluation conducted by the
DHS Science and Technology Directorate's operational test
authority. The successful completion of these tests, along with
other analyses in consultation with the National Academy of
Sciences, will then inform the Secretary's certification
decisions for ASP this fall.
In addition, during the execution of our test campaigns, we
have cooperated with the Government Accountability Office and
the National Academy of Sciences to provide both groups with
information and visibility into our testing and analysis
processes. The resulting review as issued by the GAO and the
NAS provide a valuable external assessment of the ASP program.
The 2008-2009 set of ongoing tests reflect a number of
steps DNDO has taken to reform test processes. For example, one
of our most fundamental improvements has been to standardize
test event planning. We now utilize detailed instructions with
a six-milestone process to ensure that test planning is done
openly, that partner inputs are included, and that test events
are designed to provide the data required to meet the
objectives of the test and to help make programmatic decisions.
Additionally, some improvements in the ASP program have been
the result of DHS-wide enhancements to program management. The
changes put in place within DHS further ensure that any
eventual certification or acquisition decisions are consistent
with DHS priorities and made with a strong acquisition
management foundation.
Specifically, we are implementing DHS's new management
directive 102-01 for large acquisition programs which requires
a complete set of analysis, testing and documentation as input
for an acquisition decision. This set includes, for example,
analysis of alternatives, concept of operations, operational
requirements, mission need statements, cost-benefit analysis
and others. We are acutely aware that we must integrate end-
user requirements and conduct tests and evaluation campaigns.
We recognize that the development and deployment of new systems
can be expensive and the cost to taxpayers must be justified by
the increased capabilities of the equipment. We feel that the
plans and procedures now in place for the ASP program provide a
sound foundation for future certification and acquisition
decisions. ASP systems have been under review and evaluation
for over three years, and further improvements will always be
possible, but I believe that after the plan testing and
analysis is complete and the requirements of MD 102-01 have
been fulfilled, DHS will be in a position to make an informed
decision.
As a final note, I would like to emphasize that while we
are diligently working to characterize and evaluate the
performance of ASP to ensure that operators receive appropriate
equipment for their mission, the ASP program is only one piece
of the multi-layered solution we call the global nuclear
detection architecture through which we seek to integrate
efforts across the government into an overarching strategy to
improve the Nation's nuclear detection capabilities.
Accordingly, we plan to continue deployments of detection
systems at our official ports of entry while also dedicating
increased time and effort to a wider range of pathways. In
addition, we have been working to strengthen the architecture
by filling gaps, improving technologies, building necessary
infrastructure and raising awareness about radiological and
nuclear threats and the role of detection systems.
I look forward to continuing to work with our partners
within DHS, other federal departments and State and local
agencies and the Members of this subcommittee and the Congress
to help keep the Nation safe from radiological and nuclear
terrorism.
This concludes my statement. I thank you for your
attention. I am happy to answer any questions.
[The prepared statement of Dr. Hagan follows:]
Prepared Statement of William K. Hagan
Introduction:
Good morning Chairman Miller, Ranking Member Broun, and
distinguished Members of the Subcommittee. As Acting Deputy Director of
the Domestic Nuclear Detection Office (DNDO) at the Department of
Homeland Security (DHS), I would like to thank the Committee for the
opportunity to share lessons learned and progress to date on our
Advanced Spectroscopic Portal (ASP) program. I would also like to thank
the Committee for its support of DNDO's mission to reduce the risk of
radiological and nuclear terrorism to the Nation.
DNDO was established to improve the Nation's capability to detect
and report attempts to import, possess, store, develop, or transport
nuclear or radiological material for use against the Nation, and to
further enhance this capability over time. To that end, our work is
guided by our development of a global nuclear detection architecture
(GNDA). DNDO has developed a time-phased, multi-layered, defense-in-
depth GNDA that is predicated on the understanding that no single layer
of defense can detect all radiological or and nuclear (rad/nuc)
threats. For this reason, the GNDA provides multiple detection and
interdiction opportunities overseas, at our borders, and within the
United States to effectively increase the overall probability of system
success. DNDO has worked with intra- and interagency partners to
develop time-phased strategies and plans for improving the probability
of detecting and interdicting nuclear threats. DNDO will continue to
enhance the GNDA over time by developing better detection technologies,
working with our operational partners to improve concepts of operations
(CONOPs), enabling real-time reporting of detection events, and
supporting effective response to real threats.
My testimony today will include a status update and lessons learned
in DNDO's efforts to address one aspect of the GNDA--scanning cargo
containers at ports of entry. Specifically, I will focus on the ASP
program--a program to improve the detectors used to perform this task.
Role of Container Scanning and ASP in the GNDA:
The United States border is the first layer within the GNDA where
the United States has full control over detection and interdiction. As
such, considerable effort and resources have been placed at this layer
to provide comprehensive radiological and nuclear detection
capabilities, particularly at ports of entry (POEs). After 9/11,
considerable concern was raised about the possibility that terrorists
could use the enormous volume of cargo flowing into the United States
as a pathway for bringing in nuclear material or a nuclear weapon. By
far, the largest mode for incoming cargo is maritime shipping
containers, with approximately 11 million containers coming into the
country every year. Additionally, in the Security and Accountability
for Every (SAFE) Port Act of 2006, Congress mandated that all
containers coming in through the top 22 ports, by volume, be scanned
for radiation by the end of 2007.
A key consideration in rad/nuc detection is the ability to
effectively detect threats without impeding the flow of legitimate
trade and travel across the border. United States Customs and Border
Protection (CBP) currently scans cargo entering at our nation's POEs
using polyvinyl toluene (PVT)-based radiation portal monitors (RPMs)
that can detect radiation, but cannot distinguish between threat
materials and naturally occurring radioactive material (NORM), such as
kitty litter and ceramic tiles. Narrowing down alarms to just those for
dangerous materials is especially important for POEs that have a high
volume of containers, or those that see a high rate of NORM. To address
this limitation, DNDO is developing next-generation technology--the ASP
program. The ASPs have shown significantly improved capability to
distinguish rad/nuc threats from non-threats over the hand-held
instruments currently used in secondary screening. Thus, the
introduction of ASP systems is expected to not only reduce the number
of unnecessary referrals and false positives in primary but increase
the probability of detecting dangerous materials in secondary.
As you know, DNDO initiated the ASP program in 2006, building on
previous work within CBP and the Science and Technology Directorate.
ASP systems are the next generation of radiation portal monitors. ASP
units are now being developed by two separate vendors. These units have
been subjected to rigorous tests and both systems will complete several
rounds of performance testing and field validation at POEs. Following
these performance tests, both systems will complete independent
operational testing and evaluation conducted by the DHS Science and
Technology Directorate's Operational Testing Authority. Test data will
be analyzed and provided in support of the Secretary's Certification
decision. DNDO is also engaged with the National Academy of Sciences
(NAS) to allow them to review ASP testing and inform the certification
process, as required in the FY 2008 and 2009 Homeland Security
Appropriations bills. Indeed, in its most recent report on ASP testing,
the Government Accountability Office (GAO) acknowledged the many
enhancements and lessons that DNDO has incorporated into its testing
programs.
Reviews to date and lessons learned:
Since 2006, the ASP program has undergone extensive review from
outside agencies, including the GAO, an Independent Review Team (IRT)
established by the previous DHS Secretary, and, most recently, the
National Academy of Sciences. We have taken each review seriously,
valued the recommendations that have been provided, and, where we felt
appropriate, we have incorporated their recommendations into the next
stages of the program.
The first reviews conducted by the GAO of the ASP program focused
on testing conducted in 2005 as part of the original ASP vendor
selection and the initial cost-benefit analysis used to evaluate
potential deployment options for ASP systems. In its report, released
in September 2006, GAO questioned the methods used by DNDO to quantify
performance capabilities of new systems, and insisted that ASP
performance be evaluated against system requirements prior to full
scale deployment. DNDO concurred with the need for additional testing
prior to full scale deployments, which were underway prior to the
release of GAO's report. These tests were conducted throughout 2007 and
focused specifically on evaluating the performance of ASP systems in a
number of testing environments.
In September 2007, GAO recommended that DHS establish an
independent body to conduct additional testing of ASP systems, which
DNDO agreed to, launching the ASP-Independent Review Team (IRT), a team
of independent experts, drawn from a wide range of institutions and
backgrounds. The ASP-IRT Report, delivered in February 2008, provides a
valuable independent assessment of the ASP program, and served as an
important source of information, albeit based on the data that was
available at the time.
In addition to the reviews of 2007 testing, at the conclusion of
initial ``field validation'' testing in 2007, CBP identified a number
of functional improvements, unrelated to detector performance, that
required modifications to ASP systems prior to deployment. DNDO
postponed efforts to seek certification at that time, initiated new
efforts to develop these requested changes , and conducted a new series
of tests in 2008 and early 2009 to ensure that these changes did not
detract from detector performance.
The 2008-2009 test campaign transitions the program from
developmental to functionality and performance testing, culminating in
full operational tests. This testing includes:
1. System Qualification Testing, designed to demonstrate that
ASP units are manufactured in accordance with processes and
controls that meet the specified design requirements;
2. Performance Testing at the Nevada Test Site (NTS), designed
to evaluate ASP, PVT, and radioisotope identification devices
(RIID) detection and identification performance against
controlled, realistic threat materials, shielding and masking
scenarios;
3. Integration Testing, designed to determine whether the ASP
systems are capable of operating and interfacing with the other
equipment found in operational settings;
4. Field Validation Testing, designed to exercise the ASP in a
stream of commerce environment at POEs;
5. Operational Test and Evaluation (OT&E), designed to measure
the operational effectiveness and suitability of ASP. The OT&E
will be independently conducted by the DHS Science and
Technology Directorate (S&T).
Recently, the GAO released its latest report, focusing on the
testing conducted in 2008 and 2009. GAO acknowledged improvements in
ASP testing, but raised some concerns. DNDO agrees that analysis and
review of test data is necessary. DNDO plans to continue study the
results of testing as the ASP program progresses. However, DNDO
believes that the data analysis performed to date and the anticipated
data from ongoing testing will be sufficient to inform an ASP
certification decision in the future.
In addition to the GAO's reviews, DNDO and CBP have provided NAS
with regular testing updates. Recently, the NAS delivered an interim
report to the Department and the Appropriations Committees. Like the
NAS, DNDO sees the intrinsic value of continued testing and incremental
deployment of ASP systems. However, DNDO must balance the need to
better understand a complex system like ASP and the need to further
reduce the risk of certain significant vulnerabilities and operational
burdens. DNDO believes that the criteria outlined for certification are
a sufficient threshold for determining when we have reached the point
where deployment should begin.
Ultimately, these reviews provide for a valuable external
assessment of the ASP program. It is only through initial deployments
that we will continue to learn more about the performance of these
systems, and most rapidly bring about the improvements that are needed
to address current limitations. ASP systems have been under review and
evaluation for over three years now, and, while further improvements
will always be possible, we should not delay the implementation of
substantially improved capabilities.
Acquisition lessons learned and changes made in response:
DNDO has taken a number of steps to reform processes to ensure the
success of ASP, as well as other development and acquisition programs.
At the same time, these reforms are accompanied by a number of similar
improvements to DHS-wide program management processes.
With regard to testing, DNDO has taken a number of steps to improve
internal procedures based on lessons learned from earlier tests.
One of our most fundamental improvements has been through the
standardized implementation of DNDO Operating Instruction 1, ``Test
Event Planning.'' This detailed instruction lays out a six-milestone
process that ensures that test planning is done openly, that partner
inputs are included, and that test events are designed to provide the
data required to meet the objectives of the test, and ultimately to
help make programmatic decisions.
We have also taken considerable steps to ensure that any ASP
testing is responsive not just to DNDO requirements and objectives, but
to all DHS partners. Prior to the 2008-2009 testing, DHS created a test
planning working group that included DNDO, CBP, the DHS Operational
Testing Authority (OTA), the Office of the Under Secretary for
Management (USM), and the National Laboratories. Collectively, this
group laid out the test campaign and assigned responsibilities for each
test event to respective components.
Finally, in the 2008-2009 ASP test campaign, based on lessons
learned in the 2007 test campaign series, we instituted strict entrance
and exit criteria for each of the test events. These criteria were
developed long before testing began, and were developed jointly with
our operational partner, CBP. This has given us confidence that as the
ASP systems have continued through this series of test events,
requirements are met prior to completion.
DNDO has also made a number of program management changes based on
lessons learned throughout the program. In late 2008, DHS decided
against exercising the next contract option for one of three ASP
vendors. This decision reduced costs for carrying forward multiple,
parallel development efforts.
Finally, DNDO and the ASP program have benefited from a number of
DHS-wide improvements to program management. DNDO has adopted
Management Directive (MD) 102-01, which outlines the acquisition
management process for DHS programs. ASP program plans have been
adapted to be consistent with the rigorous process outlined in the
directive. This will further ensure that any eventual certification and
acquisition decisions are consistent with DHS priorities and made with
a strong acquisition management foundation.
There is an important distinction between two key milestones for
the ASP program--Secretarial certification and the MD 102-01 milestone
decision for purchase and deployment of ASP. The former is a rather
loosely defined milestone, which we have clarified by defining a
``significant increase in operational effectiveness,'' requiring that
the Secretary formally state that she believes the ASP system is
``better than'' the current system. The latter is a very well defined
milestone that was developed for all large programs within the
Department of Homeland Security. Items such as mission needs,
operational requirements, analysis of alternatives, etc., are part of
the MD 102-01 process. No ASP production units can be purchased and
deployed without successfully navigating the MD 102-01 process. The
Secretarial certification requirement is in addition to and in advance
of the MD 102-01 deployment decision.
Together, the lessons learned in testing and program management,
along with the introduction and adherence to MD 102-01 have
significantly improved the ASP program. At the same time, it is
important to note that these lessons learned have not only benefited
the ASP program; they are being applied across DHS.
ASP Path Forward:
The plans and procedures in place for the ASP program provide a
sound path forward for ASP certification and future acquisition
decisions. The current path to certification includes testing,
accompanied by the analysis of results, to ensure that Secretary
Napolitano has sufficient information for ASP certification. ASP
systems have been under review and evaluation for over three years now,
and, while further improvements will always be possible, a
certification decision will determine whether or not the systems
address increase the probability of detecting dangerous materials while
minimizing the operational burdens.
ASP was designed to improve capabilities in both primary and
secondary inspections. For primary applications, we have defined a
``significant increase in operational effectiveness'' as being a
quantified reduction in unwanted referrals to secondary inspection,
while maintaining similar or better sensitivity in detecting materials
of concern. Ultimately, the degree to which ASP systems meet this
objective is driven by sensitivity thresholds at which the systems will
be operated, similar to the threshold used for current PVT systems.
Therefore, the advantage in primary can be viewed as an improvement in
efficiency of operations (less unwanted referrals), while maintaining
the same or better detection efficiency. The relative degree to which
we realize these benefits with respect to the current systems will vary
by port, depending on the operational thresholds at which the current
PVT systems are set.
The evaluation of ASP systems in secondary inspection is more
direct. The ability of ASP systems to identify and resolve the source
of radiation is directly compared to the ability of current
capabilities to perform the same functions. In this instance, a
``significant increase in operational effectiveness'' is defined as a
quantified improvement in the ability to identify materials of concern.
Both of these definitions were developed through coordination
between DNDO, CBP, and USM. This definition has been approved by DHS
senior leaders and has served as the guide for developing test
campaigns to meet these test objectives. Again, because of the rigor
that has gone in to developing and quantifying these improvements, we
are seeking to remove ambiguity in the evaluation process, and ensure
that any certification and acquisition decisions are made consistent
with DHS priorities and objectives.
Conclusion:
DNDO will continue to work with CBP and other partners within and
beyond DHS to improve the Nation's ability to detect radiological and
nuclear threats at our ports and borders. DHS is facing a challenge at
our ports and borders as the Department balances facilitating the flow
of goods and commerce with the need to sufficiently scan cargo for
radiological or nuclear threats as it enters our nation. As both the
President and Secretary have said, the Nation will need more technology
to meet its security challenges and the technologies that DNDO is
pursuing, of which ASP is but one example, are a critical component in
addressing that challenge.
Our efforts to develop and evaluate ASP systems are sound. Current
test results are capturing the benefits of ASP systems, and the reviews
to date have provided a valuable assessment of the program and
identified a number of key lessons learned.
I welcome and appreciate the Committee's active engagement with
this program, and look forward to continuing our cooperation as we move
forward together. Chairman Miller, Ranking Member Broun, and Members of
the Subcommittee, I thank you for your attention and will be happy to
answer any questions that you may have.
Biography for William K. Hagan
Dr. William Hagan serves as Acting Deputy Director of the
Department of Homeland Security's (DHS) Domestic Nuclear Detection
Office (DNDO). Prior to that, he was Assistant Director for the
Transformational Research and Development (R&D) Directorate at DNDO. In
that role Dr. Hagan was responsible for long-term R&D seeking
technologies that can make a significant or dramatic positive impact on
the performance, cost, or operational burden of detection components
and systems.
Prior to DNDO, he was a Senior Vice President at Science
Applications International Corporation (SAIC). Business areas included
nuclear technology (analysis, detection, and applications),
telecommunications, optics, transportation, system integration, and
technology assessments during his thirty years at SAIC.
Dr. Hagan earned a Bachelor of Science in Engineering Physics in
1974, Master of Science in Physics in 1975, and Master of Science in
Nuclear Engineering in 1977 from the University of Illinois at Urbana.
He received his Ph.D. in Physics from the University of California-San
Diego in 1986. He holds three patents. Dr. Hagan was appointed to
Senior Executive Service in 2006.
Chairman Miller. Thank you, Dr. Hagan.
Mr. Owen for five minutes.
STATEMENT OF MR. TODD C. OWEN, ACTING DEPUTY ASSISTANT
COMMISSIONER, OFFICE OF FIELD OPERATIONS, U.S. CUSTOMS AND
BORDER PROTECTION, DEPARTMENT OF HOMELAND SECURITY
Mr. Owen. Good morning, Chairman Miller, Ranking Member
Broun, distinguished Members of the Subcommittee. As the Acting
Deputy Assistant Commissioner for the Office of Field
Operations with U.S. Customs and Border Protection, I am
honored to be here this morning alongside Dr. Hagan to discuss
the detection of radioactive and nuclear material in cargo
containers and the future role that Advanced Spectroscopic
Portal technology will have on CBP operations.
I would also like to express my gratitude to the Congress
for its continued support of CBP initiatives. Among the
numerous priorities that were recognized in the American
Recovery and Reinvestment Act, Congress provided CBP with $100
million worth of stimulus funding towards non-intrusive
inspection equipment. This funding will allow CBP to upgrade
and expand its successful Non-Intrusive Inspection (NII)
program and more efficiently inspect containers and vehicles
crossing the border, allowing them to enter our country and its
commerce in a safe and prompt manner.
CBP has made tremendous progress in ensuring that supply
chains importing goods into the United States are more secure
against potential exploitation by terrorist groups aiming to
deliver weapons of mass effect. CBP uses a multi-layered
approach to ensure the integrity of supply chains from point of
stuffing through the arrival in the U.S. ports of entry. This
multi-layered defense is built upon interrelated initiatives,
which include the 24-hour rule in the Trade Act of 2002, the
automated targeting system, the use of Non-Intrusive Inspection
equipment and radiation portal monitors, our container security
initiative and the Customs Trade Partnership Against Terrorism
program. These complementary layers enhance security and
protect our nation.
Prior to 9/11, not a single radiation portal monitor and
only 64 large-scale non-intrusive inspection systems were
deployed to our nation's borders. By October of 2002, CBP had
deployed the first RPM at the Ambassador Bridge in Detroit.
Today CBP has well over 1,200 RPMs and 227 large-scale NII
technology systems deployed nationwide. NII technology allows
the officers to detect possible anomalies, anomalies which may
indicate the presence of a weapon of mass effect or some other
contraband, and to date in fiscal year 2009, CBP NII systems
have conducted over 3.1 million exams resulting in over 7,800
narcotic seizures with a total weight of over 2.6 million
pounds as well as $6.2 million in currency seized.
In addition to the significant strides made in the area of
NII equipment, CBP also continues to deploy first-generation
radiation portal monitors to our ports of entry. Currently, 97
percent of the trucks and 93 percent of the personally owned
vehicles arriving from Canada, 100 percent of the trucks and
vehicles from Mexico and 98 percent of the arriving sea
containers are scanned by our current radiation detection
technologies. In total, CBP scans 98 percent of all cargo
arriving into the United States by land and sea using radiation
portal monitors. In addition, CBP officers scan 100 percent of
general aviation aircraft arriving to the United States from
foreign destinations using hand-held radiation identification
devices.
Since the first RPM was deployed in 2002, CBP officers have
scanned over 368 million conveyances for the presence of
radiation, and we have resolved 2.1 million radiological alarms
successfully with minimal or no impact to the flow of
legitimate trade and travel. CBP continues to closely
coordinate with key stakeholders to ensure the impact of this
activity causes minimal disruption to port operations. The
first-generation RPM systems, although very sensitive, do have
limitations. While they alert CBP officers to the presence of
radiation, a secondary exam is necessary to positively identify
the specific isotope causing the alert. In the event that a CBP
officer is unable to positively resolve the alert, scientific
reach-back is available 24 hours a day, seven days a week.
The ASP is expected to enhance our detection capability
while significantly reducing the number of secondary
examinations. This is due to its ability to distinguish between
actual threats and natural or medical radiation sources that
are not security threats. CBP has worked closely with the DNDO
in the development and operational testing of ASP, has provided
DNDO with functional requirements and has been actively engaged
in every step of the evaluation process. CBP's focus for
operational testing is to evaluate the effectiveness for
systems deployed in our operational environments. We will
continue to work with DNDO and the DHS Science and Technology
Directorate towards secretarial certification, and we are also
working within DHS to ensure that any future ASP acquisitions
and deployment decisions are consistent with DHS priorities.
The decision to purchase and deploy ASPs in the operational
arena will be based on CBP's mission needs, operational
requirements, comprehensive cost-benefit analysis to include
the full understanding of maintenance and operation costs, and
analysis alternatives and other considerations.
Mr. Chairman, Members of the Subcommittee, today I have
addressed CBP's commitment to invest in new technologies and
emerging technology aimed at enhancing cargo security. We must
continue to maintain our tactical edge by integrating new
technology into our ports of entry.
Thank you again for the opportunity to be here this morning
and I would be happy to answer any questions.
[The prepared statement of Mr. Owen follows:]
Prepared Statement of Todd C. Owen
Chairman Miller, Ranking Member Broun, esteemed Members of the
Subcommittee, it is a privilege and an honor to appear before you today
to discuss the work of U.S. Customs and Border Protection (CBP),
particularly the detection of radioactive and nuclear material in cargo
containers and the future role that the Advanced Spectroscopic Portal
(ASP) program will have on our operations. CBP strives to continually
improve the security of cargo entering our borders and facilitate the
flow of legitimate trade and travel. Included in this process, over 98
percent of all arriving maritime containerized cargo is presently
scanned for radiation through radiation portal monitors.
I want to begin by expressing my continuing gratitude to Congress
for its continued support for the mission and people of CBP. It is
clear that the Congress is committed to providing CBP the resources we
need in order to increase and maintain the security of our borders. We
appreciate your efforts and assistance.
CBP is the largest uniformed, federal law enforcement agency in the
country. We station over 20,000 CBP officers at access points around
the Nation, including at air, land, and sea ports. As of mid-May, we
have deployed over 19,000 Border Patrol agents between the ports of
entry. These forces are supplemented with 1,058 Air and Marine agents,
2,318 agricultural specialists, and other professionals. These
personnel are key players in the implementation of Secretary
Napolitano's Southwest Border Security Initiative.
CBP continues to execute all of its responsibilities, which include
stemming the illegal flow of drugs, contraband and people, protecting
our agricultural and economic interests from harmful pests and
diseases, protecting American businesses from theft of their
intellectual property, enforcing textile agreements, tracking import
safety violations, regulating and facilitating international trade,
collecting import duties, facilitating legitimate travel, and enforcing
United States trade laws. CBP facilitates lawful immigration, welcoming
visitors and new immigrants, while making certain those entering this
country are indeed admissible and taking appropriate action when an
individual fears being persecuted or tortured if returned to their home
country. At the same time, our employees maintain a vigilant watch for
terrorist threats. In FY 2008, CBP processed more than 396 million
pedestrians and passengers, 122 million conveyances, 29 million trade
entries, examined 5.6 million sea, rail, and truck containers,
performed over 25 million agriculture inspections, apprehended over 720
thousand illegal aliens between our ports of entry, encountered over
220 thousand inadmissible aliens at the ports of entry, and seized more
than 2.8 million pounds of illegal drugs.
We must perform our important security and trade enforcement work
without stifling the flow of legitimate trade and travel that is so
important to our nation's economy. These are our twin goals: border
security and facilitation of legitimate trade and travel.
CBP OVERVIEW
I am pleased to appear before the Subcommittee today to highlight
key accomplishments related to container security, particularly those
related to new and emerging technology. CBP has made tremendous
progress in securing the supply chains bringing goods into the United
States from around the world to prevent their potential use by
terrorist groups that seek to deliver weapons of mass effect. The use
of cutting-edge technology has greatly increased the ability of front
line CBP Officers to successfully detect and interdict illicit
importations of nuclear and radiological materials. CBP uses a multi-
layered approach to ensure the integrity of the supply chain from the
point of stuffing through arrival at a U.S. port of entry. This multi-
layered approach includes:
[bullet] Advanced Information
24-Hour Rule
Automated Targeting Systems
Importer Security Filing
[bullet] The Customs Trade Partnership Against Terrorism
[bullet] The Container Security Initiative
[bullet] The Secure Freight Initiative
[bullet] Use of Non-Intrusive Inspection Technology and
Mandatory Exams for All High-Risk Shipments
I will discuss each one of these layers in greater detail with
particular focus on our radiation and nuclear detection capabilities.
ADVANCE INFORMATION
CBP requires advanced electronic cargo information as mandated in
the Trade Act of 2002 (including the 24-hour rule for maritime cargo).
Advanced cargo information on all inbound shipments for all modes of
transportation is effectively evaluated using the Automated Targeting
System (ATS) before arrival in the United States.
ATS provides decision support functionality for CBP officers
working in Advanced Targeting Units (ATUs) at our ports of entry and
CSI ports. The system provides uniform review of cargo shipments for
identification of the highest threat shipments, and presents data in a
comprehensive, flexible format to address specific intelligence threats
and trends. ATS uses a rules-based program to highlight potential risk,
patterns, and targets. Through rules, the ATS alerts the user to data
that meets or exceeds certain pre-defined criteria. National targeting
rule sets have been implemented in ATS to provide threshold targeting
for national security risks for all modes: sea, truck, rail, and air.
The Importer Security Filing interim final rule, also known as ``10
+ 2,'' went into effect earlier this year and has already yielded some
promising results. This program will provide CBP timely information
about cargo shipments that will enhance our ability to detect and
interdict high risk shipments. Comments on aspects of this rule were
accepted until June 1, 2009, and implementation using informed
compliance will continue until January of next year. Shipments
determined by CBP to be high-risk are examined either overseas as part
of our Container Security Initiative or upon arrival at a U.S. port.
Customs Trade Partnership Against Terrorism
The Customs Trade Partnership Against Terrorism (C-TPAT) is an
integral part of the CBP multi-layered strategy, in that CBP works in
partnership with the trade community to better secure goods moving
through the international supply chain. C-TPAT has enabled CBP to
leverage supply chain security throughout international locations where
CBP has no regulatory reach. In 2009, CBP will continue to expand and
strengthen the C-TPAT program and ensure that certified member
companies are fulfilling their commitment to the program by securing
their goods moving across the international supply chain to the United
States. To carry-out this critical tenet of C-TPAT in 2009, teams of
Supply Chain Security Specialists (SCSS) will conduct validations and
revalidations of C-TPAT members' supply chains to ensure security
protocols are reliable, accurate, and effective.
As C-TPAT has evolved, we have steadily added to the rigor of the
program. CBP has strengthened the C-TPAT program by clearly defining
the minimum-security requirements for all categories of participants
wishing to participate in the program and thereby gain trade
facilitation benefits. As of June 18, 2009, there are 9,286 companies
certified into the C-TPAT program. CBP's goal is to validate all
partners within one year of certification, revalidate all companies not
less than once every three years and revalidate all U.S./Mexico highway
carriers on an annual basis, based on the risk associated with the
Southern Border Highway Carrier sector of C-TPAT.
Container Security Initiative
To prevent terrorists and their weapons from entering the United
States, CBP has also partnered with other countries through our
Container Security Initiative (CSI). In FY 2008 CBP Officers stationed
at CSI ports reviewed over 11 million bills of lading and conducted
over 74,000 exams in conjunctions with their host country counterparts.
Because of the sheer volume of sea container traffic, containerized
shipping is uniquely vulnerable to terrorist exploitation. Under CSI,
which is the first program of its kind, we are partnering with foreign
governments to identify and inspect high-risk cargo containers at
foreign ports before they are shipped to our seaports and pose a threat
to the United States and to global trade.
CBP Officers stationed at foreign CSI ports review 100 percent of
the manifests originating and/or transiting those foreign ports for
containers that are destined for the United States. In locations where
the tremendous volume of bills prevents the CSI team at the port itself
from performing 100 percent review, or during port shutdowns, CSI
targeters at the National Targeting Center provide additional support
to ensure that 100 percent review is accomplished. Utilizing the
overseas CSI team and the CSI targeters at our National Targeting
Center, CBP is able to achieve 100 percent manifest review for the CSI
program.
Today, CSI is operational in 58 ports covering 86 percent of the
maritime containerized cargo shipped to the United States.
Secure Freight Initiative
The Secure Freight Initiative (SFI) is an unprecedented effort to
build upon existing port security measures by enhancing the United
States Government's ability to scan containers for nuclear and
radiological materials in seaports worldwide and to better assess the
risk of inbound containers. Secure Freight will provide carriers of
maritime containerized cargo with greater confidence in the security of
the shipment they are transporting, and it will increase the likelihood
of an uninterrupted and secure flow of commerce. This initiative is the
culmination of our work with other government agencies, foreign
governments, the trade community, and vendors of leading edge
technology.
Moving forward, CBP will prioritize future deployments of scanning
systems to locations of strategic importance by identifying seaports
where non-intrusive imaging and radiation detection data would be most
practical and effective in deterring the movement of weapons of mass
destruction via containerized cargo. Under this strategy, the
additional scan data provided by SFI will enhance DHS' risk-based and
layered approach to securing maritime containerized cargo. We will
continue to work with Congress to enhance the safety of our nation's
ports and the security of incoming cargo.
Non-Intrusive Inspection/Radiation Detection Technology
Today I will specifically address large-scale X-ray and gamma
imaging systems and radiation detection devices; technologies that play
a critical role in our layered enforcement strategy.
The deployment of imaging systems and radiation detection equipment
has contributed to CBP's tremendous progress in ensuring that supply
chains bringing goods into the United States from around the world are
secure against exploitation by terrorist groups that seek to deliver
weapons of mass effect.
Non-Intrusive Inspection (NII) technology serves as a force
multiplier that allows officers to detect possible anomalies between
the contents of the container and the manifest. CBP relies heavily on
the use of NII as it allows us to work smarter and more efficiently in
recognizing potential threats.
Prior to 9/11, not a single Radiation Portal Monitor (RPM), and
only 64 large-scale NII systems, were deployed to our nation's borders.
By October of 2002, CBP had deployed the first RPM at the Ambassador
Bridge in Detroit. Today, CBP has deployed 1,250 operational RPMs at
seaports, land border ports, and mail facilities, 227 large-scale gamma
ray or x-ray imaging systems and 3,000 small scale NII systems
nationwide. Additionally, CBP has deployed over 1,382 Radiation Isotope
Identifier Devices (RIID) and over 17,542 Personal Radiation Detectors
(PRD). These devices allow CBP to inspect 100 percent of all identified
high-risk cargo.
Currently, 97 percent of trucks and 93 percent of personally owned
vehicles arriving through northern border ports, 100 percent of
vehicles arriving through southern border ports, and 98 percent of
arriving sea containers are scanned by our radiation detection
technologies. CBP uses RPMs to scan 98 percent of all cargo arriving in
the U.S. by land and sea. In addition, CBP officers now use hand-held
radiation identification devices to scan 100 percent of private
aircraft arriving in the U.S. from foreign destinations. As of May
2009, CBP officers scanned over 368 million conveyances and
successfully adjudicated 2.1 million radiological alarms.
It is important to distinguish these deployments from the 100
percent mandate. These deployments refer to CBP's domestic RPM
deployments, which perform radiation detection (not imaging) of
containers that are scanned in the U.S. but prior to release into the
commerce. The 100 percent mandate requires scanning in a foreign port
and both imaging and radiation detection.
The first generation RPM systems, although very sensitive, do have
limitations. While they alert CBP officers to the presence of
radiation, a secondary exam is necessary to positively identify the
location and specific isotope causing the alert. In the event that a
CBP officer is unable to positively resolve the alert, scientific reach
back is available on a 24/7 basis through the National Targeting Center
and CBP's Laboratory & Scientific Services Division located in the
northern Virginia area.
Understanding these limitations and the need for more precise
radiological detection architecture, the DNDO was chartered to develop
new technologies that will improve CBP's radiation and nuclear
detection capabilities. One of these new technologies is the next
generation RPM, or the Advanced Spectroscopic Portal (ASP).
The ASP is able to distinguish between actual threats and natural
or medical radiation sources that are not security threats. In doing
so, the ASP is expected to enhance our detection capability, while
significantly reducing the burden of responding to the numerous benign,
nuisance alarms that are mostly generated by everyday products. This
will allow CBP to focus our staffing and resources on high-risk
shipments and other border security initiatives.
CBP COORDINATION WITH THE DOMESTIC NUCLEAR DETECTION OFFICE (DNDO)
In the course of our collaboration with DNDO, CBP brings knowledge
of how our ports work, of the support needs of our front-line officers,
and of the operational requirements for new technologies that must work
consistently in a broad array of environments. Additionally, we must
remain attuned to critical factors such as throughput and capacity as
we seek to maintain an appropriate balance between security and the
facilitation of cross-border travel and trade.
CBP has worked closely with DNDO in the developmental and
operational testing of the ASP. A complete independent operational
testing and evaluation will be conducted by the DHS S&T Director, T&E
and Standards Director, Operational T&E, when the system is ready.
CBP's objective for operational testing is ensuring that systems are
operationally acceptable and effective and can be deployed in our
operational environments. Specifically, CBP provided DNDO with
functional requirements for the ASP and has been actively engaged in
every step of testing, including performance testing at the Nevada Test
Site and Integration testing currently ongoing at a mock port of entry
at the Pacific Northwest National Laboratory.
During integration testing, CBP works closely with DNDO to assess
each system's performance as an integrated unit, including reach back
capability and ancillary equipment such as traffic lights and automated
gate arms that are essential to maintain positive control of vehicles
at our congested ports of entry. In addition, CBP works with DNDO to
assess and categorize each system's defects to ascertain their
technological impact on performance and their operational impact on
front-line CBP officers--the users of the system.
CBP will continue to work with DNDO towards Certification by the
Secretary, which is dependent on demonstrating a ``significant increase
in operational effectiveness'' over existing first generation radiation
detection systems. Only after this Certification has been reach can the
discussion then turn to potential acquisition and deployments of the
ASP systems. The decisions to purchase and deploy ASPs in the
operational arena will be based on mission needs, operational
requirements, and a full understanding of maintenance and operational
costs, to include a comprehensive cost benefit analysis, an analysis of
alternatives, etc.
CONCLUSION
In conclusion, I would like to say that technology plays an
enormous role in securing the supply chain. Security technology is
continuously evolving, not only in terms of capability but also in
terms of compatibility, standardization, and integration with
information systems. It is important to note that there is no single
technological solution to improving supply chain security. As
technology matures, it must be evaluated, and adjustments to
operational plans must be made. Priority should be given to effective
security solutions that complement and improve the business processes
already in place, and which build a foundation for secure 21st century
global trade.
Mr. Chairman, Members of the Committee, today I have addressed
CBP's commitment to investing its efforts in the areas of new and
emerging detection technology, as well as some of the steps we have
taken towards enhancing cargo security.
Thank you again for this opportunity to testify. I will be happy to
answer any of your questions.
Biography for Todd C. Owen
Todd C. Owen is the Executive Director of the Cargo and Conveyance
Security Office within U.S. Customs and Border Protection's Office of
Field Operations. As the Executive Director for the Cargo and
Conveyance Security (CCS) Office since May 2006, Mr. Owen is directly
responsible for all cargo security programs and policies for CBP,
including the ``100 percent scanning initiative'' announced in October,
2007. Included within Mr. Owen's responsibilities are the Container
Security Initiative (CSI), the Customs-Trade Partnership Against
Terrorism Program (C-TPAT) Office, all non-intrusive inspection
technology and radiation portal monitor deployments, the National
Canine Enforcement Program, and the National Targeting Center, Cargo.
Mr. Owen also coordinates CBP's maritime cargo enforcement policies
and activities with the U.S. Coast Guard, and all air cargo efforts
with the Transportation Security Administration.
Previously, Mr. Owen was the Director of the C-TPAT program from
January 2005 through May 2006. During his tenure as C-TPAT Director,
this 9,000 member strong industry partnership program was strengthened
by more clearly defining the security measures which must be adopted
for a member to be eligible to receive the trade facilitation benefits
afforded by CBP. Strong management controls were implemented and hiring
was increased, allowing for a significant increase in the level of
foreign site assessments performed worldwide under this program.
Prior to arriving in Washington in January 2005, Mr. Owen was the
CBP Area Port Director in New Orleans. As the Area Port Director, Mr.
Owen was directly responsible for all CBP operations in New Orleans and
throughout Louisiana. Two hundred forty officers are stationed in the
seven Louisiana CBP port offices managed by the Area Director.
Mr. Owen began his career as an Import Specialist in Cleveland,
Ohio in 1990, and transferred to Miami in 1992. Through his eight years
in South Florida, Mr. Owen held various trade related positions within
Field Operations and the Office of Strategic Trade, before being
selected for the New Orleans Area Port Director position in 2000.
Mr. Owen, a career member of the Senior Executive Service, is a
graduate of John Carroll University in Cleveland, Ohio, and was a
senior executive fellow at Harvard University's John F. Kennedy School
of Government. Mr. Owen also holds a Master's degree in Public
Administration from St. Thomas University in Miami, Florida.
Discussion
Chairman Miller. I now recognize myself for five minutes.
Dr. Hagan, you have heard a lot of criticisms of this program
this morning, that is the Maginot Line of terrorism. It is
pretty unlikely that terrorists will actually try to go through
this system when it would be relatively easy to go around it,
that a lot of the testing seems to be an afterthought to
justify a conclusion that was already reached. The end-user,
the Customs and Border Patrol, think that they are doing
perfectly well and that their existing technology could be
improved upon relatively cheaply compared to the investment
that would be required, the spending that would be required for
ASP. And both of the previous witnesses very much questioned
the validity, the thoroughness of the testing that was being
used to justify the decision to deploy ASP. What is the
urgency? In view of all of that, what is the urgency about
putting a certification decision before the DHS Secretary in
October?
Dr. Hagan. I guess there is a couple things I would like to
say. First is that there is a sense of urgency regarding some
of the limitations of the current system. We can't talk about
these in open session, but we would be happy to do that at a
later time. But having said that, I don't believe that today
that there is a rush to deployment nor is there a `do it by
October or something bad happens.' So what we are doing though
is pushing forward as aggressively as we can to get to the
decision point about deployment. In other words, we are not
saying we have got to deploy these right now. We are saying we
have got to get, as soon as possible, to the point where we
make that decision, and as was said by the earlier witnesses,
the information that is required to make that decision, as I
mentioned in my oral statement, there is a lot of information,
a lot of data, a lot of test results that have yet to be
analyzed and brought together and integrated in this cost-
benefit analysis, and that would be an important part of the
decision that will be made in October. So, I guess I would
disagree that we are rushing to deployment. I would say that we
are rushing more towards getting all the information as fast as
we can but doing it in a thoughtful, deliberate, disciplined
way and through this MD 102-01 process that I mentioned, to get
to that decision point.
Chairman Miller. Mr. Owen, we heard again from GAO and the
National Academies of Science that the work to this point did
not really instill confidence in them that this new system
would perform as reliably, as effectively as we would like or
even as the existing technology works. They seem not concerned
about a delay. They in fact encouraged a delay. Do you agree
with their recommendation to stick with what we have got and
not rush either to deploy or to certify or to decide to deploy?
Mr. Owen. Sir, as the operator, the end-user of the
systems, we, as front-line officers, need technology that will
not only reduce the possibility of a missed threat but as well
as reduce the rate of innocent alarms that our officers are
spending their time on. In the port of Long Beach, for example,
we have between 400 and 600 radiation alarms every single day.
We have a team of about 100 officers over the three shifts that
work on just resolving these innocent alarms throughout the
course of their workday. So having a new piece of technology or
an enhancement to technology that will allow us to not only
identify any missed potential threats, but reduce the number of
innocent alarms, is something that we do support. But as I
mentioned in our hearing, our testimony, we have been doing
this for some time with the existing PVTs. We have been able to
resolve those 2.1 million alarms without a negative throughput
with the flow of commerce in our ports of entry. So the
technology that we have is effective but again, as we have
matured the technology, we are looking for that technology
which will give us an edge, if you will, a step up from what we
have, particularly in terms of missing any threats that the
PVTs may not capture right now.
Chairman Miller. With respect to the existing technologies,
the PVTs with the follow-up searches, are there enhancements,
improvements in that technology that seem available,
achievable, and have those improvements, are those being
pursued?
Mr. Owen. Our current system with the PVT which will
alert--the container will then go in a secondary where we will
perform the analysis with the radiation isotope identifier and
we push that information back to our laboratory and scientific
folks--it is a process that we have crafted and we feel
comfortable with. What we have seen in the operational area, in
terms of the energy windowing that you heard from the first
panel, is we have seen operational benefits to being able to
make adjustments to the existing PVTs to account for the
recurring, burdensome, naturally occurring radioactive
shipments that come through in legitimate cargo. So we have
seen the benefits from energy windowing in the operational
environment.
Now, as to have we pushed those benefits as far as we can
through energy windowing, we really need to defer to the
scientists, to DNDO and our partners at Pacific Northwest
National Laboratories to tell us if we have gone as far as we
can with energy windowing. But again, as the operator of these
systems, we have seen benefits in the ports of entry where we
have deployed energy windowing. Have we maximized that? I would
really defer to the scientists to give us that answer.
Chairman Miller. I understand the principal problem with
the current technology is as you have pointed out, not so much
that it would miss--it is less a concern for missing radiation
than the large number of false positives. Perhaps not false in
the sense that it is not radiation, but it is not radiation
that is a problem. How--I understand that the PVT system has
been tested for that as well. How did it fare compared to the
existing--I am sorry. The ASP system, how did it fare compared
to the PVT system in false positives?
Dr. Hagan. I will take that. The actual test data is
classified, but I can answer in a qualitative way, that in--
there are two places in which the ASP is being tested and
considered. One is in primary inspection and the other is
secondary inspection. Do you want me to explain what I mean by
that?
Chairman Miller. Well, for the record at least, sure.
Dr. Hagan. Okay.
Chairman Miller. Briefly.
Dr. Hagan. So a conveyance comes in and goes through
primary inspection or screening, and goes through an RPM or
radiation portal monitor of some sort. If that monitor alarms,
then that conveyance is referred to secondary inspection where
it is further examined either with another monitor, a radiation
portal monitor or a hand-held RIID. And ASP is being considered
for deployments to both of those, both primary inspection and
secondary inspection. In the primary application, the ASP is
a--has the ability to identify the type of--the source of the
radiation. The PVT is not able to do that. So PVT simply says
there is radiation, better go to secondary and off it goes. In
the case of ASP, because of the spectroscopic nature of the
system, it can discriminate between a threat material and non-
threat material, or if it can't discriminate, then it sends it
to secondary as well. So it can dismiss and reduce the number
of referrals to secondary. In the secondary application, if you
have ASP there, then you are comparing essentially the
performance of a large spectroscopic portal which is very, very
large compared to a very small hand-held detector in which case
the--again, I can't talk about the specifics of the results but
the ASP is far superior to the hand-held detector in terms of
its ability to, in real time, identify the source of the
radiation and dismiss it as appropriate.
Chairman Miller. Is it classified to say which did better,
the PVT or the ASP?
Dr. Hagan. No. I am sorry. I should have said, yes. So the
ASP does far better in the secondary application of being able
to identify----
Chairman Miller. Well, which--I am sorry. Which system
produced more false positives, ASP or PVT?
Dr. Hagan. In the most recent----
Chairman Miller. February.
Dr. Hagan. Okay. In the most recent--you are talking about
field validation testing, I think.
Chairman Miller. Yes.
Dr. Hagan. In field validation tests, the ASP system sent
more--had more referrals to secondary than the PVT did.
However, we have now--it is actually not surprising. It was the
first time that ASP had been able to operate in a real
operating environment, and so just as with PVT, adjustments
were made to what are called thresholds in the ASP. So we are
not making software changes, we are simply tuning the system as
you do with the PVT, and then we have done a lot of analysis
and simulation, as was mentioned earlier, to verify that these
changes will in fact improve the performance and give us the
expected performance.
Chairman Miller. Okay. It appears that my time has really
and truly expired.
Dr. Broun for five minutes.
Mr. Broun. Thank you, Mr. Chairman. I don't watch the clock
very closely, as you very well know, and I will always give
you--almost always give you a lot of leeway unless there is
some particular reason not to.
But Dr. Hagan, what you are saying, it is my understanding
that the ASPs--you just made a statement that they are better.
It is my understanding that that is only with the lightly
shielded radiation. Isn't it true that with heavily shielded or
moderately shielded radiation, that there is not much
difference between the two programs? And isn't it also true
that if anybody is going to bring radiation-type special
nuclear materials in any shipment, aren't they going to be
shielded and very highly so?
Dr. Hagan. The answer to the second part first. One might
speculate that that would be the case but you can't really know
for sure, and so yes, it is probably likely that someone would
shield it, but a knowledgeable adversary might not do that for
other reasons which I will talk about later. But going back to
the first part of your question, yes, the two systems, both PVT
monitors and ASP monitors are what are called passive detection
systems. They both suffer from the limitation that if a nuclear
material or a weapon is shielded enough, then there just is no
signal for either detector to detect. Now, so what we are--the
long-range plan for this, and it is already being implemented
in part, is to say that if a--by having these passive
detectors, we are forcing the adversary, knowledgeable
adversary, anyway, to heavily shield the object. Well, that
heavily shielded object, then, is a very--well, I shouldn't say
very but is an easy target or an easy image or easy thing to
identify in a radiographic image or non-intrusive inspection
system which Mr. Owen talked about. So by having complementary,
in the sense, orthogonal type of systems, one that is passive
system that will detect lightly shielded or unshielded
material, and another system that will detect shielded
material, then you have covered--you have done a good job of
covering the complete spectrum of possibilities. Any system, of
course, can be defeated by a knowledgeable adversary, but by
having those two systems combined, then you really do a much
better job of dealing with that problem. That being said, the
amount of shielding that is needed, the effectiveness of PVT
versus ASP for lightly or moderately shielded materials is--it
depends on what it is you are shielding, and we would have to
go into particulars and specifics about different types of
objects, which we can't do here.
Mr. Broun. Thank you, Dr. Hagan. My time is about up. I
will yield back.
Chairman Miller. Ms. Dahlkemper for five minutes.
Ms. Dahlkemper. Thank you, Mr. Chairman.
I live on Lake Erie and so I go back and forth to Canada a
fair amount and actually I felt much safer a few years ago when
were pulled over. My father had had a stress test two weeks
prior so I do actually have some personal experience with those
innocent alarms. I didn't understand what was going on at the
time. It was the first time I had ever been pulled over.
Mr. Owen. I hope we didn't cause too much stress.
Ms. Dahlkemper. No, no, we were fine, but I actually felt
much better coming back home and telling people who had no idea
that this was actually being detected as we all would drive
back home from Canada. But DHS has spent almost half a
million--a billion dollars deploying PVT monitors at border
crossings and ports across the country, including the one I
came across, obviously. Does, Mr. Owen, the Customs and Border
Protection believe that the current system, using PVTs and
doing the secondary inspection with hand-held detectors and
other means, is working well in terms of keeping us safe and
moving commerce along? And I guess I am wondering what is the
cost of manpower versus what is the cost of ASP? You know, I am
looking at the dollars spent here, knowing we have limited
resources and where are we best spending our money.
Mr. Owen. And I would just respond, ma'am, that again two
parts to that equation is, is the PVT finding all of the
threats or are there threats that could be slipping through
just because of the physics involved, and that is where ASP can
help us in that regard. The second piece of that would be, can
we respond to the number of secondaries that are caused by the
existing PVT systems, and the answer is yes, we demonstrate
that. There is a resource impact on this. Again, we would look
at Los Angeles, we look at what we do up in the Peace Bridge
there in Buffalo where you probably came through. It is a
timely process. The officers do have to go through each
secondary exam and treat it as if it is a real threat. We
cannot let our guard down and just assume it was your father-
in-law, that he had medical testing, let you go down. We have
to take you out of the vehicle, we have to go through that
whole process. So having a system that reduces the number of
innocent alarms that are sent into secondary will help us do
that. As to is the cost of the manpower savings offset by the
ASP, I think that is some of the work that the DNDO is doing
with their overall cost-benefit analysis. It will take into
consideration those variables.
Ms. Dahlkemper. I wanted to ask you too about the
maintenance and operation costs of the ASPs and they are
estimated to be anywhere from five to twelve times more
expensive than the costs of running the PVTs, and so in the
worst case ASPs will cost about $100,000 a year to run as
compared to $8,000 a year for the PVTs. Does Customs have the
budget to support this kind of system if it were deployed
nationally, and have you thought about what you would have to
give up in terms of personnel or other, you know, equipment to
run this?
Mr. Owen. Yes, and that is a very good question. That is a
concern that we do have now. The ASP systems, depending on how
the final outcome of the cost-benefit and the cost of the
systems, I think it is well accepted that it is going to cost
more than what we have with the PVTs. Currently we have about
1,250 PVTs deployed nationwide. We continue to expand along the
northern border. We are going to get--up to 100 percent of the
cargo coming in from Canada this year will be covered by PVTs.
We will end up with about 1,500 PVTs by the end of this
calendar year. There is an extensive operation and maintenance
tail that comes with that. I think any deployment decisions
going forward with ASPs needs to be cognizant of the impact
that will have on the operator. There is much talk about the
initial acquisition cost and the deployment cost to buy it, put
it in the ground, but I don't think we can forget about the
operation and maintenance tail that comes along with this, that
will then fall on the backs of the operator, in this case CBP.
So it is something that we are concerned with. It is something
that again is going into the overall cost adjustments for what
we can expect with the ASP systems.
Ms. Dahlkemper. Do you have any thoughts on what this would
do in terms of personnel? Would it free up great numbers for
other purposes?
Mr. Owen. Well, my view on this is, when you look at the
way the layouts are in the seaports, you have to have still an
officer man those exit gates where you have the radiation
portal monitors. So whether it is manned with two officers or
one officer because the alarms are less frequent, there will
still be a cost figure associated with the manpower. There may
be some operational adjustments that we can make in the
secondary areas but I think the cost savings are something that
we still need to measure once we are confident that the ASPs
are delivering as we hope they will do. So there will be some
savings from that. Will that offset the increase in operation
maintenance costs associated with the ASP? That is something
that we have to take very careful consideration of.
Ms. Dahlkemper. Thank you. My time is up. I yield back.
Chairman Miller. Thank you, Ms. Dahlkemper. That will be
the last round of questions. Dr. Hagan, I trust that when DHS
has completed its cost-benefit analysis, that we will get a
copy that is still warm from the printer.
Dr. Hagan. Okay.
Chairman Miller. So under the rules--before we bring the
hearing to a close, I do want to thank all of our witnesses for
testifying before the Subcommittee today. Under the rules of
the Committee, the record will remain open for two weeks for
additional statements from the Members and for answers to any
follow-up questions any Member of the Committee may have for
witnesses, and I understand that Dr. Broun does have questions.
The witnesses are excused and the hearing is now adjourned.
[Whereupon, at 11:41 a.m., the Subcommittee was adjourned.]
Appendix:
----------
Additional Material for the Record
[SKIP PAGES = 000]
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
HEARING ON THE SCIENCE OF SECURITY, PART II: TECHNICAL PROBLEMS
CONTINUE TO HINDER ADVANCED RADIATION MONITORS
----------
TUESDAY, NOVEMBER 17, 2009
House of Representatives,
Subcommittee on Investigations and Oversight,
Committee on Science and Technology,
Washington, D.C.
The Subcommittee met, pursuant to call, at 1:05 p.m., in
Room 2310 of the Rayburn House Office Building, Hon. Brad
Miller [Chairman of the Subcommittee] presiding.
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
hearing charter
SUBCOMMITTEE ON INVESTIGATIONS AND OVERSIGHT
COMMITTEE ON SCIENCE AND TECHNOLOGY
U.S. HOUSE OF REPRESENTATIVES
The Science of Security, Part II:
Technical Problems Continue to
Hinder Advanced Radiation Monitors
Tuesday, November 17, 2009
1:00 p.m.-3:00 p.m.
2318 Rayburn House Office Building
Purpose
The Subcommittee on Investigations and Oversight meets on November
17, 2009, to examine continuing problems with the Department of
Homeland Security's (DHS) efforts to acquire its next generation
radiation monitors known as Advanced Spectroscopic Portals (ASPs). This
is a follow-up to the hearing the Subcommittee held on June 25, 2009,
titled: The Science of Security: Lessons Learned in Developing, Testing
and Operating Advanced Radiation Monitors. Since the Domestic Nuclear
Detection Office (DNDO), a DHS component, was created in 2005 they have
been responsible for researching, developing, testing and managing the
program.
The ASP program is estimated to cost $2-to-$3 billion and has been
under scrutiny since 2006 for failing to have clear-cut requirements,
an adequate test plan, sufficient timelines, development milestones or
a transparent and comprehensive cost benefit analysis. These problems
have been identified by the Government Accountability Office, National
Academy of Sciences, the Homeland Security Institute, a Federally
Funded Research and Development Center for DHS, and the National
Institute of Standards and Technology.
In July, one month after the Subcommittee's last hearing, the ASPs
went through a second round of Field Validation Tests. During the tests
the ASPs exhibited several ``false positive'' alarms for special
nuclear material that did not exist. In another disturbing incident
during the tests, one ASP monitor stopped working altogether yet the
system operator remained unaware of this malfunction. Two dozen cargo
trucks were permitted to go through the non-functioning portal monitor
in order to be screened for potential radioactive and nuclear material
until the problem became apparent. DNDO considered this a ``Mission
Critical Failure.'' No new plans have yet been scheduled to re-test the
ASPs for the third time. The Subcommittee will examine the results from
the most recent tests, continuing technical problems with the ASPs,
supply shortages of a key component for radiation monitors that may
hinder the eventual deployment of the ASPs and further drive up its
potential cost, and potential enhancements to the current fleet of
radiation monitors in use today.
Background
Since the September 11, 2001 terrorist attacks, protecting the
Nation from a nuclear or radiological attack has been a top national
security priority. In 2002, to help address this threat, the U.S.
Customs and Border Protection (CBP) agency began deploying radiation
monitors at U.S. border sites and ports of entry to screen the more
than 23 million cargo containers that enter the country every year for
radiological and nuclear materials.
Polyvinyl toluene (PVT) radiation portal monitors have been used to
screen this cargo since then. They are able to detect the presence of
radioactive sources, but unable to identify the type of radiation
present. The PVT monitors, while relatively inexpensive, robust and
highly reliable, are unable to distinguish between radioactive sources
that might be used to construct a nuclear bomb, such as Highly Enriched
Uranium (HEU), and non-threatening naturally occurring radiological
materials (NORM) contained in ceramic tiles, zirconium sand or kitty
liter, for instance. As a result, any time a PVT detects a radioactive
source the cargo is sent to ``secondary'' screening where CBP agents
verify the detection of the source with a second PVT monitor and use
hand-held Radioactive Isotope Identification Devices called RIIDs to
help identify the source of radiation.
This method of operation leads to many ``secondary'' inspections
for naturally occurring radioactive material or radioactive material
intended for benign purposes, such as radioactive medical isotopes. At
the Los Angeles/Long Beach port of entry, for instance, PVT monitors
routinely send up to 600 conveyances of cargo to secondary inspection
each day. In addition, the RIIDs used in secondary inspections are
limited in their abilities to locate and identify potential radioactive
material in large cargo containers.
In order to help improve the flow of commerce by eliminating many
of the unnecessary alarms that send cargo for secondary screening and
to more accurately identify radioactive or nuclear material, the
Department of Homeland Security (DHS) began developing Advanced
Spectroscopic Portals (ASPs) in 2004. The ASPs were intended to both
detect and identify radioactive material. In April 2005, the Domestic
Nuclear Detection Office was created by National Security Presidential
Directive-43/Homeland Security Presidential Directive-14 to, among
other things, research, develop, test and acquire radiation detection
equipment to be used by CBP and other federal agencies.
In July 2006, then-Secretary of Homeland Security Michael Chertoff
and the former Director of DNDO, Vayl Oxford, announced contract awards
to three companies worth an estimated $1.2 billion to develop the ASPs,
including the Raytheon Company and the Thermo Electron Company (now
called Thermo Fisher Scientific Inc.) both headquartered in Waltham,
Massachusetts and Canberra Industries from Connecticut. Canberra is no
longer a contractor on the DNDO program.
ASP Requirements/Criteria
One of the key reasons for replacing the existing radiation
monitors with newly developed ASPs in the first place, as articulated
by Secretary of Homeland Security, Michael Chertoff in July 2006 was to
``have fewer false positives.'' In September 2007, Vayl Oxford, then
the director of DNDO reiterated that point in testimony to Congress
where he emphasized that the ASPs would reduce the number of false
alarms from the nearly 600 experienced each day by the PVTs at the port
of Long Beach in California, for instance, to 20-to-25 per day with the
new ASP monitors. That was the hope, but it has not been the reality
during testing of the ASPs and other serious security questions about
the performance reliability of the ASPs have emerged in the most recent
round of tests.
As the House Committee on Appropriations has said in the past,
procurement of the Advanced Spectroscopic Portal monitors should not
proceed until they are deemed to add a ``significant increase in
operational effectiveness'' over the current PVT system already in
place. In July 2008, CBP, DNDO and the DHS management directorate
jointly issued criteria for determining this increase in effectiveness
in both ``primary'' and ``secondary'' screening. In primary screening
the criteria requires ASPs to detect potential threats as well as or
better than PVTs, show improved detection of Highly Enriched Uranium
and reduce innocent alarms. In secondary screening the criteria
requires ASPs to reduce the probability of misidentifying special
nuclear material (HEU or plutonium) and reduce the average time to
conduct secondary screenings. The Secretary of Homeland Security must
certify to Congress that the ASPs have met these criteria before
funding for full-scale procurement of the ASPs goes forward. The
criteria to measure this improvement, however, are weak and rather
vague.
Testing Regime
Significant hurdles remain before ASPs can be certified and fully
deployed. Both contractors have passed ``integration testing.'' They
must now successfully make it through Field Validation Tests where they
operate at ports of entry in tandem with PVT units. So far, only one of
the two ASP vendors has made it to this stage. The one vendor that has
made it to this stage will need to make its third attempt to
successfully pass the Field Validation Tests before it can move
forward. If and when they successfully pass this stage of testing they
will then go to ``Solo Operations,'' where they will be tested at a
port-of-entry operating independently of the PVTs. If they pass those
two critical tests, then the DHS Directorate of Science & Technology
which has been mandated the Operational Testing Authority (OTA) of the
ASPs will put them through a separate series of tests to ensure they
meet the specified requirements, do not suffer from technical glitches
and operate efficiently. Once that testing is completed and the S&T
Directorate signs off on the performance and reliability of the ASPs
then the DHS Secretary must make a determination about whether the
costs of the ASPs and the capabilities they provide justifies a
decision to invest in their full scale deployment. Along the way DNDO
is supposed to provide a final cost-benefit-analysis of the ASP program
to help inform the Secretary's decision. This document has been
promised many times but not yet completed.
Masking & Shielding
If terrorists were to try to smuggle nuclear or radiological
materials into the U.S. via containerized cargo they would likely try
to shield and/or mask those materials in an attempt to make it more
difficult to detect, identify and locate the material of concern.
Shielding requires that lead or other types of metal enclose the
radioisotopes to hide its radioactive signature. Potential terrorists
may also attempt to ``mask'' threatening radioactive material by
placing it together with or alongside other non-threatening material
that has a natural radioactive signature, such as ceramic material,
kitty liter or even bananas. Most nuclear security experts believe
smuggled radioactive or nuclear material would be both shielded and
masked in order to conceal it from being located and properly
identified. These efforts would make it harder to detect.
Many of DNDO's previous tests of the ASPs have been criticized for
being less than realistic. In one series of tests the ASP portals did
prove more effective than the PVTs in detecting HEU materials concealed
by ``light shielding.'' However, differences between the ASPs and PVTs
became less notable when shielding was slightly increased or decreased.
In other tests there was virtually no difference in the performance of
the two machines with regard to detecting other kinds of radioactive
isotopes, such as those used for medical or industrial purposes,
according to the GAO, except in one case where the ASPs performed worse
than the PVTs. In the most recent round of tests in July DNDO says the
ASPs detected one radioactive source that the PVTs missed.
In previous attempts to detect HEU during tests, the ASPs performed
better only in one narrowly defined scenario, which many experts see as
an unrealistic portrayal of a true attempted nuclear smuggling
incident. None of the tests run by DNDO, for instance, included
scenarios that utilized both ``shielding'' and ``masking'' as a means
of attempting to smuggle radioactive or nuclear material. In addition,
only one of the vendors has made it to field validation testing. But as
the contractor has attempted to fix problems that occurred during
previous tests new, more serious technical issues have emerged.
Field Validation Tests
The Raytheon ASPs went through their first round of field tests
last February, but technical issues hampered their performance. They
had a large number of false alarms on several radioactive isotopes.
Overall, in fact, the ASPs sent more cargo for secondary inspection
than the currently operating PVTs did. Adjustments were made to prepare
them for another round of field tests. Since the Subcommittee's last
hearing on the ASP program in June, the ASPs have gone through a second
Field Validation Test at four U.S. ports of entry in L.A. Long Beach,
California; the New York Container Terminal in Newark, New Jersey; Port
Huron, Michigan; and Laredo, Texas.
On average, the PVTs refer one out of every 40 cargo containers to
secondary inspection placing a large a burden on the staffing resources
of CBP. The ASPs are required to send only one out of every 1,000
inspections to secondary inspection in order to help lessen that
logistical burden. This is one of the key requirements that must be met
in order for the Secretary of Homeland Security to permit full scale
production of the ASPs to proceed. During the Field Validation Testing
last February, however, the ASPs sent more than five times that number
of cargo conveyances to secondary inspection based on false alarms.
During the most recent Field Validation Tests in July the ASPs
reportedly reduced the number of false alarms compared to the PVTs by
69 percent bringing them much closer to the 80 percent reduction in
false alarms that they are required to meet. But new, more serious
problems also emerged during the field validation tests in July.
During this second round of field tests the ASPs again failed to
perform as expected. This time they falsely identified several cargo
conveyances as having special nuclear material, when they actually had
none. This is a critical issue, since the actual smuggling of special
nuclear material presents a serious threat. If it is detected at a
port-of-entry Customs and Border Protection officers have extensive
response requirements they must implement. DNDO and the contractor are
still unclear why the ASPs falsely identified special nuclear material
during these tests. Their intended fix to this problem has been to
decrease the sensitivity of the ASP monitors to specific radioactive
isotopes. The hope is that this will correct the problem, reduce the
number of false alarms and still ensure that the ASPs are able to
detect these isotopes. It is a delicate and difficult balance. It also
decreases the ostensible advantage of having the ASPs replace the PVTs
in the first place.
Most unsettling, in one instance during the July tests one ASP
monitor stopped working altogether yet the system operator remained
unaware of this malfunction. Two dozen cargo trucks were permitted to
go through the ASP in order to be screened for potential radioactive
and nuclear material while it was not operating. DNDO considered this a
``Mission Critical Failure.'' Fortunately, during these tests all
trucks that went through the ASP also went through a PVT monitor. If
this had occurred during ``solo'' testing of the ASPs or during actual
deployment of the ASPs, cargo carrying radiological or nuclear threat
material could have sailed past port security and into the United
States unchecked. The cause of this problem has reportedly been
rectified by the contractor.
Energy Windowing
Many experts believe significant improvements can be made to the
existing fleet of PVT radiation monitors without investing billions of
dollars into new ASPs. Energy windowing is a mathematical algorithm
that can help improve the sensitivity of PVT radiation monitors,
enhancing their ability to detect radioactive sources resulting in
improved operations and capabilities. The technology is currently used
in some radiation monitors. Both GAO and CBP believe that DNDO should
much more aggressively invest in this research to improve the
performance of the currently operating radiation detection monitors.
Although energy windowing may only lead to modest enhancements in the
performance of PVTs, that improvement could be significant in terms of
improving their performance to be more on par with what ASPs are
supposed to be capable of and at a far less financial cost. Reducing
the sensitivity of the ASPs to certain types of special nuclear
material, which was done to resolve the problems that emerged during
the July tests, should not prevent them from alarming for isotopes that
were not there in the first place. The only result would be to reduce
the odds that the ASPs will identify those isotopes when they are
actually present.
A Dwindling Supply of Helium-3 (He-3)
The future deployment of both PVT and ASP monitors is dependent on
the supply of Helium-3 (He-3), a non-radioactive gas that is a
byproduct of tritium decay. Tritium is a critical component in nuclear
weapons used to boost the yield of nuclear warheads. Helium-3 gas is
used in neutron detector tubes, a component of both PVT and ASP
radiation portal monitors used to help identify plutonium. He-3 is also
used in medical imaging, such as MRI machines, the oil and gas industry
and for high energy research. During the cold war the U.S. had a steady
supply of He-3 as a result of its nuclear weapons production
operations. With the end of the cold war the production of nuclear
weapons ceased and this supply diminished. At the same time, since 9/11
the demand for radiation monitors skyrocketed and demand for He-3 soon
out-paced the supply.
There are no readily available alternatives to He-3. In addition,
no other technology matches the stability, sensitivity, and ability to
detect neutron radiation that He-3 neutron tubes currently offers. DNDO
has estimated that the anticipated supply-to-demand ratio of Helium-3
in coming years is expected to be 1-to-10. Costs for the rare isotope
have already begun to rise. By one estimate, a few years ago the cost
of He-3 was around $100 per liter. Today, He-3 is estimated to cost as
much as $2,000 per liter. According to a recent Department of Energy
report, new ASP radiation monitors will use nearly three times more He-
3 as current PVT monitors do, about 132 liters compared to 44 liters.
These facts should be carefully considered by the Secretary of DHS when
making cost-benefit decisions about whether or not to proceed with
producing the ASPs.
Cost Benefit Analysis
Even if the technical abilities of the ASPs are proven, their
relative technical capabilities and increased costs must be carefully
weighed in comparison to the existing radiation monitoring system in
place today. Replacing a proven, less-costly system that has the
confidence of its operators, must be given careful consideration. The
DNDO has not yet provided an updated cost-benefit-analysis that would
validate a decision to procure the multi-billion dollar ASP equipment.
Virtually any high-technology research and development program
experiences bumps in the road, technical troubles and occasional set-
backs. However, well managed programs have clear technical requirements
and strategic goals. They ensure that the new technology being
developed is thoroughly tested and adequately integrated into the
operational plans and procedures of those who must operate them in the
field. When these vital components are short changed, when the test
plan is insufficient and the program's research, development and
testing methods are marred by scanty scientific rigor then the
technical tools being developed are bound to suffer as a result.
Cutting critical corners in the development process serves no one's
interests. Yet, at the start of the ASP program many of the DNDO
leaders seemed more interested in fielding this technology then in
effectively validating its performance and effectiveness. At the July
2006 press conference unveiling the contractors on the ASP program,
Vayl Oxford then the Director of DNDO said: ``the priority for the
first year . . . is to get units out immediately.'' Three years later,
none of the ASPs have yet cleared field validation tests.
Witnesses:
Mr. Gene Aloise, Director, Natural Resources and Environment,
Government Accountability Office (GAO)
Dr. Timothy M. Persons, Chief Scientist, Government Accountability
Office (GAO)
Mr. Todd Owen, Executive Director for Cargo and Conveyance Security,
U.S. Customs and Border Protection (CBP), Department of Homeland
Security (DHS)
Dr. William Hagan, Acting Deputy Director, Domestic Nuclear Detection
Office (DNDO), Department of Homeland Security (DHS)
Chairman Miller. Welcome to today's hearing entitled The
Science of Security, Part II: Technical Problems Continue to
Hinder Advanced Radiation Monitors.
Soon after the September 11 attacks, Customs and Border
Protection, CBP, began operating radiation portal monitors to
screen cargo entering the United States for radiological and
nuclear material. They have purchased approximately 1,500
polyvinyl toluene monitors and deployed them at ports and
border crossings throughout the United States. Mercifully,
polyvinyl toluene monitors are generally referred to as PVTs.
PVTs indicate the presence of a radiation source, but they
cannot identify the nature of the source. As a result, any
cargo container that provokes a warning from a PVT is then sent
to a secondary inspection where customs officers use other
technology and information to determine what sort of material
is in the container. There are plenty of innocent sources of
radiation: kitty litter, medical isotopes, ceramics, bananas,
and many of these secondary inspections can be handled quickly.
However, some secondary inspections require that the container
be opened, and even emptied, in the search for a source. That
is time consuming, but the PVT seems to be working well to meet
customs' dual mission to keep us safe while maintaining a
steady flow of commerce.
The Department of Homeland Security (DHS) has been
developing a new radiation portal monitor, championed by the
Domestic Nuclear Detection Office, DNDO, that advocates believe
should replace the PVTs. The new monitor, the Advanced
Spectroscopic Portal monitor, or ASP, would detect the presence
of radiation in cargo, but it would also identify the type of
radiation. That would allow more harmless cargo to pass
unimpeded through the port and require far fewer secondary
referrals. If it worked as advertised, the ASPs would be more
likely to identify highly enriched uranium or other materials
of concern and enhance the flow of commerce while freeing up
customs officers to tend to other duties besides secondary
inspections.
Despite a $230 million investment of taxpayer dollars thus
far for development, the ASPs haven't performed as expected,
and the results from recent tests are still worrisome. Last
June we learned of problems in the first field test in February
2009. At our June hearing we heard that those issues had been
fixed and that the July field test would allow the department
to move forward towards a cost-benefit analysis and
certification decision for the Secretary. So far, we have seen
neither.
The July test highlighted yet another problem with the
ASPs. The devices detected nuclear materials when none were
present.
The ASPs had numerous false positive hits for special
nuclear material in July, each one of which would have resulted
in the implementation of mandated security responses by
Customs, potentially shutting down port operations.
Fortunately, Customs was also running the PVTs which saw no
radiation presence, and there was none, and cleared up the
issue in secondary fairly quickly.
DNDO has told our staff that they intend to fix this new
problem by changing the sensitivity of the ASPs to detect
uranium, but it isn't clear why that should be reassuring,
changing the sensitivity setting. If we lower the sensitivity
setting to the very materials the ASP is supposed to be better
at monitoring or detecting, would the ASP still be better than
PVTs at detecting those materials? And would we have to go back
to the Nevada Test Site to prove that you can still detect
levels of special nuclear material more accurately than the
PVTs can?
Second, if the machines detect special nuclear material
when there is none there, how does changing the sensitivity
make any difference at all? Detecting ghost isotopes is a
problem with the operating system, not with the sensitivity
level, it would appear anyway.
Since the taxpayers have spent $200,000 for each of the
1,500 PVTs that are deployed now, the case for ASPs, which will
run about $800,000 per unit, a total cost of $2 to $3 billion,
needs to be clear both in terms of better detection performance
and better support for Customs operations. Add to that greater
acquisition cost, an annual operating expense of ASPs that is
at least five times more expensive per unit than PVTs, and the
need for a convincing case is even greater. As it stands, it is
hard to see why ASPs should be more than a secondary inspection
tool.
In fact, that is the role they play in the Department of
Energy's Megaport program. The DOE already runs a program that
inspects cargo leaving 27 major foreign ports for destinations
anywhere in the world. DOE, which developed portal radiation
detection technology, uses PVTs for primary inspection and then
uses the ASPs for secondary inspection to help identify the
type of isotope to which the PVT responded in the first place.
The Department of Energy's approach to identifying radiation
should be instructive to DHS.
I want to thank all of our witnesses for appearing today. I
particularly want to thank GAO for continuing their work on
this matter and for their continued assistance to this
committee and Congress, and the Appropriations Committee as
well. It is very helpful for the Appropriations Committee to be
asking the right questions, and you have helped them and helped
us ask the right questions. I suspect that this will not be the
last time that we gather on this subject--we do seem to come
back to the same subjects again and again--nor the last time we
hear from witnesses that we still face a long list of tests and
validations before we can think about replacing the PVTs with
the ASPs.
I would now recognize the Ranking Member, Dr. Broun, for
his opening comment.
[The prepared statement of Chairman Miller follows:]
Prepared Statement of Chairman Brad Miller
Soon after the September 11 attacks, Customs and Border Protection
(CBP) began operating radiation portal monitors to screen cargo
entering the United States for radiological or nuclear material. They
have purchased approximately 1,500 polyvinyl toluene (PVT) monitors and
deployed them at ports and border crossings throughout the United
States.
PVTs indicate the presence of a radiation source, but they cannot
identify the nature of the source. As a result, any cargo container
that provokes a warning from a PVT is then sent to a ``secondary''
inspection where Customs officers use other technology and information
to determine what sort of material is in the container. There are
plenty of innocent sources of radiation--from kitty litter to medical
isotopes--and many of these secondary inspections can be handled
quickly. However, some secondary inspections require that the container
be opened, and even emptied, in the search for a source. While this is
time consuming, the PVT seems to be working well to meet Customs' dual
mission to keep us safe while maintaining a steady flow of commerce.
The Department of Homeland Security has been developing a new
radiation portal monitor, championed by the Domestic Nuclear Detection
Office (DNDO), that advocates believe should replace the PVTs. This new
monitor--the Advanced Spectroscopic Portal monitor or ASP--would detect
the presence of radiation in cargo, but it would also identify the type
of radiation. This would allow more harmless cargo to pass unimpeded
through the port and require far-fewer secondary referrals. If it
worked as advertised, the ASPs would be more likely to identify highly
enriched uranium and other materials of concern and enhance the flow of
commerce while freeing up Customs officers to tend to other duties
besides secondary inspections.
Despite a $230 million investment of taxpayer dollars for
development, the ASPs haven't performed as expected, and the results
from recent field tests are worrisome. Last June we learned of problems
in the first field test of February 2009. At our June hearing we heard
that those issues had been fixed and that the July field test would
allow the Department to move towards a cost-benefit analysis and
certification decision for the Secretary. To date, we have seen
neither.
The July field test highlighted yet another problem with the ASPs:
the devices detected nuclear materials when none were present.
The ASPs had numerous false positive hits for special nuclear
material in July--each one of which would have resulted in the
implementation of mandated security responses by Customs, potentially
shutting down port operations. Fortunately, Customs was also running
the PVTs (which properly saw no radiation present) and cleared up the
issue in secondary.
DNDO has told our staff that they intend to fix this new problem by
changing the sensitivity of the ASPs to detecting uranium, but it isn't
clear that should be reassuring. If you lower the sensitivity to the
very materials the ASP was supposed to be better at detecting, why
would the ASP will still be better than PVTs at detecting those
materials? Would you have to go back to the Nevada Test Site to prove
that you can still detect levels of special nuclear material more
accurately than the PVTs can?
Second, if the machines detect special nuclear material where it
doesn't exist, why should changing the sensitivity make any difference
at all? Detecting ghost isotopes is a problem with the operating system
itself, not with the sensitivity level for a particular isotope.
Since the taxpayers have spent $200,000 for each of the 1,500 PVTs
already deployed, the case for ASPs, which will run approximately
$800,000 per unit--for a total cost of $2-3 billion--needs to be clear
both in terms of better detection performance and better support for
Customs operations. Add to this greater acquisition cost, an annual
operating expense of ASPs that is at least five times more expensive
per unit than PVTs, and the need for a convincing case is even greater.
As it stands, it is hard to see why ASPs should be more than a
secondary inspection tool.
In fact, that is the role they play in the Department of Energy's
Megaport program. DOE already runs a program that inspects cargo
leaving 27 major foreign ports for destinations anywhere in the world.
DOE, which developed portal radiation detection technology, uses PVTs
for primary inspection and reserves ASPs for secondary inspections to
help identify the type of isotope to which the PVT responded. The
Department of Energy's approach to identifying radiation should be
instructive to DHS.
I want to thank our witnesses for attending today. I particularly
want to thank GAO for their continuing work on this matter and for
their continuing assistance to this committee and Congress. I suspect
that this will not be the last time we gather on this subject, nor the
last time we hear from witnesses that we still face a long list of
tests and validations before we can even speak sensibly about replacing
PVTs with ASPs.
Mr. Broun. Thank you, Mr. Chairman. I want to welcome the
witnesses here today and thank you all for participating in our
follow-up hearing on the Department of Homeland Security's
Advanced Spectroscopic Portal program. It is hard for a
Southerner to say quickly.
This afternoon we will be brought up to date on the
Department's ongoing development of the next generation
radiation portal monitors and get an update from the GAO on
their continuing work. As I said in our earlier hearing this
past summer, this program is certainly not out of the woods.
The latest field validation tests reveal additional problems
that will have to be overcome before moving forward. I hope
DNDO will be able to give us some insight today on what we can
expect from this program in terms of the future paths forward.
With considerable taxpayer money on the line, questionable
improvements over current capabilities and outstanding cost
benefit analysis and a confusing acquisitions history that
unfortunately has morphed the R&D with procurement, this
program is rapidly approaching a point where the Federal
Government has to decide whether it wants to fish or cut bait.
I am concerned with the fact that considerable public funding
has been expended on developing a technology that the private
sector was developing in parallel on its own dime. DHS as a
whole, and DNDO, CBP, DHS, S&T individually, should be focusing
on long-term, high-risk, high-reward technology, not providing
seed money for commercial, off-the-shelf equipment. That being
said, I realize that DHS's mission is vastly different from
DOE's, the Department of Defense's and that they have
additional requirements that demand a more robust system. GAO
and the Academy made several recommendations over the past few
years. I trust that DNDO and CBP will be able to update this
committee on how they are responding to those recommendations
and where they plan to go from here. The Nation expects a lot
from the Department, and I hope that we aren't developing
tunnel vision by focusing too much on one method of conveyance
and not seeing the forest for the trees. The Department has an
enormous task of securing our borders and not just at points of
entry but all along our borders. Spending billions of dollars
to secure the front door of our house doesn't seem very
rational if we are just going to leave the back door open and
all the windows and have a gaping hole in the walls, too. That
is not to say that we should do nothing at all, but rather,
everything we do should be put in context of a well-thought-out
global nuclear detection architecture.
As I said earlier this summer, many of these issues we are
dealing with today could have been prevented by engaging the
end-users earlier in the process. Clearly defining the
requirements, developing clear architectural priorities and
simply following a clear acquisition process. This Committee is
no stranger to programs that have set aside these best
practices for working in expediency's sake.
I look forward to working with the Department and the
Majority to make sure any decision that is made is in the best
interest of our nation's security, the taxpayer and our
economy.
With that, Mr. Chairman, I yield back my time and I thank
you.
[The prepared statement of Mr. Broun follows:]
Prepared Statement of Representative Paul C. Broun
Thank you, Mr. Chairman. I want to welcome the witnesses here
today, and thank them for participating in our follow-up hearing on the
Department of Homeland Security's (DHS) Advanced Spectroscopic Portal
(ASP) program. This afternoon we will be brought up to date on the
Department's ongoing development of next generation Radiation Portal
Monitors and get an update from the General Accountability Office (GAO)
on their continuing work.
As I said at our earlier hearing this past summer, this program is
certainly not out of the woods. The latest Field Validation Test
revealed additional problems that will have to be overcome before
moving forward. I hope DNDO will be able to give us some insight today
on what we can expect from this program in terms of future paths
forward. With considerable. taxpayer money on the line, questionable
improvements over current capabilities, an outstanding cost-benefit
analysis, and a confusing acquisitions history that unfortunately has
morphed Research and Development (R&D) with procurement, this program
is rapidly approaching a point where the Federal Government has to
decide to ``fish or cut bait.''
I'm also concerned with the fact that considerable public funding
has been expended on developing a technology that the private sector
was developing in parallel on its own dime. DHS as a whole (and DNDO,
CBP, and DHS S&T individually) should be focusing on long-term high-
risk high-reward technology, not providing seed money for Commercial
Off-The-Shelf (COTS) equipment. That being said, I realize that DHS'
mission is vastly different from the Department of Energy's (DOE) and
the Department of Defense's (DOD), and that they have additional
requirements that demand a more robust system.
GAO and the Academy made several recommendations over the last few
years. I trust that DNDO and CBP will be able to update this committee
on how they are responding to those recommendations, and where they
plan to go from here. The Nation expects a lot from the Department, and
I hope that we. aren't developing tunnel vision by focusing too much on
one method of conveyance and not seeing the forest through the trees.
The Department has an enormous task of securing our borders, not just
at points of entry, but all along our borders. Spending billions of
dollars to secure the front door or our house, doesn't seem very
rational if we are just going to leave the back door open: That is not
to say we should do nothing at all, but rather everything we do should
be put in the context of a well thought out Global Nuclear Detection
Architecture.
As I said earlier this summer, many of the issues we are dealing
with today could have been prevented by engaging the end-users earlier
in the process, clearly defining requirements, developing clear
architectural priorities, and simply following a clear acquisition
process. This committee is no stranger to programs that have set aside
these best practices for expediency's sake. I look forward to working
with the Department and the majority to make sure any decision made is
in the best interest of our nation's security, the taxpayer, and our
economy.
With that, Mr. Chairman, I yield back my time.
Thank you.
Chairman Miller. Thank you, Dr. Broun. I ask unanimous
consent that all additional opening statements submitted by
Members be included in the record. Without objection, it is so
ordered. Also, there has been an e-mail exchange between our
staff and the staff of DNDO, Kimberly Koeppel, and without
objection, I move that the printed versions of the e-mailed
questions and answers also be entered into the record. Without
objection, so ordered.
[The information follows:]
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
Chairman Miller. It is my pleasure to introduce our
witnesses at this time. Mr. Gene Aloise is the Director of
Natural Resources and Environment at the Government
Accountability Office, GAO. He is an expert in international
nuclear proliferation and safety issues and holds degrees in
political science, economics and public administration. Mr.
Aloise is a recipient of GAO's Meritorious Service and
Distinguished Services Awards and has served several
Congressional committees and offices within GAO. This is not
the first time he has appeared before us on this topic.
With Mr. Aloise is Dr. Timothy Persons, GAO's Chief
Scientist. Before entering GAO, before joining GAO, Dr. Persons
was the Technical Director of the Intelligence Advanced
Research Projects Activity, I-ARPA, and previously served as a
technical director at the National Security Agency. Mr.
Persons, if people ask you at a cocktail party what you did for
a living, did you just say you worked for the government? He
holds degrees in nuclear physics, computer science and
biomedical engineering and has been involved in evaluations of
many high-tech U.S. Government programs and projects, including
GAO's work on the Advanced Spectroscopic Portals, ASP monitors.
We look forward to hearing his testimony today as well.
Mr. Todd C. Owen is the Acting Deputy Assistant
Commissioner of the Office of Field Operations for the U.S.
Customs and Border Protection, Department of Homeland Security.
Also a mouthful. He also served as the Executive Director of
the Cargo and Conveyance Security Office, Office of Field
Operations, since May of 2006. Mr. Owen began his career with
the Customs and Border Protection in 1990 and has previously
held the positions of Area Port Director in New Orleans and
Director of the Customs Trade Partnership Against Terrorism, C-
TPAT Program.
And then finally, Dr. William Hagan is the Acting Deputy
Director of the Domestic Nuclear Detection Office at the
Department of Homeland Security. Dr. Hagan holds degrees in
physics and nuclear engineering as well as three patents. He
spent 30 years with the Science Applications International
Corporation before coming to DNDO and serving as the Assistant
Director of the Transformational Research and Development
Directorate.
I think all of our witnesses would need to use small print
for their business cards.
As our witnesses should know, you will each have five
minutes for your spoken testimony. Your written testimony will
be included in the record for the hearing. When you all have
completed your spoken testimony, we will begin with questions.
Each Member will have five minutes to question the panel. It is
the practice of this subcommittee to receive testimony under
oath. This is an Investigations and Oversight Subcommittee. Do
any of you have an objection to taking an oath? Let the record
reflect that none of the witnesses had any objection. You also
have the right to be represented by counsel. Do any of you have
counsel here? Let the record reflect that all of the witnesses
said no. We ask you these questions to put you at ease.
Would you all now please stand and raise your right hand?
Mr. Broun. And we are not going to water board them.
Chairman Miller. Well, that would be the next hearing.
Please stand and raise your right hand. Do you swear to tell
the truth and nothing but the truth? Let the record reflect
that each of the witnesses did take the oath.
We will begin with Mr. Gene Aloise. Mr. Aloise, please
begin.
STATEMENT OF MR. GENE ALOISE, DIRECTOR, NATURAL RESOURCES AND
ENVIRONMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE (GAO)
Mr. Aloise. Mr. Chairman and Members of the Subcommittee, I
am pleased to be here today to discuss DHS's plans to develop
and test advanced portal monitors, known as ASPs, for use at
the Nation's borders to prevent nuclear materials from being
smuggled into the United States. My testimony today focuses on
the results of DNDO's testing of the ASPs, including the July
2009 field testing.
According to DHS, the current system of radiation detection
equipment, the PVTs, is effective and does not impede the flow
of commerce. However, DHS wants to improve the capabilities of
the existing equipment with ASPs. One of the major drawbacks of
the ASP, as you know, is a substantially higher cost compared
to the existing equipment. We estimated that the life cycle
cost of each standard cargo version of the ASP to be about
$822,000 compared to about $308,000 for the PVT standard cargo
portal and that the total program cost would be about $2
billion.
Earlier this year I testified before this subcommittee on
DNDO's 2008 round of ASP performance testing at the Nevada Test
Site. That testing showed that the ASPs performed better than
the PVTs in detecting certain nuclear materials and met DOE's
threat guidance. However, the ASP's performance rapidly
deteriorated once shielding was slightly increased. These test
results showed what we reported in 2006, that any increase in
detection of certain nuclear materials would be marginal.
While the ASP may have met DOE's threat guidance for
shielded nuclear material, the threat guidance is not a
realistic approximation of how a terrorist might shield nuclear
material to successfully smuggle it undetected across our
borders.
The latest round of field testing conducted in July
revealed two critical performance problems with ASPs that the
Chairman mentioned. First, the ASPs had an unacceptably high
number of false positives for the detection of high-risk
nuclear material. In other words, the ASP was seeing nuclear
material that wasn't there and alarming. CBP officials told us
that any alarm for this type of nuclear material is very
disruptive to a port or border crossing and could effectively
shut down operations until the source of the alarm is found.
Furthermore, repeated false alarms for nuclear materials could
have the undesired effect of causing CBP officers to doubt the
reliability of the ASPs and be skeptical about the credibility
of future alarms.
The second critical failure of the ASPs noted in the July
testing stemmed from a problem with the key component of the
equipment which led an ASP to in essence shut down. Of great
concern was the fact that the ASP did not alert the CBP
official that it had shut down and was not scanning cargo. If
this were not a controlled test, the CBP officer would have
allowed the cargo to enter the United States thinking it had
been scanned when it had not.
DNDO's proposed solutions to these critical failures raise
questions about whether the ASPs will provide any meaningful
increase in the ability to detect certain nuclear materials.
Specifically, to address the problem of false positives, DNDO
is modifying the ASP to make it less sensitive to certain
nuclear materials. While this may fix the problem of false
alarms, it diminishes even further the ASP's ability to detect
the nuclear material we are most concerned about.
To address the second failure, DNDO plans to, among other
things, install an indicator light on the ASP that will alert
CBP officials that the ASP has a mission-critical failure. In
our view, an indicator light is not the solution. The ASP must
be stable and secure enough to avoid these shut-downs.
Furthermore, DNDO has not completed efforts to improve the
PVTs to detect high-risk material through energy windowing. CBP
has repeatedly urged the completion of this research, because
an improved PVT could be the more cost-effective way to improve
detecting certain nuclear materials and have a similar
performance to a working ASP.
In closing, the concerns raised by the results of the July
2009 field testing provide even greater reason for DNDO to
implement our recommendations from our May 2009 report. In
particular, our recommendation that DNDO assess whether the
ASPs meet the criteria for significant increase in operational
effectiveness based on a valid comparison with the PVT's full
performance. This is especially relevant given that ASPs
seemingly will no longer be as effective in detecting certain
nuclear materials.
Mr. Chairman, that concludes my remarks. Dr. Persons and I
would be happy to respond to any questions you may have.
[The prepared statement of Mr. Aloise follows:]
Prepared Statement of Gene Aloise
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today to discuss GAO's work on the
Department of Homeland Security's (DHS) testing of advanced
spectroscopic portal (ASP) radiation detection monitors. One mission of
U.S. Customs and Border Protection (CBP), an agency within DHS,
includes screening cargo and vehicles coming into this country for
smuggled nuclear or radiological material that could be used in an
improvised nuclear device or radiological dispersal device (a ``dirty
bomb''). To screen cargo at ports of entry, CBP conducts primary
inspections with radiation detection equipment called portal monitors--
large stationary detectors through which cargo containers and vehicles
pass as they enter the United States. When radiation is detected, CBP
conducts secondary inspections using a second portal monitor to confirm
the original alarm and a hand-held radioactive isotope identification
device to identify the radiation's source and determine whether it
constitutes a threat.
The polyvinyl toluene (PVT) portal monitors CBP currently uses for
this screening can detect radiation but cannot identify the type of
material causing an alarm. As a result, the monitors' radiation alarms
can be set off even by shipments of bananas, kitty litter, or granite
tile because these materials contain small amounts of benign, naturally
occurring radioactive material. To address the limitations of current-
generation portal monitors, DHS's Domestic Nuclear Detection Office
(DNDO) in 2005 began to develop and test ASPs, which are designed to
both detect radiation and identify the source.\1\ DNDO hopes to use the
new portal monitors to replace at least some PVTs currently used for
primary screening, as well as PVTs and hand-held identification devices
currently used for secondary screening.
---------------------------------------------------------------------------
\1\ DNDO was established within DHS in 2005; its mission includes
developing, testing, acquiring, and supporting the deployment of
radiation detection equipment at U.S. ports of entry. CBP began
deploying portal monitors in 2002, prior to DNDO's creation, under the
radiation portal monitor project.
---------------------------------------------------------------------------
Since 2006, we have been reporting on issues associated with the
cost and performance of the ASPs and the lack of rigor in testing this
equipment. For example, we found that tests DNDO conducted in early
2007 used biased test methods that enhanced the apparent performance of
ASPs and did not use critical CBP operating procedures that are
fundamental to the performance of current hand-held radiation
detectors.\2\ In addition, in 2008 we estimated the life cycle cost of
each standard cargo version of the ASP (including deployment costs) to
be about $822,000, compared with about $308,000 for the PVT standard
cargo portal, and the total program cost for DNDO's latest plan for
deploying radiation portal monitors--which relies on a combination of
ASPs and PVTs and does not deploy radiation portal monitors at all
border crossings--to be about $2 billion.\3\
---------------------------------------------------------------------------
\2\ Combating Nuclear Smuggling: Additional Actions Needed to
Ensure Adequate Testing of Next Generation Radiation Detection
Equipment. GAO-07-1247T, (Washington, D.C.: Sept. 18, 2007).
\3\ Combating Nuclear Smuggling: DHS's Program to Procure and
Deploy Advanced Radiation Detection Portal Monitors Is Likely to Exceed
the Department's Previous Cost Estimates. GAO-08-1108R, (Washington,
D.C.: Sept. 22, 2008).
---------------------------------------------------------------------------
Concerned about the performance and cost of the ASP monitors,
Congress required the Secretary of Homeland Security to certify that
the monitors will provide a ``significant increase in operational
effectiveness'' before DNDO obligates funds for full-scale ASP
procurement.\4\ In response, CBP, DNDO, and the DHS management
directorate jointly issued criteria for determining whether the new
technology provides a significant increase in operational
effectiveness. The primary screening criteria require that the new
portal monitors detect potential threats as well as or better than
PVTs, show improved performance in detection of highly enriched uranium
(HEU), and reduce by 80 percent the number of innocent alarms that are
sent to secondary inspection. To meet the secondary screening criteria,
the new portal monitors must reduce the probability of misidentifying
special nuclear material (e.g., HEU and plutonium) and the average time
to conduct secondary screenings.
---------------------------------------------------------------------------
\4\ Consolidated Appropriations Act, 2008, Pub. L. No. 110-161, 121
Stat. 1844, 2069 (2007); Consolidated Security, Disaster Assistance,
and Continuing Appropriations Act, 2009, Pub. L. No. 110-329, 121 Stat.
3574, 3679 (2008); Department of Homeland Security Appropriations Act,
2010, Pub. L. No. 111-83, 123 Stat. 2142, 2167 (2009).
---------------------------------------------------------------------------
DNDO designed and coordinated a new series of tests, originally
scheduled to run from April 2008 through September 2008, to determine
whether the new portal monitors meet the certification criteria and are
ready for deployment. Key phases of this round of testing include
concurrent testing led by DNDO of the new and current equipment's
ability to detect and identify threats and of ASPs' readiness to be
integrated into operations for both primary and secondary screening at
ports of entry; field validation testing led by CBP at four northern
and southern border crossings and two seaports; and an independent
evaluation, led by the DHS Science and Technology Directorate at one of
the seaports, of the new portal monitors' effectiveness and
suitability.
In May 2009, we reported on the results of the then-current round
of ASP testing.\5\ The findings from that report were based on
completed tests and preliminary results available at the time. Testing
on ASPs has continued since that report was issued. Today my testimony
will (1) discuss the principal findings and recommendations from our
May report and (2) update those findings based on the results of DNDO's
July 2009 ASP field validation testing. The findings we are presenting
today are based on our previous ASP reports and updated with
information collected during interviews with DNDO and CBP officials. We
also reviewed testing results in a report on the July 2009 tests from
the ASP Field Validation Advisory Panel, a panel made up of officials
from CBP, DNDO, and a national laboratory established to examine
testing results and provide recommendations. On November 12, 2009, we
briefed DHS, CBP, and DNDO officials on the findings of our updated
work. During the briefing, CBP and DNDO officials provided oral
comments and offered additional information and clarifications we
included in this testimony as appropriate. Both our prior work and our
updated work were conducted in accordance with generally accepted
government auditing standards. Those standards require that we plan and
perform the audit to obtain sufficient, appropriate evidence to produce
a reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our statement today.
---------------------------------------------------------------------------
\5\ Combating Nuclear Smuggling: DHS Improved Testing of Advanced
Radiation Detection Portal Monitors, but Preliminary Results Show
Limits of the New Technology, GAO-09-655 (Washington, D.C.: May 21,
2009).
Improved Testing Rigor Discussed in Our May 2009 Report Demonstrates
Limitations of ASPs
Our May 2009 report on the then-current round of ASP testing found
that DHS increased the rigor of ASP testing over that of previous
tests, and that a particular area of improvement was in the performance
testing at the Nevada Test Site, where DNDO compared the capability of
ASP and current-generation equipment to detect and identify nuclear and
radiological materials. For example, unlike in prior tests, the plan
for the 2008 performance test stipulated that the contractors who
developed the equipment would not be involved in test execution. This
improvement addressed concerns we previously raised about the potential
for bias and provided increased credibility to the results.
Nevertheless, based on the following factors, in our report we
questioned whether the benefits of the new portal monitors justify the
high cost:
[bullet] The DHS criteria for a significant increase in
operational effectiveness. Our chief concern with the criteria
is that they require only a marginal improvement over current-
generation portal monitors in the detection of certain weapons-
usable nuclear materials during primary screening. DNDO
considers detection of such materials to be a key limitation of
current-generation portal monitors. The marginal improvement
required of ASPs to meet the DHS criteria is problematic
because the detection threshold for the current-generation
portal monitors does not specify a level of radiation shielding
that smugglers could realistically use. Officials from the
Department of Energy (DOE), which designed the threat guidance
DHS used to set the detection threshold, and national
laboratory officials told us that the current threshold is
based not on an analysis of the capabilities of potential
smugglers to take effective shielding measures but rather on
the limited sensitivity of PVTs to detect anything more than
certain lightly shielded nuclear materials. DNDO officials
acknowledge that both the new and current-generation portal
monitors are capable of detecting certain nuclear materials
only when unshielded or lightly shielded. The marginal
improvement in detection of such materials required of ASPs is
particularly notable given that DNDO has not completed efforts
to fine-tune PVTs' software using a technique called ``energy
windowing'' that could improve the PVTs' sensitivity to nuclear
materials. DNDO officials expect they can achieve small
improvements in sensitivity through energy windowing, but DNDO
has not yet completed efforts to fine-tune the PVTs' software.
In contrast to the marginal improvement required in detection
of certain nuclear materials, the primary screening requirement
to reduce the rate of innocent alarms by 80 percent could
result in hundreds of fewer secondary screenings per day,
thereby reducing CBP's workload. In addition, the secondary
screening criteria, which require ASPs to reduce the
probability of misidentifying special nuclear material by one-
half, address the limitations of relatively small hand-held
devices in consistently locating and identifying potential
threats in large cargo containers.
[bullet] Results of performance testing and field validation.
The results of performance tests that DNDO presented to us were
mixed, particularly in the ASPs' capability to detect certain
shielded nuclear materials during primary screening. The
results of performance testing at the Nevada Test Site showed
that the new portal monitors detected certain nuclear materials
better than PVTs when shielding approximated DOE threat
guidance, which is based on light shielding. In contrast,
differences in system performance were less notable when
shielding was slightly increased or decreased: both the PVTs
and ASPs were frequently able to detect certain nuclear
materials when shielding was below threat guidance, and both
systems had difficulty detecting such materials when shielding
was somewhat greater than threat guidance. With regard to
secondary screening, ASPs performed better than hand-held
devices in identification of threats when masked by naturally
occurring radioactive material. However, the differences in the
ability to identify certain shielded nuclear materials depended
on the level of shielding, with increasing levels appearing to
reduce any ASP advantages over the hand-held identification
devices. Other phases of testing uncovered multiple problems in
meeting requirements for successfully integrating the new
technology into operations at ports of entry. Of the two ASP
contractors participating in the current round of testing, one
has fallen behind due to severe problems encountered during
testing of ASPs' readiness to be integrated into operations at
ports of entry (``integration testing''); the problems may
require that the vendor redo previous test phases to be
considered for certification. The other vendor's system
completed integration testing, but CBP suspended field
validation testing in January 2009 after two weeks because of
serious performance problems resulting in an overall increase
in the number of referrals for secondary screening compared
with existing equipment.
[bullet] DNDO's plans for computer simulations. As of May
2009, DNDO did not plan to complete injection studies--computer
simulations for testing the response of ASPs and PVTs to
simulated threat objects concealed in cargo containers--prior
to the Secretary of Homeland Security's decision on
certification even though delays to the ASP test schedule have
allowed more time to conduct the studies. According to DNDO
officials, injection studies address the inability of
performance testing to replicate the wide variety of cargo
coming into the United States and the inability to place
special nuclear material and other threat objects in cargo
during field validation. DNDO had earlier indicated that
injection studies could provide information comparing the
performance of the two systems as part of the certification
process for both primary and secondary screening. However, DNDO
subsequently decided that performance testing would provide
sufficient information to support a decision on ASP
certification. DNDO officials said they would instead use
injection studies to support effective deployment of the new
portal monitors.
[bullet] Lack of an updated cost-benefit analysis. DNDO had
not updated its cost-benefit analysis to take into account the
results of ASP testing. An updated analysis that takes into
account the testing results, including injection studies, might
show that DNDO's plan to replace existing equipment with ASPs
is not justified, particularly given the marginal improvement
in detection of certain nuclear materials required of ASPs and
the potential to improve the current-generation portal
monitors' sensitivity to nuclear materials, most likely at a
lower cost. DNDO officials said they were updating the ASP
cost-benefit analysis and planned to complete it prior to a
decision on certification by the Secretary of Homeland
Security.
Our May report recommended that the Secretary of Homeland Security
direct DNDO to (1) assess whether ASPs meet the criteria for a
significant increase in operational effectiveness based on a valid
comparison with PVTs' full performance potential and (2) revise the
schedule for ASP testing and certification to allow sufficient time for
review and analysis of results from the final phases of testing and
completion of all tests, including injection studies. We further
recommended that, if ASPs are certified, the Secretary direct DNDO to
develop an initial deployment plan that allows CBP to uncover and
resolve any additional problems not identified through testing before
proceeding to full-scale deployment. DHS agreed to a phased deployment
that should allow time to uncover ASP problems but disagreed with GAO's
other recommendations, which we continue to believe remain valid.
Results from July 2009 Testing Raise Continuing Issues
The results of DNDO's most recent round of field validation
testing, which it undertook in July 2009, after our May report was
released, raise new issues. In July 2009, DNDO resumed the field
testing of ASPs at four CBP ports of entry that it initiated in January
2009 but suspended because of serious performance problems. However,
the July tests also revealed ASP performance problems, including two
critical performance deficiencies. First, the ASP monitors had an
unacceptably high number of false positive alarms for the detection of
certain high-risk nuclear materials. According to CBP officials, these
false alarms are very disruptive in a port environment in that any
alarm for this type of nuclear material would cause CBP to take
enhanced security precautions because such materials (1) could be used
in producing an improvised nuclear device and (2) are rarely part of
legitimate or routine cargo. Furthermore, once receiving an alarm for
this type of nuclear material, CBP officers are required to conduct a
thorough secondary inspection to assure themselves that no nuclear
materials are present before permitting the cargo to enter the country.
Repeated false alarms for nuclear materials are also causes for concern
because such alarms could eventually have the effect of causing CBP
officers to doubt the reliability of the ASP and be skeptical about the
credibility of future alarms.
Secondly, during the July testing the ASP experienced a ``critical
failure,'' which stemmed from a problem with a key component of the ASP
and caused the ASP to shut down. Importantly, during this critical
failure, the ASP did not alert the CBP officer that it had shut down
and was no longer scanning cargo. As a result, were this not in a
controlled testing environment, the CBP officer would have permitted
the cargo to enter the country thinking the cargo had been scanned,
when it had not. According to CBP officials, resolving this issue is
important in order to assure the stability and security of the ASP.
In addition to these key performance problems, the ASP was not able
to reduce referrals to secondary inspection by 80 percent as required
by the DHS criteria for a significant increase in operational
effectiveness. According to the report from the ASP Field Validation
Advisory Panel, a panel made up of officials from CBP, DNDO, and a
national laboratory, the ASP was able to reduce referrals to secondary
inspection by about 69 percent rather than the 80 percent as required
by the DHS criteria.
While the performance of the ASP during the July field validation
testing raises issues about its potential readiness for deployment,
DNDO's proposed solutions to address these performance problems raise
additional questions about whether this equipment will provide any
meaningful increase in the ability to detect certain nuclear materials.
Specifically, to address the problem of false positive alarms
indicating the presence of certain nuclear materials, according to DNDO
officials, DNDO has modified the ASP to make this equipment less
sensitive to these nuclear materials. While this may address the issue
of false positive alarms, it will also diminish the ASP capability of
detecting a key high-risk nuclear material. Since the ASP modification,
DNDO conducted computer simulations using a vendor-provided system
called a ``replay tool'' to examine the effect of the modification on
the ASP's performance. According to DNDO officials, the replay tool
demonstrated that the modified ASP will still be able to detect certain
nuclear materials better than the PVT. However, at this point, DNDO
does not plan to retest the ASP at the Nevada Test Site where it can
examine the effects of these modifications using actual nuclear
materials. As we reported earlier this year, the results of the testing
at the Nevada Test Site demonstrated that the ASPs represented a
marginal improvement in detecting certain nuclear materials. By
reducing the sensitivity to these materials and not retesting the
modified ASPs against actual nuclear materials, it is uncertain exactly
what improvement in detecting certain nuclear materials these costly
portal monitors are providing.
While DNDO is reducing the sensitivity of ASPs to certain nuclear
materials, it has yet to complete efforts to improve the PVT's ability
to detect these same materials through energy windowing. For several
years, CBP officials have repeatedly urged DNDO officials to complete
this research. However, it was not apparent from our discussions with
DNDO officials if this effort is making meaningful progress with the
development of energy windowing or when it will be completed.
Furthermore, CBP officials stated that, depending on the outcome of
this research, energy windowing could be the more cost effective way to
improve detection of certain nuclear materials. In our view, ASPs being
modified to diminish their capabilities to detect certain nuclear
materials raises questions about whether energy windowing might be able
to achieve a similar level of performance against these same materials
from the PVTs that are already in place.
Beyond reducing the sensitivity of ASPs to certain nuclear
materials, DNDO also plans to address the issue of critical failures
by, among other things, installing an indicator light on the ASP that
will alert CBP officers that the ASP has experienced a mission-critical
failure and is no longer scanning cargo. While this should address the
issue of CBP officers not knowing that the ASP has suffered a critical
failure, CBP officials stressed to us the need for the ASP to be stable
and secure enough to avoid these shutdowns.
In closing, the issues raised by the results of the July 2009 field
validation tests provide even greater reason for DNDO to address
recommendations from our May 2009 report. In particular, we reiterate
the importance of our prior recommendation for DNDO to assess whether
ASPs meet the criteria for a significant increase in operational
effectiveness based on a valid comparison with PVTs' full performance
potential, given that the ASPs will no longer be as effective in
detecting certain nuclear materials.
Mr. Chairman, this completes my prepared statement. I would be
happy to respond to any questions that you or other Members of the
Subcommittee may have at this time.
Staff Acknowledgments
Dr. Timothy Persons (Chief Scientist), Ned Woodward (Assistant
Director), Joseph Cook, and Kevin Tarmann made key contributions to
this testimony. Kendall Childers, Karen Keegan, Carol Kolarik, Jonathan
Kucskar, Omari Norman, Alison O'Neill, and Rebecca Shea also made
important contributions.
Biography for Gene Aloise
Gene Aloise is a Director in the Natural Resources and Environment
team at GAO. He is GAO's recognized expert in international nuclear
nonproliferation and safety issues and completed training on these
subjects at the University of Virginia and Princeton University. His
work for GAO has taken him to some of Russia's closed nuclear cities
and the Chernobyl reactor in Ukraine as well as numerous nuclear
facilities around the world and in the United States. Mr. Aloise has
had years of experience developing, leading, and managing GAO domestic
and international engagements. His diverse experience includes
assignments with congressional committees as well as various offices
within GAO. He has received numerous awards for his leadership and
expertise including GAO's Meritorious Service and Distinguished Service
Awards. Mr. Aloise received his Bachelor's degree in political science/
economics from Rowan University and holds a Master of Public
Administration from Temple University. Mr. Aloise is also a graduate of
the Senior Executive Fellows Program, John F. Kennedy School of
Government, Harvard University.
STATEMENT OF DR. TIMOTHY M. PERSONS, CHIEF SCIENTIST, U.S.
GOVERNMENT ACCOUNTABILITY OFFICE (GAO)
Chairman Miller. Thank you, Mr. Aloise. Dr. Persons. I
think I said Parsons earlier. Persons. You are here to answer
questions but the testimony of Mr. Aloise is your testimony as
well?
Dr. Persons. That is correct.
Biography for Timothy M. Persons
Dr. Timothy M. Persons was appointed the Chief Scientist of the
United States Government Accountability Office (GAO--the investigative
arm of the U.S. Congress) in July of 2008. In this role he is an expert
advisor and chief consultant to the GAO, Congress, and other federal
agencies and government programs on cutting-edge science and technology
(S&T), key highly-specialized national and international systems,
engineering policies, best practices, and original research studies in
the fields of engineering, computer, and the physical and biological
sciences to ensure efficient, effective, and economical use of science
and technology in government programs. The Chief Scientist also works
with GAO's Chief Technologist to lead the production of Technology
Assessments for the U.S. Congress.
Prior to joining GAO, Dr. Persons was the Technical Director of the
Intelligence Advanced Research Projects Activity (IARPA) from November
2002 to July 2008. IARPA's mission is to invest in high-risk/high-
payoff research with potential to revolutionize the business of
intelligence collection, processing, analysis, and dissemination. From
July 2001 to November of 2002, he served as the Technical Director for
the National Security Agency's (NSA) Human Interface Security Group
which researches, designs, and tests next-generation biometric
identification and authentication systems. He has also served as a
radiation physicist with the University of North Carolina at Chapel
Hill.
Dr. Persons is a 2007 Director of National Intelligence S&T Fellow
whose research focuses on computational imaging systems. He was also
selected as the James Madison University (JMU) Physics Alumnus of 2007.
He received his B.Sc. (Physics) from JMU, a M.Sc. (Nuclear Physics)
from Emory University, and a M.Sc. (Computer Science) and Ph.D.
(Biomedical Engineering) degrees from Wake Forest University. He is a
senior member of the Institute for Electrical and Electronic Engineers,
Association for Computing Machinery, and the Sigma Xi research honor
society and has authored an array of journal, conference, and technical
articles. He is also a Ruling Elder in the Presbyterian Church in
America and lives happily in Maryland with his wife Gena and their two
children.
Chairman Miller. Okay. Mr. Owen.
STATEMENT OF MR. TODD C. OWEN, ACTING DEPUTY ASSISTANT
COMMISSIONER, OFFICE OF FIELD OPERATIONS, U.S. CUSTOMS AND
BORDER PROTECTION, DEPARTMENT OF HOMELAND SECURITY
Mr. Owen. Chairman Miller, Ranking Member Broun,
distinguished Members of the Subcommittee, I am honored to be
here this afternoon to provide an update on the role that U.S.
Customs and Border Protection plays in protecting our nation
from the illicit introduction of radiological or nuclear
materials in cargo containers, and on the future role that
Advanced Spectroscopic Portal technology would have on CBP
operations.
I would like to begin by expressing my gratitude to
Congress for its continued support toward CBP initiatives.
Among the numerous priorities that were recognized in the
American Recovery and Reinvestment Act of 2009, Congress
provided CBP with $100 million of stimulus funding toward
upgrading and adding non-intrusive inspection equipment. This
funding will allow CBP to upgrade and expand its successful NII
program and more effectively inspect containers and vehicles
crossing our border, allowing them to enter our country and its
commerce in a safe and prompt manner.
Everyday over 57,000 maritime containers, truck trailers
and rail cars cross our border. CBP uses a multi-layered
approach to ensure the integrity of supply chains from points
of stuffing through arrival at U.S. ports of entry. This multi-
layer defense is built upon interrelated initiatives which
include the 24-Hour Rule and the Trade Act of 2002, the
Automated Targeting System, Non-Intrusive Inspection (NII)
equipment and radiation portal monitors, the Container Security
Initiative and the Customs Trade Partnership Against Terrorism,
C-TPAT program. These complementary layers enhance security and
protect our nation.
Prior to 9-11, not a single radiation portal monitor and
only 64 large-scale NII systems were deployed to our nation's
borders. By October of 2002, CBP had deployed the first RPM to
the Ambassador Bridge in Detroit, and today CBP has just under
1,400 RPMs and 232 large-scale NII systems deployed nationwide,
and that is an increase of almost 200 more RPMs and five
additional large-scale NII systems since I last testified
before this subcommittee in June.
NII technology allows the officers to detect possible
anomalies, anomalies which may indicate the presence of weapons
of mass effect or some other contraband. In Fiscal Year 2009,
CBP conducted over 5.2 million examinations using NII
technology, allowing CBP to meet our twin goals of enhance
security and trade facilitation.
In addition to the significant strides made in the area of
NII equipment, the CBP also continues to deploy first-
generation radiation portal monitors to our nation's ports of
entry. Since last testifying before this subcommittee, CBP and
DNDO have now completed deployments of RPMs along the northern
border and now scan 100 percent of trucks and passenger
vehicles arriving from Canada, 100 percent arriving from Mexico
and 98 percent of arriving sea containers for the presence of
nuclear radiological materials. Since the first RPM was
deployed in 2002, CBP officers have scanned over 404 million
conveyances for the presence of radiation and 2.2 million
radiological alarms have been successfully adjudicated with
minimal or no impact on the flow of legitimate trade and
travel. CBP continues to closely coordinate with key
stakeholders to ensure the impacts of this activity causes
minimal disruption on port operations.
The first generation RPM systems, although very sensitive,
do have limitations. While they alert the CBP officers to the
presence of radiation, a secondary exam is necessary to
positively identify the specific isotope causing the alert. In
the event that a CBP officer is unable to positively resolve
the alarm, scientific reach-back is available 24 hours a day,
seven days a week.
The ASP is expected to enhance our detection capability
while significantly reducing the number or secondary
examinations. This is due to its ability to distinguish between
actual threats and natural or medical radiation sources that
are not security threats. CBP has worked closely with the DNDO
in the development and operational testing of ASP, has provided
DNDO with functional requirements and has been actively engaged
in every step of systems evaluation. CBP will continue to work
with DNDO, and with the DHS Science and Technology Directorate,
S&T, toward Secretarial Certification, and is also working with
DHS to ensure that any future acquisition and deployment
decisions are consistent with DHS priorities. The decision to
purchase and deploy ASPs in the operational arena will be based
on CBPs mission needs and operational requirements, a
comprehensive cost benefit analysis to include a full
understanding of the maintenance and operational cost and
analysis of alternatives and other considerations.
Mr. Chairman and Members of the Subcommittee, today I have
addressed CBP's commitment to investing in new and emerging
detection technology aimed at enhancing cargo security. We must
continue to maintain our tactical edge by integrating new
technology into our ports of entry.
Thank you again for the opportunity to testify. I would be
happy to answer any questions.
[The prepared statement of Mr. Owen follows:]
Prepared Statement of Todd C. Owen
Chairman Miller, Ranking Member Broun, esteemed Members of the
Subcommittee, it is a privilege and an honor to appear before you today
to discuss the work of U.S. Customs and Border Protection (CBP),
particularly the detection of radioactive and nuclear material in cargo
containers and the potential future role that the Advanced
Spectroscopic Portal (ASP) program will have on our operations. CBP is
responsible for ensuring the security of cargo entering the United
States at our borders and facilitating the flow of legitimate trade and
travel. As part of this process, over 98 percent of all arriving
maritime containerized cargo is currently scanned for radiation through
radiation portal monitors.
I want to begin by expressing my gratitude to Congress for its
continued support for the mission and people of CBP. It is clear that
Congress is committed to providing CBP the resources we need in order
to increase and maintain the security of our borders. We appreciate
your efforts and assistance.
CBP is the largest uniformed federal law enforcement agency in the
country. We station over 21,000 CBP officers at access points around
the Nation, including air, land, and sea ports. We have deployed over
20,000 Border Patrol agents between the ports of entry. These forces
are supplemented with 1,266 Air and Marine agents, 2,392 agricultural
specialists, and other professionals.
CBP's responsibilities include stemming the illegal flow of drugs,
contraband and people, protecting our agricultural and economic
interests from harmful pests and diseases, protecting American
businesses from theft of their intellectual property, enforcing textile
agreements, tracking import safety violations, regulating and
facilitating international trade, collecting import duties,
facilitating legitimate travel, and enforcing U.S. trade laws. CBP
facilitates lawful immigration, welcoming visitors and new immigrants,
while making certain those entering this country are indeed admissible
and taking appropriate action when an individual fears being persecuted
or tortured if returned to their home country. At the same time, our
employees maintain a vigilant watch for terrorist threats. In FY 2009,
CBP processed more than 361 million pedestrians and passengers, 109
million conveyances, 25.8 million trade entry summaries, examined 5.2
million sea, rail, and truck containers, performed over 26.8 million
agriculture inspections, apprehended over 732,000 illegal aliens
between our ports of entry, encountered over 224,000 inadmissible
aliens at the ports of entry, and seized more than 2.8 million pounds
of illegal drugs.
We must perform our important security and trade enforcement work
without stifling the flow of legitimate trade and travel that is so
important to our nation's economy. These are our twin goals: border
security and facilitation of legitimate trade and travel.
CBP OVERVIEW
I am pleased to appear before the Subcommittee today to highlight
the continued progress on radiation detection technology. I would also
like to take this opportunity to bring attention to CBP's holistic
cargo security programs that are applied to all environments. CBP has
made tremendous progress toward securing the supply chains that bring
goods into the United States from around the world, and preventing
their potential use by terrorist groups, by: using cutting-edge
technology to increase the ability of front-line CBP Officers to
successfully detect and interdict illicit importations of nuclear and
radiological materials; moving resources where they are most needed;
integrating all CBP offices; sharing information, including actionable
intelligence, across all aspects of CBP; and utilizing a multi-layered
approach to ensure the integrity of the supply chain from the point of
stuffing, through arrival at a U.S. port of entry. This multi-layered
approach includes the following comprehensive cargo security programs
that are applied to all modes of transportation:
[bullet] Advance Information
24-Hour Rule
Automated Targeting Systems
Importer Security Filing (commonly known as
``10+2'')
[bullet] The Customs Trade Partnership Against Terrorism (C-
TPAT) and Free and Secure Trade (FAST)
[bullet] Container Security Initiative (CSI)
[bullet] Secure Freight Initiative
[bullet] Use of non-intrusive inspection technology and
mandatory exams for all high risk shipments.
I will discuss each one of these layers in greater detail with
particular focus on our radiation and nuclear detection capabilities.
ADVANCE INFORMATION
CBP requires advanced electronic cargo information, as mandated in
the Trade Act of 2002, for all inbound shipments in all modes of
transportation. This advanced cargo information is evaluated using the
Automated Targeting System (ATS) before the cargo arrives in the United
States.
ATS provides decision support functionality for CBP officers
working in Advanced Targeting Units at our ports of entry and Container
Security Initiative ports abroad. The system provides uniform review of
cargo shipments for identification of the highest threat shipments, and
presents data in a comprehensive, flexible format to address specific
intelligence threats and trends. Through rules, the ATS alerts the user
to data that meets or exceeds certain pre-defined criteria. National
targeting rule sets have been implemented in ATS to provide threshold
targeting for national security risks for all modes of transportation:
sea, truck, rail, and air. The DHS Science and Technology Directorate
is exploring additional methodologies for conducting risk assessment.
The Importer Security Filing interim final rule, also more commonly
known as ``10+2,'' went into effect earlier this year and has already
yielded promising results. This program will increasingly provide CBP
timely information about cargo shipments that will enhance our ability
to detect and interdict high risk shipments. Comments on aspects of
this rule were accepted until June 1, 2009, and implementation using
informed compliance will continue until January of next year. Shipments
determined by CBP to be high-risk are examined either overseas as part
of our Container Security Initiative, or upon arrival at a U.S. port.
CUSTOMS TRADE PARTNERSHIP AGAINST TERRORISM (C-TPAT)
CBP works with the trade community through the Customs Trade
Partnership Against Terrorism (C-TPAT) to better secure goods moving
through the international supply chain. C-TPAT has enabled CBP to
leverage supply chain security throughout international locations where
CBP has no regulatory reach. Under the C-TPAT program, a prospective
member submits basic company information and a security profile via an
Internet-based portal system. CBP conducts records checks on the
company in its law enforcement and trade databases and evaluates the
security profile, ensuring the company meets the security criteria for
its particular business sector. Members who pass initial vetting are
certified into the program. Using a risk-based approach, CBP Supply
Chain Security Specialists conduct on-site visits of foreign and
domestic facilities to confirm that the security practices are in place
and operational.
C-TPAT is a voluntary public-private partnership program wherein
CBP works with the trade community in adopting tighter security
measures throughout their international supply chain. In return for
making these enhancements members are afforded benefits to include
reduced exams, dedicated cargo lanes (FAST), Stratified Exam, an
assigned Supply Chain Security Specialist (SCSS). Potential members
initially complete an online application. The SCSS then conducts
internal record checks and evaluates the security profile provided.
Upon completion of vetting the company then becomes a Certified member
who will then be physically validated within a year of certification.
In 2009, CBP continued to expand and strengthen the C-TPAT program
and ensure that certified member companies are securing their goods
moving through the international supply chain to the United States.
Teams of Supply Chain Security Specialists conducted validations and
re-validations of C-TPAT members' supply chains to ensure that security
protocols are reliable, accurate, and effective.
As C-TPAT has evolved, we have steadily increased both the rigor of
the program and program membership. CBP has strengthened the C-TPAT
program by clearly defining the minimum security requirements for all
categories of participants wishing to take part in the program. As of
Nov. 6, 2009, there were 9,509 companies certified into the C-TPAT
program. CBP's goal is to validate all partners within one year of
certification, re-validate all companies not less than once every three
years, and re-validate all U.S./Mexico highway carriers on an annual
basis, due to the risks of compromise of trailers associated with the
Southern Border Highway Carrier sector of C-TPAT.
C-TPAT's 9,509 Certified Partners include 4,330 importers, 2,583
carriers, 821 brokers, 784 consolidators/third party logistic
providers, 56 Marine Port Authority and Terminal Operators and 935
foreign manufacturers. C-TPAT has conducted 13,246 on-site validations
of manufacturing and logistics facilities in 90 countries. 301 C-TPAT
importer partners have been designated Tier 3, meaning they have
exceeded the minimum security criteria and have been granted the
highest level of program benefits.
CONTAINER SECURITY INITIATIVE
Because of the sheer volume of sea container traffic, containerized
shipping is uniquely vulnerable to terrorist exploitation. To prevent
terrorists and their weapons from entering the United States, CBP has
also partnered with foreign governments through our Container Security
Initiative (CSI). CSI, which is the first program of its kind, was
announced in January 2002 and is currently operational in 58 foreign
seaports--covering more than 80 percent of the maritime containerized
cargo shipped to the United States. The program enables CBP to identify
and inspect high-risk cargo containers at foreign ports before they are
shipped to our seaports and pose a threat to the United States and to
global trade. CSI stations multidisciplinary teams of CBP officers to
work with host country counterparts to identify and examine containers
that are determined to pose the highest risk for terrorist activity.
CBP officers stationed at foreign CSI ports review 100 percent of
the manifests originating and/or transiting those foreign ports for
containers that are destined for the United States. In locations where
the tremendous volume of bills prevents the CSI team at the port itself
from performing 100 percent review, or during port shutdowns, CSI
targeters at the National Targeting Center provide additional support
to ensure that 100 percent review is accomplished. Utilizing the
overseas CSI team and the CSI targeters at our National Targeting
Center, CBP is able to achieve 100 percent manifest review for the CSI
program. In FY 2009, CBP officers stationed at CSI ports reviewed over
nine million bills of lading and conducted over 56,000 exams in
conjunction with their host country counterparts.
SECURE FREIGHT INITIATIVE
The Secure Freight Initiative (SFI) is an effort to build upon
existing port security measures by enhancing the U.S. Government's
ability to scan containers for nuclear and radiological materials in
seaports worldwide and to better assess the risk of inbound containers.
SFI provides carriers of maritime containerized cargo greater
confidence in the security of the shipment they are transporting, and
increases the likelihood of an uninterrupted and secure flow of
commerce. This initiative is the culmination of our work with other
U.S. Government agencies, foreign governments, the trade community, and
vendors of leading-edge technology.
CBP will prioritize future deployments of scanning systems to
locations of strategic importance by identifying seaports where non-
intrusive imaging and radiation detection data would be most practical
and effective in deterring the movement of weapons of mass destruction
via containerized cargo. The additional scan data provided by SFI will
enhance DHS' risk-based and layered approach to securing maritime
containerized cargo. We will continue to work with Congress to enhance
the safety of our nation's ports and the security of incoming cargo.
NON INTRUSIVE INSPECTION/RADIATION DETECTION TECHNOLOGY
The deployment of imaging systems and radiation detection equipment
has contributed tremendously to CBP's progress in ensuring that supply
chains bringing goods into the United States from around the world are
secure against exploitation by terrorist groups. Non-Intrusive
Inspection (NII) technology serves as a force multiplier that allows
officers to detect possible anomalies between the contents of a
container and the manifest. CBP relies heavily on the use of NII, as it
allows us to work smarter and more efficiently in recognizing potential
threats.
Prior to 9/11, not a single Radiation Portal Monitor (RPM), and
only 64 large-scale NII systems were deployed to our nation's borders.
By October 2002, CBP had deployed the first RPM at the Ambassador
Bridge in Detroit. Today, CBP uses RPMs to scan 99 percent of all cargo
arriving in the U.S. by land and sea. CBP, in partnership with the DHS
Domestic Nuclear Detection Office (DNDO) and Pacific Northwest National
Laboratory (PNNL), has deployed 473 RPMs at northern border land ports
of entry; 385 RPMs at southern border land ports of entry; 433 RPMs at
seaports; and 55 RPMs at mail facilities. Currently, CBP has 232 large-
scale NII systems deployed and 5,816 small-scale NII units.
Additionally, CBP has deployed 1,515 Radiation Isotope Identifier
Devices (RIIDs) and 19,365 Personal Radiation Detectors (PRDs). These
devices allow CBP to examine 100 percent of all identified high-risk
cargo. To date, CBP has used the deployed systems to conduct over 38
million examinations, resulting in over 8,300 narcotic seizures, with a
total weight of over 2.6 million pounds, and over $28.6 million in
undeclared currency seizures. Since RPM program inception in 2002, CBP
has scanned over 404 million conveyances for radiological contraband
resulting in over 2.2 million alarms. CBP's Laboratories and Scientific
Services spectroscopy group at the National Targeting Center has
responded to some 21,599 requests from the field for technical
assistance in resolving alarms. To date 100 percent of all alarms have
been successfully adjudicated as innocent.
CBP is pleased to report that the final installation of RPMs along
our shared northern border was commissioned on Oct. 29 at the Trout
River, N.Y., port of entry. This milestone represents another critical
step in the Department's efforts to strengthen the interconnected U.S.
border security network by deploying technology and personnel to key
border locations to meet modern-day security needs. This recent
milestone provides CBP with the ability to utilize radiation detection
technologies to scan 100 percent of trucks and personally owned
vehicles arriving through both northern and southern border ports, and
98 percent of arriving sea containers. In addition, CBP officers now
use hand-held radiation identification devices to scan 100 percent of
private aircraft arriving in the U.S. from foreign destinations.
The first generation RPM systems, although very sensitive, do have
limitations. While they alert CBP officers to the presence of
radiation, a secondary exam is necessary to identify the location and
specific isotope causing the alert. In the event that a CBP officer is
unable to positively resolve the alert, scientific reach back is
available on a 24/7 basis through the National Targeting Center and
CBP's Laboratory & Scientific Services Division located in the northern
Virginia area.
Understanding these limitations and the need for more precise
radiological detection architecture, DNDO was created in 2005 to focus
on radiological and nuclear threats and develop new technologies that
will improve the Nation's ability to detect and identify radiological
and nuclear weapons and material. One of these new technologies is the
next generation RPM, or the Advanced Spectroscopic Portal (ASP).
The ASP is able to distinguish between actual threats and natural
or medical radiation sources that are not security threats. In doing
so, the ASP would enhance our detection capability, while significantly
reducing the burden of responding to the numerous benign alarms that
are mostly generated by everyday products. This would allow CBP to
focus our staffing and resources on high-risk shipments and other
border security initiatives.
CBP COORDINATION WITH DNDO
In our collaboration with DNDO, CBP brings knowledge of how our
ports work, of the needs of our front-line officers, and of the
operational requirements for new technologies that must work
consistently in a broad array of environments. Additionally, CBP is
attuned to critical factors such as throughput and capacity as we seek
to maintain an appropriate balance between security and the
facilitation of cross-border travel and trade.
CBP has worked closely with DNDO in the developmental and
operational testing of the ASP. A complete, independent operational
testing and evaluation will be conducted by the DHS Science and
Technology Directorate's Test and Evaluation and Standards Division,
along with the Director of Operational Test and Evaluation, when the
system completes the current course of testing. CBP's objective for
operational testing is to ensure that systems are suitable and
effective in our operational environments. CBP provided DNDO with
functional requirements for the ASP and has been actively engaged in
every step of testing, including performance testing at the Nevada Test
Site and integration testing currently ongoing at a mock port of entry
at the Pacific Northwest National Laboratory.
During CBP-led field validation and integration testing, CBP has
been working closely with DNDO to assess each ASP system's performance
as an integrated unit, including reach back capability and ancillary
equipment such as traffic lights and automated gate arms that are
essential to maintaining positive control of vehicles at our ports of
entry. In addition, CBP works with DNDO to assess and categorize each
system's issues to ascertain their technological impact on performance
and their operational impact on front-line CBP officers--the users of
the system.
CBP will continue to work with DNDO toward certification by the
Secretary, which is dependent on demonstrating a ``significant increase
in operational effectiveness'' over existing first-generation radiation
detection systems. At such time, the discussion could then turn to
potential acquisition and deployment of ASP systems. Any decision to
purchase and deploy ASPs in the operational arena will be based on
mission needs, operational requirements, and a full understanding of
maintenance and operational costs, to include a comprehensive cost-
benefit analysis and an analysis of alternatives.
CONCLUSION
Technology plays an enormous role in securing the supply chain.
Security technology is continuously evolving, not only in terms of
capability but also in terms of compatibility, standardization, and
integration with information systems. It is important to note that
there is no single technological solution to supply chain security. As
technology matures, it must be evaluated, and adjustments to
operational plans must be made. Priority should be given to effective
security solutions that complement and improve the business processes
already in place, and which build a foundation for secure 21st century
global trade.
Mr. Chairman, Members of the Subcommittee, thank you again for this
opportunity to testify about CBP's commitment to investing in new and
emerging detection technology, as well as some of the steps we have
taken toward enhancing cargo security. I will be happy to answer any of
your questions.
Biography for Todd C. Owens
Mr. Todd C. Owen was appointed to Executive Director of the Cargo
and Conveyance Security office, Office of Field Operations (OFO), in
May 2006. Mr. Owen is responsible for all cargo security programs and
policies for CBP, including the Container Security Initiative, the
Customs-Trade Partnership Against Terrorism Program (C-TPAT) office,
all non-intrusive inspection technology and radiation portal monitor
deployments, the national canine enforcement program, the National
Targeting Center-Cargo, and the 100 percent scanning initiative. Mr.
Owen also coordinates U.S. Customs and Border Protection's (CBP)
maritime cargo enforcement policies and activities with the U.S. Coast
Guard and all air cargo efforts with the Transportation Security
Administration.
Mr. Owen previously served as the Director of the C-TPAT program
from January 2005 until May 2006. As Director, he strengthened
management control and increased hiring, allowing for a significant
enhancement in the level of foreign site assessments performed
worldwide under this 9,000 member partnership program.
Before arriving in Washington, D.C., Mr. Owen was the Area Port
Director in New Orleans, where he was directly responsible for all CBP
operations throughout Louisiana, managing 240 officers at seven port
offices. Before being selected as Area Port Director, Mr. Owen spent
eight years in South Florida where he held various trade related
positions within OFO and the Office of Strategic Trade.
Mr. Owen began his career with U.S. Customs Service in 1990 as an
Import Specialist in Cleveland, Ohio.
Mr. Owen is a career member of the Senior Executive Service and was
a senior executive fellow at Harvard University's John F. Kennedy
School of Government.
He is a graduate of John Carroll University in Cleveland, Ohio, and
holds a Master's degree in Public Administration from St. Thomas
University in Miami, Florida.
Chairman Miller. Thank you, Mr. Owen. Dr. Hagan for five
minutes.
STATEMENT OF DR. WILLIAM K. HAGAN, ACTING DEPUTY DIRECTOR,
DOMESTIC NUCLEAR DETECTION OFFICE, DEPARTMENT OF HOMELAND
SECURITY
Dr. Hagan. I am sorry. I am losing my voice. Good afternoon
Chairman Miller, Ranking Member Broun and the distinguished
Members of the Subcommittee. As Acting Deputy Director of the
Domestic Nuclear Detection Office, DNDO, at the Department of
Homeland Security, I am honored to be here with the U.S.
Customs and Border Protection Executive Director for Cargo and
Conveyance Security, Mr. Todd Owen.
I would like to thank the Committee for the opportunity to
provide this update on our Advanced Spectroscopic Portal
Program, or ASP. Since our last hearing, ASP systems have
undergone another round of field validation conducted by CBP.
The purpose of field validation is to produce information about
the operational characteristics of the system and correct any
issues that must be resolved before moving onto the last test
event for the system, the Operational Test and Evaluation
(OT&E).
The July field validation identified several issues. We
worked with CBP to classify the severity of these operational
and technical issues and DNDO has since addressed all the
issues initially identified. Because the systems are
configuration controlled, adjustments were checked and
validated through use of a replay tool and regression testing
to ensure that sensitivity to threats was not decreased below
guidance. Although a few issues remain to be resolved, we are
optimistic that field validation can be restarted in the near
future.
After CBP completes field validation, OT&E will be
conducted by the DHS Science and Technology Directorate's
operational testing authority. The current path to
certification includes field validation and OT&E, as well as
preparation of all the departmental acquisition program
requirements, including presentation of the program before the
DHS Acquisition Review Board, or ARB, for a production
decision. We are committed to following the Department's
Management Directive 102-01. As such, we cannot speculate about
the outcome of the ARB decision. The Secretary's decision
regarding certification will follow the ARB.
Beyond testing of the ASP systems, deployment of our
current, low-rate initial production ASP units is now being
considered. The fiscal year 2010 appropriations bill signed
three weeks ago contains specific language encouraging
incremental deployments of ASP, as did the report from the
National Academy of Sciences, and we are planning discussions
with CBP regarding the best way to deploy these units once OT&E
is completed.
However, one new obstacle has emerged that will greatly
impact the path forward. The United States is facing a severe
helium-3 shortage. Helium-3 is a gas that is used for neutron
detection within both current and next-generation RPMs. Because
this decreased supply affects multiple agencies within the
Federal Government and beyond, the White House has convened an
Interagency Policy Committee, including DHS, to discuss the
issue and possible solutions. This group decided in September
that no more helium-3 will be allocated to RPMs for the time
being. We are currently leading interagency efforts to identify
alternative neutron detectors, and we are working to assess the
impact this will have on our deployment strategy and path
forward. For now, we plan to continue field validation and
operational testing evaluation in exploring our options for the
path forward beyond that.
Additionally, I have also been asked to address the status
of improvements to the currently deployed RPM systems, the PVT
systems. Some have asked if software or algorithm improvements
including improvements to the energy windowing algorithms that
are currently deployed could provide enhanced capabilities for
PVT systems. We are currently funding work at Pacific Northwest
National Laboratory and the Johns Hopkins University Applied
Physics Laboratory to improve energy windowing and conduct
related research to determine what kinds of PVT improvements
may be possible. This research is still ongoing. That being
said, it is important to note that any improvements to PVT
detectors will only improve primary inspection capabilities or
detection. Current performance limitations, and secondary
inspection where CBP officers require the ability to identify
radiological sources, are not addressed by improvements to PVT
technology. Better energy windowing algorithms will not change
this fact.
As a final note, I would like to emphasize that the ASP
program is only one piece of the multi-layered solution we have
termed the global nuclear detection architecture. This strategy
calls for the use of multiple preventive actions at our ports
as well as security measures along all potential pathways
beyond, at and within the Nation's borders.
We will continue to work with our partners within DHS,
other federal departments, State and local agencies and the
Members of this subcommittee and the Congress to keep the
Nation safe from radiological and nuclear terrorism.
This concludes my statement. I thank you for your
attention, and I will do my best to answer any questions you
may have.
[The prepared statement of Dr. Hagan follows:]
Prepared Statement of William K. Hagan
Introduction:
Good afternoon Chairman Miller, Ranking Member Broun, and
distinguished Members of the Subcommittee. As Acting Deputy Director of
the Domestic Nuclear Detection Office (DNDO) at the Department of
Homeland Security (DHS), I would like to thank the Committee for the
opportunity to provide a status update on the Advanced Spectroscopic
Portal (ASP) program. I would also like to thank the Committee for its
support of DNDO's mission to reduce the risk of radiological and
nuclear terrorism to the Nation.
In late June of this year, I provided testimony about our next-
generation radiation portal monitors (RPMs), including where RPMs are
deployed and how DHS' U.S. Customs and Border Protection (CBP) operates
this technology at our ports of entry (POEs). My testimony today will
include a status update and the path forward for the ASP program. I
will also describe operations during field validation and improvements
for poly-vinyl toluene (PVT) systems.
Energy Windowing for PVT Systems:
Before I get into an update on the ASP Program, I have been asked
to address the status of improvements to PVT systems. For some time,
there have been various questions about the possibility of exploring
energy windowing improvements to the currently deployed PVT systems.
Energy windowing is an algorithmic alarm method that can be applied to
plastic scintillator-based RPM systems to improve operational
sensitivity to certain threat sources while reducing the alarm rates
from naturally occurring radioactive material. Some have asked if
software or algorithm improvements could provide enhanced capabilities
for PVT systems that would achieve performance similar to that of the
ASP systems. Algorithms that provide energy windowing for PVT systems
were introduced into the currently deployed systems in 2007, and DNDO
is currently funding work both at the Johns Hopkins University Applied
Physics Laboratory (APL) and at Pacific Northwest National Laboratory
(PNNL) to determine if further gains can be made through additional
energy windowing techniques. We have also asked that, wherever
possible, the labs work cooperatively on these efforts. Funding for the
work at PNNL is approximately $1.6 million of Fiscal Year (FY) 2008
appropriations, and investments in APL for studies related to energy
windowing are approximately $90,000 of FY 2009 appropriations.
Scientists are thus far uncertain whether additional gains in
operational performance can be coaxed from PVT RPMs by using more
energy windowing, because the limits of passive detection with plastic-
based detectors are coupled with a need to keep false alarms at a
minimum. As new techniques are studied and evaluated, it is important
to note that the scintillation properties of PVT detectors are
fundamentally different than the ASP technology and we continue to
evaluate the operational effectiveness of next-generation systems
concurrently.
Current Status of the ASP Program:
DNDO continues to enhance the capability to detect and report
attempts to import, possess, store, develop, or transport nuclear or
radiological material for use against the Nation. Part of our work
involves working with DHS partners to provide the equipment and systems
they need to perform their missions, including scanning cargo for
possible radiological or nuclear threats. After 9/11, considerable
concern was raised about the possibility that terrorists could use the
enormous volume of cargo flowing into the United States as a means to
bring in nuclear material or a nuclear weapon. By far, the largest mode
for incoming cargo is maritime shipping containers, with approximately
11 million containers coming into the country every year. Additionally,
in the Security and Accountability for Every (SAFE) Port Act of 2006,
Congress mandated that all containers coming in through the 22 top
volume ports be scanned for radiation by the end of 2007. Thus,
considerable effort and resources have been devoted to this mode of
transportation to provide comprehensive radiological and nuclear
detection capabilities, particularly at POEs.
CBP currently scans cargo entering at our nation's POEs using PVT-
based radiation portal monitors (RPMs) that can detect radiation, but
cannot distinguish between threat materials and naturally occurring
radioactive materials (NORM), such as kitty litter and ceramic tiles.
Narrowing down alarms to just those for dangerous materials is
especially important for POEs that have a high volume of containers, or
those that see a high rate of NORM.
Building on previous work within CBP and the DHS Science and
Technology Directorate (S&T), DNDO initiated the ASP program in 2006 to
develop next-generation technology that can both detect radiological
material and distinguish between threat and non-threat materials. ASP
systems have shown significantly improved capability to distinguish
radiological threats from non-threats over the hand-held instruments
currently used in secondary screening. Thus, the introduction of ASP
systems is expected to not only reduce the number of unnecessary
referrals and false positives in primary scanning but increase the
probability of detecting dangerous materials in secondary. I want to be
clear here that ASP cannot and will not be acquired and deployed until
an Acquisition Review Board (ARB) has been conducted and the Secretary
of Homeland Security has certified that the technology provides a
significant improvement in operational effectiveness over current
systems. Although we are getting closer to these decision points,
testing and evaluation still remains so that all the data required to
make informed decisions has been accumulated, analyzed, and documented.
To date, two ASP vendors have developed systems that have completed
the following 2008-2009 tests: System Qualification Testing, designed
to demonstrate that ASP units are manufactured in accordance with
processes and controls that meet the specified design requirements; and
Performance Testing at the Nevada Test Site (NTS), designed to evaluate
ASP, PVT, and radioisotope identification devices (RIID) detection and
identification performance against controlled, realistic threat
materials, shielding, and masking scenarios. One vendor has also
completed Integration Testing, designed to determine whether the ASP
systems are capable of operating and interfacing with the other
equipment found in operational settings. This vendor has now begun
field validation testing, designed to exercise ASP systems in a stream
of commerce environment at POEs.
The second vendor has experienced technical and accounting issues
that have caused its testing schedule to lag behind. DNDO is in the
process of determining the best path forward with this vendor. The
remainder of my testimony will focus solely on the first vendor.
Field Validation:
Since our last hearing, CBP conducted an additional round of field
validation at operational POEs. You may recall that CBP operated the
ASP systems at four field validation sites in January and February of
2009 for a period of two weeks. During this time, the systems were run
in tandem with the PVT systems to scan incoming cargo conveyances, and
were able to collect data in an operational port environment with real
flow-of-commerce. During these operations, the ASP systems showed
higher than expected alarm rates for three industrial sources.
Following this first round of field validation, the ASP thresholds
were adjusted to more effectively eliminate alarms on benign sources
and were retested to ensure that the sensitivity for detection of
special nuclear material and threats did not fall below guidance
requirements. Utilizing a computer-based replay tool, we were able to
determine that the adjustments solved the problem without decreasing
the probability that the system will detect and identify threats.
Additionally, the systems were put through regression testing to
determine if the problem had been appropriately addressed.
After the threshold adjustments were fine-tuned for the three
specific isotopes that were problematic in the winter, CBP restarted
field validation in July. This testing identified two additional
issues. First, there was a single mission-critical fault at one field
validation site in which the software database system failed to scan
multiple conveyances, and did not immediately notify the CBP Officer of
the problem. There was no security threat at this port because ASP
systems were operating in parallel with the current generation systems.
A technical representative from the vendor was summoned and the issue
was rectified. The cause of the problem has since been identified and
fixed. To assure that the fix did not create any unintended problems,
the software has been successfully regression tested.
The second technical issue identified during July field validations
was a higher than expected rate of false alarms for certain special
nuclear material (SNM) threats. Since ending this second round of
testing, the ASP thresholds were adjusted to eliminate this problem,
and the systems were retested to ensure that the sensitivity for
detection of special nuclear material and threats was not reduced below
guidance requirements. Use of the replay tool and regression testing
validated that the false alarm rate will be reduced below guidance
requirements, and it is anticipated that another round of tandem
operation field validation will verify the effectiveness of the
settings. In this case, the threshold adjustments resulted in an
acceptable reduction in the sensitivity of the system, while remaining
more sensitive than required by the original specifications.
The fact that the SNM false alarms did not occur at the same rate
in the first round of field validation as seen in the second round is a
good example of the need for robust, multi-stage testing, since it may
have revealed a sensitivity to seasonal changes and cargo contents that
was not expected. Such oddities are typical when a new system
transitions from the lab to the real world, and we will continue to
learn and improve as testing progresses. At present, DNDO and CBP are
in discussions to establish the ground rules for starting the next
period of tandem field validation evaluations. Following the completion
of tandem operations, ASP systems will undergo solo field validation
evaluations.
Path Forward:
Once all the phases of field validation are completed to the
satisfaction of both DNDO and CBP, ASP systems will complete
independent operational testing and evaluation (OT&E) conducted by the
S&T Operational Testing Authority. Test data will be analyzed and
provided to inform the Secretary's certification decision. DNDO is also
engaged with the National Academy of Sciences (NAS) to perform an
additional review of ASP testing and inform the certification process,
as required in the FY 2008, 2009, and 2010 Homeland Security
Appropriations bills.
Additionally, DNDO is working to develop a cost-benefit analysis
that will analyze the cost effectiveness of deploying ASP systems in
several different configurations. This cost-benefit analysis
methodology is still being processed with the available data and
requires additional data from the remaining tests before it can undergo
review within the Department and be finalized.
The current path to certification includes testing as described
above, accompanied by the analysis of results, to ensure that Secretary
Napolitano has sufficient information and all the Departmental
acquisition program requirements are met. ASP systems have been under
review and evaluation for over three years now, but we are focused on
informing a decision to go forward with acquisition and deployment with
the appropriate processes. Such a decision will be made only when it
has been determined that ASP will increase the probability of detecting
dangerous materials while minimizing operational burdens, rather than
based on a pre-determined timeline.
The changes and continued diligence that DNDO exercises in
conjunction with the ASP program will ensure that any eventual
certification and acquisition decisions are consistent with DHS
priorities and made with a documented acquisition management
foundation. The ASP program will also be presented to the DHS ARB for
an MD 102-01 milestone decision for purchase and deployment of ASP.
This is a very well-defined milestone that was developed for all large
programs within the Department of Homeland Security. Items such as
mission needs, operational requirements, analysis of alternatives,
etc., are part of the MD 102-01 process--no production units can be
purchased and deployed without successfully navigating the process. To
be clear, the Secretarial certification requirement is in addition to
the MD 102-01 deployment decision. DNDO intends to present both the
Secretary and the ARB with all the necessary information to make
decisions about the operational effectiveness and potential optimal
deployment of ASP systems.
Conclusion:
DNDO will continue to work with CBP and other partners within and
beyond DHS to improve the Nation's ability to detect radiological and
nuclear threats at our ports and borders. DHS is facing a challenge; we
must balance facilitating the flow of commerce at our ports and borders
with the need to sufficiently scan cargo for radiological or nuclear
threats before it enters our Nation. As both the President and
Secretary have said, the Nation needs more technology to meet its
security challenges and the technologies that DNDO is pursuing, of
which ASP is but one example, are a critical component in addressing
that challenge.
Our efforts to develop and evaluate ASP systems are based on sound,
proven testing and evaluation processes used with proven success across
government and academia. Current test results are capturing the
benefits of ASP systems, and the reviews to date have provided a
valuable assessment of the program and identified a number of key
lessons learned. As we collect more operational data for the ASP
systems, we are better able to determine the optimized settings for
detecting and identifying threats, while facilitating the flow of
legitimate commerce.
I welcome and appreciate the Committee's active engagement with
this program, and look forward to continuing our cooperation as we move
forward together. Chairman Miller, Ranking Member Broun, and Members of
the Subcommittee, I thank you for your attention and will be happy to
answer any questions that you may have.
Biography for William K. Hagan
Dr. William Hagan serves as the Acting Deputy Director for the
Department of Homeland Security's (DHS) Domestic Nuclear Detection
Office (DNDO). Prior to January 2009, Dr. Hagan served as the Assistant
Director of the Transformational Research and Development (R&D) within
DNDO. Dr. Hagan was responsible for long-term R&D, seeking technologies
that can make a significant or dramatic positive impact on the
performance, cost, or operational burden of detection components and
systems.
Before his work with DNDO, he was a Deputy Business Unit Manager at
Science Applications International Corporation (SAIC). Business areas
included nuclear technology (analysis, detection, and applications),
telecommunications, optics, transportation, system integration, and
technology assessments during his thirty years at SAIC.
Dr. Hagan earned a Bachelor of Science in Engineering Physics in
1974, Master of Science in Physics in 1975, and Master of Science in
Nuclear Engineering in 1977 from the University of Illinois at Urbana.
He received his Ph.D. in Physics from the University of California-San
Diego in 1986. He holds three patents. Dr. Hagan was appointed to
Senior Executive Service in 2006.
Discussion
Chairman Miller. Thank you, Dr. Hagan. I now recognize
myself for five minutes.
CBP Procedures After a Primary Alarm and the Effect of False
Positives
Mr. Owen, I hope that CBP's procedure is, if you identify
nuclear materials, the kind of nuclear material that would be
used in a nuclear weapon or for a dirty bomb, that your
procedures would require that you act like your hair was on
fire, that you act with great urgency. Without getting into
anything that is classified in this setting, could you describe
what your procedures are?
Mr. Owen. Yes, sir. Whenever we have any alarm of a PVT on
primary, the cargo or the passenger vehicle is then sent into
secondary because again, the PVTs at this point only detect
radiation, they do not identify what it is.
In the secondary, we use the hand-held Radiation Isotope
Identification Device. If that determination is that we have
special nuclear materials, we then implement what we call SIN
procedures: secure, isolate, notify. We secure the area, we
isolate the conveyance, and then we begin our notifications
back to our Laboratory and Scientific Services folks at our
National Targeting Center for cargo here in Northern Virginia.
The CBP officers would transmit the radiation spectra to the
Laboratory and Scientific Services people to make a
determination as to what exactly what we have there. So as this
is going on, again, we are in secondary. It is a more
controlled arena. Depending on the footprint of the port,
depending on the layout, the time of day, the traffic, will
impact how much of a disruption it will be to some port
operations. But clearly, whenever we have a higher level threat
such as the special nuclear materials, we do act accordingly.
There is some disruption to the port operations until we
resolve that alarm.
Chairman Miller. You said secure, secure and identify,
secure the area. How big an area do you secure?
Mr. Owen. Mostly it is just in the secondary area with what
we have. We will create a perimeter of secure area if you will.
It will not require an automatic shutdown of neighboring
activities or anything like that. As we do the initial
readings, send it back to our Laboratory and Scientific
Services folks--clearly, if the issue has escalated, if it is
of grave concern to us, your exclusion zone would expand and
the level of activity would increase.
Chairman Miller. Okay. Do you have any idea how many hours
ports like New York, New Jersey, Los Angeles, Long Beach, would
be closed or how they would be affected by false positives for
special nuclear materials?
Mr. Owen. I don't have any data as to how many times this
has happened or was shut down, but those are the procedure. And
really again, based on the layout of the port, if it is a large
seaport, you can have just one terminal affected by an alert of
special nuclear materials through one of our detection devices.
If it is a smaller land border crossing, you could
theoretically have a much greater impact on that type of
operation. So there is an impact depending on the different
variables as to how much of an impact, but clearly there is an
impact whenever we have an alert of a special nuclear material.
Chairman Miller. Okay. Well, obviously one of the most
immediate problems with false positives is disruption of
commerce, disruption of ports and borders. But I remember from
college that the first time the fire alarm went off in the
dorm, everyone rolled right out of bed and went outside, but by
the time it happened three or four times because of a prank,
all anyone did was just put a pillow over their head to make
the noise--so they could sleep through the noise. I don't ever
want it to become routine if we have an alert that there is a
special nuclear material. Obviously, if there is a real fire,
the fire alarm would have done no good at all.
Mr. Aloise, the July round of field validation tests, there
were, you said, false positives. The machinery, the device,
ASPs detected materials that were not there at all. Can you
give us the details of how many false positives or was it one
or two machines, was it all the machines? What happened?
Mr. Aloise. Yes, I believe, and CBP can confirm this, there
was four per thousand at the smaller ports and eight per
thousand, eight false positives per thousand conveyances at the
larger ports during the testing in July.
Chairman Miller. And was that across all machines or was
that just one or two machines that seemed not to be quite
right?
Mr. Aloise. I believe it was across all machines, yes.
Chairman Miller. So it was a problem with technology
generally not with a problem with a specific----
Mr. Aloise. With any one machine.
Chairman Miller. Okay. My time has almost expired, so I
will recognize Dr. Broun for five minutes.
Mr. Broun. Thank you, Chairman. I still am going to be very
generous with time. You have been very generous with me, and I
will continue to be generous with you at any point. I want to
make the Chairman----
Chairman Miller. After serving for two years with Dana
Rohrabacher as a Member of the Subcommittee, it is really not a
problem.
Steps Taken to Reduce False Positives and Negatives
Mr. Broun. Thank you, Mr. Chairman. Some of the field
validation tests reveal high levels of false positives for the
special nuclear materials. This is a serious problem because
any identification of any special nuclear materials requires a
robust response from CBP officers. What is DNDO doing to
address this issue? And you might also--I am concerned not only
about false positives but false negatives. And I really haven't
heard anything about that so if you could deal with that, too?
Dr. Hagan. Regarding what we are doing about the false
positives, the field validation showed a large number or
unacceptably high number of calls for SNM (Special Nuclear
Material). We have adjusted the threshold levels for the ASP
and run a replay tool to demonstrate that in fact the number of
false alarms, if these threshold settings had been used instead
of what was used in July, that almost all of those false alarms
would have been eliminated. The number would have dropped from
about, according to my data here, from about 49 to seven. Now,
no system is flawless or perfect. And so it is not likely that
we will ever build a system that will produce no false alarms
in primary. But the levels that we are operating now according
to the replay tool are within the acceptability range.
Regarding false negatives, in the flow of real commerce in
the field validation activity, there is really no way to know
if you have dismissed something that you shouldn't have. Now,
we have testing at another place, the Nevada Test Site, where
we tested that, and I don't recall what the false negative rate
was, but it was quite low.
Mr. Broun. As a physician, I am concerned about, and in
science, as you know, talk about specificity and sensitivity.
Several independent sources have already indicated that ASP
only marginally increases our ability to detect lightly
shielded special nuclear materials. If we manipulate those
thresholds, won't this decrease that margin even more, and if
this is the case, how would that affect any future cost benefit
analysis?
Dr. Hagan. The change in--when the threshold settings were
changed, again using the replay tool, we were able to calculate
what the difference or see what the difference was in the
sensitivity. It is a reduced sensitivity as with any detector.
When you make your system--when you reduce, make changes to
reduce the false alarm rate, the sensitivity is usually
affected. So this is true of this detector as well. But when we
compare the sensitivity of the system before the threshold
settings were made to that after, while it was reduced, the
reduction was rather slight and the performance of the system
is still within the specification and the guidance.
Mr. Broun. In order to verify and validate the threshold
changes made to ASP systems, a replay tool was developed. Has
this tool been validated by an independent party?
Dr. Hagan. Yes, it has.
Mr. Broun. Okay. And who is that party?
Dr. Hagan. The Johns Hopkins APL, Applied Physics Lab.
Mr. Broun. And what was the results of that?
Dr. Hagan. Okay. Well, maybe the best way to convey that is
to comment or to give you some of the results from the replay
tool validation exercise. So when we say a replay tool, we mean
we want a tool that when given the same input data as the real
system, it produces the same results as the real system. And so
the replay tool was given the real data for 11,000 occupancies,
and the results were compared to what actually came out of the
system. The results were identical for all 11,000 except for
eight. Those eight that were found, the reason that the
differences were slightly different, the results were slightly
different, is that the codes were run on two different
computers, and there was a slight round-off error difference
between the two computers. And because of that, there were
eight occupancies that yielded slightly different results.
Other than that, the results were identical.
Mr. Broun. Thank you. Mr. Aloise or Dr. Persons, do you all
have a response to that?
Mr. Aloise. We would say this and then I will let Dr.
Persons answer as well. As you recall in June when we
testified, we said there was a marginal improvement in the
Nevada testing over what the PVTs were, and that marginal
improvement, because of this reducing of the thresholds has now
gotten smaller. So when they do the cost benefit analysis, all
this has to be factored into that cost benefit analysis because
now the ASP seemingly does not have the performance that it
initially did when we testified here earlier this year. So that
is one point.
The other point is we have not looked at the replay tool.
We have not talked to Johns Hopkins, so we can't really comment
on the replay tool. Dr. Persons, would you like to----
Dr. Persons. Well, just to confirm what Mr. Aloise was
saying, we have not been able to look at that, but one of our
key points on this is just the idea of an independent
verification or validation of such a tool to do that. And in
interviews with CBP officials, for example, there was a concern
about perhaps having an institution such as the American
National Standards Institute, or ANSI, do something more
standardized oriented body to do that. There is also of course
when you are dealing with software development issues which are
implied in these systems with this level of complication, then
you would look at of course the software models, the software
standards, how those were constructed, how they execute the
algorithms and perform under test.
Mr. Broun. Dr. Hagan, I would like for you to, if you
would, submit that report for the record for us----
Dr. Hagan. Sure, in the form of a letter.
Mr. Broun.--and a discussion about it. Okay. Good. Thank
you, sir. Mr. Chairman, I yield back.
Chairman Miller. Thank you, Dr. Broun. The Chair joins Dr.
Broun in that request.
Ms. Dahlkemper for five minutes.
Mission Critical Failure
Ms. Dahlkemper. Thank you, Mr. Chairman. Thank you to all
those here testifying today. I want to go to one of the most
basic problems and I guess one of the most alarming was the
mission critical failure when two dozen cargo trucks were
permitted to go through the ASP in order to be screened and
actually were not operating at that time. And supposedly this
has been rectified by the contractor with an indicator light.
Is that correct?
Dr. Hagan. No.
Ms. Dahlkemper. Nothing has been rectified?
Dr. Hagan. It has been rectified but not with an indicator
light. There is a message that gets shown to the operator on
screen that says that the system is in mission critical failure
and should not be operated.
Ms. Dahlkemper. So prior to this there was no way to detect
if there was----
Dr. Hagan. There was a message but it was confusing and not
clear, and so that is what led to the incident. By the way, I
should comment that this only happened one time in one system
in all the testing in validation runs that have been going on
for approximately for a year. So we don't think it is going to
happen again, but we modified the software to properly notify
the operator.
Ms. Dahlkemper. I can buy a $15 smoke detector for my house
and it will tell me when it is beginning to not work, and when
it doesn't work, it will beep so I have to get up in the middle
of the night and take the battery out. But anyway, I just
wanted to make sure that that problem is rectified at this
point.
Dr. Hagan. It has been rectified, it has been regression
tested, and we are confident that the message will be properly
displayed. We are struggling a bit with this one error or issue
that has come up because it has only happened once. So we don't
think it will happen again. But that is the point of field
validation, is to shake out some of the issues and problems
before you move on to the actual operational test.
Ms. Dahlkemper. How may of these are in operation in this
field testing? How many?
Dr. Hagan. Six.
Ms. Dahlkemper. Six? And how many of the PVTs do we have
currently?
Mr. Owen. Just under 1,400 deployed nationwide at the land
borders and seaports and some mail facilities.
Energy Windowing to Improve PVT Performance
Ms. Dahlkemper. Okay. One of the things I wanted to ask,
Mr. Owen, is on the GAO's previous work on the ASPs, they found
that the ASPs provide a marginal improvement of the PVTs to
attack the certain types of radioactive sources under a handful
of very specific scenarios. But the GAO has also recommended
that DNDO invest in ways to increase performance of the
existing radiation monitors known as PVTs by a software
improvement process called energy windowing. Do you believe
that more efforts should be made to investigate the benefits of
the energy windowing?
Mr. Owen. Yes, our position remains that we need to fully
maximize what the detection capabilities of the existing
technology of the PVTs is. So we believe there are gains that
have already been made with energy windowing, and we would rely
on our scientific partners to tell us if we have reached the
edge of that envelope as to how much they can do, either with
the existing algorithms or with advanced algorithms that may be
there. So as the operator and end-user of the equipment, yes,
we would like to etch out as much detection capabilities as we
can from the PVTs, recognizing that even with energy windowing,
there will still not be an identification device, just a
detection device.
Ms. Dahlkemper. Mr. Aloise, how much has been spent on
energy windowing work by DNDO or DHS in the last few years
compared to how much has been spent on developing ASPs?
Mr. Aloise. I don't have that information, but I do know
that we have been told that further research has sort of been
slow-balled, and we would have liked to have seen them a lot
further by now in where the energy windowing is with the PVTs.
Ms. Dahlkemper. Could I get that information? Could that
information be found, Mr. Owen, in terms of how much work has
been done?
Dr. Hagan. Yeah, I can give you the numbers right now.
Ms. Dahlkemper. Okay.
Dr. Hagan. For energy windowing, we have funded a total of
about $1.9 million again to Pacific Northwest National
Laboratory and the Johns Hopkins University Applied Physics
Laboratory.
Ms. Dahlkemper. And how does that compare to how much has
been spent on ASPs?
Dr. Hagan. It is a lot less. I don't know the exact ratios
or anything.
Ms. Dahlkemper. What is the total? You don't know what the
total has been spent?
Dr. Hagan. Well, I think the number that was quoted I
wouldn't argue with. I think it was $230 million over the
last----
Ms. Dahlkemper. It is $1.9 million over the last how many
years?
Dr. Hagan. Which----
Ms. Dahlkemper. Last few years. When did we start? I am new
to Congress so when did we start working on ASPs.
Dr. Hagan. Oh, ASPs? 2004 or 2005. I actually don't know. I
wasn't here then.
Ms. Dahlkemper. So we have spent $230 million in that time
period on that?
Dr. Hagan. Right.
Ms. Dahlkemper. And the energy windowing, you started
working on that?
Dr. Hagan. That was in fiscal year 2009. I think the actual
work started in July of '09 at PNNL.
Ms. Dahlkemper. So since just this past July we spent $1.9
million?
Dr. Hagan. Oh, we haven't spent it. We have obligated it.
Ms. Dahlkemper. Obligated it?
Dr. Hagan. I don't think it has been actually expended yet,
but the work is ongoing.
Ms. Dahlkemper. Okay. And that just started this year, in
July?
Dr. Hagan. Yes, in July, right.
Ms. Dahlkemper. My time is expired. I yield back.
Chairman Miller. Thank you, Ms. Dahlkemper. Mr. Bilbray for
five minutes.
Mr. Bilbray. Thank you, Mr. Chairman. I apologize for being
late, and so if my questions have already been answered, you
can remind me my tardiness was inappropriate.
The CBP Inspection System
Mr. Owen, the issue of inspection, do we have a formal,
multi-tier inspection system and when we talk about the
inspection is there any pre-entrance inspection before they get
to the port facilities?
Mr. Owen. We do not pre-inspect cargo. What we do have is
part of our layered cargo security strategy. We receive
advanced information on all cargo and all modes before it
arrives in the United States, different timelines, whether it
is coming by air, by truck, by rail or maritime. With that
advanced information, we make an initial assessment if the
cargo poses a risk. In our maritime environment through our
container security initiative, we have officers deployed in 32
countries at 58 seaports that can then conduct an inspection on
the highest risk maritime cargo before it is put on a vessel
coming to the United States. We do not have a similar program
in air or in land. So even though we have that advanced
information before the truck or the train or the airplane
arrives in the United States, we can identify the highest risk,
but then it would be looked at at the port of arrival. So we do
not have pre-inspection overseas for cargo.
Mr. Bilbray. Okay. So we do have it for the maritime. Do we
have any structure at all that if there was the ability to
basically modify the arrival system, in other words hold them
offshore until we could board the craft and be able to inspect
it?
Mr. Owen. Yes, sir. Absolutely. We work very closely with
the Coast Guard. If there are any threats on a vessel, we will
keep the vessel out at sea to address those threats or concerns
with the crew or whatever the case may be. Likewise with the
air cargo, we receive the information four hours before the
plane lands, so in most cases if we have a concern that
aircraft can be diverted to have address that threat, yes, sir.
Mr. Bilbray. And I am glad to hear that because, let us
face it, the port facilities tend to be much more centrally
located in high population areas than either the port of
entries for vehicles or with the aircraft, so thats preemptive.
The Helium-3 Shortage and Potential Alternative Materials
The issue of helium-3, as we are fighting to try to make
what we have stretch, any discussion of the crisis and what we
would need to do to crank up the breeder so we can produce more
and make it available for you?
Dr. Hagan. There has been a lot of discussion. This issue
has really risen quickly to the top of a lot of people's lists,
including the White House. They have formed an Interagency
Policy Committee to address this that involves not only DHS but
of course the White House folks, DOE, DOD and so forth. The
problem is rather severe. The demand will probably outstrip the
supply by a factor of ten. There is just not very much. And
there are other uses for this material, for helium-3 that
involve medical uses and so forth that frankly will probably
end up having higher priority, although I can't say that for
sure.
Mr. Bilbray. So the fact that this is a strategic supply
problem that we need to address as a nation?
Dr. Hagan. Yes. I believe that is the case. Now, there
are--so for some applications, like low-temperature physics
research, helium-3 is pretty much the only--you are almost
researching the material itself, so it can't be replaced.
For our applications in neutron detection, there are other
potential alternatives which we are, DNDO, is taking the lead
for the interagency group to explore. We have been funding
research in this area for several years already. So we are
looking at alternative means of detecting neutrons. But because
we don't know what those means are today, there are several, as
I say, potential alternatives. But it can still take a year or
two probably to sort of test those, evaluate them and make sure
that the designs work. So it is a real problem for us. That is
why it has thrown a real glitch into our planning process.
Mr. Bilbray. Are there any other jurisdictions or agencies
around the world that are specifically looking at developing
this capability?
Dr. Hagan. To make helium-3?
Mr. Bilbray. Yeah.
Dr. Hagan. Well, there are places----
Mr. Bilbray. We don't have any others domestically. Is
there anybody domestically looking at it?
Dr. Hagan. There is some thought to perhaps going into
natural gas wells and separating some of the helium from that
and then distilling that, pulling out the helium-3 from that.
There is a possibility of a source in--well, several sources in
other countries. I probably shouldn't say much more than that.
Mr. Bilbray. Okay, Mr. Chairman, I know my time has expired
but in all fairness, there are a whole lot of issues that the
Federal Government gets involved with that states, counties,
cities, international bodies could work on or whatever. Here is
one where this really is in our lap. I just can't perceive a
state or a county or a coalition of states being able to
address this issue, and here is one that I would sure like to
see the Federal Government make some priority decisions and say
this is one that only the Federal Government can do so the
Federal Government has to be more aggressive on this. And I
will remind all of us as we talk about other issues that could
be handled by other agencies that we keep forgetting about the
ones that only we can handle. And maybe we need to focus on
these items that other people can't handle. So I appreciate
your testimony and yield back, Mr. Chairman.
Chairman Miller. Thank you. Mr. Rothman was here and I
think will be back shortly. I will recognize him out of turn if
he returns for a first round rather than for a second round.
In What Circumstances Should ASPs Be Deployed?
The Chair of the Appropriations Subcommittee for Homeland
Security is my colleague from North Carolina, David Price. His
district adjoins mine. In fact, they kind of wrap around each
other like a couple playing Twister. If he should ask me in
what circumstances the new technology of the ASP should be
deployed, what should I tell him? What are the markers for when
it is reliable enough to replace the PVTs? Should it never
replace the PVTs? Should it always be used unless there is some
breakthrough that we haven't seen yet, some improvement in the
technology? Should it always be used for secondary screening
after a PVT has identified something to be concerned about? Or
is this worth it at all? If the PVTs seem to be working, there
is some disruption of having a truck pull over to the side and
going over it by hand or cargo. But is this worth it? Mr.
Aloise.
Mr. Aloise. Well, our position has consistently been that
nothing should be acquired in large scale until all the testing
is complete, all the results are in, we know whether the system
works or not, a cost benefit analysis has been done, and we can
see whether it is worth it given all the other demands we have
in our architecture, all the gaps in that architecture. After
that cost benefit analysis is done, we will have a better
answer to know whether it is worth to pursue with the ASPs
versus the PVTs. Then the question is where? Does it make sense
more in secondary? Does it make sense in other places?
So our position, again, we need to get the testing done,
all of it done, we need to do the cost benefit analysis. And
until then we really don't have enough information to make this
decision on.
Chairman Miller. Well, the original idea was to replace
PVTs with the ASPs. Given what we know, even assuming that the
testing improves, we figure out what all the problems are so
that it doesn't sort of go off without--it doesn't become non-
functional without us noticing it until after a lot of trucks
have gone through or it doesn't produce an amount of false
positives that is going to shut down major ports for a
significant period of time. After we have worked out those
kinks, does it still make sense at all for it to be a primary
detection device instead of the PVTs?
Mr. Aloise. We have our doubts.
Chairman Miller. Mr. Owen.
Mr. Owen. Well, again, from our perspective, assuming it
meets all of the technical and scientific endeavors and the
machines work, then we have a serious concern about the
operation and maintenance costs, the tail that will come with
each one of these devices. As you mentioned in your opening
statement, we are looking at perhaps as much as five times or
more what it costs to currently operate it. So we also are
waiting anxiously to see the cost benefit analysis as well as
the life cycle costs to see if we can even as an agency afford
these devices should they continue to develop and mature and be
proven scientifically to work.
Chairman Miller. Dr. Hagan.
Dr. Hagan. I agree with the GAO on this. Until we get the
cost benefit analysis and a lot of other data, by the way, that
goes into what we call an Acquisition Review Board, it is
really not prudent to try to presuppose the result of that. We
absolutely adhere to the notion that doing the testing and
doing the validation, doing the cost benefit analysis, life
cycle cost estimates, all that has to be done and fed into this
process. We have a very rigorous process within the department
to do that. I wouldn't want to speculate about whether it is
going to make sense in primary or secondary, either or both.
But we at DNDO at least are indifferent to that. The point is
it has to be driven by, you know, real data and analysis.
Chairman Miller. Three years ago, Vayl Oxford, who was then
Director of DNDO, said the priority for the first year is to
get units out immediately. That seems like not the best idea
right now. Does anyone disagree with that? Does anyone agree
still with what Vayl Oxford said then, three years ago? Not a
priority to get this in place immediately? Okay.
When do we expect we will get the cost benefit analysis?
Expectations of a Cost-benefit Analysis on ASPs
Dr. Hagan. The cost benefit analysis is--we are actually
struggling with this right now because of the helium-3 shortage
and the impact of that. The cost benefit analysis of course
depends on what the cost of the system is. We will have to most
likely replace to helium-3 neutron detectors with some
alternative. At this point, we don't know what that alternative
is. As I said before, we have several potential candidates for
that. But until we know what that is, it is going to be
difficult to generate a cost benefit analysis because we won't
know what the cost of the systems is. It may be possible to
work around that, but frankly we don't know that yet. We are
still looking at all the various options. We can try to get to
a CBA, cost benefit analysis, but without that information and
of course without the results from the operational test, we
can't do that.
Chairman Miller. Dr. Persons, actually, most of my
questions I have directed initially to Mr. Aloise and he used
the first person plural, and then you nodded. So I didn't
direct an additional question at you, but you have worked with
many new technologies. Are there best practices? Are there
lessons to be learned for how to develop new practices, how to
develop and deploy new technologies and have those lessons,
have those best practices been applied with respect to this
technology?
Dr. Persons. Yes, sir. I worked with--coming out of an
environment where, if you are familiar with the Defense Advance
Research Projects Agency, it is the idea about rapid technology
development, in the absence of requirements that is normally
for a very high-risk type, high-reward research and development
paradigm. And this particular case, as you mentioned earlier,
there is sort of some relatively mature technologies, it was
just relatively immature, the concept of operations it was
going to be asked to be placed under, in other words, a port
operation with high volume and so on.
And so I think some of the lessons learned from this is
what you put your finger on just a moment ago, was what I would
call the mistake of scheduling invention, in other words,
setting artificial deadlines. We want to get these out and then
we will worry about sort of the technology development later
on. GAO best practices for many years on this topic have shown
that that is a mistake to do that. You want to risk mitigate
the way you develop your programs by investing and particularly
empowering your science and technology folks to do that and do
that outside a procurement operation. Even though there needs
to be close ties there, it still needs to be this idea that the
science and tech folks can do the technology and development,
and the procurement folks can worry about how to do the
procurement
And so keeping that sort of balance of power, doing that,
the requirements definitions is hugely important. I think in
this case, the customer ultimately was CBP and what were they
going to be comfortable with in an operational paradigm? Was
the con-op really going to be an ASP and primary alone in solo
mode, or is there going to be a reduction in their manpower
requirements and so on. or was it an SNM detection paradigm
alone? Those are some issues that drive requirements through
CBP out to the end of the program.
And then thirdly, of course, the independent testing and
evaluation, we have had a lot of discussion on that. Again, the
key thing is there is the independence of that and how that is
done.
Chairman Miller. Okay.
Dr. Persons. And so we have been pleased to see DNDO
talking about interacting with DHS's OT&E office and so on----
Chairman Miller. Okay.
Dr. Persons.--at this phase.
Chairman Miller. One of the lessons learned is that, at
some point, you look for an off-ramp that if a technology
seemed like a good idea but just didn't work, that you
determine is not a matter of just working out kinks, but this
was not as good idea as it seemed. Are there measures for when
you should look for an off-ramp?
Dr. Persons. Right. Well, GAO best practices recommend the
use of readiness levels, technology readiness levels and so on.
I think one of the things here that I think could perhaps be
used is the idea of software readiness levels and so on because
software an algorithms are such a critical part. I think that
has manifested itself in the recent conclusion of the field
validation tests and so on.
So you have these metrics for doing that, and I think
organizationally as I believe DHS likely has is some idea about
what is an acceptable risk of this technology or this system
for deployment. And there are ways to measure that and just
give your managers informed decisions so that they can
determine the risk of something, whether or not it needs to
move forward and--mode.
Chairman Miller. Well that is not quite a Dana Rohrabacher
five minutes, but I am over the traditional five minutes. And
with that I recognize Dr. Broun for five minutes or more.
Metrics and Timelines for Making Decisions About ASPs
Mr. Broun. Thank you, Chairman, and I am going to dovetail
back into what the Chairman was asking. I would like to know
the metrics. It seems to me that we keep fooling around here
and have excuses of not implementing a technology, and not only
a valid technology that is not only useful but may be
preferable over our current technology, but there are other
things that CBP is already doing. And going back to the
Chairman, you are talking about an off-ramp, it seems to me we
are getting to the point of fishing or cutting bait or one of
the two. I mean, we have got to do something. If you all would
give us some metrics about when we can expect the decision-
making process, if you can help us to understand at what point
do we need to pull the plug on the ASP program or implement
something else or what have you with all of you, and I would
like to--because I don't have a good feeling about this,
frankly. I don't know about other Members but I would
appreciate it if you all would give us some metrics or if you
have them, please tell me right now. Mr. Aloise?
Mr. Aloise. Well, you know, DHS came up with their
``significant increase in operational effectiveness'' as their
metric for going forward with this new equipment, and we from
the beginning thought it was a pretty low bar. I mean, I will
go back to what I said. You know, even if this ASP works, it is
a marginal increase. Now, with the adjustments and thresholds,
that margin has gotten more narrow. So what GAO's role is here
to evaluate what DHS is doing and provide this information to
the Congress, of course, and we have made recommendations all
along to make improvements in the testing, in the reporting,
but the bottom line, I have to go back to what I have said,
once the testing is done, we need to do that cost benefit
analysis and decide whether or not it is worth to go ahead with
this.
Now, it has taken longer. You know, you mentioned in 2007
they were ready to deploy, but the Congress who has oversight
and getting GAO involved has got us to where we are today. Now,
where do we go from here? I assume a third round of field
validation testing. The first round, they made adjustments. The
second round, we may be seeing the results of those
adjustments, and we don't know what is going to happen in the
third round. The question is, is this equipment worth it? And
that is a decision that DHS has to make. And I mean, we have
got a lot of information right now, and somebody can make a
decision or they can put this back into what we belong--where
it should have been for a long while, and that is R&D, in
development, and see where--what is the most you can get out of
this promising piece of equipment. But when you know, we
started down the path or just started down the path of
acquisition, deployment and spiral development, they have
called it, and it has got us to where we are today, where the
equipment has a lot of problems, it doesn't work very well
right now. And to take a risk and replace the equipment we have
got now on the borders that we know works with equipment we
don't know works, you are always going to have GAO saying,
whoa, step back. We need to look at this again.
So I don't know. It is kind of a long-winded answer to your
question, but we need more information before we could come to
a conclusion, and that has to be the cost benefit analysis.
Mr. Broun. Another question with cost benefit analysis, how
can you do one without first knowing whether the ASP was used
as a primary screening tool or as a secondary screening tool? I
think that needs to be determined, too, just doing that cost
benefit analysis because otherwise you are just shooting in the
dark.
Frankly, I see the private sector developing screening
methods without the taxpayer being on the hook for them, and we
have this, we have spent a bunch of money and a lot of it is
sitting in a warehouse and not being deployed and we are
testing. Mr. Chairman, I think it might be time to have that
offer out from everything that I can see, but I don't know.
And I would just like to ask Mr. Owen, would you possibly
agree with me that it is time to maybe look at other
technologies?
Mr. Owen. Well, again, as you mentioned, the PVTs do meet
our needs right now. They are effective. We can handle the
level of alarms that we have got there. Hopefully, with any
advances through the energy windowing, they give us a little
bit greater technology detection capabilities than we have
right now. Whether we can say we should pull the plug or not, I
think again from the CBP standpoint, the jury is still out. We
need that cost benefit analysis. We need to see how much better
these in fact perform over the technology that we have before
we could even make a decision as to should they just be in
secondary or should they be primary and secondary.
But I would like to just close by saying again, through all
of our testing, we have kept the PVTs up and running. So we
have always had that safety net, that effective equipment that
we know has worked so that those 27 trucks did not get through
when the machine went down.
Mr. Broun. Thank you, Mr. Chairman. I yield back.
Chairman Miller. Thank you.
Dr. Hagan. Could I respond to any of this?
Chairman Miller. Yes. Dr. Hagan.
Dr. Hagan. Thank you. First of all, I wanted to comment
that the cost benefit analysis addresses both primary and
secondary screening, so we are not pre-determining that
outcome. Second, the ASP, what we have been talking about here
primarily is about ASP and PVT in primary, and you know, the
debate will rage on. As I said before, I agree that the CBA is
the way to determine that, you know, whether that is effective
or not and worth the money.
The ASP operating in secondary is another matter. The
limitations of the current system--CBP officers are using the
current equipment as well as they possibly can. There is no
question about that. But there are limitations to those systems
in both primary and secondary, and let me just focus on
secondary. In secondary, there are limitations which
improvements to PVT cannot address. Only ASP can do that. I
can't talk about the limitations or the changes or improvements
of ASP compared to hand-helds in an open session. But the
differences are dramatic. And so I think this talk of an off-
ramp would be very premature and perhaps a closed session would
be the way to deal with that. But I would urge you to consider
the fact that ASP has considerable application and benefit in
secondary.
Chairman Miller. It is not really my turn, but I do hold
the gavel. Mr. Aloise, Dr. Persons, do you all have security
clearances? Do you have that top secret security, the top? Are
you familiar with all of the analyses of the comparative
benefits of PVT and ASP and primary and secondary versus hand-
held? Are you familiar with all of that as part of the work you
have done?
Mr. Aloise. We are familiar with everything we have been
shown, yes.
Chairman Miller. Okay. Have you asked for everything?
Mr. Aloise. Yes, we have.
Chairman Miller. If there is something you haven't been
shown, you have asked for it and not been shown it?
Mr. Aloise. Right, and we don't know that that is the case
but----
Chairman Miller. Okay. Right. And you do understand that
GAO is part of Congress and when they ask, we are asking and we
need for you to give them anything they have asked for.
Mr. Aloise. Absolutely.
Chairman Miller. Okay. Assuming that you have seen
everything, is there anything--if you know all the tippy-top
secret stuff, is it really different? Is analysis a lot
different in the way that Dr. Hagan suggests?
Mr. Aloise. I would say this. I think you pretty much know
the story as I have testified and GAO has reported on.
Now, in regards to secondary, just in discussions among
ourselves, the ASP is bigger in radiation detection size
matters. So it is a bigger instrument than the hand-held. So
just for that--you have longer dwell times in secondary. So
there might be uses. There might be a possibility where it make
sense to put it into secondary, but we still don't have all
that information.
Chairman Miller. Okay. What is a dwell time? I am sorry.
Mr. Aloise. You know, the container can sit a little bit
longer in a secondary inspection, so you have a better account.
Chairman Miller. I see. You are not trying to move things
through.
Mr. Aloise. You are not driving through.
Chairman Miller. All right. I now recognize Ms. Dahlkemper.
Ms. Dahlkemper. Staying on this line of questioning, if it
was used in the secondary manner, how many units would we need?
Mr. Owen. Yeah, I don't have the date available as to how
many secondary locations we have in all of the ports of entry,
but it is far fewer than what we have in primary obviously. But
even with that, the volume through secondary is relatively low.
I mean, we have about a one percent alarm rate on the land
border and about 2.5, 2.6 percent in seaports. So we would have
to again look at the life cycle costs, the cost benefits as to
how much better does it offset what it is costing us to buy
this equipment versus the time savings or the fewer officers to
address the secondaries.
Ms. Dahlkemper. Would anyone else like to address that? I
don't know if anyone else can give you a better--more answer on
that?
Dr. Hagan. I can give you sort of a range. Like Mr. Owen, I
don't have the numbers in front of me, but I believe the number
of secondary sites is on the order of 400 to 500. Not clear
that one would put an ASP at every one of those by any means.
But as Mr. Owen says, it is far fewer than the number of
primary screening locations. So the----
Ms. Dahlkemper. I am certain there would be certain points
that would be much more crucial. Maybe some of the borders--I
mean, I live near the Canadian border. Some of those ports of
entry are much more crucial than others.
Dr. Hagan. Right.
Ms. Dahlkemper. I did want to ask, though, I did want to
kind of go back to this helium-3 issue because I think this is
a crucial issue even beyond this hearing today. Who is in
charge of the interagency task force on this?
Dr. Hagan. It is the White House. I can't remember the name
of the organization. I can give you the name of the person.
That is about all I could do.
Ms. Dahlkemper. Who is that?
Dr. Hagan. Her name is Julie Bentz, B-e-n-t-z.
Ms. Dahlkemper. Okay. My understanding is the ASP need
three times the helium-3 as the PVTs?
Dr. Hagan. The way it was designed was to have even more
sensitivity for neutrons than the PVT systems did. So we put
helium-3 tubes in the ASP design. But if we wanted to maintain
the same level of sensitivity as the PVT for neutrons, it would
require the same amount of helium-3. There is nothing that
links the neutron detection technology to whether it is an ASP
or a PVT.
Ms. Dahlkemper. As it is currently designed?
Dr. Hagan. It was designed to have more sensitivity to
neutrons than the PVT system because more sensitivity is
better.
Ms. Dahlkemper. So how much more helium-3 is being utilized
in an ASP than a PVT?
Dr. Hagan. About three times more.
Ms. Dahlkemper. About three times?
Dr. Hagan. What I am saying though is if we want to go back
to the same level of sensitivity as we currently have with the
PVT, then the amount of helium-3 that would be required for an
ASP unit would be the same as for a PVT unit.
Ms. Dahlkemper. Well, then why would we have to change the
unit if the sensitivity wasn't increased?
Dr. Hagan. Because this is for neutron detection only.
Ms. Dahlkemper. Okay.
Dr. Hagan. There is another part of the system that detects
gamma rays, and that is the part that is different. So maybe I
should say that the difference between an ASP and a PVT is the
gamma ray detection part. In one case, with the PVT, it is a
big block of plastic, and in the case of an ASP it is chunks of
sodium iodide. But in both cases, the neutron detection part is
done with tubes of helium-3 gas. And that part is essentially
the same with both systems.
Ms. Dahlkemper. So you could redesign it so it would use
the same amount of helium-3 and not really jeopardize the
security?
Dr. Hagan. Right. You wouldn't even have to redesign it.
You could just take the tubes out, cut the number of tubes down
by pulling them out.
Ms. Dahlkemper. Okay.
Dr. Hagan. It would be simple to do that.
Ms. Dahlkemper. Okay. That answers my question. Thank you.
I yield back.
Chairman Miller. Thank you. I thought there was a Steve
Rothman sighting, but that was perhaps a false positive. I
think we now had sufficient questions for now at least. We may
have an additional hearing. On this, you know, we do have--all
Members have top-secret clearance. We have in this building and
in the Cannon Building we have--leaded walls and Members go in
and have to drop off their Blackberries and their cell phones
before they go in, and they bring down a cone of silence and we
hear all the top-secret stuff. And if we need to do that, we
will, but if we go through that exercise and don't hear
anything, it is going to be pretty irritating. So let us not do
that right away, but before we close the hearing out, Dr.
Hagan, can you tell us exactly where you think we will be on
April 1? Will we have completed successful field tests, will
there be a cost benefit analysis, where will we be on April 1?
Three years ago the then-head of your unit said let us get this
stuff deployed right now. Where will we be in April?
Dr. Hagan. Because of the helium-3 issue, I cannot give you
an answer about the cost benefit analysis. We know how
important it is, and we may be able to do something, but
without knowing how much the helium-3, or the neutron detection
part of the system will cost, it makes that difficult. But we
are looking at ways to try to get around that. Going back to
field validation--well, maybe I should let Mr. Owen respond
about the schedule. But by April, we will be, I suspect, in
some phase of field validation or operational test.
Mr. Owen. In terms of the field validation, we have those
two critical areas that we have been discussing. We feel those
have been addressed by DNDO. There are a few other minor issues
that we are working through. Our objective is to restart field
validation in the first quarter of calendar year 2010. So
within the next month, we should restart field validation for a
third round and go forward from there.
Chairman Miller. Okay. Before we bring this hearing to a
close, I want to thank our witnesses for testifying, and I
guess it is a little early to say I look forward to seeing you
the next time. Under the rules of the Committee, the record
will remain open for two weeks for additional statements from
the Members and for any answers to any follow-up questions the
Committee may have for the witnesses. The witnesses are excused
and the hearing is now adjourned.
[Whereupon, at 2:27 p.m., the Subcommittee was adjourned.]
Appendix:
----------
Answers to Post-Hearing Questions
[SKIP PAGES = 000]
Responses by William K. Hagan, Acting Deputy Director, Domestic
Nuclear Detection Office, Department of Homeland
Security; and Todd C. Owen, Acting Deputy Assistant
Commissioner, Office of Field Operations, U.S. Customs
and Border Protection, Department of Homeland Security
Questions submitted by Representative Paul C. Broun
Questions for Dr. Hagan, and Mr. Owen:
Q1. LDo you believe computer modeling through replay tools,
injection studies, or regression tools are enough for
certification, or do you believe real-world field testing is
required?
A1. Certification requires that the ASP system be both
effective and suitable for the operation it is designed to
perform and that it meets the significant increase in
operational effectiveness (SIOE) criteria. Each of the elements
listed in the question play a part.
Real-world testing is vital. During the ASP program the
systems have undergone or will undergo real-world and near-real
world testing during controlled performance tests at the Nevada
Test Site (NTS, real-world targets of interest), Integration
testing (real-world interface and interoperability testing),
and both Field Validation activities and Operational Testing
(real-world stream of commerce and Customs Officer
involvement), as well as any initial deployments. Real-world
events allow the system to be exposed to a variety of traffic,
cargo and radiological conditions that cannot be duplicated in
any other way. Because real-world data is so important, it is
vital that the data relating to traffic, cargo and radiological
conditions be collected so that the impact of certain system
changes can be evaluated without having to re-experience all
real-world environments. For ASP systems all of this data is
collected in what is known as ICD-1 files.
Replay tools provide the developer with an opportunity to
re-create the real-world sensor data from previous data
collection events (ICD-1 files), and to explore how changes in
systems parameters would have impacted the results from those
events. Replay tools are not computer modeling. These types of
tools are used extensively for many types of systems,
especially when data collection/real-world testing is
expensive, dangerous, or time consuming. In this case, Replay
tools can determine the performance of the detector system for
a previous data collection event under various system parameter
changes, such as threshold settings. In another example, Replay
tools are used to determine if the system can continue to
perform if one or more of its elements are faulty. Since ICD-1
data files are organized by detector type, the replay tools can
be used on modified ICD-1 files that have had this data
removed. These sorts of regression analysis are extremely
useful and can be used to provide information toward a
certification decision. Injection studies are a way of
extending the types and amounts of threat sources into stream
of commerce data (i.e., ICD-1 files) so that system response to
a broad spectrum of sources can be estimated. Injection studies
are a type of computer modeling, and should therefore be
checked against real, controlled experiments to validate that
the injection tools closely match the output of real systems
given the same input. Injection tools are useful to point
toward potential improvements in systems performance, and
should be used during continuous improvement planning--For
certification, the best approach to confidently understanding
system performance for threat sources is to perform
measurements that span the threat space as much as practicably
possible.
Q2. LHas other ASP equipment been developed outside of this
process by the private sector?
A2. DNDO is aware of other vendors, outside the ASP program,
that have continued to develop spectroscopic portals.
Q3. LHow much money has DNDO spent on ASP R and D to date?
A3. $181M.
Q4. LWhat is the proper role for DNDO in this process?
Should DNDO be simply testing moderate changes to
Commercial Off The Shelf Technology, or should it be focusing
on next generation technology?
A4. The Congressional and Executive guidance establishing DNDO,
specifically the SAFE Port Act of 2006, the National Security
Presidential Directive 43, and Homeland Security Presidential
Directive 14, charge DNDO with maturing ``transformational
technologies'' as well as to ``develop . . . and enhance
national nuclear and radiological capabilities.''
Q5. LWho should be testing COTS, CBP or DNDO?
A5. Three entities within DHS cooperate in the testing of COTS
(and GOTS) systems: DNDO, the end-user (in the case of ASP this
would be CBP), and S&T. CBP must ensure that the testing of
systems incorporates realistic operational scenario, DNDO must
ensure that the testing properly addresses the detection and
identification requirements of the system within the GNDA and
also must ensure that any developmental testing has been
completed prior to Operational Testing. S&T is the DHS
Operational Testing Authority and as such must assess the
suitability and effectiveness of the final system configuration
in real operational environments.
Q6. LIn addressing previous problems indentified in Field
Validation Testing, has the agency changed the underlying
algorithms that were tested at the Nevada Test Site (NTS)?
A6. No.
Q7. LWill simply changing isotope thresholds be enough to fix
these problems, or will algorithms need to be changed?
A7. As demonstrated by replay and regression testing, we are
convinced that threshold adjustments, while reducing
sensitivity, will reduce the false alarm rate while maintaining
adherence to the ASP specification.
Q8. LIf algorithms are changed, will the equipment have to go
back to the NTS?
A8. If the Department determines that changes, if they are to
occur, warrant a return, to NTS, then such testing will be
conducted.
Q9. LIf so, how much will this cost and how long will it take?
Is the Department only changing threshold levels (and not
changing algorithms) simply because it does not want to go back
to NTS?
A9. The cost and schedule of such tests, if needed, would be
dictated by the type of testing required. ASP thresholds were
changed because the algorithm is performing well, and because
no reasons to make changes to the algorithm have been
identified. The thresholds were changed because real-world
field validation showed that they were not previously set in
the best configuration. The Department has not intentionally
avoided making any necessary changes to the algorithm to avoid
returning to NTS.
Q10. LAt our previous hearing, we discussed the possibility of
deploying the Low Rate Production (LRP) ASPs that are currently
in storage in tandem with PVTs in order to get more data.
Does the Department currently have the authority to do
this?
A10. The FY 2010 Appropriations Bill, and accompanying
Conference Report, addresses low rate initial production ASP
systems. Specifically, H.Rept. 111-298 states that, ``the
conferees encourage DNDO to undertake deployment of low rate
initial production ASP systems, as appropriate, and use data
from such deployments to inform future portal monitor
decisions.''
Q11. LHas the Department made any progress on this front?
A11. The Department is working through the logistics of
completing operational testing to get to Secretarial
certification before making an LRIP deployment decision.
Q12. LHow much would this cost and how long would it take?
A12. There are eight (8) LRIP systems available for deployment.
Approximately $3 million would be needed for installing and
maintaining the systems for 1 year. Installation of the 8
systems would take place over a six (6) month window. Data
collection and data analysis would take approximately 6-12
months and would cost $100K-$200K per system. Total for all
activities would be less than $5 million.
Q13. LCBP issued a RFI for ASPs several years ago only to have
it rescinded by UNDO after the office was formed.
In retrospect, do you believe the Request For Information
(RFI) should have gone forward, with an eventual Request for
Proposal (RFP) after that?
A13. Given the urgency to address perceived limitations in
fielded systems at the time, DHS still believes that those
decisions were appropriate. It is difficult to gauge what may
have come from an RFI/RFP process from 2003. CBP believes that
the success of any research and development program, leading to
possible deployments of next generation equipment, must begin
with requirements defined by the end user/operator, and robust
field evaluations/development.
Q14. LDo you believe the requirements developed by DNDO will
create a system that can stand up to a cost-benefit analysis?
A14. An initial cost-benefit analysis, based on performance
assumptions and systems requirements developed by DHS,
warranted a decision to proceed with the program. Performance
testing has shown potential benefits for ASP systems over
current systems, particularly in secondary screening. However,
the Department will not make any predictions as to the results
of a final cost-benefit analysis, based on performance data,
until such all testing and analysis is complete
Q15. LShould the cost-benefit analysis consider upgrades in
current Radioisotope Identification Device (RIID) technology
and Polyvinyl Toluene (PVT) ``energy windowing'' as options?
A15. No. The CBA will assess the cost and benefits of the ASP
vs. the current system (which currently utilizes energy
windowing). Should the ``current system'' change before the CBA
is complete, the CBA would be adjusted accordingly.
Q16. LHow is the Department responding to the recommendations
of GAO and the National Academy of Sciences?
A16. DNDO has appropriately incorporated or responded to the
recommendations of both organizations as follows.
a. Responding to the GAO
Recommendation regarding significant improvement to
operational effectiveness (SIOE): The GAO Recommended DNDO
assess whether ASPs meet SIOE criteria based on valid
comparison with PVT's full performance potential, including use
of energy windowing to increase sensitivity to threats. DNDO
has structured ASP testing to validate that ASP meets SIOE
criteria. The SIOE criteria require that the comparison be made
against the ``current system.'' DNDO will and should follow
this path. We stress that the current PVT based system contains
energy windowing algorithms that were updated in 2007. DNDO has
also funded additional research at both Pacific Northwest
National Laboratory and Johns Hopkins University Applied
Physics Laboratory to further investigate the potential to
improve performance of PVT systems based upon algorithm
enhancements, including new approaches to energy windowing. On
a side note, energy windowing techniques may also be applied to
spectroscopic systems like the ASP, which may enhance
performance there as well.
Recommendation to revise ASP testing and Certification
schedules: The GAO recommended altering schedules to allow
sufficient time for review and analysis of results from final
testing including injection studies. The ASP program schedule
incorporates time following completion of Field Validation and
Operational Testing of
ASP, before presenting ASP to the DHS Acquisition Review
Board for approval and then to the DHS Secretary for
certification. This time interval prior to a decision allows
all stake holders to thoroughly review test results and for
proper consideration of the ASP cost-benefit analysis.
b. Responding to the NAS
Recommended approach for testing and evaluation: The NAS
recommended an ``iterative approach with modeling and physical
testing complimenting each other.'' In this approach, ``theory
and models of threat objects, radiation transport, and detector
response'' should be used to ``simulate performance and predict
outcome,'' using ``physical experiments to validate the
predictions and allow critique of the models.'' We believe that
the ASP program, through development activities carried out by
vendors, to test programs executed by the government, to replay
tools and injection tools used for analysis, has followed the
spirit of this recommendation. The performance of the ASP
systems in controlled performance tests has mirrored
predictions extremely well. The issue that remains for ASP is
one of threshold settings against real-world cargo which must
be determined through empirical evidence.
Recommended Approach of the Procurement Process: The NAS
recommended that DNDO employ an ``iterative'' approach for the
procurement process. Rather than a ``onetime certification
decision in the near future,'' the NAS recommended a continuous
process of improvement and adaptation of the system, beginning
with the deployment of LRIP systems. We agree with the NAS in
principle and are evaluating this approach. This approach is
consistent with the ASP Acquisition Plan and will be reflected
in actual deployments when they begin.
Recommended Approach for Cost-Benefit Analysis: The NAS
Committee recommended that DHS not proceed with further
procurement until ``ASP is shown to be a favored option in the
cost-benefit analysis.'' This recommendation is incorporated
into the governance structure of the ASP program. Additionally
the NAS made recommendations about the nature of the CBA
itself, stating that it ``needs to include three key elements:
(1) a clear statement of the objectives of the screening
program: (2) an assessment of meaningful alternatives to
deploying ASPs; and (3) a comprehensive, credible and
transparent analysis of in-scope benefits and costs.'' We
continue to agree with the NAS that a complete CBA is necessary
before any decisions are made and the elements suggested by NAS
are factored into the CBA.
Q17. LSetting aside the increased costs of ASPs over the
current PVT/RIDD combination, how much more money will ASP
Operations and Maintenance (0 and M) cost CBP?
A17. Current PVT RPM units cost approximately $12,000 per year
to operate and maintain. While CBP has not seen the final ASP
maintenance cost estimate, this figure is expected to range
from $65-100k each year per unit.
Q18. LGiven dwindling budget estimates, how will these
increased O&M costs affect overall security?
A18. If no increase in O&M funds accompanies the deployment of
ASPs, CBP would have to secure funding from other programs or
staff positions.
Q19. LWill CBP even be able to operate ASPs in the current
budget environment if they are certified?
A19. CBP anticipates that the appropriate O&M funding would be
included should the ASPs be deployed.
Q20. LIf the ASP systems are certified and deployed, will the
Department be able to find cost savings?
A20. The deployment of ASPs will result in an overall increase
in life cycle costs.
Q21. LDoes the ASP system require less CBP personnel to
operate?
A21. There is no appreciable savings in required personnel over
the entire deployed architecture. In primary scanning
operations, for example, land border crossings would not see a
staff reduction. In secondary, the ASPs may require less
physical labor and less time to operate; however, a CBP Officer
will still be required to operate the ASPs. It should be noted
that the ASP will not eliminate the need for RIIDs which will
be used for more detailed inspections and alarm adjudication.
Only very limited reductions likewise would be seen in
seaport deployments.
Q22. LDoes the ASP system allow for greater flow of commerce,
or would other factors at ports and border crossings still
impede commerce?
A22. No appreciable gains to the flow of commerce would be
realized over the entire deployed architecture. At land border
crossings, impediments to the flow of commerce are due to non-
radiation scanning processes (i.e.--passport control and
antismuggling inspections). At seaports, the flow of commerce
is mostly determined by individual terminal procedures.
Answers to Post-Hearing Questions
Responses by Gene Aloise, Director, Natural Resources and Environment,
U.S. Government Accountability Office (GAO); Todd C. Owen,
Acting Deputy Assistant Commissioner, Office of Field
Operations, U.S. Customs and Border Protection, Department of
Homeland Security; and William K. Hagan, Acting Deputy
Director, Domestic Nuclear Detection Office, Department of
Homeland Security
Questions submitted by Representative Paul C. Broun
Questions for Mr. Aloise, Dr. Hagan, and Mr. Owen:
Q1. Do you believe computer modeling through replay tools, injection
studies, or regression tools is enough for certification, or do you
believe real world field testing is required?
A1. The current campaign of ASP testing, which started in early 2008,
involved several different types of testing each for its own purpose.
The August 2008 testing at the Nevada Test Site (NTS) was designed to
test sensitivity of the ASPs (and PVTs) against actual physical sources
of special nuclear materials (e.g., uranium and plutonium). For the
current campaign of testing, the NTS phase of testing was particularly
important given that the ASPs were expected to have an improved
capability of detecting special nuclear materials, and, for security
reasons, NTS was the only place where DNDO could fully physically test
the ASPs capability of detecting these very dangerous materials.
To this point, the field validation tests have shown an
unacceptable level of false alarms for the presence of certain nuclear
materials. To resolve this, DNDO officials told us that they have made
changes to the thresholds of the ASP, but not to the underlying
algorithms (or software) that allows these machines to detect
radiation. As we testified, this change makes the ASP less capable of
detecting certain nuclear materials, raising questions about whether
the ASPs represent a significant improvement in operational
effectiveness over the PVTs in detecting certain nuclear materials.
While DNDO officials told us that they were not making changes to
the ASP algorithms, the report of the Field Validation Advisory Panel
stated that ``correcting algorithm issues will change ASP referral
rate'' suggesting that there will be future changes to the ASP
algorithms. So, there is some confusion about exactly what changes will
need to be made in order to successfully complete field validation
testing. This represents a subtle, but important distinction--we regard
``algorithm corrections'' as algorithm changes and not simple threshold
adjustment.
In our view, prior to any certification decision to move forward
with procurement of ASPs, there should be additional testing at NTS
against actual physical sources of special nuclear materials in order
to (1) understand exactly how the modified ASPs performance compares
against the performance of PVTs in detecting certain nuclear materials,
and (2) assure that there are not any unintended consequences in
detection capabilities resulting from the modifications made to the
ASPs since the August 2008 testing at NTS.\1\ This is to say, at this
stage in the development of the ASP, major systems acquisition best
practices require performance testing with actual special nuclear
material because of the expected corrections to the software (beyond
simple threshold adjustment).
---------------------------------------------------------------------------
\1\ Without tests at NTS we will not have True Positive or False
Negative data with the new software.
Q2a. Has other ASP equipment been developed outside of this process by
---------------------------------------------------------------------------
the private sector?
A2a. Based on the number of proposals submitted to DNDO's initial RFP
for the ASP, it is quite possible that spectroscopic radiation
detection systems have been developed outside of DNDO sponsorship. The
issue here is determining the extent to which those COTS vendors have
focused development on spectroscopic radiation detection systems that
could operate in a high-impact port environment (large format and high
traffic throughput) in support of CBP's mission. Moreover, only DNDO's
two developers have been afforded the ability to test their systems
with actual special nuclear material at the NTS.
Q2b. How much money has DNDO spent on ASP R&D to date?
A2b. DNDO would be the best source for this information.
Q2c. What is the proper role for DNDO in this process? Should DNDO be
simply testing moderate changes to Commercial off the Shelf Technology
or should it be focusing on next generation technology?
A2c. We believe the Secretary of DHS should determine whether DNDO
should be a technology development or a major systems acquisition
organization. If she chooses to keep both functions within the
organization, then the Secretary should establish a strong separation
and independent operational construct between the technology
development and major systems acquisition missions. Ultimately, DNDO's
goal should be toward addressing the mission needs of the end-user (CBP
in the case of the ASP program). Organizational best practices hold
that DNDO should regularly assess the extent to which these needs may
be met with existing commercial-off-the-shelf technologies, as well as
the nature and extent of development necessary for emerging or next-
generation technologies. In the case of ASPs, the mission need has
become less clear based upon the lessening of the criteria for a
``significant increase in operational effectiveness.'' This is to say,
the primary mission need DNDO put forward was to better detect certain
special nuclear materials, and the expectation was established that
ASPs would vastly outperform PVTs in this area as manifested by a
dramatic reduction of referrals to secondary screening due to NORM.
However, recent testing has raised questions about both the ASPs
capabilities to reliably and efficiently detect these materials as well
as the exact significance of the increase in operational effectiveness
of these next generation systems that are 2.5 times more expensive per
unit than the current generation of technology. Moreover, it is not
clear that the PVT technology has reached its limit of performance
based upon energy windowing algorithms or other advances in materials
science for nuclear detection.
Q2d. Who should be testing COTS, DNDO, or CBP?
A2d. GAO sees three parts of a testing regime for these systems: (1)
software development testing, (2) physical performance testing at NTS
(supported by follow-up modeling and simulation work), and (3) field
validation testing. In our view, the first is managed by DNDO and its
contractors with independent testing and evaluation done by a federally
funded research and development center (FFRDC), university affiliated
research center (UARC), or the National Institute of Standards and
Technology (NIST). The second is also managed by DNDO in partnership
with the National Nuclear Security Administration's staff at NTS as
well as the OT&E group of DHS and/or NIST (simulation work can be
supported by national labs or UARCs). The third should be managed by
CBP with support from DHS OT&E as the end-user since they are
responsible for screening operations at a working port of entry.
Questions for Mr. Aloise and Dr. Hagan:
Q3a. In addressing previous problems identified in Field Validation
Testing, has the agency changed underlying algorithms that were tested
at the Nevada Test Site (NTS)?
A3a. As discussed above, while DNDO officials told us that they were
not making changes to the ASP algorithms, the report of the Field
Validation Advisory Panel stated that ``correcting algorithm issues
will change ASP referral rate'' suggesting that there will be future
changes to the ASP algorithms. So, there is some confusion about
exactly what changes will need to be made in order to successfully
complete field validation testing.
Q3b. Will simply changing isotope thresholds be enough to fix these
problems, or will algorithms need to be changed?
A3b. Only additional testing can answer this. However, once the problem
with false alarms is fixed, the ability of the ASPs to detect certain
nuclear materials will be diminished. Understanding whether the ASP can
detected certain nuclear materials better than the PVTs will be an
important part of deciding whether to move forward with procurement of
the ASPs.
Q3c. If algorithms are changed, will the equipment have to go back to
the NTS?
A3c. In our view, prior to any decision to move forward with
procurement of ASPs, there should be additional testing at NTS against
actual physical sources of special nuclear materials in order to (1)
understand exactly how the modified ASPs performance compares against
the performance of PVTs in detecting certain nuclear materials, and (2)
assure that there are not any unintended consequences in detection
capabilities resulting from the modifications made to the ASPs since
the August 2008 testing at NTS.
Q3d. If so, how much will this cost and how long will it take?
A3d. DNDO would be the best source for this information. However, we
were previously told by DNDO that NTS testing costs about $500,000 per
week. In our view, this testing, while expensive, would be worthwhile
prior to DNDO committing billions of dollars toward procurement of the
ASPs.
Q3e. Is the Department only changing threshold levels (and not
changing the algorithms) simply because it does not want to go back to
NTS?
A3e. DHS would be the best source for this information.