[Federal Register Volume 66, Number 189 (Friday, September 28, 2001)]
[Notices]
[Pages 49718-49725]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 01-24172]



[[Page 49718]]

=======================================================================
-----------------------------------------------------------------------

OFFICE OF MANAGEMENT AND BUDGET


Guidelines for Ensuring and Maximizing the Quality, Objectivity, 
Utility, and Integrity of Information Disseminated by Federal Agencies

AGENCY: Executive Office of the President, Office of Management and 
Budget.

ACTION: Final guidelines, with request for comments.

-----------------------------------------------------------------------

SUMMARY: These guidelines implement section 515 of the Treasury and 
General Government Appropriations Act for Fiscal Year 2001 (Public Law 
106-554). Section 515 directs the Office of Management and Budget (OMB) 
to issue government-wide guidelines that ``provide policy and 
procedural guidance to Federal agencies for ensuring and maximizing the 
quality, objectivity, utility, and integrity of information (including 
statistical information) disseminated by Federal agencies.'' Within one 
year after OMB issues these guidelines, agencies must issue their own 
implementing guidelines that include ``administrative mechanisms 
allowing affected persons to seek and obtain correction of information 
maintained and disseminated by the agency'' that does not comply with 
the OMB guidelines. OMB is also requesting additional comment for 30 
days on the ``capable of being substantially reproduced'' standard 
(paragraphs V.3.B, V.9, and V.10) which is issued on an interim final 
basis.

DATES: Effective Date: October 1, 2001.
    Comment Date: Comments on the ``capable of being substantially 
reproduced'' standard in paragraphs V.3.B, V.9, and V.10 must be 
submitted by October 29, 2001.

ADDRESSES: Please submit comments to Brooke J. Dickson of the Office of 
Information and Regulatory Affairs, Office of Management and Budget, 
Washington, DC 20503. Comments can also be e-mailed to 
[email protected].

FOR FURTHER INFORMATION CONTACT: Brooke J. Dickson, Office of 
Information and Regulatory Affairs, Office of Management and Budget, 
Washington, DC 20503. Telephone (202) 395-3785.

SUPPLEMENTARY INFORMATION: In section 515(a) of the Treasury and 
General Government Appropriations Act for Fiscal Year 2001 (Public Law 
106-554; H.R. 5658), Congress directed the Office of Management and 
Budget (OMB) to issue, by September 30, 2001, government-wide 
guidelines that ``provide policy and procedural guidance to Federal 
agencies for ensuring and maximizing the quality, objectivity, utility, 
and integrity of information (including statistical information) 
disseminated by Federal agencies * * *.'' Section 515(b) goes on to 
state that the OMB guidelines shall:
    ``(1) apply to the sharing by Federal agencies of, and access to, 
information disseminated by Federal agencies; and
    ``(2) require that each Federal agency to which the guidelines 
apply--
    ``(A) issue guidelines ensuring and maximizing the quality, 
objectivity, utility, and integrity of information (including 
statistical information) disseminated by the agency, by not later than 
1 year after the date of issuance of the guidelines under subsection 
(a);
    ``(B) establish administrative mechanisms allowing affected persons 
to seek and obtain correction of information maintained and 
disseminated by the agency that does not comply with the guidelines 
issued under subsection (a); and
    ``(C) report periodically to the Director--
    ``(i) the number and nature of complaints received by the agency 
regarding the accuracy of information disseminated by the agency; and
    ``(ii) how such complaints were handled by the agency.''
    These guidelines are to be issued ``under sections 3504(d)(1) and 
3516'' of the Paperwork Reduction Act of 1995; pursuant to section 3503 
of that Act, the authorities of the OMB Director are carried out by the 
Administrator of the Office of Information and Regulatory Affairs.

Background

    The focus of section 515 is on the Federal Government's information 
dissemination activities. Indeed, Federal agencies have disseminated 
information to the public for decades. Until recently, agencies have 
disseminated information principally by making paper copies of 
documents available to the public. In recent years, however, Federal 
information dissemination has grown due to the advent of the Internet, 
which has ushered in a revolution in communications. The Internet has 
enabled Federal agencies to disseminate an ever-increasing amount of 
information. Congress has strongly encouraged the Executive Branch's 
dissemination efforts in statutes that include particular dissemination 
activities and in the government-wide dissemination provisions of the 
Paperwork Reduction Act of 1995 (44 U.S.C. chapter 35) (the PRA). In 
addition, the Executive Branch's strong support for information 
dissemination is reflected in the dissemination provisions of OMB 
Circular A-130, ``Management of Federal Information Resources,'' as 
well as in the provisions in OMB Circular A-110, ``Uniform 
Administrative Requirements for Grants and Agreements With Institutions 
of Higher Education, Hospitals, and Other Non-Profit Organizations,'' 
related to a Freedom of Information Act request for research data 
relating to published research findings produced under an award that 
were used by the Federal Government in developing an agency action that 
has the force and effect of law (64 FR 54926; October 8, 1999).
    Section 515 builds upon the existing agency responsibility to 
ensure information quality. According to the PRA, agency Chief 
Information Officers (CIOs) must manage information resources to 
``improve the integrity, quality, and utility of information to all 
users within and outside the agency, including capabilities for 
ensuring dissemination of public information, public access to 
government information, and protections for privacy and security.'' 
Before an agency collects information from 10 or more persons, the 
agency must seek public comment ``to enhance the quality, utility, and 
clarity of the information to be collected.'' The agency then must 
obtain OMB approval that is based upon an evaluation of the agency's 
need for the information, the ``practical utility'' of the information 
to be collected, and the minimization of burden that would be imposed 
on the public in responding to the collection. The CIO must certify to 
OMB that the agency, ``to the maximum extent practicable, uses 
information technology to reduce burden and improve data quality.''
    In developing these guidelines to implement section 515, OMB 
recognized that Federal agencies disseminate many types of information 
in many different ways. A few examples can only begin to describe the 
breadth of information disseminated by the Federal government. Agencies 
disseminate statistical information, such as the aggregated information 
from the 2000 Census and the monthly and quarterly economic reports 
issued by the Bureau of Economic Analysis and the Bureau of Labor 
Statistics. Agencies disseminate information that aids members of the 
public in their daily activities, such as the National Weather 
Service's weather reports and the FAA's air travel advisories. Agencies 
disseminate information about health, safety, and environmental risks 
and information that they collect from regulated entities, such as 
EPA's

[[Page 49719]]

dissemination of Toxic Release Inventory information. Agencies also 
disseminate technical information that they create or obtain in the 
course of developing regulations, often involving scientific, 
engineering, and economic analysis. Agencies disseminate information 
when they issue reports and studies. Moreover, agencies provide the 
public with basic descriptions of agency authorities, activities and 
programs, along with the contact information for the public to interact 
with and access that information or those services.

Underlying Principles

    In accordance with section 515, OMB has designed the guidelines to 
help agencies ensure and maximize the quality, utility, objectivity and 
integrity of the information that they disseminate (meaning to share 
with, or give access to, the public). It is crucial that information 
Federal agencies disseminate meets these guidelines. In this respect, 
the fact that the Internet enables agencies to communicate information 
quickly and easily to a wide audience not only offers great benefits to 
society, but also increases the potential harm that can result from the 
dissemination of information that does not meet basic information 
quality guidelines. Recognizing the wide variety of information Federal 
agencies disseminate and the wide variety of dissemination practices 
that agencies have, OMB developed the guidelines with several 
principles in mind.
    First, OMB designed the guidelines to apply to a wide variety of 
government information dissemination activities that may range in 
importance and scope. OMB also designed the guidelines to be generic 
enough to fit all media, be they printed, electronic, or in other form. 
OMB sought to avoid the problems that would be inherent in developing 
detailed, prescriptive, ``one-size-fits-all'' government-wide 
guidelines that would artificially require different types of 
dissemination activities to be treated in the same manner. Through this 
flexibility, each agency will be able to incorporate the requirements 
of these OMB guidelines into the agency's own information resource 
management and administrative practices.
    Second, OMB designed the guidelines so that agencies will meet 
basic information quality standards. Given the administrative 
mechanisms required by section 515 as well as the standards set forth 
in the PRA, it is clear that agencies should not disseminate 
substantive information that does not meet a basic level of quality. We 
recognize that some government information may need to meet higher or 
more specific information quality standards than those that would apply 
to other types of government information. The more important the 
information, the higher the quality standards to which it should be 
held, for example, in those situations involving ``influential 
scientific or statistical information'' (a phrased defined in these 
guidelines). The guidelines recognize, however, that information 
quality comes at a cost. Accordingly, the agencies should weigh the 
costs (for example, including costs attributable to agency processing 
effort, respondent burden, maintenance of needed privacy, and 
assurances of suitable confidentiality) and the benefits of higher 
information quality in the development of information, and the level of 
quality to which the information disseminated will be held.
    More specifically, the OMB guidelines state that ``agencies shall 
have a basic standard of quality (including objectivity, utility, and 
integrity) as a performance goal * * *''. We note, in the scientific 
context, that in 1996 the Congress, for health decisions under the Safe 
Drinking Water Act, has already adopted a basic standard of quality for 
the use of science in agency decisionmaking. Under 42 U.S.C. 300g-
1(b)(3)(A), an agency is directed, ``to the degree that an Agency 
action is based on science,'' to use ``(i) the best available, peer-
reviewed science and supporting studies conducted in accordance with 
sound and objective scientific practices; and (ii) data collected by 
accepted methods or best available methods (if the reliability of the 
method and the nature of the decision justifies use of the data).'' We 
also note that the OMB guidelines call for an additional level of 
quality ``in those situations involving influential scientific or 
statistical information.'' The additional level of quality concerns a 
standard of care for scientific or statistical analytical results, a 
``capable of being substantially reproduced'' standard that is 
discussed below.
    We further note that in the 1996 amendments to the Safe Drinking 
Water Act the Congress adopted a basic quality standard for the 
dissemination of public information about risks of adverse health 
effects. Under 42 U.S.C. 300g-1(b)(3)(B), the agency is directed, ``to 
ensure that the presentation of information [risk] effects is 
comprehensive, informative, and understandable.'' The agency is further 
directed, ``in a document made available to the public in support of a 
regulation [to] specify, to the extent practicable--(i) each population 
addressed by any estimate [of applicable risk effects]; (ii) the 
expected risk or central estimate of risk for the specific populations 
[affected]; (iii) each appropriate upper-bound or lower-bound estimate 
of risk; (iv) each significant uncertainty identified in the process of 
the assessment of [risk] effects and the studies that would assist in 
resolving the uncertainty; and (v) peer-reviewed studies known to the 
[agency] that support, are directly relevant to, or fail to support any 
estimate of [risk] effects and the methodology used to reconcile 
inconsistencies in the scientific data.'' We urge each agency in 
developing its guidelines to evaluate whether adopting or adapting 
these basic Congressional standards would be appropriate for judging 
the quality of disseminated scientific or statistical information.
    Third, OMB designed the proposed guidelines so that agencies can 
apply them in a common-sense and workable manner. It is important that 
these guidelines do not impose unnecessary administrative burdens that 
would inhibit agencies from continuing to take advantage of the 
Internet and other technologies to disseminate information that can be 
of great benefit and value to the public. In this regard, OMB 
encourages agencies to incorporate the standards and procedures 
required by these guidelines into their existing information resources 
management and administrative practices rather than create new and 
potentially duplicative or contradictory processes. The primary example 
of this is that the guidelines recognize that, in accordance with OMB 
Circular A-130, agencies already have in place well-established 
information quality standards and administrative mechanisms that allow 
persons to seek and obtain correction of information that is maintained 
and disseminated by the agency. Under the OMB guidelines, agencies need 
only ensure that their own guidelines are consistent with these OMB 
guidelines, and then ensure that their administrative mechanisms 
satisfy the standards and procedural requirements in the new agency 
guidelines. Similarly, agencies may rely on their implementation of the 
Federal Government's computer security laws (formerly, the Computer 
Security Act, and now the computer security provisions of the PRA) to 
establish appropriate security safeguards for ensuring the 
``integrity'' of the information that the agencies disseminate.

Summary of OMB Guidelines

    These guidelines apply to Federal agencies subject to the Paperwork 
Reduction Act (44 U.S.C. chapter 35). Agencies are directed to develop 
information resources management

[[Page 49720]]

procedures for reviewing and substantiating (by documentation or other 
means selected by the agency) the quality (including the objectivity, 
utility, and integrity) of information before it is disseminated. In 
addition, agencies are to establish administrative mechanisms allowing 
affected persons to seek and obtain, where appropriate, correction of 
information disseminated by the agency that does not comply with the 
OMB or agency guidelines. Consistent with the underlying principles 
described above, these guidelines stress the importance of having 
agencies apply these standards and develop their administrative 
mechanisms so they can be implemented in a common sense and workable 
manner. Moreover, agencies must apply these standards flexibly, and in 
a manner appropriate to the nature and timeliness of the information to 
be disseminated, and incorporate them into existing agency information 
resources management and administrative practices.
    Section 515 denotes four substantive terms regarding information 
disseminated by Federal agencies: quality, utility, objectivity, and 
integrity. It is not always clear how each substantive term relates--or 
how the four terms in aggregate relate--to the widely divergent types 
of information that agencies disseminate. The guidelines provide 
definitions that attempt to establish a clear meaning so that both the 
agency and the public can readily judge whether a particular type of 
information to be disseminated does or does not meet these attributes.
    In the guidelines, OMB defines ``quality'' as the encompassing 
term, of which ``utility,'' ``objectivity,'' and ``integrity'' are the 
constituents. ``Utility'' refers to the usefulness of the information 
to the intended users. ``Objectivity'' focuses on whether the 
disseminated information is being presented in an accurate, clear, 
complete, and unbiased manner, and as a matter of substance, is 
accurate, reliable, and unbiased. ``Integrity'' refers to security--the 
protection of information from unauthorized access or revision, to 
ensure that the information is not compromised through corruption or 
falsification. OMB modeled the definitions of ``information,'' 
``government information,'' ``information dissemination product,'' and 
``dissemination'' on the longstanding definitions of those terms in OMB 
Circular A-130, but tailored them to fit into the context of these 
guidelines.
    In addition, agencies have two reporting requirements. The first 
report, implemented no later than one year after the issuance of these 
OMB guidelines (no later than October 1, 2002), must provide the 
agency's information quality guidelines that describe administrative 
mechanisms allowing affected persons to seek and obtain, where 
appropriate, correction of disseminated information that does not 
comply with the OMB and agency guidelines. The second report is an 
annual fiscal year report to OMB (to be first submitted on January 1, 
2004) providing information (both quantitative and qualitative, where 
appropriate) on the number, nature, and resolution of complaints 
received by the agency regarding its perceived or confirmed failure to 
comply with these OMB and agency guidelines.

Public Comments and OMB Response

    Section 515(a) required OMB to provide the public and the Federal 
agencies the opportunity to comment on these guidelines. OMB worked 
with Federal agencies, through a working group and through an inter-
agency comment process, in the development of the proposed guidelines. 
The proposed guidelines were published in the Federal Register on June 
28, 2001 (66 FR 34489) providing a public comment period of 45 days. 
OMB received a total of 100 comments from academic institutions (36), 
Federal agencies (26), individual members of the public (7), 
associations affiliated with academia (5), associations affiliated with 
medical, social science or science interests (15), associations 
affiliated with Federal Government interests (4), and associations 
affiliated with industry interests (7).
    General Concerns. Many comments expressed support for the idea of 
government-wide quality standards for information disseminated by 
Federal agencies. Comments also expressed support for OMB's commitment 
to creating flexible general guidelines and to minimizing the 
administrative costs and burdens that these guidelines will impose. The 
majority of comments focused on two aspects of the proposed guidelines: 
suggestions for placing limitations on the administrative correction 
mechanisms requirements of the statute; and the need to clarify 
specific definitions and other terms found in the guidelines.
    Many comments raised questions and concerns about how these 
guidelines interact with existing statutes and policies, including the 
Paperwork Reduction Act and the Government Performance and Results Act. 
We have attempted to draft these guidelines in a way that addresses the 
requirements of section 515, but does not impose a completely new and 
untried set of standards upon Federal agencies. We encourage agencies 
to consider the effect of relevant existing statutes and policies in 
the development of their own guidelines.
    Administrative Mechanisms. These guidelines require agencies to 
establish administrative mechanisms allowing affected persons to seek 
and obtain, where appropriate, correction of information maintained and 
disseminated by the agency that does not comply with the OMB 
guidelines. Many comments suggested that limits be imposed on the types 
of information that should be subject to these guidelines, in 
particular, information that is disseminated by agency libraries. OMB 
agrees that archival information disseminated by Federal agency 
libraries (for example, Internet distribution of published articles) 
should not be covered by these guidelines, given that libraries do not 
endorse the information that they disseminate. Moreover, an agency's 
dissemination of public filings (for example, corporate filings with 
the Securities and Exchange Commission) is not covered by these 
guidelines. In each of these situations, the agencies have not authored 
these documents and have not adopted them as representing the agencies' 
views. By disseminating these materials, the agencies are simply 
ensuring that the public can have quicker and easier access to 
materials that are publicly available. In developing its implementing 
guidelines, and in accordance with the criteria set forth in these 
guidelines, each agency should evaluate and identify the types of 
information that it disseminates that will be subject to its 
guidelines.
    In addition, comments also raised the concern that the guidelines 
would apply to ``preliminary'' information, and they recommended that 
the guidelines exclude such information. OMB appreciates the concerns 
that these comments have raised. However, OMB does not believe that an 
exclusion for ``preliminary'' information is necessary or appropriate. 
It is still important that the quality of preliminary information be 
ensured and that preliminary information be subject to the 
administrative complaint-and-correction process.
    A few comments stated that affected information should be limited 
to information used in agency rulemaking. While this has been the 
position of previous policies which these guidelines are not intended 
to modify or replace (see, e.g., section __.36(d) in OMB Circular A-
110), we believe the

[[Page 49721]]

plain meaning and intent of section 515 covers the larger government 
information universe.
    Based on the public comments received, these guidelines allow 
agencies to determine the appropriate level of correction for a 
complaint received. Several comments suggested that agencies use 
disclaimers to distinguish the status of information, a practice that 
agencies should consider adopting as they consider their information 
holdings.
    OMB received detailed discussion on the requirement that agencies 
develop administrative mechanisms allowing for affected persons to 
``seek and obtain correction of information that does not comply with 
OMB's guidelines.'' Members of the scientific community expressed 
strong concerns about the possibility of a Federal agency that would 
``correct'' scientific information without carrying out the scientific 
analysis to support the correction. Comments from all fields suggested 
in various ways that challenging individuals should be ``required to 
openly state his/her relationship with the data/information 
(familiarity/expertise) and provide information [as] to his/her 
interest in it.''
    Comments also pointed out great potential for abuse of this 
process. As one association summarized, ``This could be seen to provide 
grounds for interested parties to demand access to underlying data, to 
compel the government to replicate research findings (at great expense 
and with unnecessary delay), or in other ways impede, discredit, harass 
or stymie research.'' For example, the National Oceanic and Atmospheric 
Administration (NOAA) explained that they receive numerous complaints 
from the public when they miss a weather forecast. ``Does this mean 
that the NWS [National Weather Service] could be requested to change a 
forecast after the fact? Or could someone with an economic interest 
challenge official observational data which could affect the value of 
an insurance payment?'' asks NOAA.
    Overall, OMB does not envision administrative mechanisms that would 
burden agencies with frivolous claims. Instead, the correction process 
should serve to address the genuine and valid needs of the agency and 
its constituents without disrupting agency processes. Agencies, in 
making their determination of whether or not to correct information, 
may reject claims made in bad faith or without justification, and are 
required to undertake only the degree of correction that they conclude 
is appropriate for the nature and timeliness of the information 
involved, and explain such practices in their annual fiscal year 
reports to OMB.
    Numerous comments provided language to clarify or limit the term, 
``affected persons.'' One academic institution suggested that the term, 
``affected persons,'' reflects a criterion of ``direct measurable 
impact with significant personal consequence.'' Other academic 
institutions suggested that ``affected persons should not be permitted 
to challenge the substance of information without showing that a 
qualified scientist has found fault with its quality or integrity.'' 
Similarly, some comments argued that the ability to correct scientific 
information should be limited only to other scientists. Several 
associations suggested that OMB identify the types of information that 
could be challenged rather than to focus on the characteristics of a 
``legitimate'' challenger. OMB considered these comments at length. Our 
conclusion is that ``affected persons'' are people who may benefit or 
be harmed by the disseminated information. This includes persons who 
are seeking to address information about themselves as well as persons 
who use information. However, each agency should consider how persons 
(which includes groups, organizations and corporations, as defined by 
the Paperwork Reduction Act) will be affected by the agency's 
information. Agencies should address the issue of ``affected persons'' 
in consultation with their constituents through the public comment 
process that agencies will provide after drafting their proposed 
guidelines and before submitting them for OMB review.
    These guidelines require that an agency official be designated to 
receive and resolve complaints regarding information that does not 
comply with either the OMB guidelines or the agency's guidelines. In 
the proposed guidelines, we required, with a limited exception, that 
the Chief Information Officer (CIO) of the agency have this 
responsibility. Of the government agencies that commented on this 
provision, many pointed to their specific agency practices on 
information quality and their designation of a ``quality official'' who 
was not necessarily working under the agency CIO. Recognizing that some 
agencies may have specific officials in place to address quality 
issues, the final guidelines allow agencies to designate an appropriate 
official. Agencies may also designate multiple officials, i.e., based 
on the needs of individual agency components, as long as there is a 
single official with these overall responsibilities designated at the 
agency level. The authorized official also needs to consult with the 
CIO on quality matters pertaining to information disseminated by the 
agency.
    Agencies need to respond to complaints in a manner appropriate to 
the nature and extent of the complaint. Examples of appropriate 
responses, as suggested by comments, include personal contacts via 
letter or telephone, form letters, press releases or mass mailings that 
correct a widely disseminated error or address a frequently raised 
complaint. Agencies may want to utilize other methods of response under 
existing agency practices. For example, for agencies with a high volume 
of complaints, it is acceptable for the agency to describe a sample of 
those complaints in the annual fiscal year report to OMB. For 
categories of inconsequential or trivial complaints identified in the 
agency guidelines, an agency may decide that no response is necessary. 
Agencies should describe to OMB as part of the annual fiscal year 
report the chosen response mechanisms and how they are working.
    Definitions and Other Terms. Section 515 denotes four substantive 
terms regarding information disseminated by Federal agencies: quality, 
utility, objectivity, and integrity. We have defined ``quality'' as an 
encompassing term. The organizations and individuals that submitted 
comments did not object to having ``quality'' defined as an 
encompassing term, but suggested that we should discuss each term 
separately. The principles laid out in the proposed guidelines, stated 
one comment, create ``subjective definitions'' of the four terms. This 
comment warned OMB that ``subjective definitions of quality, 
objectivity, utility, and integrity could cause agencies to delay data 
release or disregard data for fear of challenge.'' Other comments 
expressed similar views, or as one association observed, ``Science does 
not recognize a sliding scale of quality.''
    These guidelines reflect OMB's determination that ``quality,'' 
``utility,'' ``objectivity,'' and ``integrity'' are closely 
interrelated concepts in the context of these guidelines. Collectively, 
these terms address the following three aspects of the information that 
is to be disseminated: whether the information is useful to the 
intended users of the information; whether the disseminated information 
is being presented in an accurate, clear, complete, and unbiased manner 
in both presentation and as a characteristic that should be inherent to 
quality information; and whether the information has been protected 
from unauthorized access or revision.

[[Page 49722]]

    Some comments stated that OMB was ``exceeding the statutory 
mandate'' and going beyond ``Congressional intent'' in specifying 
scientific and statistical information in these guidelines. Others felt 
that we should simply acknowledge that the scientific and statistical 
communities already have practices and standards for their information, 
rather than create another set of standards for these information 
types. OMB does not agree with those comments that said the proposed 
guidelines went beyond the statute in covering statistical and 
scientific information. Section 515 expressly states that its scope 
includes statistical information. Moreover, section 515 has no 
exclusion for scientific information, and in many respects it is very 
similar to (and overlaps with) statistical information. OMB, however, 
does appreciate the concerns that the comments raised about the 
guidelines not creating another set of standards for statistical and 
scientific information. Our guidelines do not seek to impose new 
standards on these communities, but to reiterate the standards that are 
already held in those communities.
    Recognizing public interest in medical and public health 
information, we have specifically added a provision stating, ``Agencies 
shall adopt specific standards of quality that are appropriate for the 
various categories of information they disseminate.'' For example, OMB 
encourages agencies, in crafting their agency-specific guidelines, to 
promote objectivity in information quality in ways that protect the 
confidentiality of research subjects and encourage public participation 
in research. These concerns are particularly salient in medical and 
public health research.
    A number of comments regarded our discussion of ensuring that 
scientific information be ``substantially reproducible'' as requiring 
agencies to replicate original data and to perform independent analysis 
upon all scientific information disseminated by the agency. We have 
responded to these concerns in a number of ways. First, we make it 
clear that what we now refer to as the ``capable of being substantially 
reproduced'' standard applies to analytical results that are 
disseminated, and does not apply to the original or supporting data. 
Thus, replication of original data is not required. Second, the 
``capable of being substantially reproduced'' standard is applicable 
only to ``influential'' scientific and statistical information as 
defined in the guidelines. Third, the guidelines call for the agency to 
determine that ``influential'' analytical results be capable of being 
substantially reproducible by independent analysis. We intend this 
standard to say that, if appropriately qualified persons used the same 
or a similar methodology, they would be expected to achieve similar 
findings and results.
    Based on the concerns expressed in the comments, we expanded upon 
our discussion of ``capable of being substantially reproduced'' in our 
definition of ``objective,'' and added two explanatory definitions. We 
state, in paragraphs V.3.B, V.9, and V.10:
    In addition, ``objectivity'' involves a focus on ensuring accurate, 
reliable, and unbiased information. In a scientific or statistical 
context, the original or supporting data shall be generated, and the 
analytical results shall be developed, using sound statistical and 
research methods.
    i. If the results have been subject to formal, independent, 
external peer review, the information can generally be considered of 
acceptable objectivity.
    ii. In those situations involving influential scientific or 
statistical information, the results must be capable of being 
substantially reproduced, if the original or supporting data are 
independently analyzed using the same models. Reproducibility does not 
mean that the original or supporting data have to be capable of being 
replicated through new experiments, samples or tests.
    iii. Making the data and models publicly available will assist in 
determining whether analytical results are capable of being 
substantially reproduced. However, these guidelines do not alter the 
otherwise applicable standards and procedures for determining when and 
how information is disclosed. Thus, the objectivity standard does not 
override other compelling interests, such as privacy, trade secret, and 
other confidentiality protections.
    ``Influential'' when used in the phrase ``influential scientific or 
statistical information'' means the agency expects that information in 
the form of analytical results will likely have an important effect on 
the development of domestic or international government or private 
sector policies or will likely have important consequences for specific 
technologies, substances, products or firms.
    ``Capable of being substantially reproduced'' means that 
independent reanalysis of the original or supporting data using the 
same methods would generate similar analytical results, subject to an 
acceptable degree of imprecision.
    As a general matter, in the scientific and research context, we 
regard technical information that has been subjected to formal, 
independent, external peer review as presumptively objective. An 
example of a formal independent external peer review is the review 
process used by scientific journals. However, depending on the nature 
and timeliness of the information involved, an agency may decide that 
peer review is not necessary or appropriate. On the other hand, in 
those situations involving influential scientific or statistical 
information, the substantial reproducibility standard is added as a 
quality standard above and beyond some peer review quality standards. 
In the definition of ``influential,'' when used in the phrase 
``influential scientific or statistical information,'' we note that the 
manner in which people perceive the scientific or statistical 
information can have important consequences for specific policies, 
technologies, substances, products, and firms.
    Based on concerns with the ``substantially reproducible'' standard, 
a number of comments suggested that OMB should repropose this standard 
for additional public comment, rather than going final at this time. 
While, in deference to the statutory deadline, OMB is issuing the 
``capable of being substantially reproduced'' standard (paragraphs 
V.3.B, V.9, and V.10), OMB is doing so on an interim final basis. We 
specifically request public comments on this standard by October 29, 
2001. In addition, OMB wants to stress that the guidelines published 
today should be understood as a beginning of an evolutionary process 
that will include draft agency guidelines, public comment, final agency 
guidelines, development of experience with OMB and agency guidelines, 
and continued refinement of both OMB and agency guidelines.
    OMB modeled the draft definitions of ``information,'' ``government 
information,'' ``information dissemination product,'' and 
``dissemination'' on the longstanding definitions of those terms in OMB 
Circular A-130, but tailored them to fit into the context of these 
guidelines. Information that is disseminated on behalf of an agency 
(through a contract or a grant) is considered to be sponsored by the 
agency and is subject to these guidelines. Consistent with the PRA 
concept of agency ``sponsorship'' of a collection of information, 
information is considered to be disseminated on behalf of an agency by 
a contractor or grantee if the dissemination is done at the agency's 
specific request or with the agency's specific approval. See 5 CFR 
1320.3(d). Finally, it should be noted

[[Page 49723]]

that these guidelines focus primarily on the dissemination of 
substantive information (i.e. reports, studies, summaries) rather than 
information pertaining to basic agency operations.
    We have clarified two terms for the final guidelines. The proposed 
guidelines included ``opinions'' in the definition of ``information.'' 
We agree with comments that indicated agencies should not be 
accountable for correcting someone's opinion, but in the agency's 
presentation of the information, it should be clear that what is being 
offered is someone's opinion rather than facts or the agency's views. 
``Opinion'' has therefore been removed from the definition of 
``information'' in the final guidelines. The definition for 
``dissemination'' was also revised after discussions with two Federal 
agencies that correspond frequently with individual members of the 
public regarding their participation in the agency's programs. In 
addition, in the definition of ``dissemination,'' we changed the 
exclusion for ``judicial process'' to ``adjudicative process'' to make 
it clear that these guidelines do not apply to the issuance of agency 
adjudicative decisions.
    Reporting Requirements. Agencies have two reporting requirements. 
The first report, taking effect no later than one year after the 
issuance of these OMB guidelines, must provide the agency's information 
quality guidelines that describe administrative mechanisms allowing 
affected persons to seek and obtain, where appropriate, correction of 
disseminated information that does not comply with these OMB 
guidelines. During the year that agencies have to complete their agency 
guidelines, agencies must publish the draft reports in the Federal 
Register for a period of public comment, and no later than nine months 
after the issuance of OMB's guidelines, submit their draft reports to 
OMB for review. Upon completion of OMB's review, final agency 
guidelines must be published in the Federal Register and made available 
through the agency website. The entire process must be completed by no 
later than one year after the issuance of the OMB guidance (no later 
than October 1, 2002).
    The second report is an annual fiscal year report to OMB (to be 
first submitted on January 1, 2004) providing information on the 
number, nature, and resolution of complaints received by the agency 
regarding its perceived or confirmed failure to comply with these OMB 
and agency guidelines. Regarding the proposed guidelines, we received 
detailed comments on the required report to OMB describing the number 
and nature of complaints received by the agency and how such complaints 
were resolved. Two Federal agencies stated that it would be burdensome 
to report to OMB on every single complaint they received and responded 
to, particularly because many of the complaints may be received in 
phone calls and given informal responses that address the callers' 
concerns. Recognizing that agencies may deal with large volumes of 
complaints on particular types of information disseminated by the 
agency, OMB's guidelines allow the agency to provide qualitative and/or 
quantitative descriptions of complaints received and how they were 
resolved (or not). OMB also recognizes that a large number of comments 
about a specific document may only demonstrate that the information is 
controversial, not that its quality is flawed.
    In conclusion, issuance of these final guidelines meets the 
statutory requirement that section 515 imposed on OMB. As we stated 
earlier in this preamble, and in connection with the proposed 
guidelines, OMB has sought in developing these guidelines to make them 
flexible enough so that Federal agencies can apply them in a common 
sense, workable, and appropriately tailored manner to the wide variety 
of dissemination activities that the Federal Government undertakes. In 
addition, in drafting guidelines that will apply on a government-wide 
basis, OMB has been sensitive to the problem of unintended consequences 
and has tried to anticipate and address issues that could arise during 
the implementation of these guidelines. In this respect, the public and 
agency comments that we received on the proposed guidelines were very 
helpful and are greatly appreciated. As we explained above, we made a 
number of revisions to the guidelines to address the concerns raised in 
the comments, and we also believe that these and other concerns can be 
addressed as well in the implementing guidelines that each agency will 
develop in the coming months. In addition, OMB is issuing the ``capable 
of being substantially reproduced'' standard (paragraphs V.3.B, V.9, 
and V.10) on an interim final basis. We specifically request public 
comments on this standard over the next 30 days.
    Moreover, over time as the agencies and the public gain further 
experience with the OMB guidelines, we would appreciate receiving any 
suggestions for how OMB could improve them. Just as OMB requested 
public comment before issuing these final guidelines, OMB will refine 
these guidelines as experience develops and further public comment is 
obtained.

    Dated: September 24, 2001.
John D. Graham,
Administrator, Office of Information and Regulatory Affairs.

Guidelines for Ensuring and Maximizing the Quality, Objectivity, 
Utility, and Integrity of Information Disseminated by Federal 
Agencies

I. OMB Responsibilities

    Section 515 of the Treasury and General Government Appropriations 
Act for FY2001 (Public Law 106-554) directs the Office of Management 
and Budget to issue government-wide guidelines that provide policy and 
procedural guidance to Federal agencies for ensuring and maximizing the 
quality, objectivity, utility, and integrity of information, including 
statistical information, disseminated by Federal agencies.

II. Agency Responsibilities

    Section 515 directs agencies subject to the Paperwork Reduction Act 
(44 U.S.C. 3502(a)) to--
    1. Issue their own information quality guidelines ensuring and 
maximizing the quality, objectivity, utility, and integrity of 
information, including statistical information, disseminated by the 
agency no later than one year after the date of issuance of the OMB 
guidelines;
    2. Establish administrative mechanisms allowing affected persons to 
seek and obtain correction of information maintained and disseminated 
by the agency that does not comply with these OMB guidelines; and
    3. Report to the Director of OMB the number and nature of 
complaints received by the agency regarding agency compliance with 
these OMB guidelines concerning the quality, objectivity, utility, and 
integrity of information and how such complaints were resolved.

III. Guidelines for Ensuring and Maximizing the Quality, Objectivity, 
Utility, and Integrity of Information Disseminated by Federal Agencies

    1. Overall, agencies shall adopt a basic standard of quality 
(including objectivity, utility, and integrity) as a performance goal 
and should take appropriate steps to incorporate information quality 
criteria into agency information dissemination practices. Quality is to 
be ensured and established at levels appropriate to the nature and 
timeliness of the information to be disseminated. Agencies shall adopt 
specific standards of quality that are appropriate for the various 
categories of information they disseminate.
    2. As a matter of good and effective agency information resources

[[Page 49724]]

management, agencies shall develop a process for reviewing the quality 
(including the objectivity, utility, and integrity) of information 
before it is disseminated. Agencies shall treat information quality as 
integral to every step of an agency's development of information, 
including creation, collection, maintenance, and dissemination. This 
process shall enable the agency to substantiate the quality of the 
information it has disseminated through documentation or other means 
appropriate to the information.
    3. To facilitate citizen review, agencies shall establish 
administrative mechanisms allowing affected persons to seek and obtain, 
where appropriate, timely correction of information maintained and 
disseminated by the agency that does not comply with OMB or agency 
guidelines. These administrative mechanisms shall be flexible, 
appropriate to the nature and timeliness of the disseminated 
information, and incorporated into agency information resources 
management and administrative practices.
    4. The agency's pre-dissemination review, under paragraph III.2, 
shall apply to information that the agency first disseminates on or 
after October 1, 2002. The agency's administrative mechanisms, under 
paragraph III.3, shall apply to information that the agency 
disseminates on or after October 1, 2002, regardless of when the agency 
first disseminated the information.

IV. Agency Reporting Requirements

    1. Agencies must designate the Chief Information Officer or another 
official to be responsible for agency compliance with these guidelines.
    2. The agency shall respond to complaints in a manner appropriate 
to the nature and extent of the complaint. Examples of appropriate 
responses include personal contacts via letter or telephone, form 
letters, press releases or mass mailings that correct a widely 
disseminated error or address a frequently raised complaint.
    3. Each agency must prepare a draft report, no later than April 1, 
2002, providing the agency's information quality guidelines and 
explaining how such guidelines will ensure and maximize the quality, 
objectivity, utility, and integrity of information, including 
statistical information, disseminated by the agency. This report must 
also detail the administrative mechanisms developed by that agency to 
allow affected persons to seek and obtain appropriate correction of 
information maintained and disseminated by the agency that does not 
comply with the OMB or the agency guidelines.
    4. The agency must publish a notice of availability of this draft 
report in the Federal Register, and post this report on the agency's 
website, to provide an opportunity for public comment.
    5. Upon consideration of public comment and after appropriate 
revision, the agency must submit this draft report to OMB for review 
regarding consistency with these OMB guidelines no later than July 1, 
2002. Upon completion of that OMB review and completion of this report, 
agencies must publish notice of the availability of this report in the 
Federal Register, and post this report on the agency's web site no 
later than October 1, 2002.
    6. On an annual fiscal-year basis, each agency must submit a report 
to the Director of OMB providing information (both quantitative and 
qualitative, where appropriate) on the number and nature of complaints 
received by the agency regarding agency compliance with these OMB 
guidelines and how such complaints were resolved. Agencies must submit 
these reports no later than January 1 of each following year, with the 
first report due January 1, 2004.

V. Definitions

    1. ``Quality'' is an encompassing term comprising utility, 
objectivity, and integrity. Therefore, the guidelines sometimes refer 
to these four statutory terms, collectively, as ``quality.''
    2. ``Utility'' refers to the usefulness of the information to its 
intended users, including the public. In assessing the usefulness of 
information that the agency disseminates to the public, the agency 
needs to consider the uses of the information not only from the 
perspective of the agency but also from the perspective of the public. 
As a result, when reproducibility and transparency of information are 
relevant for assessing the information's usefulness from the public's 
perspective, the agency must take care to ensure that reproducibility 
and transparency have been addressed in its review of the information.
    3. ``Objectivity'' involves two distinct elements, presentation and 
substance.
    A. ``Objectivity'' includes whether disseminated information is 
being presented in an accurate, clear, complete, and unbiased manner. 
This involves whether the information is presented within a proper 
context. Sometimes, in disseminating certain types of information to 
the public, other information must also be disseminated in order to 
ensure an accurate, clear, complete, and unbiased presentation. Also, 
the agency needs to identify the sources of the disseminated 
information (to the extent possible, consistent with confidentiality 
protections) and, in a scientific or statistical context, the 
supporting data and models, so that the public can assess for itself 
whether there may be some reason to question the objectivity of the 
sources. Where appropriate, supporting data should have full, accurate, 
transparent documentation, and error sources affecting data quality 
should be identified and disclosed to users.
    B. In addition, ``objectivity'' involves a focus on ensuring 
accurate, reliable, and unbiased information. In a scientific or 
statistical context, the original or supporting data shall be 
generated, and the analytical results shall be developed, using sound 
statistical and research methods.
    i. If the results have been subject to formal, independent, 
external peer review, the information can generally be considered of 
acceptable objectivity.
    ii. In those situations involving influential scientific or 
statistical information, the results must be capable of being 
substantially reproduced, if the original or supporting data are 
independently analyzed using the same models. Reproducibility does not 
mean that the original or supporting data have to be capable of being 
replicated through new experiments, samples or tests.
    iii. Making the data and models publicly available will assist in 
determining whether analytical results are capable of being 
substantially reproduced. However, these guidelines do not alter the 
otherwise applicable standards and procedures for determining when and 
how information is disclosed. Thus, the objectivity standard does not 
override other compelling interests, such as privacy, trade secret, and 
other confidentiality protections.
    4. ``Integrity'' refers to the security of information--protection 
of the information from unauthorized access or revision, to ensure that 
the information is not compromised through corruption or falsification.
    5. ``Information'' means any communication or representation of 
knowledge such as facts or data, in any medium or form, including 
textual, numerical, graphic, cartographic, narrative, or audiovisual 
forms. This definition includes information that an agency disseminates 
from a web page, but does not include the provision of hyperlinks to 
information that others disseminate. This definition does not include 
opinions, where the agency's presentation makes it clear that what is

[[Page 49725]]

being offered is someone's opinion rather than fact or the agency's 
views.
    6. ``Government information'' means information created, collected, 
processed, disseminated, or disposed of by or for the Federal 
Government.
    7. ``Information dissemination product'' means any book, paper, 
map, machine-readable material, audiovisual production, or other 
documentary material, regardless of physical form or characteristic, an 
agency disseminates to the public. This definition includes any 
electronic document, CD-ROM, or web page.
    8. ``Dissemination'' means agency initiated or sponsored 
distribution of information to the public (see 5 CFR 1320.3(d) 
(definition of ``Conduct or Sponsor''). Dissemination does not include 
distribution limited to government employees or agency contractors or 
grantees; intra- or inter-agency use or sharing of government 
information; and responses to requests for agency records under the 
Freedom of Information Act, the Privacy Act, the Federal Advisory 
Committee Act or other similar law. This definition also does not 
include distribution limited to correspondence with individuals or 
persons, press releases, archival records, public filings, subpoenas or 
adjudicative processes.
    9. ``Influential'' when used in the phrase ``influential scientific 
or statistical information'' means the agency expects that information 
in the form of analytical results will likely have an important effect 
on the development of domestic or international government or private 
sector policies or will likely have important consequences for specific 
technologies, substances, products or firms.
    10. ``Capable of being substantially reproduced'' means that 
independent reanalysis of the original or supporting data using the 
same methods would generate similar analytical results, subject to an 
acceptable degree of imprecision.

[FR Doc. 01-24172 Filed 9-27-01; 8:45 am]
BILLING CODE 3110-01-P