[Federal Register Volume 65, Number 193 (Wednesday, October 4, 2000)]
[Notices]
[Pages 59200-59214]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 00-25397]
=======================================================================
-----------------------------------------------------------------------
DEPARTMENT OF JUSTICE
Confidentiality in Federal Alternative Dispute Resolution
Programs; Evaluation of Federal Alternative Dispute Resolution Programs
AGENCY: Department of Justice/Federal Alternative Dispute Resolution
Council.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: This notice solicits public comment on two documents designed
to assist Federal agencies in developing alternative dispute resolution
(ADR) programs: ``Confidentiality in Federal Alternative Dispute
Resolution Programs'' and ``Evaluation of Federal Alternative Dispute
Resolution
[[Page 59201]]
Programs.'' These documents were created by the Federal ADR Steering
Committee, a group of subject matter experts from federal agencies with
active ADR programs. They were approved for publication in draft form
by the Federal ADR Council, a group of high level government officials
chaired by the Attorney General. The first document contains detailed
guidance on the nature and limits of confidentiality in Federal ADR
programs and also includes a statement on these issues for Federal
neutrals to use in ADR proceedings. The second document contains
detailed recommendations for agencies to follow when evaluating their
ADR programs.
All interested individuals or organizations are invited to submit
comments on these documents for the consideration of the Federal ADR
Council before they are published in final form at the end of this
year.
DATES: All comments must be postmarked by November 1, 2000, in order to
receive consideration.
ADDRESSES: Address all comments to Jeffrey M. Senger, Deputy Senior
Counsel for Dispute Resolution, United States Department of Justice,
950 Pennsylvania Ave. NW, Room 4328, Washington, D.C., 20530
Dated: September 27, 2000.
Jeffrey M. Senger,
Deputy Senior Counsel for Dispute Resolution, Department of Justice.
SUPPLEMENTARY INFORMATION: In recent years, the government and the
private sector increasingly have been using techniques known as
alternative dispute resolution (ADR). Our experience has shown that ADR
can resolve disputes in a manner that is quicker, cheaper, and less
adversarial than traditional processes such as litigation. In ADR,
parties meet with each other directly, under the guidance of a neutral
professional who is trained and experienced in handling disputes. They
talk about the problems that led to the complaint and the resolution
that will work best for them in the future. While litigation often
silences the parties and severely restricts their control over the
outcome of their own dispute, ADR allows them instead to work
collaboratively to find creative, effective solutions that are
agreeable to all sides.
The Administrative Dispute Resolution Act of 1996 (ADRA), 5 U.S.C.
571-584, requires each Federal agency to promote the use of ADR and
calls for the establishment of an interagency committee to assist
agencies in the use of ADR. Pursuant to this Act, a Presidential
Memorandum dated May 1, 1998, created the Interagency ADR Working
Group, chaired by the Attorney General, to ``facilitate, encourage, and
provide coordination'' for Federal agencies. In the Memorandum, the
President charged the Working Group with assisting agencies with
training in ``how to use alternative means of dispute resolution'' and
evaluation ``to ascertain the benefits of alternative means of dispute
resolution.'' The following two documents are designed to serve these
goals.
The first document describes the nature and limits of
confidentiality in Federal ADR proceedings. Confidentiality is vital
for the success of ADR for several reasons. Parties must be free to
engage in candid, informal discussions of their interests in order to
reach the best possible settlement of their claims. Guarantees of
confidentiality permit parties to speak openly, without fear their
statements will be used against them later. Confidentiality also
facilitates ADR by encouraging parties to avoid the posturing that
often occurs when proceedings are on the record. Further,
confidentiality gives parties the ability to trust the mediator because
they are assured he or she will not later take sides and talk publicly
in favor of one party or the other. At the same time, members of the
public have a general right to know what happens in government
proceedings and do not want ADR to be used to shield improper activity
that involves public business. The ADRA is designed to strike the
appropriate balance between the public interest in access to government
decision-making and the necessity for certain guarantees of
confidentiality in ADR in order for the process to be effective.
Understandably, there has been a great deal of interest in
understanding what statements made in the context of a Federal ADR
proceeding are confidential and what statements are not. This document
is designed to give a detailed explanation of the reasonable
expectations of confidentiality for parties who participate in ADR
involving the government. The first section of the report reprints the
confidentiality provisions of the ADRA. Next, the report contains a
section-by-section analysis of these confidentiality provisions. Then
the report sets forth, in question-and-answer format, an expanded
analysis of the issues likely to arise in practice. Finally, the report
presents a model confidentiality statement suitable for use by neutrals
in Federal ADR proceedings.
The second document contains detailed guidance for agencies to use
when conducting evaluations of their ADR programs. Proponents of ADR
have described many benefits from its use, including savings of time
and money, increased party satisfaction with the process and its
outcome, increased settlement rates, and improved relationships. In
order to ensure the growth of ADR programs, these benefits must be
rigorously documented and communicated to the public. If evaluations
determine problems with ADR programs, these must be remedied.
Evaluation is a vital part of any ADR program, and it is consistent
with the obligations of all Federal agencies under the Government
Performance and Results Act (Pub. L. 103-62).
The first part of this document is a two-page description of
general evaluation recommendations for Federal ADR programs. It sets
forth specific data that agencies should capture and gives a brief
introduction to other important concepts, such as validity,
reliability, and presentation of data. The remainder of the report is a
twenty-page detailed description of evaluation, including planning and
design, methodology, and communicating results. The report concludes
with a bibliography of additional resources in this area.
The Federal ADR Council encourages all interested parties to submit
comments on these documents. The Council will consider all comments in
connection with its review of the final versions of these documents at
the end of 2000.
Nothing in these guidance documents shall be construed to create
any right or benefit, substantive or procedural, enforceable at law or
in equity, by a party against the United States, its agencies, its
officers or any other person.
The Federal ADR Council
Chair: Janet Reno, Attorney General, Department of Justice
Vice Chair: Erica Cooper, Deputy General Counsel, Federal Deposit
Insurance Corporation
Members: Leigh A. Bradley, General Counsel, Department of Veterans
Affairs; Meyer Eisenberg, Deputy General Counsel, Securities and
Exchange Commission; Mary Anne Gibbons, General Counsel, U.S. Postal
Service; Gary S. Guzy, General Counsel, Environmental Protection
Agency; Jeh C. Johnson, General Counsel, Department of the Air Force;
Harold Kwalwasser, Deputy General Counsel, Department of Defense;
Rosalind Knapp, Acting General Counsel, Department of Transportation;
Anthony N. Palladino, Director, Office of Dispute Resolution, Federal
Aviation Administration, Department of Transportation; Janet S.
[[Page 59202]]
Potts, Counsel to the Secretary, Department of Agriculture; Harriett S.
Rabb, General Counsel, Department of Health and Human Services; Henry
L. Solano, Solicitor, Department of Labor; John Sparks, Principal
Deputy General Counsel, Department of the Navy; Peter R. Steenland,
Jr., Senior Counsel for Dispute Resolution, U.S. Department of Justice;
Mary Ann Sullivan, General Counsel, Department of Energy; Robert Ward,
Dispute Resolution Specialist, Environmental Protection Agency.
Report on the Reasonable Expectations of Confidentiality Under the
Administrative Dispute Resolution Act of 1996
Table of Contents
I. Introduction
II. Administrative Dispute Resolution Act
III. Section-By-Section Analysis of Confidentiality Provisions
IV. Questions & Answers on Confidentiality under the Administrative
Dispute Resolution Act (ADR Act)
V. Model Confidentiality Statement for Use by Neutrals
I. Introduction
The Administrative Dispute Resolution Act of 1996 (``ADR Act'')
contains provisions that affect the confidentiality of administrative
ADR proceedings. Neutrals and participants in federal dispute
resolution proceedings need to have an accurate understanding of these
provisions. The Federal ADR Council directed the Interagency ADR
Working Group Steering Committee to review the ADR Act confidentiality
provisions and provide the Council with a report outlining reasonable
expectations of confidentiality for parties in federal dispute
resolution. This report, the product of that effort, describes the ADR
Act confidentiality provisions principally located at 5 U.S.C. Section
574.
The report has four sections: (1) A reprint of the confidentiality
provisions of the ADR Act; (2) a section-by-section analysis of the
confidentiality provisions; (3) a set of questions and answers designed
to expand upon the analysis and address issues likely to arise in
practice; and (4) a model confidentiality statement suitable for use by
neutrals in federal ADR proceedings.
During preparation of this report, several issues emerged regarding
implementation of the ADR Act that are not fully addressed in this
report. These issues are important to the practice of federal ADR and
would benefit from further investigation and study. As federal sector
experience with ADR evolves, some issues addressed in this report will
be refined and new issues are likely to arise. It is also important to
note that the ADR Act is not the only means of maintaining
confidentiality and other laws, regulations, and agency policies may
impact confidentiality. A complete analysis of all such authorities is
beyond the scope of this report.
II. Administrative Dispute Resolution Act
Definitions (5 U.S.C. 571)
For the purposes of this subchapter, the term--
(1) ``agency'' has the same meaning as in section 551(1) of this
title;
(2) ``administrative program'' includes a Federal function which
involves protection of the public interest and the determination of
rights, privileges, and obligations of private persons through rule
making, adjudication, licensing, or investigation, as those terms are
used in subchapter II of this chapter;
(3) ``alternative means of dispute resolution'' means any procedure
that is used to resolve issues in controversy, including, but not
limited to, conciliation, facilitation, mediation, factfinding,
minitrials, arbitration, and use of ombuds, or any combination thereof;
(4) ``award'' means any decision by an arbitrator resolving the
issues in controversy;
(5) ``dispute resolution communication'' means any oral or written
communication prepared for the purposes of a dispute resolution
proceeding, including any memoranda, notes or work product of the
neutral, parties or nonparty participant; except that a written
agreement to enter into a dispute resolution proceeding, or final
written agreement or arbitral award reached as a result of a dispute
resolution proceeding, is not a dispute resolution communication;
(6) ``dispute resolution proceeding'' means any process in which an
alternative means of dispute resolution is used to resolve an issue in
controversy in which a neutral is appointed and specified parties
participate;
(7) ``in confidence'' means, with respect to information, that the
information is provided--
(A) with the expressed intent of the source that it not be
disclosed; or
(B) under circumstances that would create the reasonable
expectation on behalf of the source that the information will not be
disclosed;
(8) ``issue in controversy'' means an issue which is material to a
decision concerning an administrative program of an agency, and with
which there is disagreement--
(A) between an agency and persons who would be substantially
affected by the decision; or
(B) between persons who would be substantially affected by the
decision;
(9) ``neutral'' means an individual who, with respect to an issue
in controversy, functions specifically to aid the parties in resolving
the controversy;
(10) ``party'' means--
(A) for a proceeding with named parties, the same as in section
551(3) of this title; and
(B) for a proceeding without named parties, a person who will be
significantly affected by the decision in the proceeding and who
participates in the proceeding;
(11) ``person'' has the same meaning as in section 551(2) of this
title; and
(12) ``roster'' means a list of persons qualified to provide
services as neutrals.
Confidentiality (5 U.S.C.574)
(a) Except as provided in subsections (d) and (e), a neutral in a
dispute resolution proceeding shall not voluntarily disclose or through
discovery or compulsory process be required to disclose any dispute
resolution communication or any communication provided in confidence to
the neutral, unless--
(1) all parties to the dispute resolution proceeding and the
neutral consent in writing, and, if the dispute resolution
communication was provided by a nonparty participant, that participant
also consents in writing;
(2) the dispute resolution communication has already been made
public;
(3) the dispute resolution communication is required by statute to
be made public, but a neutral should make such communication public
only if no other person is reasonably available to disclose the
communication; or
(4) a court determines that such testimony or disclosure is
necessary to--
(A) prevent a manifest injustice;
(B) help establish a violation of law; or
(C) prevent harm to the public health or safety, of sufficient
magnitude in the particular case to outweigh the integrity of dispute
resolution proceedings in general by reducing the confidence of parties
in future cases that their communications will remain confidential.
(b) A party to a dispute resolution proceeding shall not
voluntarily disclose or through discovery or
[[Page 59203]]
compulsory process be required to disclose any dispute resolution
communication, unless--
(1) the communication was prepared by the party seeking disclosure;
(2) all parties to the dispute resolution proceeding consent in
writing;
(3) the dispute resolution communication has already been made
public;
(4) the dispute resolution communication is required by statute to
be made public;
(5) a court determines that such testimony or disclosure is
necessary to--
(A) prevent a manifest injustice;
(B) help establish a violation of law; or
(C) prevent harm to the public health and safety, of sufficient
magnitude in the particular case to outweigh the integrity of dispute
resolution proceedings in general by reducing the confidence of parties
in future cases that their communications will remain confidential;
(6) the dispute resolution communication is relevant to determining
the existence or meaning of an agreement or award that resulted from
the dispute resolution proceeding or to the enforcement of such an
agreement or award; or
(7) except for dispute resolution communications generated by the
neutral, the dispute resolution communication was provided to or was
available to all parties to the dispute resolution proceeding.
(c) Any dispute resolution communication that is disclosed in
violation of subsection (a) or (b), shall not be admissible in any
proceeding relating to the issues in controversy with respect to which
the communication was made.
(d)(1) The parties may agree to alternative confidential procedures
for disclosures by a neutral. Upon such agreement the parties shall
inform the neutral before the commencement of the dispute resolution
proceeding of any modifications to the provisions of subsection (a)
that will govern the confidentiality of the dispute resolution
proceeding. If the parties do not so inform the neutral, subsection (a)
shall apply.
(2) To qualify for the exemption established under subsection (j),
an alternative confidential procedure under this subsection may not
provide for less disclosure than the confidential procedures otherwise
provided under this section.
(e) If a demand for disclosure, by way of discovery request or
other legal process, is made upon a neutral regarding a dispute
resolution communication, the neutral shall make reasonable efforts to
notify the parties and any affected nonparty participants of the
demand. Any party or affected nonparty participant who receives such
notice and within 15 calendar days does not offer to defend a refusal
of the neutral to disclose the requested information shall have waived
any objection to such disclosure.
(f) Nothing in this section shall prevent the discovery or
admissibility of any evidence that is otherwise discoverable, merely
because the evidence was presented in the course of a dispute
resolution proceeding.
(g) Subsections (a) and (b) shall have no effect on the information
and data that are necessary to document an agreement reached or order
issued pursuant to a dispute resolution proceeding.
(h) Subsections (a) and (b) shall not prevent the gathering of
information for research or educational purposes, in cooperation with
other agencies, governmental entities, or dispute resolution programs,
so long as the parties and the specific issues in controversy are not
identifiable.
(i) Subsections (a) and (b) shall not prevent use of a dispute
resolution communication to resolve a dispute between the neutral in a
dispute resolution proceeding and a party to or participant in such
proceeding, so long as such dispute resolution communication is
disclosed only to the extent necessary to resolve such dispute.
(j) A dispute resolution communication which is between a neutral
and a party and which may not be disclosed under this section shall
also be exempt from disclosure under section 552(b)(3).
III. Section-by-Section Analysis of Confidentiality Provisions (5
U.S.C. 574)
Section 574(a)
In general, a neutral in a dispute resolution proceeding is
prohibited from disclosing any dispute resolution communication or any
communication provided to him or her in confidence. Unless the
communication falls within one of the exceptions listed below, the
neutral cannot voluntarily disclose a communication and cannot be
forced to disclose a communication through a discovery request or by
any other compulsory process.
The exceptions to this general rule are found in subsections
574(a)(1)-(4), 574(d) and 574(e).
Section 574(a)(1)
A neutral may disclose a communication if all parties and the
neutral agree in writing to the disclosure. If a nonparty provided the
communication, then the nonparty must also agree in writing to the
disclosure.
Section 574(a)(2)
A neutral may disclose a communication if the communication has
already been made public.
Section 574(a)(3)
A neutral may disclose a communication if there is a statute which
requires it to be made public. However, the neutral should not disclose
the communication unless there is no other person available to make the
disclosure.
Section 574(a)(4)
A neutral may disclose a communication if a court finds that the
neutral's testimony, or the disclosure, is necessary to:
A. prevent a manifest injustice;
B. help establish a violation of law; or
C. prevent harm to the public health and safety.
In order to require disclosure, a court must determine that the
need for disclosure is of sufficient magnitude to outweigh the
detrimental impact on the integrity of dispute resolution proceedings
in general. The need for the information must be so great that it
outweighs a loss of confidence among other potential parties that their
dispute resolution communications will remain confidential in future
proceedings.
Section 574(b)
Unless the communication falls within one of the exceptions listed
below, the party cannot voluntarily disclose a communication and cannot
be forced to disclose a communication through a discovery request or by
any other compulsory process.
Section 574(b)(1)
The party who makes a statement or communication is free to
disclose it.
Section 574(b)(2)
A party may disclose a communication if all the parties agree in
writing to the disclosure.
Section 574(b)(3)
A party may disclose a communication if the communication has
already been made public.
Section 574(b)(4)
A party may disclose a communication if there is a statute which
requires it to be made public.
[[Page 59204]]
Section 574(b)(5)
A party may disclose a communication if a court finds that the
party's testimony, or the disclosure, is necessary to:
A. prevent a manifest injustice;
B. help establish a violation of law; or
C. prevent harm to the public health and safety.
In order to require disclosure, a court must determine that the
need for disclosure is of sufficient magnitude to outweigh the
detrimental impact on the integrity of dispute resolution proceedings
in general. The need for the information must be so great that it
outweighs a loss of confidence among other potential parties that their
dispute resolution communications will remain confidential in future
proceedings.
Section 574(b)(6)
(1) Parties may use dispute resolution communications to show that
a settlement agreement was in fact reached or to show what the terms of
this agreement mean.
(2) Parties may also use communications in connection with later
issues regarding enforcing the agreement.
(3) Communications may only be revealed to the extent that they
meet the above purposes.
Section 574(b)(7)
(1) There is no confidentiality protection for parties' dispute
resolution communications that were available to everyone in the
proceeding. For example, in a joint mediation session with all parties
present, statements made and documents provided by parties are not
confidential.
(2) Communications coming from the neutral are confidential. For
example, early neutral evaluations or settlement proposals from the
neutral are protected.
(3) A party may not use this provision to gain protection for a
communication by providing it to the neutral who then provides it to
the other party.
Section 574(c)
No one may use any dispute resolution communication in a related
proceeding, if that communication was disclosed in violation of Section
574 (a) and (b).
Section 574(d)(1)
(1) Parties may agree to alternative confidentiality procedures to
limit disclosure by a neutral.
(2) Parties must inform the neutral of the alternative procedures
before the dispute resolution proceeding begins.
(3) If parties do not inform the neutral of the alternative
procedures, the procedures outlined in Section 574(a) will apply.
Section 574(d)(2)
(1) Dispute resolution communications covered by alternative
confidentiality procedures may be protected from disclosure under FOIA.
(2) To qualify for this protection, the alternative procedures must
provide for as much, or more, disclosure than the procedures provided
in Section 574.
(3) Dispute resolution communications covered by alternative
confidentiality procedures do not qualify for protection from
disclosure under FOIA if they provide for less disclosure than those
outlined in Section 574.
Section 574(e)
(1) A neutral who receives a demand for disclosure, in the form of
a discovery request or other legal process, must make reasonable
efforts to notify the parties and any affected non-party participants
of the demand.
(2) Parties and non-party participants who receive a notice of a
demand for disclosure from a neutral:
a. must respond within 15 days and offer to defend a refusal to
disclose the information; or
b. if they do not respond within 15 days, will have waived their
objections to disclosure of the information.
Section 574(f)
Evidence that is otherwise discoverable or admissible is not
protected from disclosure under this Section merely because the
evidence was presented during a dispute resolution proceeding.
Section 574(g)
The provisions of Section 574 (a) and (b) do not affect information
and data that are necessary to document agreements or orders resulting
from dispute resolution proceedings.
Section 574(h)
Information from and about dispute resolution proceedings may be
used for educational and research purposes as long as the parties and
specific issues in controversy are not identifiable.
Section 574(i)
(1) Dispute resolution communications may be used to resolve
disputes between the neutral in a dispute resolution proceeding and a
party or non-party participant.
(2) Dispute resolution communications may be disclosed only to the
extent necessary to resolve a dispute between a neutral and party or
non-party participant.
Section 574(j)
A dispute resolution communication between a neutral and a party
that is protected from disclosure under this section is also protected
from disclosure under FOIA (Section 552(b)).
IV. Questions and Answers on Confidentiality under the
Administrative Dispute Resolution Act of 1996 (ADR Act)
General Confidentiality Rules
1. What communications are confidential?
Subject to certain exceptions, the following two types of
communications are potentially confidential under the ADR Act:
A. A dispute resolution communication. A dispute resolution
communication is any oral or written statement made by a party or a
neutral that occurs during a dispute resolution proceeding and any
writing prepared specifically for the purposes of a dispute resolution
proceeding. Written agreements to enter into a dispute resolution
proceeding and any written final agreement reached as a result of the
proceeding are not dispute resolution communications. Citation: 5
U.S.C. 571(5).
B. A ``communication provided in confidence to the neutral.'' A
``communication provided in confidence to the neutral'' is any oral
statement or document provided to a neutral during a dispute resolution
proceeding. The communication must be made: (1) With the express intent
that it not be disclosed, or (2) provided under circumstances that
would create a reasonable expectation that it not be disclosed.
Citation: 5 U.S.C. 571(7) and 574(a).
2. What confidentiality protection is provided for dispute resolution
communications?
Generally, neutrals and parties may not voluntarily disclose or be
compelled to disclose dispute resolution communications. The ADR Act
contains specific exceptions to the general rule. Citation: 5 U.S.C.
574(a), (b).
3. What confidentiality protection applies to a ``communication
provided in confidence'' by a party to a neutral?
A neutral may not disclose any communication provided in
confidence. Citation: 5 U.S.C. 574(a).
[[Page 59205]]
4. What is a dispute resolution proceeding?
A dispute resolution proceeding is any process involving the
services of a neutral that is used to resolve an issue in controversy
arising from an agency's program, operations, or actions. A dispute
resolution proceeding includes any stage of such a dispute resolution
process. Citation: 5 U.S.C. 571(6) and (8). See also, Question 10.
5. Who is a Neutral?
A neutral is anyone who functions specifically to aid the parties
during a dispute resolution process. A neutral may be a private person
or a federal government employee who is acceptable to the parties.
There may be more than one neutral during the course of a dispute
resolution process (e.g., an ``intake'' neutral, a ``convener''
neutral, as well as the neutral who facilitates a face-to-face
proceeding). It is important that agencies clearly identify neutrals to
avoid misunderstanding.
The ADR Act supports a broad reading of the term ``neutral.'' An
intake or convening neutral is included in this definition as ``an
individual who * * * functions specifically to aid the parties in
resolving the controversy'' because such neutrals take the necessary
first steps toward a potential resolution of a dispute.
In situations where an intake neutral is identified by an agency, a
party's willingness to contact and/or work with the intake neutral to
initiate an ADR process is an indication that the intake neutral is
acceptable to the party. Citation: 5 U.S.C 571(9), 571(6), 571(3),
573(a).
Example: An employee contacts an agency ADR program and
describes a dispute to an intake person. The conversation is
confidential only if the intake person has been appropriately
identified as a neutral by the agency to aid parties in resolving
such disputes.
6. Who Is a Party?
A party is any person or entity who participates in a dispute
resolution proceeding and is named in a legal proceeding or will be
affected significantly by the outcome of the proceeding. The
obligations of parties extend to their representatives and agents.
Citation: 5 U.S.C. 571(10).
7. What Constitutes Disclosure?
Disclosure is not defined in the ADR Act. Disclosure occurs when a
neutral, a party, or a non-party participant makes a communication
available to some other person by any method.
8. May a Party or Neutral Disclose Dispute Resolution Communications in
Response to Discovery or Compulsory Process?
In general, neither a neutral nor a party can be required to
disclose dispute resolution communications through discovery or
compulsory process. Compulsory processes include any administrative,
judicial or regulatory process that compels action by an individual.
(See also Question 15) Citation: 5 U.S.C. 574(a) & 574(b).
9. What Confidentiality Protection Is Provided for Communications by a
Nonparty Participant in a Dispute Resolution Proceeding?
A nonparty participant in a dispute resolution proceeding is an
individual other than a party, agent or representative of a party, or
the neutral. This could be an individual who is asked by the neutral to
present information for use of the neutral or parties. A nonparty
participant has an independent right to protect his or her
communications from disclosure by a neutral. A neutral needs to obtain
the consent of all parties and the nonparty participant to disclose
such a communication. Citation: 5 U.S.C. 574(a)(1).
10. When in an ADR Process do the Confidentiality Protections of ADR
Act Apply?
Confidentiality applies to communications when a person seeking ADR
services contacts an appropriate neutral. A communication made by a
party to a neutral is covered even if made prior to a face-to-face ADR
proceeding. Confidentiality does not apply to communications made after
a final written agreement is reached, or after resolution efforts aided
by the neutral have otherwise ended. Citation: 5 U.S.C. 571(6), 574(a)
and (b).
Exceptions to Confidentiality Protection
11. What Communications Are Not Protected by the ADR Act?
A. A party's own communications made during a dispute resolution
proceeding. A party may disclose any oral or written communication
which the party makes or prepares for a dispute resolution proceeding.
Citation: 5 U.S.C. 574(b)(1).
B. A dispute resolution communication that has ``already been made
public.'' The ADR Act's confidentiality protections do not apply to a
communication that has already been made public. Examples of
communications that have ``already been made public'' include:
1. The communication has been discussed in a Congressional hearing;
2. The communication has been posted on the Internet;
3. The communications has been released to the media;
4. The communication has been placed in a court filing or testified
about in a court in a proceeding not under seal;
5. The communication has been reported in the newspapers;
6. The communication has been discussed in an open meeting;
7. The communication has been released under FOIA.
Citation: 5 U.S.C. 574(a)(2) & 574(b)(3).
C. Communications required by statute to be made public. FOIA is an
example of a federal statute which requires agency records to be made
public under certain circumstances. NOTE: A protected dispute
resolution communication which is between a neutral and a party is
exempt from disclosure under FOIA. (See Question 23) Citation: 5 U.S.C.
574(a)(3), 574(b)(4), & 574(j).
D. When a court orders disclosure. A federal court may override the
confidentiality protections of ADR Act in three limited situations. In
order to override the confidentiality protections, a court must
determine that testimony or disclosure of a communication is necessary
to either (1) prevent a manifest injustice, (2) help establish a
violation of law, or (3) prevent harm to the public health or safety.
The court must also determine that the need for the information is of a
sufficient magnitude in the particular case to outweigh the integrity
of dispute resolution proceedings in general by reducing the confidence
of parties in future cases that their communications will remain
confidential. There are no cases as of August 2000 that have
interpreted these provisions. Citation: 5 U.S.C. 574(a)(4) & (b)(5).
E. In order to resolve a dispute over the existence or meaning of a
settlement arrived at through a dispute resolution proceeding. The ADR
Act creates an exception to the general rule of nondisclosure for the
limited purpose of determining the existence or meaning of an agreement
arrived at through a dispute resolution proceeding. Parties may also
disclose communications as required to enforce an agreement arrived at
through a dispute resolution proceeding. Citation: 5 U.S.C. 574(b)(6).
Example: Parties may disclose dispute resolution communications
as required to show that a settlement agreement was reached or to
show what the terms of this agreement were.
[[Page 59206]]
F. Parties' communications in joint session, with all parties
present. A neutral may not disclose communications made in joint
session. However, there is no prohibition against a party disclosing
communications available to everyone in the proceeding. Citation: 5
U.S.C. 574(b)(7).
G. Information sought for specific purposes. The ADR Act allows for
the disclosure of information for educational and research purposes, in
cooperation with agencies, governmental entities, or dispute resolution
programs. It is essential that the parties and specific issues in
controversy not be identifiable, however. Citation: 5 U.S.C. 574(h).
Example: An individual who has served as a neutral in a number
of agency ADR proceedings may share collected experiences when
participating in a training program provided that the parties and
specific issues are not identifiable.
Example: An ADR program administrator may collect statistics to
monitor the results of the program.
H. Communications required to resolve disputes that arise between
the neutral and a party. If there is a dispute between a neutral and a
party regarding the conduct of a dispute resolution proceeding, both
may disclose information to the extent necessary to resolve the
dispute. Citation: 5 U.S.C. 574(i)
Example: If a party refuses to pay the neutral for services, the
neutral can disclose communications to the extent necessary to
establish that payment is due.
12. Are a neutral's communications to parties in joint session or
provided to all parties confidential?
Yes. ADR Act protects communications by a neutral.
Example: Early neutral evaluations or settlement proposals
provided to the parties by a neutral are protected.
Note: A party, however, may not use this provision to gain
protection for a communication by providing it to the neutral who
then provides it to the other party. The statute says that the
communication must be ``generated'' by the neutral, not just passed
along by the neutral. Citation: 5 U.S.C. 574 (b)(7). (See H. Rept.
104-841,142 Cong. Rec. H11108-11 (September 25, 1996).
13. Can confidentiality attach to communications that are provided to
or available to fewer than all of the parties?
Yes. The ADR Act does not prohibit disclosure of dispute resolution
communications that are ``provided to or * * * available to all parties
to the dispute resolution proceeding.'' Under a plain reading of the
statute, communications are not protected when provided to, or
available to, all parties; thus, they remain protected if they are
provided to, or are available to, some (but not all) of the parties in
a dispute.
The legislative history states, ``A dispute resolution
communication originating from a party to a party or parties is not
protected from disclosure by the ADR Act.'' H.R. Rep. No. 104-841, 142
Cong. Rec. H11, 110 (Sept. 25, 1996). The plain language of the statute
is not inconsistent with this piece of legislative history, in that it
can be interpreted to mean both parties in a two-party (``party to the
other party'') or all parties in a multi-party dispute (``party to all
other parties''). Citation: 5 U.S.C. 574(b)(7).
14. Does ADR Act provide confidentiality protection for all evidence
used in the course of a dispute resolution proceeding?
No. All evidence that is otherwise discoverable is not protected
merely because it was presented at a dispute resolution proceeding.
Citation: 5 U.S.C. 574(f).
15. Does the ADR Act protect against the disclosure of dispute
resolution communications in response to requests by federal entities
for such information?
Section 574 of the ADR Act prohibits a neutral or a party from
disclosing, voluntarily or in response to discovery or compulsory
process, any protected communication. The ADR Act further states that
neutrals and parties shall not ``be required'' to disclose such
communications. However, a number of federal entities have statutory
authority to request disclosure of documents from federal agencies and
employees. Examples of such statutes include, but are not limited to,
The Inspector General Act (5 U.S.C. App.); The Whistleblower Protection
Act (5 U.S.C. Section 1212(b)(2)); and the Federal Service Labor-
Management Relations Act (5 U.S.C. Section 7114(4)). None of the
exceptions to the ADR Act's confidentiality provisions directly applies
to requests for disclosure of information from federal entities. For
example, these statutes do not require information to be made public
under ADR Act Section 574 (a)(3) & (b)(4). In addition, the judicial
override procedure outlined in Section 574 (a)(4) & (b)(5) is not
always available to federal entities with authority to access
information. Some federal entities may lack jurisdiction to seek a
court order to compel disclosure. Other federal entities may have such
jurisdiction, but may seek disclosure under other statutory authority.
In summary, a tension between these statutory authorities exists.
The issues of statutory interpretation of these differing authorities
have not yet been considered in an appropriate forum. We do not
anticipate that there will be many occasions when such requests will be
directed to neutrals or participants. However, it is important for
agencies, neutrals and participants to be aware of the potential for
requests.
In order to prevent unnecessary disputes over requests for
information pursuant to an access statute and to mitigate damage to ADR
programs, we recommend:
Agency ADR programs should enter into a dialogue with
potential requesting entities so that each may be educated about their
respective missions.
Procedures should be established for access to information
that recognize the importance of confidentiality in dispute resolution
processes and protect the integrity of the agency's ADR program.
ADR programs should identify classes of information that
are not confidential.
Requesting entities should use non-confidential
information as a basis for information requests.
Requesting entities should seek confidential information
only after other potential sources have been exhausted.
Requesting entities should seek information from a neutral
only as a last resort.
The ADR program and requesting entities should agree to
procedures to resolve specific disagreements that arise with regard to
the disclosure of information.
If a federal employee party or neutral receives a request
for disclosure, he or she should contact the agency's ADR program as
soon as possible to discuss appropriate courses of action. Neutrals
must also notify parties of any such request (See Question 19).
Alternative Procedures To Establish Confidentiality Protection
16. May parties agree to confidentiality procedures which are different
from those contained in ADR Act?
Yes. Parties may agree to more, or less, confidentiality protection
for disclosure by the neutral or themselves than is provided for in the
Act.
Subsection 574(d)(1) provides that the parties can agree to
alternative confidential procedures for disclosures by a neutral. While
there is no parallel provision for parties, the exclusive wording of
this subsection should not be construed as indicating Congressional
intent to limit alternative
[[Page 59207]]
procedures by parties. Parties have a general right to sign
confidentiality agreements, and there is no reason this should change
in a mediation context.
If the parties agree to alternative confidentiality procedures
regarding disclosure by a neutral, they must so inform the neutral
before the dispute resolution proceeding begins or the confidentiality
procedures in the ADR Act will apply. An agreement providing for
alternative confidentiality procedures is binding on anyone who signs
the agreement. (See Questions 23 and 24 for potential FOIA
implications.)
Example: Parties to an ADR proceeding can agree to authorize the
neutral to use his or her judgment about whether to voluntarily
disclose a protected communication, as long as the neutral is
informed of this agreement before the ADR proceeding commences.
Example: Parties to an ADR proceeding can agree that they, and
the neutral, will keep everything they say to each other in joint
session confidential.
Issues Regarding the Disclosure of Protected Communications
17. What restrictions are put on the use of confidential communications
disclosed in violation of the ADR Act?
If the neutral or any participant discloses a confidential
communication in violation of Sections 574(a) or (b), that
communication may not be used in any proceeding that is related to the
subject of the dispute resolution proceeding in which the protected
communication was made. A dispute resolution communication that was
improperly disclosed may not be protected from use in an unrelated
proceeding. Citation: 5 U.S.C. 574(c).
18. What is the penalty for disclosing confidential communications in
violation of the statute?
The ADR Act does not specify any civil or criminal penalty for the
disclosure of a protected communication in violation of the Act.
However, such disclosure may violate other laws, regulations or
agreements of the parties.
19. What must a neutral do when he or she receives a ``demand for
disclosure'' of confidential communications?
A demand for disclosure is a formal request for confidential
information. The demand must be made by a discovery request or some
other legal process. Upon receiving a demand for disclosure of a
confidential communication, a neutral must make a reasonable effort to
notify the parties and any affected non-party participants of the
demand. Notice must be provided even if the neutral believes that there
is no basis for refusing to disclose the communication.
Notice should be delivered to the last address provided by a party.
Parties have fifteen days, from the date they receive the notice, in
which to offer to defend the neutral against disclosure. Therefore,
notice should be sent by a process that provides certification of
delivery. For example, delivery could be by registered mail, by any
carrier that provides tracking and certification of delivery, or by
courier. Use of telephone or email communications as notice could be
problematic. Since the parties must respond within 15 days or waive
their right to object to disclosure, there must be a written record of
when the notice was sent and when it was received. Citation: 5 U.S.C.
574(e).
Example: A colleague asks a neutral what happened in a
mediation. The neutral must simply refuse to discuss the matter. The
neutral does not need to notify the parties of the request.
Example: A neutral receives a formal discovery request for
information on what happened in a mediation. The neutral must notify
the parties of this demand for disclosure using the procedures
described above.
20. What can/must parties do when they receive notice of a demand for
disclosure from the neutral?
If a party has no objection to the disclosure of confidential
communications, it need not respond to the notice. On the other hand,
if a party believes that the sought-after communications should not be
disclosed, it should notify the neutral and make arrangements to defend
the neutral. Where the party is a federal agency, it should develop
departmental procedures for processing the notice.
21. What responsibilities do agencies have for ensuring that the
notification requirement is met?
In some federal ADR programs, the neutral may be a federal employee
performing collateral duty. Imposing an obligation upon these neutrals
to keep records of parties to dispute resolution proceedings may be
unduly onerous and ineffective. Agencies should develop administrative
procedures to assure that the notification functions are fulfilled.
22. May a neutral refuse to disclose communications even when the
parties have failed to agree to defend the neutral?
Yes. The ADR Act permits, but does not compel, a neutral to
disclose if the parties have waived objections to disclosure under
Section 574(e). While the statute is clear that a neutral ``shall not''
disclose where a party objects, the statute does not say that a neutral
must disclose if a party does not object.
The effectiveness and integrity of mediation and other ADR
processes is largely dependent on the credibility and trustworthiness
of neutrals. In order to safeguard the integrity of ADR programs and to
eliminate the potential for eroding confidence in future ADR
proceedings, neutrals should be allowed to rely on established codes of
ethics and confidentiality standards to support a decision not to
disclose. Citation: 5 U.S.C. 574(a) & (e).
Issues Related to the Freedom of Information Act (FOIA)
23. What dispute resolution communications are protected from
disclosure under FOIA?
Dispute resolution communications between a neutral and a party
that are covered by the confidentiality protections of the ADR Act are
specifically exempted from disclosure under the Freedom of Information
Act. This includes communications that are generated by a neutral and
provided to all parties, such as an Early Neutral Evaluation. In
addition, other FOIA exemptions may apply.
Since only federal records are subject to FOIA, dispute resolution
communications that are not federal records are not subject to the
disclosure requirements of FOIA. Therefore, this subsection would not
apply to oral dispute resolution communications. Citation: 5 U.S.C.
574(j).
24. If parties agree to alternative confidentiality procedures, are
dispute resolution communications subject to FOIA?
Parties may agree to confidentiality procedures that differ from
those provided for in the ADR Act. Parties should be aware, however,
that the FOIA exemption may not apply to all the communications
protected under their agreement.
If the agreement provides for the same or more disclosure than
provided by the Act, dispute resolution communications are exempt from
disclosure under FOIA. If the agreement provides for less disclosure,
communications are not exempt from disclosure under FOIA. The ADR Act,
in effect, establishes a ceiling on the extent to which confidential
communications will be exempt. Parties cannot contract for more FOIA
protection than the ADR Act provides.
V. Model Confidentiality Statement for Use by Neutrals
The confidentiality provisions of the Administrative Dispute
Resolution Act
[[Page 59208]]
(ADR Act) apply to this process. Generally, if you tell me something
during this process, I will keep it confidential. The same is true for
written documents you prepare for this process and give to me.
[Similarly, you are generally required to keep information confidential
that you receive during conversations with other parties or me and from
writings prepared for this process. *
---------------------------------------------------------------------------
* Include for multi-party disputes.
---------------------------------------------------------------------------
Be advised, there are limits on our ability to keep information
confidential. If you say something or provide documents to all the
other parties it is not confidential. Under rare circumstances, a judge
can order disclosure of confidential information. Even though not
required by the ADR Act, information about a violation of criminal law,
or an act of fraud, waste, or abuse, or an imminent threat of serious
harm may have to be disclosed to appropriate authorities by a
participant, but not necessarily by me.
You can agree to more confidentiality if you want to. For example,
you can agree to keep confidential things you share with all the
parties. If you want to do any of that, it will require the agreement
of all parties and should be memorialized in writing. You should be
aware that if you agree to more confidentiality, written documents may
still be available to others, for example, through the Freedom of
Information Act. Confidentiality provisions other than those in the ADR
Act may also apply to this process.
ADR Program Evaluation Recommendations
I. Introduction
The alternative dispute resolution (ADR) field has long promoted
the various benefits of using non-traditional methods to resolve
disputes, such as savings of time and money, party satisfaction with
the ADR process and outcomes, high settlement rates, and improved
relationships. The ADR Council recognizes that ADR has the potential to
produce these results, and notes the value of hard data to back up the
assertion that ADR really delivers these benefits to agencies. The
Council's Core Principles for Non-binding Workplace ADR Programs [and
if approved, the ADR Pledge] identify evaluation as a key component of
successful ADR program management. Up-front and thorough evaluation
initiatives allow ADR program managers to ensure the quality of their
programs, to identify programmatic successes and difficulties, and to
make necessary improvements. Therefore, it is important that all
federal ADR programs engage in a rigorous evaluation of ADR's use and
benefits to ensure quality ADR programs and to provide the necessary
information to sustain and increase support of ADR.
As the use of ADR becomes institutionalized within federal
agencies, the government has a heightened interest in evaluating the
benefits and impact of these dispute resolution initiatives. This type
of formal evaluation is consistent with the legal obligations of all
federal programs, under the Government Performance and Results Act
(Pub. L. 103-62) which requires that agencies create a performance
plan, define goals, and track the extent to which they achieve their
desired outcomes. ADR program management best practices emphasize the
importance of an evaluation component in program design as well as
practice, and some federal agencies have initiated evaluations of their
ADR programs. However, the federal sector will benefit from agencies'
coordinated and uniform efforts at ADR program evaluation.
II. Recommendations
The Council acknowledges that throughout the government, ADR
program goals and services differ dramatically among Federal agencies.
Consequently, it is appropriate to tailor evaluation plans and methods
to meet the needs of a particular program. Even with agency-specific
tailoring, effective evaluations will include certain common elements.
Therefore, to promote consistency and coordination among Federal ADR
evaluation efforts, the Council makes the following recommendations to
agencies:
1. Importance of Evaluation. Each agency should engage in an up-
front and ongoing evaluation of its ADR programs.
2. Data to be Captured. At a minimum, evaluators should attempt to
capture and analyze in a timely manner the following information:
a. Usage: the extent to which ADR is considered and used.
b. Time Savings: the time it takes for a case to be resolved
through ADR as compared to traditional dispute resolution processes.
c. Cost Avoidance: the amount of financial savings (or costs) to
the agency, including staff time, dollars, or other quantifiable
factors, by resolving cases through ADR as compared to traditional
dispute resolution processes.
d. Customer Satisfaction: parties' satisfaction with the process
and outcomes, including the quality of the neutral.
e. Improved Relationships: where ongoing relationships are
important, to what extent relationships are improved.
f. Other Appropriate Indicators: in line with the agency's
strategic goals and objectives.
3. Validity and Reliability of Data. Methodologies should be valid
and reliable. ADR program results should be compared to results from
alternate or previously existing dispute resolution methods.
4. Presentation of Data. ADR Program Managers should present a
realistic, accurate and complete picture of the results of their
program.
5. Use of Data. ADR success stories should be summarized and
publicized, to help foster a culture in which ADR is accepted as
beneficial to Federal agencies and their customers. If areas for
improvement are identified, that information should be used to enhance
the ADR program.
6. Reporting. Federal ADR Program Managers are encouraged to report
the results of their evaluations to the Federal Interagency ADR Working
Group.
7. Potential Resources. In undertaking ADR activities, agencies
should consult: (1) The Federal ADR Program Manager's Resource Manual,
Chapter 8: Evaluating ADR Programs, and (2) The Electronic Guide to
Federal Procurement ADR. Both of these resources, as well as other
valuable information are available electronically at:
www.financenet.gov/iadrwg
Evaluating ADR Programs
I. Introduction
For the past ten years the practice of ADR, the creation of ADR
programs, and the discipline of ADR evaluation have been developing in
tandem. We have learned that organizations best design and develop ADR
programs by knowing an organization's conflict resolution culture, we
see that evaluation can and should be a reflective feedback mechanism
for ADR program development, and that evaluation belongs at the
beginning of ADR program design. While evaluation is ideally present at
the beginning of ADR program development, we recognize that there are
many ADR programs already up and running that do not have evaluation
components. This chapter will address ADR programs at any stage along
the way of program development.
II. Planning and Designing the Evaluation
Traditional ADR program evaluation is a way to determine whether an
ADR program is meeting its goals and
[[Page 59209]]
objectives. Evaluation data are useful in finding out what works and
what does not work and may be a critical factor in decisions to modify
or expand a program.
When planning and designing a federal ADR program evaluation, it is
important to understand what components of the program are essential to
comply with federal statutes and initiatives. To the extent that an ADR
program maintains compliance with federal ADR requirements, it fulfills
a necessary and useful function for your organization or agency. A good
design will build upon an existing program structure and will establish
an evaluation methodology for each program ``core'' area, core areas
being defined by statute or initiative. Overall program effectiveness
can then be determined by combining data from all function areas, with
consideration being given to intangible benefits and consumer
satisfaction.
Evaluation is an art as well as a science, even, perhaps, a state
of mind. It is almost never a linear process. Decisions made early in
the evaluation planning and design process will almost certainly need
to be reconsidered and modified as your ADR program grows and develops.
In addition, traditional cost/benefit analysis does not capture many of
the benefits derived from ADR service programs because these benefits
are often intangible and not easily quantifiable. With all of this in
mind, evaluators need to strive for a workable balance between the need
for defensible results and practical limitations.
Key questions to ask when planning and designing an ADR program
evaluation are:
What are your goals and objectives for your ADR program
evaluation?
How will you pay for your ADR evaluation?
Who will evaluate your ADR program?
Who is your audience for this evaluation?
What is your evaluation design strategy?
What are your measures of success?
A. What Are Your Goals and Objectives for Your ADR Program Evaluation?
The goals and objectives of an evaluation should link closely with
the goals and objectives of the ADR program being evaluated, should
reflect the needs and interests of those requesting the evaluation, and
should be sensitive to the needs and interests of the expected
audiences for the results. Ideally, the ADR program's goals and
objectives will have been established early on. Sometimes, however,
these goals may not have been clearly articulated, may not be
measurable as stated, or may have changed. Evaluators may need to ask
program managers and other stakeholders to provide input (and hopefully
arrive at a consensus) on the program's goals, while addressing
questions such as, how well is the program working, should changes be
made, should the program be continued or expanded, and how well is the
ADR program working in a particular federal context?
B. How Will You Pay For Your ADR Evaluation?
The cost of conducting an ADR program evaluation depends upon a
number of factors, such as the number and complexity of success
measures, the type of ADR program selected, the level of statistical
significance required of the results, the availability of acceptable
data, and who is selected to carry out the evaluation. Costs can be
controlled, however, by careful planning, appropriate adjustments in
the design phase, and a creative use of outside evaluators, from
universities, for example.
C. Who Will Evaluate Your ADR Program?
When selecting an evaluator, or a team of evaluators, a number of
qualifications should be considered. Objectivity (i.e. no stake in the
outcome) is essential for your results to be seen as credible. An
evaluator should have sufficient knowledge of the ADR process as well
as program expertise to design the evaluation, perform the data
collection process and data analysis as well as present your results to
your audience if you chose to have the evaluator present your results.
Such expertise may be found inside some agency policy and program
evaluation offices, at the U. S. General Accounting Office, or at
various outside evaluation consulting firms and university departments
specializing in social science research. Some understanding of the
organization or the context in which the program operates can be
helpful to the evaluator, as are good interpersonal and management
skills.
Evaluations can be conducted by people outside the agency, within
the agency but outside the program being evaluated, or by people
involved with the ADR program. There are advantages and disadvantages
to each option. An outside evaluator has the potential for the greatest
impartiality, lending credibility and validity to your results. In
addition, depending upon the expertise available in a particular
agency, an outside evaluator may have more technical knowledge and
experience. Outside evaluation may be relatively expensive, however,
depending upon the affiliation of the evaluators (e.g. colleges or
universities, other non-profit groups, or private sector entities such
as management consulting or social science research firms). If the
agency has evaluation capacity inside the organization where the ADR
program is being implemented, the requisite neutrality may be available
at a potentially lower cost. An inside evaluator involved in ADR
program implementation or design may be the least expensive, and offer
the best understanding of program context, but it also carries with it
potential perceptions of a lack of impartiality. One way to avoid some
of the disadvantages of each of these approaches is to use a team of
people, representing internal and external groups.
Regardless of who does the evaluation (outside or inside), it is
useful to have someone in the ADR program who can serve as a liaison
with the evaluator to ensure access to the necessary information. The
liaison might be the person responsible for planning the evaluation.
D. Who Is Your Audience For This Evaluation?
There are usually a variety of people who have an interest in the
results of a program evaluation. These audiences may be interested in
different issues and seek different types of information. Potential
audiences should be identified as early as possible, and kept in mind
while planning the evaluation, so that their questions will be
addressed.
Possible audiences for an ADR program evaluation include ADR
program officials, other agency officials, program users, members of
Congress, the general public, and others. Agency program officials may
be interested in finding out how the ADR program is working, and how it
might be improved. Their interests might focus, for example, on the
program's impact on case inventory (backlogs), the effects of ADR use
on long-term relationships among disputants, or how well information
about the program is being disseminated. Program officials involved in
the day-to-day operation may have different interests than those at
higher levels.
Other agency officials such as budget officers, staff within
offices of General Counsel and Inspector General, or managers from
other programs may also have an interest in evaluation results. Budget
officials may be interested in
[[Page 59210]]
whether cost savings have been achieved through implementation of the
program. The Inspector General may be interested in the nature of the
settlements and whether ADR use promotes long-term compliance. General
Counsels may care about how long it takes to resolve cases or the
nature of outcomes; other managers may want to know how effectively the
program was implemented.
Members of Congress and their staffs may be interested in how ADR
use affects budgets and how related laws, such as the Administrative
Dispute Resolution Act, are being implemented. Members of the public
may be interested in how efficiently the agency is resolving its
disputes, and how satisfied participants are with ADR processes.
Disputants may be interested in finding out how typical their
experience was compared to other users. Officials in other federal
agencies may find evaluation results helpful as they plan or modify
their own ADR programs. There may be other audiences whose interests or
desire for information should be considered.
Although terminology differs, evaluations are commonly
characterized as either: (1) Program effectiveness (also known as
impact, outcome, or summative) evaluations, which focus on whether a
program is meeting its goals and/or having the desired impact; or (2)
program design and administration (also known as process or formative)
evaluations, which examine how a program is operating. Program
effectiveness evaluations may be useful in determining whether a
program should be continued or expanded; program design/administration
evaluations often focus on how a continuing program can be improved.
Remember that decisions on the future of programs (or even how they
could be improved) are usually not made solely on the basis of program
evaluation results. Agency priorities, other institutional concerns,
budget limitations, and other factors will also affect program
decisions.
While it is not possible to satisfy every audience by answering all
potential questions, it is useful to figure out what the possible
questions are and then focus the evaluation on the most important ones.
Talking to members of the various potential audiences can help identify
the issues they are interested in, and may help develop consensus about
which issues to address. Such discussions also improve the likelihood
that evaluation results will be a useful and meaningful part of future
decision making processes.
E. What Is Your Evaluation Design Strategy?
ADR program design is based on an understanding that certain
components of a program are essential to comply with federal statutes
and initiatives. Program effectiveness evaluations are conducted to
answer fundamental questions about a program's utility, e.g., does the
program provide a necessary or useful function, is the program
accomplishing its goals, and is the program being administered
effectively. A comprehensive evaluation system measures tangible and
intangible benefits, including customer satisfaction, using both
quantitative and qualitative data. To be a useful and effective
management and planning tool, an evaluation system must do more than
provide comparison data. It also must provide a flexible process for
reevaluating the goals of the program, modifying the evaluation
methodology, and implementing necessary changes.
Development of an evaluation design might include the following
steps:
1. Identification and Clarification of ADR Program Goals
Clear goals and objectives mean that useful conclusions can be
drawn from the data collected.
2. Development of an Appropriate Evaluation Methodology
It is necessary to determine what is to be measured and how, what
the sources of the data are, and how the data will be collected. To do
this most effectively, core functional areas of ADR program practice
need to be identified, as do quantitative and qualitative sources of
data.
3. Development of an Analysis Plan and Research Methodologies
Traditionally-based experimental designs (time-cost benefit
analysis) provide statistically reliable results. Program analysis,
while producing quantifiable results, must go beyond a bare assessment
of program outcomes to explain the outcomes and to offer suggestions
for program improvement.
4. Collection Data Mechanisms
Status reports, case studies, time series collections, agency
databases, logs, surveys, and evaluation forms are all sources of
information, as are personal interviews.
F. What Are Your Measures of Success?
1. Program Effectiveness (Impact)
Program effectiveness measures are aimed at assessing the impact of
the program on users/participants, overall mission accomplishment, etc.
The indicators of program effectiveness can be further divided into
three categories: efficiency, effectiveness, and customer satisfaction.
Efficiency
{time} Cost to the Government of using alternative dispute
resolution vs. traditional dispute resolution processes:
Is the use of ADR more or less costly than the use of traditional
means of dispute resolution? (Cost may be measured in staff time,
dollars, or other quantifiable factors.)
{time} Cost to disputants of using alternative dispute resolution
vs. traditional dispute resolution processes:
Is the use of ADR more or less costly than the use of traditional
means of dispute resolution? (Cost may be measured in terms of staff
time, dollars, or other quantifiable factors.)
{time} Time required to resolve disputes using alternative dispute
resolution vs. traditional means of dispute resolution:
Are disputes resolved more or less quickly using ADR, compared to
traditional means of dispute resolution? Such factors as administrative
case processing, participant preparation, dispute resolution activity
timeframes, and/or days to resolution may be considered.
Effectiveness
{time} Dispute Outcomes
Number of settlements achieved through the use of mediation vs.
traditional dispute resolution processes:
Does the use of alternative dispute resolution result in a greater
or a fewer number of settlements?
Number of cases going beyond mediation steps:
Does the use of alternative dispute resolution result in a greater/
fewer number of investigations, further litigation activities, etc.?
Nature of outcomes:
What impact does the use of alternative dispute resolution have on
the nature of outcomes, e.g. do settlement agreements ``look
different''? Do settlement agreements reflect more ``creative''
solutions? Do outcomes vary according to the type of alternative
dispute resolution process used?
Correlations for cases selected for alternative dispute resolution,
between dispute outcomes and such factors as complexity or number of
issues, or number of parties:
Is there any correlation, where ADR is used, between the complexity
and/or number of parties/issues in a case and the outcome of the case?
{time} Durability of Outcomes
Rate of compliance with settlement agreements:
[[Page 59211]]
Does the use of alternative dispute resolution result in greater or
lesser levels of compliance with settlement agreements?
Rate of dispute recurrence:
Does the use of alternative dispute resolution result in greater or
lesser levels of dispute recurrence, i.e. recurrence of disputes among
the same parties?
{time} Impact on Dispute Environment
Size of case inventory:
Does the use of alternative dispute resolution result in an
increase/decrease in case inventory?
Types of disputes:
Does the use of alternative dispute resolution have an impact on
the types of disputes that arise?
Negative impacts:
Does the use of alternative dispute resolution have any negative
consequences, e.g. an inability to diagnose and correct systemic
problem/issues?
Timing of dispute resolution:
Does the use of alternative dispute resolution affect the stage at
which disputes are resolved?
Level at which disputes are resolved:
Does the use of alternative dispute resolution have any impact on
where and by whom disputes are resolved?
Management perceptions:
What are the quantitative and qualitative effects of using
alternative dispute resolution on management, e.g. how does the use of
ADR impact upon allocation and use of management time and resources?
Does the use of ADR ease the job of managing?
Public perceptions:
Is the public satisfied with alternative dispute resolution
outcomes? Is there any perceived impact of use of ADR on effectiveness
of the underlying program? ``Public'' may be defined differently,
depending on the particular program/setting involved.
Customer Satisfaction
{time} Participants' Satisfaction with Process
Participants' perceptions of fairness:
What are participant perceptions of access to alternative dispute
resolution, procedural fairness, fair treatment of parties by neutrals,
etc.?
Participants' perceptions of appropriateness:
What are participant perceptions of appropriateness of matching
decisions (i.e. matching of particular process to particular kinds of
disputes or specific cases)?
Participants' perceptions of usefulness:
What are participant perceptions of the usefulness of alternative
dispute resolution in the generation of settlement options, the
quantity and reliability of information exchanged, etc.?
Participants' perceptions of control over their own decisions:
Do participants feel a greater or lesser degree of control over
dispute resolution process and outcome through the use of alternative
dispute resolution? Is greater control desirable?
{time} Impact on Relationships Between Parties
Nature of relationships among the parties:
Does the use of alternative dispute resolution improve or otherwise
change the parties' perceptions of one another? Is there a decrease or
increase in the level of conflict between the parties? Are the parties
more or less likely to devise ways of dealing with future disputes? Are
the parties able to communicate more directly or effectively at the
conclusion of the ADR process and/or when new problems arise?
{time} Participants' Satisfaction with Outcomes
Participants' satisfaction with outcomes:
Are participants satisfied or unsatisfied with the outcomes of
cases in which alternative dispute resolution has been used?
Participants' willingness to use alternative dispute resolution in
the future:
Would participants elect to use alternative dispute resolution in
future disputes?
2. Program Design and Administration (Structure and Process)
How a program is implemented will have an impact on how effective a
program is in meeting its overall goals. Program design and
administration measures are used to examine this relationship and to
determine how a program can be improved.
The indicators of program design and administration are further
divided into three categories: program organization, service delivery,
and program quality.
Program Organization
{time} Program structure and process:
Are program structure and process consistent with underlying laws,
regulations, executive orders, and/or agency guidance? Do program
structure and process adequately reflect program design? Are program
structure and process adequate to permit appropriate access to and use
of the program?
{time} Directives, guides, and standards:
Do program directives, guides, and standards provide staff/users
with sufficient information to appropriately administer/use the
program?
{time} Delineation of responsibilities:
Does the delineation of staff/user responsibilities reflect program
design? Is the delineation of responsibilities such that it fosters
smooth and effective program operation?
{time} Sufficiency of staff (number/type):
Is the number/type of program staff consistent with program design
and operational needs?
{time} Coordination/working relationships:
Is needed coordination with other relevant internal and external
individuals and organizations taking place? Have effective working
relationships been established to carry out program objectives?
Service Delivery
{time} Access and Procedure
Participant access to alternative dispute resolution:
Are potential participants made aware of the program? Is the
program made available to those interested in using ADR?
Relationship between participant perceptions of access and usage of
alternative dispute resolution:
What impact do participants' perceptions about the availability of
the program have on the levels of program usage?
Participant understanding of procedural requirements:
Do program users understand how the program works? Did they feel
comfortable with the process in advance?
Relationship between procedural understanding and rates of usage:
Is there any relationship between the level of participant
understanding and the degree of program use, e.g. is a lack of
participant understanding serving as a disincentive to using the
program?
{time} Case Selection Criteria
Participants' perceptions of fairness, appropriateness:
Do participants feel that appropriate types of cases are being
handled in the program? Do participants or non-participants feel that
the criteria for which cases are eligible for alternative dispute
resolution are fair? Are cases being sent to the program at the
appropriate dispute stages?
Relationship between dispute outcomes and categories of cases:
Is there a correlation between the nature (size, types of
disputants, and/or stage of the dispute) of cases and the outcome of
the dispute? Are certain types of cases more likely to be resolved
through alternative dispute resolution than other types?
Program Quality
{time} Training
[[Page 59212]]
Participants' perceptions of the appropriateness of staff and user
training:
Do participants feel that they were provided with sufficient
initial information and/or training on how to use the program? Do they
feel that program staff had sufficient training and/or knowledge to
appropriately conduct the program?
Relationship between training variable and dispute outcomes:
Is there a relationship between the type/amount of training (for
participant and/or staff) and dispute outcomes?
{time} Neutrals
Participants' views of the selection process:
Are participants satisfied with the manner in which neutrals were
selected and assigned to cases? Were they involved in the selection
decision? If not, did they feel they should be?
Relationship between participants' views of the selection process,
perceptions of neutral competence and objectivity, and dispute
outcomes:
Is there any relationship between participant views about the
neutrals selection process and dispute outcomes? How do these views
affect participants' assessment of the competence and neutrality of
neutrals?
Participants' perceptions of competence (including appropriateness
of skill levels/training):
Do participants feel that neutrals were sufficiently competent or
trained? Do participants feel that more or less training was needed?
Participants' perceptions of neutrality/objectivity:
Do participants feel that neutrals were sufficiently objective? Do
participants feel that neutrals were fair in their handling of the
dispute?
G. Other Specific Program Features
Every dispute resolution program is unique. Those requesting and/or
conducting an evaluation may want to consider examining other aspects
of the program. These unique features may relate to the design of a
program, who was and continues to be involved in program design and
administration, etc. Each is likely to have at least some impact on
service delivery and the quality of the program, and should be
considered for inclusion in either a comprehensive or selected
evaluation of the program, as appropriate.
II. Presentation, Dissemination, and Use of Results
Results should be communicated in ways that will allow meaningful
decisionmaking by program administrators and decisionmakers.
It is easier to make decisions about the best way to present and
disseminate results if the people who will use the results (the
audience) have been consulted during the initial and subsequent
evaluation processes. Such consultation can avoid costly or
embarrassing errors; e.g., omission of a key area for analysis, and can
ensure the report meets the needs of those who will be using it.
A. What Is the Best Method for Communicating Your Findings?
There are a variety of ways that evaluators can communicate results
to potential audiences. Evaluators or program staff may provide
briefings, hold meetings with users, and/or prepare a written report.
Briefings and presentations allow evaluators or program staff to
convey important evaluation information quickly and selectively. In
selecting material to be presented, care should be taken to avoid bias
or presentation of material out of context. Some discussion of
methodology is important, as are appropriate cautions about the limits
and appropriate use of evaluation data. Providing for interaction with
or feedback from the audience may allow issues and potential problems
to be identified.
Written reports typically take a great deal of time to prepare, but
allow evaluators to provide considerably more detail on both
methodology and results. Legislation or executive decisions often
require a final, written report. If it is important to ensure that
there is one ``official'' source of information on evaluation
methodology and results, a formal, written report may be an important
and/or required format in addition to briefings and presentations by
evaluators or staff.
B. What Kind of Information Needs to Be Communicated?
Although the potential audiences, program content, and evaluation
objectives will vary for each ADR program evaluation, it is generally
helpful to include the following kinds of information in a report or
other type of presentation:
Description of the ADR program and how it operates;
Goals and objectives of the evaluation;
Description of the evaluator's methodology;
Presentation of evaluation findings;
Discussion of program strengths and weaknesses;
Implications for program administration (e.g., training,
budget, staff.); and
Recommendations as appropriate.
Presentation style is entirely a matter of what works for whom. It
is always important, however, to make sure that evaluation data are
presented accurately and completely, to prevent charges of
misrepresentation or overreaching, and to avoid misuse of results.
B. How Can You Enhance the Effectiveness of Your Presentation?
Variations in presentation format and style aside, we offer the
following suggestions for making the presentation of evaluation results
as effective as possible.
Involve potential users as early as possible in
determining presentation format and style:
Evaluation data should be organized and communicated in a way that
is useful for potential audiences and users.
Tailor presentation method, format, and style to audience
needs:
Select the method of presentation (e.g., oral briefing, written
report), format, and style of presentation (e.g., formal vs. informal,
briefing vs. discussion) based on who your audience is and what their
needs are. There may be multiple audiences with multiple needs. Be
flexible and willing to adapt material as appropriate.
Be clear and accurate:
Evaluation information must be presented clearly and accurately.
Always keep the audience in mind as you prepare to describe your ADR
program and present evaluation data. Avoid any gaps in describing the
program or presenting the results. A clear and accurate portrayal of
the program and evaluation results will allow the audience to draw
appropriate conclusions about program effectiveness and any need for
change.
Be honest and direct:
Sharing evaluation findings with potential users and involving them
in key decisions concerning presentation format and style does not mean
publishing only those findings that reflect well on the program or
those affiliated with it. Evaluators must present the story
objectively; too heavy an emphasis on the positive may cast doubt on
the integrity of the results as well as the integrity of the
evaluators. Data that suggest weaknesses in program design or
administration or that reveal failure to accomplish program goals or
objectives should be reported and can be used as a basis for suggesting
appropriate changes. Honest analysis and thoughtful consideration of
the information will enhance both the credibility and usefulness of the
results.
Keep the body of the report or the bulk of the
presentation simple: Reduce
[[Page 59213]]
complex data to understandable form, use graphic illustrations where
appropriate. Evaluation results must be presented so that the most
essential data are available, understandable, and useful. Too complex a
format or over-reliance on narrative may detract from evaluation
results and analysis. Organize the presentation or report for multiple
uses. Use headings and subheadings to help the audience identify useful
information quickly.
Limit the use of technical jargon. Prevent misinterpretation or
misuse by considering how the data will look if lifted from the context
of the presentation or report. Use simple graphics to illustrate
results and call attention to key findings. Use footnotes and make
technical data available in handouts or appendices so that the body of
the presentation or report is as uncomplicated as possible.
Provide an executive summary or abstract:
Evaluators should provide an overview. The ``quick take'' should be
supplemented by more detailed discussion later in the report.
Make survey instruments and other data collection tools
available: Materials can be made available as handouts, at an oral
presentation or face-to-face meeting, or as appendices to a written
report. The availability of such material enhances both understanding
and credibility. It also allows other ADR program evaluators to learn
from the experiences of their peers.
Note limitations on the interpretation and use of
evaluation data, where appropriate: Limitations on the interpretation
of the data, such as those that might relate to the ability to study
results, should be communicated to the audience. Evaluators need to
exercise caution in expressing their own views and conclusions. Where
conclusions are not an objective reflection of the data, they need to
be labeled appropriately; i.e., as the views of the evaluators and not
necessarily of officials responsible for the program.
Expect the need for follow-up; be flexible and responsive:
Have extra copies of reports and presentation handouts available.
Keep materials accessible. Provide addresses and telephone numbers for
follow-up discussion or questions. Be available for consultation. Stay
abreast of how results are being used; provide clarification or added
direction in the case of misinterpretation or misuse. Prepare
additional materials as needed. Tailor subsequent releases to customer
needs.
B. Who Is Responsible for Making Decisions Regarding the Dissemination
of Evaluation Results?
It is important to think about dissemination of the results at two
points: early in the planning process, and again as results become
available. Decisions about dissemination may be made solely by the
evaluator, solely by program officials or other entity that has
requested the evaluation, or, more typically, cooperatively. Such
decisions may be circumscribed by contract or agreement, or may be
discussed and resolved informally by evaluators and decisionmakers.
C. When Should Evaluation Results Be Made Available?
Decisionmakers need to consider the implications of releasing
evaluation results at different times. For example, if you want
publicity for the results, select slower news days. The timing of data
release may be defined by contract or agreement, or may otherwise be
discussed and resolved by evaluators and decisionmakers. Releasing
preliminary data before all data are collected or analyzed may be
risky.
D. How Widely Will Evaluation Results Be Disseminated?
Evaluation results may be disseminated widely or narrowly. Cost,
convenience, and level of interest are likely to play a role. It is
rare that either the evaluator or program officials will have complete
control over dissemination of the results.
E. How Will Evaluation Results Be Disclosed Initially?
Evaluation results can be initially disclosed in different ways,
with more or less fanfare. They may be made available to the selected
audiences by memorandum, by press release, by press conference, etc.
Typically, such decisions will be made at the executive level, by those
who have the authority to make the disclosure.
Evaluation Checklist
Is your ADR program ongoing or in the formative stage?
What are your goals and objectives for your ADR program
evaluation?
How will you pay for your ADR program evaluation?
Who will do the evaluation?
Who is your audience?
What is your evaluation design strategy?
What are your measures of success?
What do you need to know about your program effectiveness
(impact)?
What do you need to know about your program structure and
administration?
How and when will you disseminate your evaluation results?
Resources
Administrative Conference of the United States. (1995). Dispute
Systems Design Working Group. Evaluating ADR Programs: A Handbook
for Federal Agencies. Washington, D.C.: Administrative Conference of
the United States.
Brett, J. M., Barsness, Z. I., & Goldberg, S. B. (1996). The
Effectiveness of Mediation: An Independent Analysis of Cases Handled
by Four Major Service Providers. Negotiation Journal, 12(3), 259-
269.
Costantino, Cathy and Sickles-Merchant, Christine. (1996). Designing
Conflict Management Systems: A Guide to Creating Productive and
Healthy Organizations. Jossey-Bass.
Empowerment Evaluation: http://www.stanford.edu/davidf/
empowermentevaluation.html
Federal Deposit Insurance Corporation. (1999). Checklist for
Evaluation of Federal Agency ADR Programs: Short and Long Term.
Attorney General's ADR Working Group, Workplace Session Notes, 5/18/
99.
Federal Deposit Insurance Corporation. (1997). ADR Program
Evaluation Project, Annual Report.
Galanter, M. (1989). Compared to What? Assessing the Quality of
Dispute Processing. Denver University Law Review, 66(3), xi-xiv.
Honeyman, C. (1990). On Evaluating Mediators. Negotiation Journal,
23-36.
Honeyman, C. (1995). Financing Dispute Resolution. Madison, WI:
Wisconsin Employment Relations Commission.
McEwen, C. A. (1991). Evaluating ADR Programs. In F. E. A. Sander,
Emerging ADR Issues in State and Federal Courts. Washington, D.C.
Patton, Michael. (1990). Qualitative Evaluation and Research
Methods. Sage: Beverly Hills, CA.
Posovac, Emil J. and Raymond B. Carey. (1997). Program Evaluation:
Methods and Case Studies, 5th Edition. Prentice Hall Humanities/
Social Sciences.
Rossi, Peter and Howard Freeman. (1993). Evaluation: A Systematic
Approach. Sage: Beverly Hills, CA.
Scher, E. (1996). Evaluations: What for, by Whom, Who Pays?
Consensus, October 5, 7-8.
Susskind, L. E. (1986). Evaluating Dispute Resolution Experiments.
Negotiation Journal, April, 135-139.
Tyler, T. (1989). The Quality of Dispute Resolution Procedures and
Outcomes. Denver University Law Review, 66, 419-436.
Wholey, Joseph S., Harry P. Hatry, and Kathryn E. Newcomer, Eds.
(1994). Handbook of Practical Program Evaluation. Jossey-Bass.
Worthen, B.R., J.R. Sanders, and J. Fitzpatrick. (1997). Program
Evaluation: Alternative Approaches and Practical Guidelines.
Addison, Wesley, Longman.
[[Page 59214]]
This document was written by Lee Scharf, ADR Specialist at the
Environmental Protection Agency, and draws from the work of Cathy
Costantino and Christine Sickles-Merchant as well as that of the
Administrative Conference of the United States. See the Resources
section for cites.
[FR Doc. 00-25397 Filed 10-3-00; 8:45 am]
BILLING CODE 4410-AR-U