[House Hearing, 110 Congress]
[From the U.S. Government Publishing Office]
EXAMINING THE EFFECTIVENESS OF THE
VETERANS BENEFITS ADMINISTRATION'S
TRAINING, PERFORMANCE MANAGEMENT
AND ACCOUNTABILITY
=======================================================================
HEARING
before the
SUBCOMMITTEE ON DISABILITY ASSISTANCE
AND MEMORIAL AFFAIRS
of the
COMMITTEE ON VETERANS' AFFAIRS
U.S. HOUSE OF REPRESENTATIVES
ONE HUNDRED TENTH CONGRESS
SECOND SESSION
__________
SEPTEMBER 18, 2008
__________
Serial No. 110-105
__________
Printed for the use of the Committee on Veterans' Affairs
U.S. GOVERNMENT PRINTING OFFICE
44-932 WASHINGTON : 2009
----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing
Office Internet: bookstore.gpo.gov Phone: toll free(866) 512-1800; DC
area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC,
Washington, DC 20402-0001
COMMITTEE ON VETERANS' AFFAIRS
BOB FILNER, California, Chairman
CORRINE BROWN, Florida STEVE BUYER, Indiana, Ranking
VIC SNYDER, Arkansas CLIFF STEARNS, Florida
MICHAEL H. MICHAUD, Maine JERRY MORAN, Kansas
STEPHANIE HERSETH SANDLIN, South HENRY E. BROWN, Jr., South
Dakota Carolina
HARRY E. MITCHELL, Arizona JEFF MILLER, Florida
JOHN J. HALL, New York JOHN BOOZMAN, Arkansas
PHIL HARE, Illinois GINNY BROWN-WAITE, Florida
SHELLEY BERKLEY, Nevada MICHAEL R. TURNER, Ohio
JOHN T. SALAZAR, Colorado BRIAN P. BILBRAY, California
CIRO D. RODRIGUEZ, Texas DOUG LAMBORN, Colorado
JOE DONNELLY, Indiana GUS M. BILIRAKIS, Florida
JERRY McNERNEY, California VERN BUCHANAN, Florida
ZACHARY T. SPACE, Ohio STEVE SCALISE, Louisiana
TIMOTHY J. WALZ, Minnesota
DONALD J. CAZAYOUX, Jr., Louisiana
Malcom A. Shorter, Staff Director
______
SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS
JOHN J. HALL, New York, Chairman
CIRO D. RODRIGUEZ, Texas DOUG LAMBORN, Colorado, Ranking
PHIL HARE, Illinois MICHAEL R. TURNER, Ohio
SHELLEY BERKLEY, Nevada GUS M. BILIRAKIS, Florida
Pursuant to clause 2(e)(4) of Rule XI of the Rules of the House, public
hearing records of the Committee on Veterans' Affairs are also
published in electronic form. The printed hearing record remains the
official version. Because electronic submissions are used to prepare
both printed and electronic versions of the hearing record, the process
of converting between various electronic formats may introduce
unintentional errors or omissions. Such occurrences are inherent in the
current publication process and should diminish as the process is
further refined.
C O N T E N T S
__________
September 18, 2008
Page
Examining the Effectiveness of the Veterans Benefits
Administration's Training, Performance Management and
Accountability................................................. 1
OPENING STATEMENTS
Chairman John J. Hall............................................ 1
Prepared statement of Chairman Hall.......................... 31
Hon. Michael R. Turner........................................... 3
Hon. Doug Lamborn, Ranking Republican Member, prepared statement
of............................................................. 32
WITNESSES
U.S. Government Accountability Office, Daniel Bertoni, Director,
Education, Workforce, and Income Security Issues............... 4
Prepared statement of Mr. Bertoni............................ 32
U.S. Department of Veterans Affairs, Michael Walcoff, Deputy
Under Secretary for Benefits, Veterans Benefits Administration. 20
Prepared statement of Mr. Walcoff............................ 68
______
American Federation of Government Employees, AFL-CIO, Michael
Ratajczak, Decision Review Officer, Cleveland Veterans Affairs
Regional Office, Veterans Benefits Administration, U.S.
Department of Veterans......................................... 9
Prepared statement of Mr. Ratajczak.......................... 39
Bartzis, Nicholas T., Cleveland, OH.............................. 15
Prepared statement of Mr. Bartzis............................ 65
Disabled American Veterans, Kerry Baker, Assistant National
Legislative Director........................................... 11
Prepared statement of Mr. Baker.............................. 42
National Veterans Legal Services Program, Ronald B. Abrams, Joint
Executive Director............................................. 13
Prepared statement of Mr. Abrams............................. 47
Human Resources Research Organization, Patricia A. Keenan, Ph.D.,
Program Manager................................................ 14
Prepared statement of Ms. Keenan............................. 61
MATERIAL SUBMITTED FOR THE RECORD
Reports:
Veterans' Benefits: Increased Focus on Evaluation and
Accountability Would Enhance Training and Performance
Management for Claims Processors, Report No. GAO-08-561,
dated May 2008............................................. 71
Post-Hearing Questions and Responses for the Record:
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Michael Ratajczak, Decision Review Officer,
Cleveland Veterans Affairs Regional Office, letter dated
September 25, 2008, and Mr. Ratajczak's responses.......... 108
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Kerry Baker, Assistant National Legislative
Director, Disabled American Veterans, letter dated
September 22, 2008, and Mr. Baker's responses.............. 113
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Ronald B. Abrams, Joint Executive Director,
National Veterans Legal Services Program, letter dated
September 25, 2008, and Mr. Abrams responses............... 114
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Dr. Patricia Keenan, Program Manager, Human
Resources Research Organization, letter dated September 22,
2008, and Dr. Keenan's response............................ 117
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Mr. Michael Walcoff, Deputy Under Secretary for
Benefits, Veterans Benefits Administration, U.S. Department
of Veterans Affairs, letter dated September 22, 2008, and
VA responses............................................... 119
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Bradley Mayes, Director, Compensation and
Pension Service, Veterans Benefits Administration, U.S.
Department of Veterans Affairs, letter dated September 22,
2008, and VA responses..................................... 121
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Dorothy Mackay, Director, Employee Development
and Training, Veterans Benefits Administration, U.S.
Department of Veterans Affairs, letter dated September 25,
2008, and VA responses..................................... 122
EXAMINING THE EFFECTIVENESS OF
THE VETERANS BENEFITS ADMINISTRATION'S
TRAINING, PERFORMANCE MANAGEMENT AND ACCOUNTABILITY
----------
THURSDAY, SEPTEMBER 18, 2008
U.S. House of Representatives,
Committee on Veterans' Affairs,
Subcommittee on Disability Assistance
and Memorial Affairs,
Washington, DC.
The Subcommittee met, pursuant to notice, at 10:06 a.m., in
Room 340, Cannon House Office Building, Hon. John Hall
[Chairman of the Subcommittee] presiding.
Present: Representatives Hall, Lamborn, and Turner.
OPENING STATEMENT OF CHAIRMAN HALL
Mr. Hall. Good morning. The House Committee on Veterans'
Affairs Disability Assistance and Memorial Affairs Subcommittee
hearing on examining the effectiveness of the Veterans Benefits
Administration's (VBA) training, performance management, and
accountability will now come to order.
I would ask everyone to rise for the Pledge of Allegiance.
Flags are located at both ends, actually this end.
[Pledge of Allegiance.]
Mr. Hall. Thank you. And welcome again.
Our Nation's veterans understand the necessity of proper
and adequate training. Their lives have depended on it. The
military trains for its operations and everyone knows every
detail of their job prior to their mission.
These same veterans should be able to expect the same level
of competence when they seek assistance from the Veterans
Benefits Administration. Unfortunately, that is not always the
case as we have heard at other meetings and hearings throughout
the year that this Subcommittee has held regarding the U.S.
Department of Veterans Affairs (VA) disability claims
processing system.
The VA has standardized its training curriculum and
requires that all claims processors must complete 80 hours of
annual training. This is a lot of hours because, in fact, some
healthcare providers do not need to meet that level of
continuing education to maintain their clinical license or
credentialing.
The VBA training topics are identified by the Central
Office (CO) or by the individual's Regional Office (RO). New
employees go through an orientation process and there are
additional online learning tools available through the VBA's
training and performance support system.
Yet, with all this effort, VA training seems to fall short
of its intended goals. Less than 50 percent of the Ratings
Veterans Service Representatives or RVSRs passed the
certification exam even though it was an open book test.
Frankly, I have seen the training manual and it could be
measured in pounds, not pages. So I do not know how useful the
book is, especially given the workload that the people being
trained are already under. But that is sort of the crux of the
matter.
As outlined in previous hearings, there are significant
inconsistencies in ratings between the VA's 57 Regional Offices
and a high rate of remanded cases.
I am pleased that the American Federation of Government
Employees (AFGE) is here to shed light on the issue. You are a
critical link to those on the front lines working to improve
outcomes for our disabled veterans.
The U.S. Government Accountability Office (GAO) released a
report in May 2008 entitled, ``Veterans' Benefits: Increased
Focus on Evaluation and Accountability Would Enhance Training
and Performance Management for Claims Processors.'' That is a
long title, but sums it up.
[The GAO Report No. GAO-08-561, which was attached to Mr.
Bertoni's prepared statement, appears on p. 71.]
The report documented areas in which the VA needs to
improve its training and hold accountable those it does train.
According to the GAO, staff is not held accountable for
completing the required training since the VBA does not track
completion, so there are no consequences for not taking the
training.
Additionally, the VBA does not evaluate its training, so it
does not know if it is successfully designed and implemented in
educational program.
Feedback is not consistently collected from RO employees on
the training that they do receive and many have reported
difficulty in accessing training because of their stringent
productivity demands.
I look forward to hearing more from the GAO about this
report, but these are not surprising conclusions to the
Veterans Service Organizations who have complained for years
about the inadequacies of the VBA training program. So I am
grateful that they have joined us here today as well.
Training is not an issue that should be taken lightly. We
all know the importance of good training, but effective
implementation that ensures consistency and accountability can
be elusive and that is what I hope we can address today.
I have taken steps to ensure improved training outcomes, we
have on this Committee when we passed and the full House passed
H.R. 5892, the ``Veterans Disability Benefits Claims
Modernization Act.''
These policy enhancements will hopefully lead to
compensation claims processing improvements and more accurate
claims adjudication results for our veterans and their
families.
Moreover, I am not sure that the VBA's current training
regimen complements its current claims processing improvement
model or CPI. In fact, I am positive that the current coupling
detracts from increased accountability efforts.
I am pleased to report that with the help of many in this
room, H.R. 5892 passed unanimously on July 31, 2008, by the
full House. On August 1, 2008, Senator Clinton introduced
companion legislation, S. 3419, in the Senate.
So, Congress hopefully is on its way to rectifying the
inadequacies in the VBA training system that have already been
identified.
Today's oversight hearing will allow us to look deeper into
this issue and gauge where VA is in terms of its training
protocol and see what other improvements can still be made.
I look forward to hearing from our witnesses. I hope to
learn more about best practices and strategies for measuring
performance, building better training protocols, and
accountability standards.
[The prepared statement of Chairman Hall appears on p. 31.]
And Mr. Lamborn, our Ranking Member, is not present yet,
but, Mr. Turner, would you like to make an opening statement?
If so, you are recognized.
OPENING STATEMENT OF HON. MICHAEL R. TURNER
Mr. Turner. Sure. Thank you, Chairman Hall.
Thank you for this opportunity for collective discussion on
the effectiveness of the Veterans Benefits Administration's
training, VBA's performance management and accounting
requirements.
Over the course of the past several months, this
Subcommittee has examined nearly every facet of the VA benefits
claims process system in order to identify how we might help
the Department overcome the claims backlog crisis.
While the recent expansion of its workforce will certainly
have a positive impact, VA must ensure that newly hired claims
workers receive training that is commensurate with their
responsibilities. It is critical that the training it provides
meets the needs of the Department and its employees.
It is equally important that the results of the training
are evaluated. Without feedback, VA may never know whether or
not the training is accomplishing its goals.
Any viable training program should be able to identify
deficiencies and demonstrate the intended and actual outcome of
the curriculum. VA training must be connected to its vision and
mission and VA managers need to be assured that if employees
are pulled off the floor for training that it will result in
long-term benefits.
With the growing number of pending claims, there is a
certain level of trepidation that there is too much work to do
already and we will just get further behind if we have to
conduct training.
There must be clear support from the top down in order to
conduct adequate training and acquire the expected outcomes.
Certainly training new employees of everything they need to
know in order to make sound rating decisions is a daunting
task.
The VA rating schedule itself is complex and it is merely a
portion of the array of knowledge a competent adjudicator must
possess to perform his or her job.
Today's hearing is an opportunity to not only learn more
about the training and assessment program VA provides its
employees but also to reiterate to the Department that it
should be forthright about any additional resources deemed
necessary to fulfill this critical requirement.
Mr. Chairman, I thank you for hosting this hearing on VA's
training program and I yield back.
Mr. Hall. Thank you, Mr. Turner.
I would like to remind all panelists that your complete
written statements have been made a part of the hearing record
so that you can limit your remarks to 5 minutes so we can have
sufficient time for follow-up questions once everyone has had
the opportunity to testify.
Our first panel, the entire panel, is Mr. Daniel Bertoni,
Director of Education, Workforce, and Income Security Issues at
the U.S. Government Accountability Office.
Welcome, Mr. Bertoni, and you are now recognized for 5
minutes.
STATEMENT OF DANIEL BERTONI, DIRECTOR, EDUCATION, WORKFORCE,
AND INCOME SECURITY ISSUES, U.S. GOVERNMENT ACCOUNTABILITY
OFFICE
Mr. Bertoni. Mr. Chairman, Members of the Subcommittee,
good morning.
I am pleased to discuss training, performance management
for Department of Veterans Affairs' disability claims
processors.
Last year, VA paid about $38 billion in benefits to nearly
four million veterans and their families.
The disability claims process has chronically suffered from
long waits for decisions, large backlogs, and problems with
accuracy and consistency.
We have also noted that VA's program is in urgent need of
transformation, especially in regard to how it assesses work
capacity and provides interventions and support services to
veterans.
To address its management challenges, VA has hired
thousands of additional staff. However, increased staff alone
will not guarantee more timely, accurate, and consistent
decisions.
Among other things, adequate training and performance
management will be key to developing new staff and ensuring
that more experienced staff maintain needed skills.
My remarks today draw from our prior work for this
Subcommittee and focus on two areas, VA's training program for
disability claims staff and its performance management system.
Summary. Consistent with accepted training practice, VA has
taken steps to align its training with the agency's overall
mission and goals. For example, in 2004, VA established the
Training and Learning Board to ensure that training support of
VA's strategic and business plans.
Various analyses. VA has also identified the skills and
abilities needed by staff and taken steps to determine the
appropriate level of investment in their training and
prioritize funding.
Finally, we found that VA's training program for new staff
defined pertinent terms and concepts and provided many
realistic examples of claims work. However, we did identify
areas for improvement.
While the VA collects feedback to assess initial training
for new staff, not all training is evaluated to determine how
relevant or effective it is. None of the Regional Offices we
visited consistently collected feedback on training they
provided either for new or experienced staff. Thus, VA's
Central Office lacks key information on its entire training
activities.
Both new and experienced staff reported concerns with their
training. Some staff noted that VA's computer-based learning
tool, Training and Performance Support System (TPSS), was often
out of date and too theoretical.
The more experienced staff reported that they struggle to
meet VA's annual 80-hour training requirement due to workload
pressures or the lack of training relevant to their experience
level.
It is unclear what criteria was used to justify the 80-hour
requirement, but identifying the right amount of training is
crucial. Overly burdensome requirements can take staff away
from essential tasks while too little training can contribute
to errors.
Putting aside the appropriateness of the current
requirement, VA has no policies to hold staff accountable for
meeting it and may be missing an opportunity to convey the
importance of training as a means to meet individual goals as
well as broader agency performance goals.
Regarding performance management, VA's system generally
conforms to accepted practices in that individual performance
measures such as quality and productivity are also aligned with
organizational performance measures.
Staff are required to receive regular feedback on
performance and employees and other key stakeholders were
actively involved in developing the current system.
However, we are concerned that under the current rating
formula, VA's system may not make meaningful distinctions in
staff performance. Although VA has a five category rating
system in the field offices we visited, 90 percent of all staff
ended up in only two, fully successful and outstanding as noted
in the chart we provided.
We have reported that four or five category systems are
most useful for making performance distinctions. However, if
staff ratings are clustered in only one or two categories as
with VA, this may be problematic.
Broad, overlapping performance categories may deprive
managers of information needed to reward top performers,
address performance issues, deprive staff of much needed
feedback.
VA has acknowledged potential issues with the rating
formula and is considering some adjustments. However, the
agency has never examined the distribution of all staff across
the five performance categories. Absent this analysis, VA will
lack a clear picture of what adjustments are actually needed.
Conclusion. While VA recognizes the importance of
developing and maintaining high performing staff, it must
devote more attention to training and performance management.
Additional study on such issues as the effectiveness of
regional training, the appropriateness of the current training
requirement, and the usefulness of computer-based learning
tools is needed.
Additional means for holding staff and managers accountable
for completing training should be explored as well as options
for enhancing the current rating system.
We acknowledge VA's efforts to hire more staff. However, in
the longer term, more fundamental changes are needed. VA and
other Federal disability programs must adopt a more modernized
approach to determining eligibility for benefits as well as the
timing and portfolio of services that are provided.
If progress is made in this area, effective training and
performance management will be of crucial importance and will
impact the degree to which service to veterans is ultimately
improved.
Mr. Chairman, this concludes my statement. I am happy to
answer any questions you may have. Thank you.
[The prepared statement of Mr. Bertoni appears on p. 32.]
Mr. Hall. Thank you, Mr. Bertoni, for that most incisive
statement and for your written statement as well, which goes
into considerably more detail, of course.
In your opinion, what is more effective, the classroom
training or online tools?
Mr. Bertoni. I do not know what individual mode of training
is most effective. I think any training program should have a
mix of classroom training, on the job, and online, computer-
based tools.
Agencies should be looking at and evaluating each of those,
call it the three-legged stool, to ensure that they complement
each other, they are consistent in terms of the information
they convey, and they are meeting the needs of staff.
Mr. Hall. What did you think of the VA's learning
management system and training and performance support system?
Mr. Bertoni. Generally we used an in-house criteria to
assess the four components of training system and generally
there are four. Our criteria is based on various external
experts, internal experts at GAO, private-sector individuals,
nonprofits, and academia. And we came up with a criteria that
we use quite often to evaluate training and performance
management.
Generally we break this apart into four categories,
planning, design, implementation, and ultimately evaluation.
And as our report notes, I think VBA and VA are doing an
adequate job in many respects. They have aligned their training
with broader agency goals. They have aligned their performance
management system and with their measuring folks on with their
broader agency goals and mission.
So they are doing, I think, many things well relative to
accepted practices, but still are falling short in terms of in
the area of feedback. Feedback is essentially your evaluation
loop. That is how you find and determine whether your training
is relevant, whether it is effective, and whether it is being
delivered at the appropriate time. To the extent you are not
getting that feedback, I think you are missing an opportunity.
They do a great job or a better job early on for the
initial orientation training, but I think things fall off as
staff return to their home units and begin taking training. And
it is harder for us to get a handle on uniformity, consistency,
and whether it is as effective as it could be.
Mr. Hall. Thank you, Mr. Bertoni.
I understand that proper training is vital to efficiency
and accuracy, but is training the bottom line problem at VBA?
Would you correlate the problems inherent in the backlog as an
issue with training?
Mr. Bertoni. It is hard to isolate all the factors that
contribute to the backlog. There are laws, regulation that have
contributed. There are management inefficiencies that have
contributed. But training certainly is a tool to get at or to
address many of the problems that agencies face. We would not
do it if we did not think it was worthy.
And clearly when you have good training, you can see
results. Whether you can, you know, measure clear cause and
effect, that is probably not something that you can always do.
But generally a good, solid designed training system is
effective.
Beyond training, I do really believe that part and parcel
with that is a good quality assurance process. Until you know
where your soft spots are, where your problems are through your
quality assurance reviews, it is very hard to design training
that is going to really target and get at the problematic areas
in your process.
We have gone many years with timeliness, accuracy, and
consistency issues, and I have to believe based on some work
that we have done looking at the STAR system and other quality
assurance processes that having more robust quality assurance
could help VBA, VA identify some of the real root causes for
inconsistency and inaccuracies.
And I think they really need to focus on that. And it
appears from the statement I read from VA this morning they are
heading in that direction.
Mr. Hall. In reviewing VBA's performance management system,
how different was it from other Federal agencies? Three-part
question here. Are there other agencies with a better
performance management system that VA could adopt and would it
make sense to give performance credits for training as well as
for work completion so that there is not pressure to stay on
the workload and not train?
Mr. Bertoni. I do not know if it is a matter of giving
credits for workload completion. I believe their rating should
reflect the fact that they are doing a good job or a great job
in terms of processing the workloads.
Our concern with their performance management system is
that it may not be rewarding. It may, in fact, be potentially
demoralizing for some staff.
I will give you an example. We'll use three people. You
take the first person who is clearly a high flyer who rates
exceptional across all the critical and noncritical dimensions.
That person, it is a no-brainer. That person would be listed in
the outstanding category.
We will take another person who is just barely at the fully
successful level and, you know, just barely eking out a fully
successful rating, marginally falling below that, but at the
end of the performance year ends up with a fully successful
rating.
Take a third person who is clearly also a high performer.
That person gets exceptional ratings on all dimensions,
critical and noncritical. With the exception of one, that
person gets a fully successful. That individual will be dumped
into the bucket with that fully successful person.
Clearly that person falls somewhere between exceptional and
outstanding, but because of that one fully successful in a
critical dimension are dropping down into that lower bucket and
all the incentives, the pay incentives, the other monetary
incentives are not available.
So if you are trying to create a world-class organization
and incentivize people for hard work, this example really
disturbs us. We see in the four regions that we visited 90
percent of all folks in two categories and nobody in that
second excellent category. It gives me pause.
Mr. Hall. Thank you, Mr. Bertoni.
My time is expired. I will now recognize and welcome our
Ranking Member, the gentleman from Colorado, Mr. Lamborn.
Mr. Lamborn. Yeah. And thank you, Mr. Chairman.
Mr. Bertoni, you apparently have some familiarity with
other departments other than the VA because of looking at
training programs in other parts of the Federal Government.
What are some successful components that you have seen in
some of these other areas that you would suggest the VA should
imitate?
Mr. Bertoni. Again, I will hearken back to what we
identified in our report. I think you really need to foster a
rigorous system for feedback and evaluation. If you are
focusing on new employees at the initial orientation level and
getting good information there, fine. We acknowledge that.
But when these folks transfer back or go out into their
home units and are taking this, you know, training at the
regional level, there is no feedback loop that is feeding back
into VBA's headquarters so they can use this information to
identify areas where they are falling short and ultimately to
devise training to get at some of the problems that we have
talked about in the area of accuracy and consistency.
World-class organizations have completed that wheel,
planning, design, implemented, and your evaluation loop. To the
extent that all those elements are firing, you have better
systems. And there are some organizations that do better than
others.
Mr. Lamborn. Thank you.
Mr. Chairman, I yield back.
Mr. Hall. Thank you, Mr. Lamborn.
And one more question, Mr. Bertoni. In 2001, the GAO issued
a report ``Veterans Benefits Training for Claims Processors
Needs Evaluation.'' At that time, the GAO found that the TPSS
was being implemented differently in each Regional Office. I
know that the VBA has mandated some of the training, but
employees have raised concerns about the training and claim
limited time for it because of workload demands.
So what differences do you see in the VA's training process
from that time, from 2001 to the present?
Mr. Bertoni. We really did not delve into the TPSS system.
We had the prior report. We followed up our recommendations.
I know that VA has taken steps to monitor and approve that
system. I think there is an ongoing effort and I think almost a
report to be issued shortly also. But we really did not delve
into the system exclusively.
We were only able to sort of give a snapshot of what was
going on in the four Regional Offices and there was clearly
some noise that the TPSS system was not all it could be for
some folks in terms of the timeliness of the information in
there, the relevance, especially in regard to the collection of
medical evidence. I believe that came up frequently.
But this was not a look at TPSS. It was sort of a broad-
based review of all the elements and tools that are there for
staff to use.
Mr. Hall. Okay. Well, that is all the questions I have. Mr.
Lamborn, if you are done, I would thank you, Mr. Bertoni, again
for your testimony. And your panel is now excused.
Mr. Bertoni. Thank you.
Mr. Hall. And thank you so much.
And joining us now on our second panel, we would like to
welcome Mr. Michael Ratajczak--is that correct----
Mr. Ratajczak. Ratajczak. Thank you.
Mr. HALL [continuing]. Decision Review Officer at the
Cleveland Veterans Affairs Regional Office on behalf of the
American Federation of Government Employees; Mr. Kerry Baker,
Assistant National Legislative Director of the Disabled
American Veterans (DAV); Ronald Abrams, Joint Executive
Director of the National Veterans Legal Services Program
(NVLSP); Dr. Patricia Keenan, Manager of Employee Performance
Enhancement and Growth Program at the Human Resources Research
Organization (HumRRO); and Mr. Nick Bartzis, a veteran from
Cleveland, Ohio.
Welcome, all, and your statements, written statements are
in the record so that we will allow you to deviate and ask you
to stay within the 5 minutes so we have time for questions for
everybody.
Tell me one more time. Ratajczak.
Mr. Ratajczak. Ratajczak.
Mr. Hall. Ratajczak. Mr. Ratajczak, you are now recognized
for 5 minutes.
STATEMENTS OF MICHAEL RATAJCZAK, DECISION REVIEW OFFICER,
CLEVELAND VETERANS AFFAIRS REGIONAL OFFICE, VETERANS BENEFITS
ADMINISTRATION, U.S. DEPARTMENT OF VETERANS, ON BEHALF OF
AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES, AFL-CIO; KERRY
BAKER, ASSISTANT NATIONAL LEGISLATIVE DIRECTOR, DISABLED
AMERICAN VETERANS; RONALD B. ABRAMS, JOINT EXECUTIVE DIRECTOR,
NATIONAL VETERANS LEGAL SERVICES PROGRAM; PATRICIA A. KEENAN,
PH.D., PROGRAM MANAGER, HUMAN RESOURCES RESEARCH ORGANIZATION
(HUMRRO); AND NICHOLAS T. BARTZIS, CLEVELAND, OH (VETERAN)
STATEMENT OF MICHAEL RATAJCZAK
Mr. Ratajczak. Thank you.
I think I have as much practical experience with regard to
training issues at the VBA as anyone in the room. I served as
centralized challenge instructor. I worked on the certification
design Committees. I have identified trends in errors by
interacting with Board of Veterans' Appeals (BVA) personnel. I
have taken some remedial action to fix errors resulting from
what ultimately is improper and effective training.
As a Decision Review Officer every day, I kind of am
serving as a forensic pathologist where I look at a case where
a mistake was made and trying to assign a reason for that
mistake. When I have done that more often than not, I will
effectively as possible take the case to the person who may
have made the mistake and say this is what you did, let us try
and avoid that in the future.
More often than not as an explanation for their actions,
the folks who made an error will attribute the error to the
fact that, well, I just did not know that. I was not trained. I
do not even know where to find that information.
To the extent that errors are made, it is obviously a
reflection of improper training on behalf of the VBA.
And what I would like to submit is that the folks who are
making these errors, the front-line claims adjudicators,
veterans representatives, specialists, decision officers, they
have an interest in effective training. They do not want to
make mistakes because as we all do, they have service to
veterans at heart ultimately.
But beyond that, if they are making mistakes, adverse
employment action can be taken against them. They can be put on
a production improvement plan. So they have a very real
interest in effective training and participating in effective
training.
To the extent that that opportunity for effective training
is not presented to them, I think we can do better.
I was struck by the testimony in previous hearings before
this Subcommittee and others regarding complexity, increasing
complexity of the process, judicial review, explanations for
why there is a backlog in the VBA.
And all that is true. VBA no longer operates in splendid
isolation for judicial review, so the court comes back and
looks at VBA's processes and says, well, this is what you need
to do, this is what you are not doing correctly.
Those kind of decisions should be viewed more as an
opportunity than as an impediment because we can take those
decisions, take the reasoning in them, and identify things they
need to be trained on so we avoid the mistakes in the future.
Recognizing that there is a component to the backlog which
is attributable to complexity in legal review, VBA, I do not
think, has made the same leap to the fact that those same
elements have a detrimental impact on an individual employee's
ability to do their job.
Obviously if a given claim is more complex and there are
additional legal requirements placed on the Veterans Service
Representatives (VSRs) and Rating Specialist, it is going to
take them longer to do their job and maybe what we could do is
design some effective training to make them more efficient in
doing their job in the face of increasing complexity and
increasing legal requirements. I am going to suggest briefly a
couple of methods of doing that based on my experiences.
I just want to note that I did read the GAO report. I think
overall it is accurate, but I want to caution anybody who will
listen to the extent that it may imply that individual claims
processors are responsible for improper training, that cannot
be the case. They do not have the ability to go to their
management and say, hey, I need more training. You need to do
that so I can do my job better. Frankly, that is someone else's
job to do that training and give them the opportunity for it.
Several actions can be taken to include training. Number
one, centralize curriculum from Central Office implemented by
instructors that are responsible to Central Office. That would
provide a uniform voice to all Regional Offices. This is CO's
policy.
To the extent that adjudicators' experience will be
informed by that uniform training, you would expect to get some
uniformity and some more consistency in the decisions.
Also, I think Veterans Service Representatives, Rating
Specialists in particular, should be given work credit for
doing deferred actions where they have direct additional
development in a claim to the person who is responsible for
developing the evidence. That would provide feedback to people
developing the evidence and also an incentive to the Rating
Specialists.
Thank you.
[The prepared statement of Mr. Ratajczak appears on p. 39.]
Mr. Hall. Thank you, Mr. Ratajczak.
Mr. Baker, now you are recognized for your statement.
STATEMENT OF KERRY BAKER
Mr. Baker. Mr. Chairman and Members of the Subcommittee, on
behalf of the DAV, I am pleased to offer my testimony to
address VA's training, accountability, and performance
measurement system.
VA has a standard training curriculum for new claims
processors and an 80-hour annual training requirement for all
claims processors.
DAV has long maintained that the VA should invest more in
training and hold employees accountable for higher standards of
accuracy.
VA's problems caused by a lack of accountability do not
begin in the development and the rating process and in the
training program of which we can find little, if any,
measurable accountability.
For example, some employees inform DAV that many candidates
begin centralized training before they complete or even start
phase one training. Candidates are then not held responsible
via formal testing on subjects before advancing to phase two
training.
As in phase one, VA refuses to test phase two trainees.
Without such testing, VA cannot gauge the success of its
learning objectives.
During phase three training, employees work on real world
cases. That notwithstanding, no accountability, no testing, and
no oversight other than that provided locally. That oversight
is not measured nationally.
The result of such an unsupervised and unaccountable
training system is that no distinction exists between
unsatisfactory performance and outstanding performance. Lack of
accountability during training eliminates employee motivation
to excel.
The need for improvements in VA's training program became
evident when it began the skills certification test for VSRs.
The first two attempts at the test produced a 25- and a 29-
percent pass rate. The third produced a 42-percent pass rate
but only after employees completed a 20-hour prep course to
pass the test.
Mandatory, comprehensive testing designed cumulatively one
subject area to the next for which VA then holds trainees
accountable should be the number one priority of any plan to
improve VA's training program.
Further, VA should not allow trainees to advance to
subsequent stages of training until they have successfully
completed such testing.
DAV has also long stated that in addition to training,
accountability is the key to quality. Therefore, timeliness.
We believe VA's quality assurance or STAR Program is
severely inadequate. In 2006, the STAR Program reviewed just
over 11,000 compensation and pension (C&P) claims for improper
payments. There was only .72 percent of the total number of
cases available for review. However, a more reliable measure of
VA's accuracy and its lack of accountability is shown through
the Board of Veterans' Appeals' annual report.
Fiscal year 2007, the Board decided approximately 40,000
appeals. The Board reversed 21 percent of those appeals and
remanded over 35 percent. Of those remands, over 7,000 of those
cases or nearly 20 percent were remanded because of errors in
the most basic of procedures, procedures that should be
elementary to every single decision-maker.
The problem is compounded when one considers that those
errors cleared the local Board, local Rating Board and the
local Appeals Board which contain VA's most senior Rating
Specialists. These facts clearly show little incentive for many
employees to concern themselves with the quality of their work.
Congress should require the Secretary to report on how the
Department could establish a quality assurance and
accountability program that will detect, track, and hold
responsible those employees who commit errors.
We believe that effective accountability can be engineered
in a manner that would hold each VA employee responsible for
his or her work as a claim moves through the system while
simultaneously holding all employees responsible.
If such errors are discovered, employees responsible for
such errors should forfeit a portion of their work credit. This
idea is discussed more in my written testimony, but I would be
happy to explain further if the Committee has questions.
I believe this type of accountability system will draw on
the strengths of VA's performance measurement system, thereby
allowing easier and less disruptive implementation of stronger
and more effective accountability.
Mr. Chairman, that concludes my statement, and I will be
happy to try to answer any questions you might have.
[The prepared statement of Mr. Baker appears on p. 42.]
Mr. Hall. Thank you, Mr. Baker.
Now, Mr. Abrams, you are recognized for 5 minutes.
STATEMENT OF RONALD B. ABRAMS
Mr. Abrams. Good morning, Mr. Chairman and Members of the
Committee. I am pleased to have the opportunity to submit this
testimony on behalf of NVLSP.
As some of you know, we have trained thousands of service
officers and lawyers in veterans' benefits law. We have written
books, textbooks, and guides on veterans' benefits and we help
people represent veterans seeking VA benefits.
We also conduct quality reviews of the VA Regional Offices
on behalf of the American Legion. We believe after reviewing
all of this that the effectiveness of VBA training should be
measured by the quality of the work product produced by VA
adjudicators.
Therefore, the quality and timeliness of VA adjudication
should reflect the effectiveness or lack thereof of VA
training. Because the quality of VA adjudications is in our
opinion inadequate, NVLSP must conclude that VBA training is
not effective or adequate.
In the experience of NVLSP over 10 years of quality checks,
over 40 VA Regional Offices combined with extensive
representation before the Veterans Court, we find that most of
the worst VA errors are the result of premature Regional Office
decisions.
We find that many VA managers emphasize quantity over
quality, making many aspects of training not relevant. That is
because VA workers want to please the people they work for.
Training is de-emphasized because production is paramount.
VA employees consistently let us know that they are told
that time spent training reduces the time available to produce
decisions. Also training VA workers regarding the procedures
designed to protect the rights of claimants, developing the
claim fully, reviewing the entire file instead of top sheeting,
which is just looking at the first couple of pages--all takes
time. The essence of fairness in this particular process is
giving the VA worker the time to do the job.
In fact, in September 2008, courageous VA Regional Office
employees filed a grievance exposing this overemphasis on
production. That grievance is attached to my testimony. The
grievance asserts that the Regional Office created and
encouraged a culture in which quantity is emphasized to the
detriment of quality, that the RO failed to properly implement
training to assure that those reviewing claims are sufficiently
trained and that the RO failed to properly implement a fair and
impartial performance appraisal system that assures that
quantitative measures of performance are not emphasized to the
detriment of measures of quality.
These VA workers allege they were told to produce cases. It
did do not care about the quality of their work or it did not
care that much. And we think that the quality of the VA work is
much worse than what is reported by the VA.
As Kerry testified, the BVA statistics are scary. If over
50 percent of what goes up to the Board of Veterans' Appeals
has to be returned because the VA in general failed to follow
proper process, we have a problem. That is the best quality
check possible.
I want to thank you for permitting NVLSP to testify on this
important issue. I will be happy to take any questions.
[The prepared statement of Mr. Abrams appears on p. 47.]
Mr. Hall. Thank you, Mr. Abrams. We will have some
questions for you.
Dr. Keenan, you are now recognized.
STATEMENT OF PATRICIA A. KEENAN, PH.D.
Dr. Keenan. Good morning, Mr. Chairman and Members of the
Subcommittee. I am Patricia Keenan, a Program Manager at the
Human Resources Research Organization or less formerly and more
shortly is HumRRO.
HumRRO is a nonprofit 501(c)(3) research and development
organization established in 1951 that works with Government
agencies and other organizations to improve their effectiveness
through improved human capital development and management.
Our comment today is about the compensation and pension
services training program as well as on methods to reduce
training variance.
I am the project leader for HumRRO's work with VBA skills
certification programs. HumRRO has worked closely with C&P
service on skill cert programs for Veterans Service
Representatives or VSRs since 2001.
We have seen passing rates on the VSR test rise steadily
since the beginning indicating to us that C&P's training
initiatives are having a positive effect.
As part of our work, we conducted numerous focus groups and
interviews at Regional Offices. As a result, we identified
several factors that decrease rating consistency.
First, the RVSR job is very cognitively demanding. It
requires knowledge of medical conditions, regulations, and the
rating schedule, attention to detail, decisiveness, and strong
analytical ability. These requirements exacerbate the other
challenges RVSRs face.
The second factor we identified is the problem of
incomplete information in the rating schedule. While the
schedule contains over 700 diagnostic codes, by comparison, the
international classification of diseases by the medical
profession contains thousands of codes.
Many areas of the rating schedule leave room for
interpretation and RVSRs develop individual rules for matching
the medical evidence to the schedule. This allows them to
reduce the cognitive load and work more quickly, but it also is
a source of rating variance.
The rating schedule does not have diagnostic codes for some
conditions such as Parkinson's disease or carpal tunnel
syndrome. When a claim includes an unlisted condition, the RVSR
rates it by analogy to a closely related disease or injury. By
their nature, these analogous codes lack criteria for rating,
so raters have to research different conditions to make the
evaluation that may not be very straightforward.
The third challenge to the RVSR is related to workload. The
sheer volume of cases awaiting adjudication sometimes results
in a lack of attention to detail. One way this shows itself is
in cases that are not ready to rate and require additional
development. This means the RVSR has spent time reviewing a
case, but the decision must wait for new information and the
case must be reviewed a second time.
The pressure also shows itself in rating decisions that
rely too heavily on the use of templates rather than clearly
establishing the connection between the medical evidence
presented and the regulations in a way that is easily
understandable.
It is very important that veterans feel confident that
their cases have been fairly evaluated.
VBA's recent wave of hiring was the best long-term response
to the influx of claims. However, one result has been that
newly hired RVSRs do not understand the development process
well.
Although the workload of the RVSR is not going to lighten
in the near future, there are some actions that could help
improve rating consistency.
First, to address one of the workload issues, newly hired
RVSRs should work predetermination for several weeks to learn
the system. The obvious drawback to this is that it would take
longer for new RVSRs to begin rating cases. But we believe
having this additional knowledge would pay off in the longer
term.
Second, job aids that include more specific medical
information and rating schedule would ease the cognitive
complexity of the job and reduce the individual interpretation
of the ratings. However, job aids cannot help with the complex
problem of writing good rating decisions.
One long-term solution to this problem would be to develop
a selection tool that assesses an applicant's ability to
synthesize information and present it in a well-structured,
easily comprehensible document.
Finally, VBA should ensure that all RVSRs continue to
receive standardized training.
It has been HumRRO's pleasure to work with the C&P service
for the past 7 years. We are honored to be even a small part of
the valuable work the Veterans Benefits Administration does for
America's veterans. We have watched both the skill
certification program and C&P service over this time. The
effort devoted to training has been steadily improving pass
rates on the test.
Beyond training, we have identified several areas that
could further increase rating consistency. Addressing the
problems of cognitive complexities, ambiguity in the rating
schedule, and workload is something VBA should continue.
Thank you for this opportunity.
[The prepared statement of Ms. Keenan appears on p. 61.]
Mr. Hall. Thank you, Dr. Keenan.
Mr. Bartzis, you are now recognized for 5 minutes.
STATEMENT OF NICHOLAS T. BARTZIS
Mr. Bartzis. Thank you, sir.
Thank you for the invitation to speak here today as a
veteran and I am here today as a private citizen. However, I
have a few statements beyond my written statement to go
through. Most of them are in the form of----
I raise my right hand approximately 20 inches. I am going
to show approximately a half an inch with my left hand. If we
determine we have two cases, the cases make the mistake, make
the same mistake on the same issue, how long is it going to
take to go through all of the medical evidence on the case on
the right as opposed to the case on the left? The answer to
that question to me or the question itself is how much time is
it going to take?
But what I am really talking about is time. If I have to go
through 20 inches of medical evidence or if I have to go
through \1/4\ inch or \1/2\ inch of medical evidence, I am
going to spend a significantly different amount of time to find
the error.
It has been stated earlier that part of the problem is the
development issue, but the actual issue is how much time does
it take and who is going to do it. That is a question I believe
the Committee should ask.
The second issue is accountability. Who is going to do it
and when and where is a failure, if any?
The rest of the statements are in my written statement, and
I will defer to that. Thank you very much.
[The prepared statement of Mr. Bartzis appears on p. 65.]
Mr. Hall. Thank you, Mr. Bartzis.
I will recognize myself now for a round of questions.
Mr. Ratajczak, what happens when RO staff cannot complete
the 80 hours of training?
Mr. Ratajczak. To the best of my knowledge, nothing
happens. I do not think there is any detrimental effect on the
individual adjudicator if they have not concluded their
training requirements. I suppose that there would be some
method in place for them to do some remedial training. But to
the extent that they are not held accountable for ensuring the
training to begin with, nothing happens.
As far as a practical matter, what happens is they do not
get enough training and obviously we get poor decisions and
increasing appeals.
Mr. Hall. Dr. Keenan, you spoke of new hires having an
initial period of, if I understood you correctly, of training
before they start working on cases.
What amount of time do you recommend and is that something
that you have seen succeed in other similar situations in other
agencies?
Dr. Keenan. For them to work predetermination before they
start rating cases?
Mr. Hall. Right. To just have a concentrated training
period before they start rating cases.
Dr. Keenan. Well, right now they do have a 3-month training
period before they start rating, but they do not have enough
time to learn all the intricacies of predetermination which is
all the evidence gathering that is required for the RVSR then
to be able to make ratings, solid ratings. So if they could
spend an extra 6 or 8 weeks doing pre-d, just learning the
whole claims process.
What we heard in our visits to the ROs was that the people
who are promoted internally to RVSRs, it is a road running much
more than people who are hired from the outside.
Mr. Hall. Yes.
Dr. Keenan. And that makes perfect sense. It is a very
complicated process. There is a lot to learn in PR and when you
are doing a rating decision, you have to understand, you have
to be able to look at the evidence and know that you have
enough there to make your decision.
If the VSR has not developed all of that information and
you do not know this and you spend time going through your case
and then have to defer it, as Michael mentioned before, so if
they could spend more time once they have gone through their
initial training and then go through pre-d, spend some time,
they will still be working cases in pre-d, so they are helping
in that sense getting things ready to rate, they will have a
much better sense of the whole process.
Mr. Hall. So rather than 3 months, you suggest 4 or 5
months, in effect? An additional 6 to 8 weeks, you said?
Dr. Keenan. Yes.
Mr. Hall. Why do you think Dr. Keenan, so many VBA
employees had a problem in passing the open book test that you
developed?
Dr. Keenan. When VBA first told us they wanted an open book
test, we were totally dumfounded. We do a lot of certification
work and it is always closed book. You come in on a Saturday.
You take the test.
But once we realized the complexity of the job and the fact
that people really cannot do their job just with their memory,
it is just too complicated, there are too many rules, too much
money, too many regulations, they have to have training aids,
job aids, you know, regulations available to them so that they
can make these decisions.
The tests are fairly applied, so it is not just a matter of
recall which screen you use for that, although those are
important to know because it makes you work more efficiently.
The----
Mr. Hall. So you think they need more time to take the
test?
Dr. Keenan [continuing]. Well, we actually just had a
meeting about that yesterday. We are going to add----
Mr. Hall. Because it is a learning experience? The test
itself?
Dr. Keenan [continuing]. We want them to learn while they
are testing.
Mr. Hall. Right.
Dr. Keenan. We hope that they will come in knowing that.
And we had originally set the time limit. It is a two part
test, half in the morning and half in the afternoon, 2 hours
and 45 minutes in the morning and 2 and 45 in the afternoon. We
are going to expand that to a full 3 hours both times.
Mr. Hall. Thank you. I am sorry to have to interrupt you,
but I only have a little bit of time left in this round.
Mr. Baker, in your testimony, you mentioned that the VBA
does not test participants after each phase of training.
Do you think it would be better to test at these intervals
instead of the certification exam or in addition to the
certification exam?
Mr. Baker. Well, if I may clarify. It sounded like I was
pretty rough on VA employees there and I want to make sure that
you know I was not aiming at VA employees. It was the system
that I would like to see improved.
But to answer your question, I think it should be both,
absolutely testing during the training phases, cumulative
testing, so that you are responsible for what you learned in
the previous phase and the current phase and so on and so on.
I agree with the open book because my opinion this is more
of a legal system than anything else and as complex it is,
people need to know where to find it. And I would go as far as
to have those resources cited in the answers.
But if there was some incentive to do that, I think you
would end up with a lot better training right up front.
Mr. Hall. With Mr. Lamborn's patience and agreement, I am
going to just go on and ask a couple more questions even though
the red light is on.
Mr. Abrams, what steps would you recommend that VA follow
to improve their training program and, in particular, should
VBA training be contracted out to lawyers and doctors who
specialize in disability compensation cases?
Mr. Abrams. The first thing that we would recommend is that
we make upper management accountable by evaluating the BVA
reversal and remand rate. We are not confident in the STAR
Program. We do know many of the VA managers and they are good
people. They are smart. And if their promotions and raises were
contingent in part on the quality of the work performed at
their Regional Office or for Central Office, the national
figures, I have confidence that they would implement a real
good training program because it would affect them.
Right now it does not affect anybody when things are not
good at the Board or even at the court where I know that we
screen many, many cases. And the ones we take, we win over 90
percent of the time because there are many process errors.
Can you ask your second question again?
Mr. Hall. The second question, and this relates to various
aspects of the VA's work, because of the backlog, the workload
and the complexity and funding or staffing difficulties over
the years, we keep running up against privatization proposals.
But in particular, this is training I am talking about.
Mr. Abrams. Right.
Mr. Hall. Should VBA training be contracted out to lawyers
and doctors who specialize in disability compensation cases?
Mr. Abrams. Well, I will tell you, on one of our quality
trips, I was asked by the Regional Office manager to train the
Rating Boards in an area where we all agree they were making
error after error. And after I was finished training, they came
up to me and said we would like to do what you say, but we do
not have time.
When I got back to DC, I realized that we have an ethical
issue here. If we are going to sue the VA in Federal Court as a
matter of practice, if we are going to represent veterans, I am
not certain that most knowledgeable lawyers who represent
veterans before the VA, the Board, and the courts should be
involved in training the VA too. It seems to put the advocate
on both sides of the table.
I think that if the VA wants to contract, I think that the
people who do that may have to give up their other duties.
There is a real tension there to me.
Mr. Hall. Okay. Thank you so much, Mr. Abrams.
Mr. Bartzis, you testified in your written statement that
nationwide for the period of January 1, 2007, through September
5, 2008, the training and performance support system basic
rating completion report lists 2,115 RVSR employees. Only 124
have completed the TPSS portion of the training. This is a
completion rate of 17 percent.
Could you elaborate on this point, please?
Mr. Bartzis. First, I would like to state that there is a
calculation error. The error is, it is not 17 percent
completion rate. It is a 5.8 percent completion rate.
Second, it is common knowledge of myself and other people,
VA employees that I speak with that TPSS does not get
completed. What we have is a complex system and we have
multiple layers of people who get employed at different times.
One layer of employee hired at a certain date may only
complete portions of the basic completion report. Another
section might complete half of it. Another section might
complete a very small portion.
In order to elaborate, what this shows me is a very low
percentage of VA employees have completed the scheduled
training. Whether it is 5.8 percent or 17 percent or 50
percent, what we have is a significantly low portion of a
completion of training during a specified time period.
Mr. Hall. Thank you, Mr. Bartzis.
I will now recognize Mr. Lamborn.
Mr. Lamborn. Thank you, Mr. Chairman.
And I have a couple of questions for Mr. Ratajczak.
There seemed to be three main bodies of knowledge that the
VA employees who are doing this need to be aware of, the VA
rating schedule, Title 38, and precedent court decisions.
Are you finding in your review of erroneous decisions that
one or more of those areas are where there needs to be extra
training?
Mr. Ratajczak. I would say it is the application of court
decisions and continuing legal requirements to the actual
development of cases insofar as that interacts with the
veterans.
It is not hard, well, I should not say it is not hard, but
it is easier to rate or adjudicate a claim, make the decision
with regard to the entitlement service connection, evaluation,
and effective date if all the evidence is before you.
When the evidence is not before you, when the case is not
properly developed, it becomes difficult if not impossible to
do that correctly. Given the time constrains that folks have to
make these decisions, oftentimes they are not given any
incentive to actually make the case right before they make the
decision.
So I think the real crux of the problem is that the folks
who are developing the evidence need to know what they are
developing and why they are developing it and how to do that.
And that, I think, is informed by the case law and the
regulations.
Mr. Lamborn. Okay. Thank you.
And, secondly, in the area of centralized training
curriculum, which you were referring to from the Central
Office, who do you think should develop this? Should it be
experienced field employees or is there also a role for outside
experts?
Mr. Ratajczak. Well, the one thing that the GAO report
points out is that Central Office staff does a very good job
with regard to developing a curriculum for initial hires. And
to that extent, I think that the Central Office staff as it
exists now maybe with some additional personnel would be able
to design a curriculum that is relevant to new issues as they
develop and design a curriculum that sets forth the best
practice with regard to how these issues should be dealt with
by folks out in the field, some language that they can use in
their rating decisions so that they are legally sufficient.
I think that might be the better way to go simply because
those folks that are already in VBA are already familiar with
the process. If we were to go out and try to contract out, get
someone else to do it, I think that there would be a very large
lag time, number one.
And, number two, ultimately the training has to be
responsive and coordinated with what the policy of the VBA is.
Contracted lawyers, et cetera, who are not interpreting that
policy and those practices, we may get a good disconnect.
Mr. Lamborn. Okay. Thank you for your answers.
And thank you all for your testimony.
Mr. Hall. Thank you, Mr. Lamborn.
We have more questions, but what we are going to do is
submit them to you in writing. And you have provided lots of
answers in your testimony to begin with.
So, Mr. Ratajczak, Mr. Baker, Mr. Abrams, Dr. Keenan, Mr.
Bartzis, thank you for your work on behalf of our Nation's
servicemembers and veterans. And you are now excused. Thank you
very much.
And joining us in our third panel is Michael Walcoff,
Deputy Under Secretary for Benefits of the Veterans Benefits
Administration at the Department of Veterans Affairs.
Mr. Walcoff is accompanied by Dorothy Mackay, Director of
Employee Development and Training of the Veterans Benefits
Administration, and Bradley Mayes, Director of Compensation and
Pension Service of the VBA at the Department of Veterans
Affairs.
Of course, you know the routine. Your written statement is
made a part of the record, so feel free to deviate from it. Mr.
Walcoff, you are now recognized for 5 minutes.
STATEMENT OF MICHAEL WALCOFF, DEPUTY UNDER SECRETARY FOR
BENEFITS, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF
VETERANS AFFAIRS; ACCOMPANIED BY DOROTHY MACKAY, DIRECTOR,
EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS
ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; AND
BRADLEY MAYES, DIRECTOR, COMPENSATION AND PENSION SERVICE,
VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS
AFFAIRS
Mr. Walcoff. Thank you, Mr. Chairman.
Actually, I did have an oral statement that was derived
from my written statement, but instead I think because the
written statement is part of the record, I would like to use
this time just to address some of the issues that were raised
by the previous panels. Hopefully that will be okay.
I want to start with some of the testimony presented by my
old friend, Mr. Abrams. And in his testimony, he talked about
what he saw as an emphasis within VBA on productivity at the
expense of quality. And he said that he has seen that and heard
that from employees who work within VBA.
What I would like to offer concerning that is the objective
criteria that we actually use to evaluate our employees and our
managers, and I will give you three instances to refute what
Mr. Abrams said.
First of all, in our Directors' performance standards, we
have in their standards equal elements for both productivity
and quality. They are weighted exactly the same.
In our employees' performance standards, again we have two
elements, two of the elements are productivity and quality,
again weighted exactly the same.
And in our incentive award criteria, we go a step further.
In that criteria, in order for a station to qualify for what we
call level two money, they have to meet a variety of factors
and normally we will list, say, five or six factors and say you
have to meet four of them.
But in addition to the four of the six factors, they have
to meet all of their quality factors. That is one that never
can be waived. Always have to meet quality. And I think that
gives an indication of the fact that we do put a lot of value
on quality, certainly as compared to productivity.
Another statement Mr. Abrams made had to do with work sent
back from the Board. And he said that 50 percent, over 50
percent of work that is sent to the Board comes back to the VA
as incorrect.
I will tell you that is basically a totally false
statement. The remands, as he was referring to, the actual
avoidable remand rate, meaning cases that go back to the ROs
because of mistakes made by those Regional Offices is 18
percent.
Now, I am not saying 18 percent is something that is, you
know, as good as we would like it to be, but it is a far cry
from over 50 percent. And I wanted to clarify that on the
record.
Mr. Chairman, in your opening statement and then in
questions with Dr. Keenan, you talked about certification and
said that you had been told that over 50 percent of the Rating
Specialists, RVSRs, who took the certification test failed it.
And I wanted to clarify that. The actual pass rate on the
one RVSR certification test that we have given is 85 percent.
What you were referring to is the pass rate on the VSR test
which was at 49 percent.
And what I would say to you is that you have to understand
that the VSR certification test measures the employee's ability
to move into the senior technical position in authorization,
which is what a VSR does.
The person who gets that position has to be at a point
where they can authorize the work of other employees. We do not
believe that every employee necessarily will get to the point
where they can authorize the work of other employees.
So, frankly, 50 percent as a pass rate did not surprise us
and I do not believe there is anything wrong with that. The 85
percent on Rating Specialists is different because that is a
test that measures basic knowledge of rating skills. So I would
expect the pass rate to be higher in that and, of course, it
was.
I want to address something that Dr. Keenan said where she
talked about the desire that Rating Specialists should have an
extra 3 weeks of training in predetermination.
Normally in ideal circumstances, we promote people to the
Rating Specialist from the VSR position where they have been
doing that development work throughout their career. So it is
really not necessary to do additional training.
Because of the unusual circumstance that we have been going
through over the last year and a half where we were hiring so
many people, bringing most of them in as VSRs, we were put in a
position where we really could not promote the number of Rating
Specialists that we would need to do the work that is being
developed by these new VSRs all from the VSR position because
then we would be left with no experienced VSRs and we wound up
having to hire some Rating Specialists off the street.
And they are the people that she is talking about that
should spend some time in predetermination. That is a very
small percentage of the total Rating Specialists that we hire,
but I think her point is well taken and it is something that we
will definitely look at.
There are three points that I wanted to make just
addressing some of the previous remarks. And at this point, I
will turn it over and say I will be very willing to----
Mr. Hall. Even though the light is red, you are the guy
that we are here to hear from, so feel free to continue.
Mr. Walcoff [continuing]. Okay. Then let me address one
other thing that you had mentioned and that was that employees
should get credit for training.
And, in effect, they do get credit for training in that
when we measure their productivity during the day, they get
excluded time for any time that they are in training. So that
does not count against them. If anything, it makes it so that
the ratio of hours available for work and the work they do is
equal so that they do not get penalized for being in training.
And I think that is an important point.
But at this point, I think I would like to end my statement
and be available to answer any questions you have.
[The prepared statement of Mr. Walcoff appears on p. 68.]
Mr. Hall. Thank you, Mr. Walcoff.
Mr. Lamborn is double booked for most of this day, so we
are going to let him go first, the Ranking Member, Mr. Lamborn.
Mr. Lamborn. Thank you, Mr. Chairman.
At this point, I will just defer to you because I do not
have any questions.
You have done a good job in your written testimony and I
will be looking that over further. And I will submit further
questions to you if any arise. Thank you.
Mr. Walcoff. Thank you, sir.
Mr. Hall. Thank you, Mr. Lamborn.
Mr. Walcoff, we talked earlier with one of the panelists
about getting 80 hours of training and what happens to an
employee who does not complete it.
Could you answer that same question? What does VA do about
employees who do not complete their training? Are there
incentives to complete it and are there any penalties to not
completing it?
Mr. Walcoff. Let me answer that by saying that I agree with
the statement that was made by Mr. Ratajczak on the second
panel that I do not believe that it should be the
responsibility of the employee to complete the training. I
think it is management's responsibility to make sure that all
their employees complete the training because, frankly, an
employee is not going to go up to the manager and say, oh, I am
leaving my desk now to go sit in on training and I will be back
in an hour. The manager has to be the one to orchestrate that.
So the person that should be held responsible for employees
meeting their training are the managers. And I will tell you
that we have an element in the Directors' performance standards
and it is in most of the coaches' standards that talks about,
and it is in the Human Resource responsibilities, I believe, it
is named a critical element, and it talks about the fact that
the manager is responsible for doing a bunch of different
things involving Human Resources (HR), one of which is making
sure that training is done.
But I will tell you that in reviewing that and, frankly, in
reviewing it in preparation for this hearing, I spoke with the
Associate Deputy Under Secretary for Field Operations and
discussed with her the need to probably make this a little more
specific and make it so that we have in there very specifically
that our managers will be responsible for making sure that all
their employees complete the number of hours for that position
that is required because the requirement we currently have,
while I think it is specific enough, I think that a third party
might say, well, it is too generous. So we are going to make it
even more specific because we do believe that somebody should
be responsible for that and it should be the managers.
Mr. Hall. Well, that makes sense and I am happy to hear it.
In your testimony, you mentioned that last year, you
trained 2,150 new VSRs and RVSRs. What percentage of your
rating workforce does that represent approximately?
Mr. Walcoff. Well, the 2,100 is Rating Specialists and
VSRs.
Mr. Hall. Right.
Mr. Walcoff. I mean, I can tell you that we have about
1,800 Rating Specialists and we have somewhere around 4,500
VSRs. But, of course, some of those are people that have been
there for many, many years. And the numbers you are referring
to are just the people that went through the challenge training
during this year.
So it is kind of tough. It is a hundred percent of the new
employees that we hired is what I would say. You know, I know I
am not answering your specific question, but challenge is
really geared to new employees and every employee must go to
challenge training.
Ms. Mackay. We can indicate that fiscal year 2008, 727 of
the 2,000 or 2,200 people that we have trained in centralized
training were RVSRs. So that gives you an indication of what
percent they represent of the workforce.
Mr. Hall. Mr. Walcoff, you noted that VBA did training with
an independent consultant to evaluate the challenge training.
What were the results of that? You contracted with an
independent consultant. What were the results of that study?
Mr. Walcoff. I am going to ask Ms. Mackay to answer that.
Ms. Mackay. Yes. I do want to make the point that we have
actually done evaluation on a number of different programs. In
fact, we have done two different evaluations of the TPSS
program. As GAO noted, one of those was in 2007 and one was in
2008.
We also reviewed the Challenge program. Some of the
recommendations that have come out of that and the point that
was made was that it does provide value to the organization,
that the program itself is providing value to the organization.
There were some recommendations about resequencing some of
the training modules. Right now there are many that are done on
the post end of Challenge training, if you will, and we want to
move some of them to the beginning so that there is less
duplication.
The TPSS modules are done on the tail end of Challenge
training at home station training. Those are very explicit with
tests at the end of them.
In the Challenge centralized training, there is some
repetition of some of the learning that is taking place, so we
want to move some of those modules forward.
Those were some of the results that came out of the
Challenge evaluation. It is a good program. The employees like
it. We got excellent results. Of the three studies that we did,
two on TPSS and one on Challenge. We have data from 37 ROs. We
did 470 interviews and we interviewed through survey, survey
response 2,700 employees.
So we really got a very representative sample of the
population out there including, you know, subject matter
experts and bargaining unit employees in the field.
Mr. Hall. Has the VA also conducted a time and motion
study?
Mr. Walcoff. I am going to ask Brad to answer that.
Mr. Mayes. Yes, we have. We conducted one in fiscal year
2007, I think, was the last time a motion study was conducted.
Mr. Hall. And any results you would care to share with us?
Mr. Mayes. Well, what we do with information that we get
from the time and motion study is we determine if the weights
are appropriate for the end products that we use for
performance management. In other words, how many hours does it
take to do an original compensation claim, how many hours does
it take to do a reopened compensation claim or a Dependency and
Indemnity Compensation (DIC) claim. And then those end products
as employees work those types of cases, they get work credit,
the station gets credit. Ultimately that is used for resource
allocation as well.
So there were some adjustments made to the work credits
based on that study. But off the top of my head, I do not
remember exactly what the----
Mr. Hall. Would you please send the Subcommittee the study?
It is something we would like to have a look at and save you
from having to tell us or remember all the details of it.
Mr. Mayes, how long can an employee be on a performance
improvement plan?
Mr. Mayes [continuing]. Well, the----
Mr. Walcoff. Yeah. If you do not mind, I will answer that.
I am a former HR guy, so I have knowledge of that.
The performance improvement plan (PIP) as laid out in our
negotiated agreement is 90 days. And they have 90 days during
which they meet regularly with their supervisor. They get
counseling sessions in terms of how they are doing. We ask them
what help do you need in order to meet your performance
standards. Is there training you need. Is there any particular
information that you need. And then they are evaluated at the
end of the 90 days and then a decision is made on whether they
should be kept in their position or removed from their
position.
Mr. Hall. So if the employee improves, but then does not
achieve the fully successful rating on their evaluation, do you
do another improvement plan or just move them to a different
position or move them?
Mr. Walcoff. There are situations where an employee will be
not meeting their standards. They go on a PIP. They meet their
standards. They go off the PIP. They do not meet their
standards and then you put them on another PIP.
The case law actually in that situation says that if that
happens a number of times, and it is undefined what the number
is, that you may actually be looking at a conduct problem
rather than a performance problem. It is something that
occasionally we run into.
Most of the time when employees are on a PIP, hopefully we
are able to give them the assistance and the tutoring that they
need so that they can become satisfactory in their jobs and
never be on a PIP again.
Mr. Hall. Is the ASPEN system, which tracks employee
performance, integrated with the STAR Quality Review Program
and how is that information cross-referenced?
Mr. Walcoff. The ASPEN system is merely just a shell that
we use for tracking. You know, we have to do quality control,
performance evaluations of individuals because that is part of
their performance standards. We needed some vehicle to track
that.
In other words, if we are looking at Dorothy Mackay as an
employee, how many cases did she do this month, how many were
correct, and be able to track that over a year. That is all
that Aspen is.
The quality assurance program that is STAR is a totally
separate program. They are not connected in any way other than
one measures quality of an individual whereas the quality
assurance program, the STAR Program measures quality for the
organization.
Mr. Hall. The primary goal of training obviously is to
improve the accuracy of the rating decisions. However, there
are still accuracy issues at many ROs.
Should there be additional training in those offices and
are those employees still eligible for bonuses and promotions
even if the office's accuracy rate is below the national
standard?
Mr. Walcoff. Let me answer the second part of the question
in terms of the bonuses. As I had mentioned earlier, the level
two awards, which are the primary source of bonuses for
employees, is dependent on the station performance and quality.
So if a station is not making quality, that station will not
get level two money that they could give to those employees.
There is something called level one which is a smaller
amount of money that every station gets for their own use. And
if you happen to have an individual in a C&P, in a service
center who is an outstanding employee, whose individual work is
outstanding, even though the work of that division is not
outstanding, you have at least that level one money where you
can reward that individual. So----
Mr. Hall. Management discretion?
Mr. Walcoff [continuing]. Right. Exactly.
Mr. Hall. I want to ask Ms. Mackay how many actual days of
training do you provide at the Academy? Do trainees have to
sign in and out every day and how do you track their
participation?
Ms. Mackay. I assume we are talking about the Challenge
students related to C&P training because we do do training at
the Academy as well.
It is 3 weeks of training. And I noted in DAV's testimony
that they thought that the travel time really cut into the 3
weeks of training. These are bargaining unit employees, so we
do have to pay attention to the fact that, you know, we do not
want them traveling on the weekend. But we only cut one and a
half days off of that 3 weeks. So it is 1 day on the front end
and then a half day on the back end.
So it is, you know, virtually 3 weeks long of training. And
they are there. There is an accountability. There are
instructors there along with a course manager. So any coming
and going of students during that 3 weeks of centralized
training is duly noted.
It is only for emergency purposes. We have had some folks
who might have gotten sick or whatever. They are monitored. And
not only are they monitored in terms of their health and
getting them back in the classroom, but there is a
communication with the Regional Office that they have been
removed from the classroom for whatever reason. So that
notification is made to the RO as well.
So we keep a good track of the students when they are at
the Academy for that 3 weeks of training.
Mr. Hall. What is the pass/fail rate at the Baltimore
Academy?
Ms. Mackay. In terms of?
Mr. Hall. The employees you train.
Ms. Mackay. Well, if they go through the modules and the
training, the 3 weeks of training, there is not any graduation
per se of them. There is enforcement every single day.
Again, whenever you have a classroom full of students with
varying levels of ability, it is the job of the instructors to
really assess that classroom and see whether they need to pay
particular attention and give special effort and interest to
particular students, that they get it before they leave the
classroom, that particular lesson.
So in that sense, they do or should when they leave the
Academy have learned the modules and the lessons that they need
in order to go back and do the post training at their station.
Mr. Hall. If we could just talk about the training manual,
the five volume training manual for a second. It seems like it
may not be the most effective tool.
Have you consulted with the private-sector organizations
that specialize in developing training protocols or Federal and
private-sector agencies?
Ms. Mackay. I think I am going to pass that to Mr. Mayes.
Mr. Mayes. I am not familiar with the five volume training
manual. I think you might be talking about what we call our
Manual of Adjudication Procedures. We call it 11MR right now.
It should just be M21-1. We had rewritten the manual a couple
of years ago and completed it. It is a number of volumes.
The way I look at the manual is that we start with statute
and regulations and sometimes, as you know, some of these
statutes and regulations are rather difficult to apply or
understand or interpret at the Regional Office level.
So the manual then takes those procedures or those
regulations and interprets those in a more, I would call it, a
more straightforward fashion so that our VSRs and our RVSRs in
the field have a ready reference to apply the laws and
regulations properly.
And it is complex. When we did the task analysis for our
VSRs a number of years ago, actually, Admiral Cooper noted
this, and they were working with the Claims Commission, there
is something on the order of 11,000 tasks. It is complex work
and it does take a lot of rules to make sure that we comply
with the law.
Mr. Hall. We have been known to write some complicated and
challenging laws.
Mr. Mayes. I was not referring to any of your legislation.
Mr. Hall. We held a roundtable with survivors back in the
spring. At that time, we heard tales of folks who have
encountered VBA employees who did not seem to know much about
survivors' benefits such as DIC.
Do you do specialized training around those issues?
Mr. Mayes. Well, training on the Dependency and Indemnity
Compensation benefit is part of the training curriculum for our
new VSRs. So, yes, that is included in that curriculum.
And that curriculum, by the way, requires 26 weeks. So the
formal curriculum, 26 weeks. It is two to 3 weeks of
prerequisite training. Then we send our VSRs off to Challenge
where they have another 3 weeks of formal training. We utilize
the TPSS modules, the training, performance, support system
modules in this training and then they go back to the Regional
Office.
And it is a combination, as the gentleman from GAO said, it
is a combination of computer-based training and then moving
away from that and practically applying it. And DIC is part of
that process, that curriculum.
Mr. Hall. Thank you.
Now, Mr. Walcoff, I want to ask you a couple more questions
here.
VHA has had success with teleconferencing capabilities. I
am curious if VBA has sought out the same opportunities to
train with teleconferencing.
Mr. Walcoff. I am looking at Dorothy to see whether we
actually use teleconferencing for training.
Ms. Mackay. Yes.
Mr. Mayes. I can tell you most recently, we identified in
the field some challenges applying the DeLuca rules. In other
words, a court case a number of years ago that said that when
there is limitation resulting--when pain results in limitation
of function or motion, that we can adjust the evaluation
commensurately.
And so what we did recently was we had a national
conference call that was targeted at our Rating Veteran Service
Representatives. We identified this through our quality
assurance program. So when we find through our quality
assurance program that we have a targeted area that needs
focus, then frequently we will have a teleconference with RVSRs
or managers out in the field.
And we are doing that right now as part of our followup
from the findings on STAR. We are doing that monthly with the
Regional Office. So there is an example where we would use
conference calls to initiate discussion on topics that are
causing some problems in the field.
Mr. Hall. You know, that is obviously a developing and more
frequently used tool that you may find yourselves wanting to
use more.
I want to also ask what training is required of Board of
Veterans' Appeals employees.
Mr. Walcoff. Well, you know, that is something that we are
not really involved in. I mean, BVA has their own training
programs for their employees and I do not think any of us are
in a position to be able to answer that.
Mr. Hall. Then we will not ask you that question then.
So the Subcommittee staff recently conducted a site visit
to Okinawa, Guam, and Hawaii to review the claims processing
issues there. And a point that was brought up in Japan was the
role of the U.S. State Department in compiling claims
information for the Islands of Micronesia, Palau, and the
Marshall Islands. The Department of the Interior handles
veterans' claims.
How does VA inform other agencies and train them in the
claims application process and educate them on other benefits
information?
Mr. Walcoff. That is going to be yours again.
Mr. Mayes. Well, we do work with representatives from the
State Department who are involved in communicating information
about veterans' benefits. We recently had a training seminar.
Members of the Compensation and Pension Service staff conducted
that training. So we are involved in that.
Primarily what we have State Department personnel doing is
again giving out information and also they are involved in the
coordination of compensation and pension exams for veterans who
reside overseas. So we give them information and provide
training for that as well.
Mr. Hall. Okay. Lastly, I wanted to ask a question about
dose reconstruction. The Veterans' Advisory Board on Dose
Reconstruction met this month and discussed the advantages of
having the specialization and ionizing radiation claims in one
RO so the training could be targeted.
If this is true for atomic veterans' claims, would
specialization and targeted training benefit veterans with
other conditions such as Post Traumatic Stress Disorder (PTSD)
or Traumatic Brain Injury (TBI)?
Mr. Walcoff. I think that the idea of specializing in that
sense is something that we have had a lot of discussion about.
And at this point, the problem is that when somebody, for
instance, files a claim for PTSD, invariably there are going to
be other issues that they are also claiming. And it is very
difficult to send just the PTSD part of a file, you know, to
another office and say you work on that and we will leave
everything else at the home office because everybody needs
access to the entire file.
What we do believe is as we move, I hope steadily, toward
getting to the point where we go paperless and we have
electronic files, one of the real advantages of that will be
that if we want a particular office to, say, do the PTSD part
of the claim, they will be able to access the file the same
time as the home office is accessing the file because it is
electronic. And that really would make it so that we would be
able to do that type of specialization on a more frequent
basis.
The radiation claims are so complex and there are so few of
them that we felt that it made sense to do it even with the
fact that we are in paper. But ultimately I believe that when
we go paperless, it will open all kinds of doors for us exactly
as you are talking.
Mr. Hall. You are a moving target, as you know, and in our
oversight capability or legislative capability, we are trying
to encourage or fund or to help you move in directions that you
are already moving in many circumstances.
So you just brought up the paperless thing. I want to ask
you how that is going. I know even though our bill has passed
the House, we are waiting for Senate action, which hopefully we
will see. But at the same time, I am sure you are somewhere
down the road.
Any reports on that?
Mr. Walcoff. There are a lot of things that are happening.
The biggest thing right now is we are awaiting for the awarding
of a contract for our lead systems integrator, which is going
to be a very key piece as we move forward because the
integrator is going to be the person that or the organization
that works with us in coordinating all the different parts of
this, you know, project that we call paperless, you know,
including the part that deals with a veteran's ability to apply
online for benefits, including the ability for a veteran to go
into our system and be able to determine the status of his
claim without having to pick up the phone and call us,
including the part about making all the actual paper
electronic.
There are so many pieces to this and the integrator is
going to be a key player. That contract is supposed to be
awarded before the end of the fiscal year. So that is a major,
major step forward with us. So we are very, very optimistic in
terms of the future.
The Secretary, I will tell you, Mr. Chairman, the Secretary
has been relentless in terms of telling us that we will be
paperless. So we are absolutely getting a lot of encouragement.
Mr. Hall. In terms of your continuing efforts to do the
things that we are trying to tell you to do or ask you to do,
is there any report or progress? Have you been working on
updating the rating schedule at the same time?
Mr. Walcoff. Brad, you want----
Mr. Mayes. Yes. I do not know if you saw the press release.
I think it came out yesterday or the day before. But we are
creating a Federal Advisory Committee that will be tasked with
taking a look at the schedule, looking at the studies that have
been done recently.
And we have the Marsh-West study. We have the Disability
Benefits or Veterans Disability Benefits Commission, the Dole-
Shalala Commission. There are a lot of folks out here that have
done some work and looked at it.
And, of course, I think you know that we have contracted
with a firm to review the aspects of the Dole-Shalala proposed
legislation that the President sent over to Congress.
So this Advisory Committee will be looking at all of that
and then making recommendations to the Secretary on, I would
say, how do we get moving and prioritize and what is next.
And, finally, I want to mention to you that we had proposed
a new rule for traumatic brain injury. We think that is going
to go final very quickly. It is going to change the rating
criteria for TBI. We just updated the skin portion that was
proposed. We think we are close to going final on that. And we
are working on adding presumptive for Amyotrophic Lateral
Sclerosis (ALS).
Mr. Hall. And this is really the last question now. Where
do you stand with the quality of life study?
Mr. Mayes. Well, that is the study that I just mentioned.
We are in the process of going through it. Our technical
comments have been requested. So as we move through that and
provide that feedback to the contractor. Once that is complete,
then it will be published and available to the Committee.
Mr. Hall. Great. We look forward to receiving that.
Thank you all for your statements, for your answers, for
the work you do for our veterans.
And I would like to thank everybody from all the previous
panels for your insight and opinions.
And this hearing now stands adjourned.
[Whereupon, 11:45 a.m., the Subcommittee was adjourned.]
A P P E N D I X
----------
Prepared Statement of Hon. John J. Hall, Chairman,
Subcommittee on Disability Assistance and Memorial Affairs
Our Nation's veterans understand the necessity of proper and
adequate training--their lives have depended on it. The military trains
for its operations and everyone knows every detail of their job prior
to the mission. These same veterans should be able to expect the same
level of competence when they seek assistance from the Veterans
Benefits Administration. Unfortunately, that is not the case, as we
have heard at other meetings and hearings throughout the year that this
Subcommittee has held regarding the VA disability claims processing
system.
VA has standardized its training curriculum and requires that all
claims processors must complete 80 hours of annual training. This is a
lot of hours because in fact some healthcare providers don't need to
meet that level of continuing education to maintain their clinical
license or credentialing.
The VBA training topics are identified by Central Office or by the
individual's Regional Office (RO). New employees go through an
orientation process and there are additional online learning tools
available through the VBA's Training and Performance Support System.
Yet, with all of this effort, VA training seems to be falling short
of its intended goals. Less than 50 percent of the Ratings Veterans
Service Representatives (RVSRs) passed the certification exam, even
though it was an open-book test. But, frankly, I've seen the training
manual and it can be measured in pounds not pages, so I don't know how
useful their book is and that can be the crux of the matter right
there. More importantly, as outlined in previous hearings, there are
significant inconsistencies in ratings between VA's 57 ROs and a high
rate of remanded cases.
I am pleased that the AFGE is here to shed light on this issue. You
are a critical link to those on the frontlines working to improve
outcomes for our disabled veterans.
The Government Accountability Office (GAO) released a report in May
2008 entitled, ``Veterans' Benefits: Increased Focus on Evaluation and
Accountability Would Enhance Training and Performance Management for
Claims Processors'' that documented the areas in which VA needs to
improve its training and hold accountable those it does train.
According to the GAO, staff is not held accountable for completing
the required training, since VBA does not track completion, so there
are no consequences for not taking the training. Additionally, VBA does
not evaluate its training, so it does not know if it has successfully
designed and implemented an educational program. Feedback is not
consistently collected from RO employees on the training that they do
receive and many have reported difficulty in accessing training because
of their stringent productivity demands. I look forward to hearing more
from the GAO about this report.
But, these are not surprising conclusions to the Veterans Service
Organizations, which have complained for years about the inadequacies
of the VBA training program. So, I am grateful that they have joined us
here today as well.
Training is not an issue that should be taken lightly. We all know
the importance of good training, but effective implementation that
ensures consistency and accountability can be elusive and that is what
I hope we can address today. I have taken steps to ensure improved
training outcomes when I introduced the ``Veterans Disability Benefits
Claims Modernization Act'', H.R. 5892. These policy enhancements will
lead to compensation claims processing improvements and more accurate
claims adjudication results for our veterans and their families.
Moreover, I am not sure that VBA's current training regimen complements
its current Claims Processing Improvement model, or CPI. In fact, I am
positive that the current coupling detracts from increased
accountability efforts.
I am pleased to report, with the help of many in this room, H.R.
5892 passed the House on July 31, 2008, by a 429 to 0 vote. On August
1, 2008, Senator Clinton introduced companion legislation to my bill,
S. 3419. So, Congress is well on its way to rectifying the inadequacies
in the VBA that have already been identified to us.
Today's oversight hearing will allow us to drill deeper into this
issue and gauge where VA is at in its training protocol and see where
other improvements can still be made. I look forward to hearing from
our witnesses today and hope to learn more about best practices and
strategies for measuring performance and building better training
protocols and accountability standards.
Prepared Statement of Hon. Doug Lamborn, Ranking Republican Member,
Subcommittee on Disability Assistance and Memorial Affairs
Thank you Chairman Hall for yielding.
I am pleased to have this opportunity for a collective discussion
on the effectiveness of the Veterans Benefits Administration's (VBA's)
training, performance management, and accountability requirements.
Over the course of the past several months, this Subcommittee has
examined nearly every facet of the VA benefits claims processing system
in an effort to identify how we might help the Department overcome the
claims backlog crisis.
While the recent expansion of its workforce will certainly have a
positive impact, VA must ensure that newly hired claims workers receive
training that is commensurate with their responsibilities.
It is critical that the training it provides meets the needs of the
Department and its employees.
It is equally important that the results of the training are
evaluated. Without feedback, VA may never know whether or not the
training is accomplishing its goal.
Any viable training program should be able to identify deficiencies
and demonstrate the intended and actual outcome of its curriculum.
VA training must be connected to its vision and mission, and VA
managers need to be assured that if employees are pulled off the floor
for training that it will result in long-term benefits.
I'm sure that with a growing number of pending claims, there is a
certain level of trepidation that, ``There's too much work to do
already and we'll just get further behind if we have to conduct
training.''
There must be clear support, from the top down, in order to conduct
adequate training and acquire the expected outcomes.
Certainly, training new employees on everything they need to know
in order to make sound rating decisions is a daunting task.
The VA rating schedule itself is complex and it's merely a portion
of the array of knowledge a competent adjudicator must possess to
perform his or her job.
I view today's hearing as an opportunity to not only learn more
about the training and assessment program VA provides its employees,
but also to reiterate to the Department that it should be forthright
about any additional resources deemed necessary to fulfill this
critical requirement.
I look forward to our witnesses' testimony and a productive
discussion on VA's training program.
I yield back.
Prepared Statement of Daniel Bertoni, Director, Education, Workforce,
and Income Security Issues, U.S. Government Accountability Office
Improvements Needed in VA's Training and
Performance Management Systems
GAO Highlights
Why GAO Did This Study
The Department of Veterans Affairs' (VA) disability claims process
has long been a subject of concern because of long waits for decisions
and large backlogs of claims pending decisions. To address these
issues, VA has hired almost 3,000 new claims processors since January
2007. However, adequate training and performance management are
essential to developing highly competent disability claims processors
and ensuring that experienced staff maintain the skills needed to issue
timely, accurate, and consistent decisions.
The Subcommittee on Disability Assistance and Memorial Affairs,
House Veterans' Affairs Committee asked GAO to present its views on (1)
VA's training for its claims processors, and (2) VA's performance
management of this staff. This statement is based on a May 2008 report
on VA's training and performance management (GAO-08-561) and has been
updated as appropriate.
What GAO Recommends
In its May report, GAO recommended that VA collect feedback from
staff on the training provided in the regional offices and use this
feedback to improve training; hold staff accountable for meeting their
training requirement; and assess, and if necessary adjust its process
for placing staff in overall performance categories. VA concurred with
these recommendations, but has not yet reported any significant
progress in implementing them.
What GAO Found
Training for VA disability claims processors complies with some
accepted training practices, but VA does not adequately evaluate its
training and may have opportunities to improve training design and
implementation. VA has a highly structured, three-phase training
program for new staff and an SO-hour annual training requirement for
all staff. GAO found that VA has taken steps to plan this training
strategically and that its training program for new staff appears well-
designed and conforms to adult learning principles. However, while VA
collects some feedback on training for new staff, it does not collect
feedback on all the training conducted at its regional offices.
Moreover, both new and experienced staff reported problems with their
training. Some new staff told us a computer-based learning tool is too
theoretical and often out of date.
More experienced staff said they struggled to meet the annual SO-
hour training requirement because of workload pressures or could not
always find courses relevant given their experience level. Finally, the
agency does not hold claims processors accountable for meeting the
annual training requirement.
VA's performance management system for claims processing staff
generally conforms to accepted practices. For example, individual
performance measures, such as quality and productivity, are aligned
with the agency's organizational performance measures, and VA provides
staff with regular performance feedback. However, the system may not
clearly differentiate among staff performance levels. In each of the
regional offices we visited, at least 90 percent of claims processors
were placed in just two of five overall performance categories. Broad,
overlapping performance categories may deprive managers of the
information they need to reward top performers and address performance
issues, as well as deprive staff of the feedback they need to improve.
Fiscal Year 2007 Appraisals for Four Offices
Were Concentrated in Two Categories
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: VBA regional office.
Mr. Chairman and Members of the Subcommittee:
I am pleased to have the opportunity to comment on training and
performance management for Department of Veterans Affairs' (VA)
disability claims processors. In fiscal year 2007, VA paid about $37\1/
2\ billion in benefits to more than 3.6 million veterans and their
families. The disability claims process has long been a subject of
concern for VA, the Congress, and veterans' service organizations due
to long waits for decisions, large backlogs of pending claims, and
problems with the accuracy and consistency of decisions. Moreover, we
have noted that VA's current disability process is in urgent need of
re-examination and transformation, especially in regard to how it
assesses the work capacity of individuals with disabilities in today's
world and in its ability to provide timely and appropriate benefits.
With an increase in claims resulting from injuries sustained in Iraq
and Afghanistan and from an aging veteran population, these issues will
likely persist. To address them, VA added almost 3,000 new claims
processors from January 2007 to July 2008 and has plans to add even
more staff by the end of September 2008. Earlier this year, I testified
before this Subcommittee that enlarging VA's disability workforce is
likely to produce certain human capital challenges for the agency.\1\
More staff alone will not guarantee effective disability claims
processing. Among other things, adequate training and performance
management are essential to developing highly competent new disability
claims processors and ensuring that experienced staff maintain the
skills needed to issue timely, accurate, and consistent disability
decisions.
---------------------------------------------------------------------------
\1\ GAO, Veterans' Disability Benefits: Claims Processing
Challenges Persist, While VA Continues to Take Steps to Address Them,
GAO-08-473T (Washington, DC: Feb. 14, 2008).
---------------------------------------------------------------------------
My remarks today primarily draw from our May 2008 report for the
Committee on Veterans' Affairs and focus on (1) VA's training for its
disability claims processing staff and (2) its performance management
system for claims processors. We conducted our work in accordance with
generally accepted Government auditing standards.\2\ For this
testimony, we updated information from our report, as appropriate, to
reflect the current status of VA training and performance management
systems.
---------------------------------------------------------------------------
\2\ GAO, Veterans' Benefits: Increased Focus on Evaluation and
Accountability Would Enhance Training and Performance Management for
Claims Processors, GAO-08-561 (Washington, DC: May 27, 2008).
---------------------------------------------------------------------------
In summary, although we found that training for VA disability
claims processors complies with some accepted training practices, it is
not adequately evaluated, and some aspects of training design and
implementation could be improved. We found that VA has taken steps to
strategically plan its training, including the establishment of a
training board to evaluate the agency's training needs. Also, VA's
training program for new staff appears well-designed and conforms with
adult learning principles. However, while VA collects feedback on many
of the training methods and tools for new staff, not all the training
VA conducts is evaluated to determine how relevant or effective it is.
Moreover, both new and experienced staff reported problems with their
training. Some new staff members reported that a computer-based
learning tool was not useful. Also, VA requires 80 hours of training
annually for all claims processors, but some experienced claims
processors struggled to meet this requirement because of workload
pressures, and some could not always find relevant courses. It is not
clear what criteria VA uses to justify the number of required training
hours. Furthermore, individual claims processors are not held
accountable for meeting the annual training requirement, although
according to VA, the agency has implemented a new learning management
system allowing it to monitor staffs completion of the training
requirement.
VA's performance management system for claims processing staff
generally conforms to accepted performance management practices. For
example, individual performance measures, such as quality and
productivity, are aligned with the agency's organizational performance
measures, and VA provides claims processing staff with regular feedback
on their performance. However, the system may not clearly differentiate
among performance levels. Broad, overlapping performance categories may
deprive managers of the information they need to reward top performers
and address performance issues, as well as deprive staff of the
feedback they need to improve.
Background
The Veterans Benefits Administration (VEA) within VA administers
the disability compensation and pension programs, whereby VA claims
processing staff assess veterans' applications for disability
compensation and pension benefits. Aside from benefits for veterans,
VEA claims processing staff make eligibility determinations for
deceased veterans' spouses, children, and parents. In short, they are
responsible for ensuring that the decisions that lead to paying
disability compensation and pension benefits are timely, accurate, and
consistent.
The VA disability claims process involves multiple steps and
usually involves more than one claims processor. When a veteran submits
a claim to one of VEA's 57 regional offices, staff in that office are
responsible for obtaining evidence to evaluate the claim, such as
medical and military service records; determining whether the claimant
is eligible for benefits; and assigning a disability rating specifying
the severity of each of the veteran's impairments. These ratings
determine the amount of benefits eligible veterans will receive.
VA has faced questions about the timeliness, accuracy, and
consistency of its disability decisions. GAO designated Federal
disability programs, including VA and other programs, as a high-risk
area in 2003. In particular, our prior work found VA relied on outmoded
criteria for determining program eligibility that did not fully reflect
advances in medicine and technology or changes in the labor market. As
a result, VA's disability program may not recognize an individual's
full potential to work. In addition, VA has seen processing times for
their disability claims increase over the past several years, and
inconsistencies in disability decisions across locations have raised
questions about fairness and integrity.
Some have suggested that VA needs to address its training and
guidance related to claims processing in order to improve consistency
and that it should conduct periodic evaluations of decisions to ensure
the accuracy of ratings across disability categories and regions. VA
has reported that some of the inconsistency in its decisions is due to
complex claims, such as those involving post-traumatic stress disorder,
but it has also acknowledged that the accuracy and consistency of
claims decisions needs further improvement.
Training Complies With Some Accepted Practices, But VBA Does Not
Adequately Evaluate Training and May Be Falling Short in
Training Design and Implementation
To prepare newly hired staff to perform the tasks associated with
processing disability claims, VEA has developed a highly structured,
three-phase program designed to deliver standardized training. The
first phase is designed to lay the foundation for future training by
introducing new staff to topics such as medical terminology and the
computer applications used to process and track claims. The second
provides an overview of the technical aspects of claims processing,
including records management, how to review medical records, and how to
interpret a medical exam. The third includes a combination of
classroom, on-the-job, and computer-based trainings. The second and
third phases in this program are designed to both introduce new
material and reinforce material from the previous phase.
To help ensure that claims processing staff continually maintain
their knowledge after their initial training and keep up with changing
policies and procedures, VEA's Compensation and Pension Service
requires all claims processing staff to complete a minimum of 80 hours
of technical training annually. This training requirement can be met
through a mix of classroom instruction, electronic-based training from
sources such as the Training and Performance Support System (TPSS), or
guest lecturers. VBA's regional offices have some flexibility over what
courses they provide to their staff to help them meet the training
requirement. These courses can cover such topics as establishing
veteran status, asbestos claims development, and eye-vision issues.
We found that VBA has taken some steps to strategically plan its
training for claims processors in accordance with generally accepted
training practices identified in our prior work.\3\ For example, VBA
has taken steps to align training with the agency's mission and goals.
In 2004, VBA established an Employee Training and Learning Board
(board) to, among other things, ensure that the agency's training
decisions support its strategic and business plans, goals, and
objectives. Also, VBA has identified the skills and competencies needed
by its claims processing staff by developing a decision tree and task
analysis of the claims process. In addition, VBA has taken steps to
determine the appropriate level of investment in training and to
prioritize funding. The board's responsibilities include developing an
annual training budget and recommending training initiatives to the
Under Secretary of Benefits. Further, we found that VBA's training
program for new claims processing staff appears well-designed, in that
it conforms to adult learning principles by carefully defining all
pertinent terms and concepts and providing abundant and realistic
examples of claims work.
---------------------------------------------------------------------------
\3\ GAO, Human Capital: A Guide for Assessing Strategic Training
and Development Efforts in the Federal Government, GAO-04-546G
(Washington, DC: March 2004).
---------------------------------------------------------------------------
However, while VBA has developed a system to collect feedback from
new claims processing staff on their training, the agency does not
consistently collect feedback on all of the training it provides. For
example, none of the regional offices we visited consistently collected
feedback on the training they conduct. Without feedback on regional
office training, VBA may not be aware of how effective all of its
training tools are.
Moreover, both new and experienced claims processing staff we
interviewed reported some issues with their training. A number of staff
told us the TPSS was difficult to use, often out-of-date, and too
theoretical. Some claims processing staff with more experience reported
that they struggled to meet the annual training requirement because of
workload pressures or that training topics were not always relevant for
staff with their level of experience. VBA officials reported that they
have reviewed the 80-hour training requirement to determine if it is
appropriate, but they could not identify the criteria or any analysis
that were used to make this determination. Identifying the right amount
of training is crucial. An overly burdensome training requirement may
needlessly take staff away from essential claims processing duties,
while too little training could contribute to processing and quality
errors.
In addition to lacking a clear process for assessing the
appropriateness of the SO-hour training requirement, VBA also has no
policy outlining consequences for individual staff who do not complete
the requirement. Because it does not hold staff accountable, VBA is
missing an opportunity to clearly convey to staff the importance of
managing their time to meet training requirements, as well as
production and accuracy goals. In fiscal year 2008, VBA implemented a
new learning management system that allows it to track the training
hours completed by individual staff. Although VBA now has the capacity
to monitor staffs completion of the training requirement, the agency
has not indicated any specific consequences for staff who fail to meet
the requirement.
VA's Performance Management System Generally Conforms With Accepted
Practices, But May Not Clearly Differentiate Among Staff's
Performance Levels
VA's performance management system for claims processors is
consistent with a number of accepted practices for effective
performance management systems in the public sector.\4\ For example,
the elements used to evaluate individual claims processors-such as
quality, productivity, and workload management-appear to be generally
aligned with VBA's organizational performance measures. Aligning
individual and organizational performance measures helps staff see the
connection between their daily work activities and their organization's
goals and the importance of their roles and responsibilities in helping
to achieve these goals. VA also requires supervisors to provide claims
processors with regular feedback on their performance, and it has
actively involved its employees and other stakeholders in developing
its performance management system.
---------------------------------------------------------------------------
\4\ GAO, Results Oriented Cultures: Creating a Clear Linkage
between Individual Performance and Organizational Success, GAO-03-488
(Washington, DC: Mar. 14, 2003).
---------------------------------------------------------------------------
However, VA's system may not be consistent with a key accepted
practice-clear differentiation among staff performance levels. We have
previously reported that, in order to provide meaningful distinctions
in performance for experienced staff, agencies should preferably use
rating systems with four or five performance categories.\5\ If staff
members' ratings are concentrated in just one or two of multiple
categories, the system may not be making meaningful distinctions in
performance. Systems that do not make meaningful distinctions in
performance fail to give (1) employees the constructive feedback they
need to improve and (2) managers the information they need to reward
top performers and address performance issues.
---------------------------------------------------------------------------
\5\ GAO, Human Capital: Preliminary Observations on the
Administration's Draft Proposed ``Working for America Act,'' GAO-06-
142T (Washington, DC: Oct. 5, 2005).
---------------------------------------------------------------------------
VA's performance appraisal system has the potential to clearly
differentiate among staff performance levels. Each fiscal year,
regional offices give their staff a rating on each individual
performance element: exceptional, fully successful, or less than fully
successful. For example, a staff member might be rated exceptional on
quality, fully successful on productivity, and so forth. Some elements
are considered critical elements, and some are considered noncritical.
Staff members are then assigned to one of five overall performance
categories, ranging from unsatisfactory to outstanding, based on a
formula that converts a staff member's combination of ratings on the
individual performance elements into an overall performance category
(see fig. 1).
Figure 1--VA Overall Performance Appraisal Formula
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: GAO analysis of VBA information.
However, there is evidence to suggest that the performance
management system for claims processing staff may not clearly or
accurately differentiate among staffs performance. Central office
officials and managers in two of the four regional offices we visited
said that, under the formula for assigning overall performance
categories, it is more difficult to place staff in certain overall
performance categories than in others-even if staffs performance truly
does fall within that category. These managers said it is especially
difficult for staff to be placed in the excellent category. In fact, at
least 90 percent of all claims processors in the regional offices we
visited ended up in only two of the five performance categories in
fiscal year 2007: fully successful and outstanding (see fig. 2).
Figure 2--Fiscal Year 2007 Overall Performance Ratings
for Claims Processors in Four Regional Offices Were
Concentrated in the Outstanding and Fully Successful Categories
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: VBA regional offices.
Some managers told us that there are staff whose performance is
better than fully successful but not quite outstanding, but that under
VA's formula, it is difficult for these staff to be placed in the
excellent category. To be placed in the excellent category, a staff
member must be rated exceptional in all the critical elements and fully
successful in at least one noncritical element. However, managers told
us that virtually all staff who are exceptional in the critical
elements are also exceptional in the noncritical elements, and they are
appropriately placed in the outstanding category. On the other hand, if
a staff member is rated fully successful on just one critical element,
even if all other elements are rated as exceptional, the staff member's
overall performance category falls from outstanding to fully
successful.
Neither VBA nor VA central office officials have examined the
distribution of claims processing staff across the five overall
performance categories. However, VA has acknowledged that there may be
an issue with its formula, and the agency is considering changes to its
performance management system designed to allow for greater
differentiation in performance. Absent additional examination of the
distribution of claims processors among overall performance categories,
VA lacks a clear picture of whether its system is working as intended
and whether any adjustments are needed.
In conclusion, VA appears to have recognized the importance of
developing and maintaining high performing claims processors. It needs
to devote more attention, however, to ensuring that its training and
performance management systems are better aligned to equip both new and
experienced staff to handle a burgeoning workload. Specifically, in our
May 2008 report, we recommended that VA should collect feedback from
staff on training provided in the regional offices in order to assess
issues such as the appropriateness of the 80-hour annual training
requirement and the usefulness of TPSS. We also recommended that the
agency should use information from its new learning management system
to hold staff members accountable for meeting the training requirement.
In addition, we recommended that VA should assess whether its
performance management system is making meaningful distinctions in
performance. In its comments on our May 2008 report, VA concurred with
our recommendations, but it has not yet reported making any significant
progress in implementing them.
While hiring, training, and evaluating the performance of staff is
essential, commensurate attention should be focused on reviewing and
aligning disability benefits and service outcomes to today's world. In
prior work, we have noted that VA and other Federal disability programs
must adopt a more modern understanding of how technology and labor
market changes determine an individual's eligibility for benefits, as
well as the timing and portfolio of support services they are provided.
To the extent progress is made in this area, effective training and
performance management systems will be of crucial importance. Moreover,
the way VA's larger workforce is distributed and aligned nationwide can
also significantly impact the degree to which it succeeds in meeting
the agency's responsibilities to veterans in the future. In short, VA
should seize this opportunity to think more strategically about where
to best deploy its new staff and how to develop and maintain their
skills.
GAO Contact and Staff Acknowledgments
Mr. Chairman, this concludes my remarks. I would be happy to answer
any questions that you or other Members of the Subcommittee may have.
For further information, please contact Daniel Bertoni at (202)
512-7215 or [email protected]. Also contributing to this statement were
Clarita Mrena, Lorin Obler, David Forgosh, and Susan Bernstein.
Related GAO Products
Veterans' Benefits: Improved Management Would Enhance VA's Pension
Program. GAO-08-112. Washington, DC: February 14, 2008.
Veterans' Disability Benefits: Claims Processing Challenges
Persist, while VA Continues to Take Steps to Address Them. GAO-08-473T.
Washington, DC: February 14, 2008.
Disabled Veterans' Employment: Additional Planning, Monitoring, and
Data Collection Efforts Would Improve Assistance. GAO-07-1020.
Washington, DC: September 12, 2007.
Veterans' Benefits: Improvements Needed in the Reporting and Use of
Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.
Washington, DC: September 30, 2003.
Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government. GAO-03-893G. Washington,
DC: July 2003.
Results-Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success. GAO-03-488.
Washington DC: March 14, 2003.
Major Management Challenges and Program Risks: Department of
Veterans Affairs. GAO-03-110. Washington, DC: January 1, 2003.
Veterans' Benefits: Claims Processing Timeliness Performance
Measures Could Be Improved. GAO-03-282. Washington, DC: December 19,
2002.
Veterans' Benefits: Quality Assurance for Disability Claims and
Appeals Processing Can Be Further Improved. GAO-02-806. Washington, DC:
August 16, 2002.
Veterans' Benefits: Training for Claims Processors Needs
Evaluation. GAO-01-601. Washington, DC: May 31, 2001.
Veterans Benefits Claims: Further Improvements Needed in Claims-
Processing Accuracy. GAO/HEHS-99-35. Washington, DC: March 1, 1999.
Prepared Statement of Statement of Michael Ratajczak, Decision
Review Officer, Cleveland Veterans Affairs Regional Office,
Veterans Benefits Administration, U.S. Department of Veterans,
on Behalf of American Federation of Government Employees, AFL-CIO
Dear Chairman and Members of the Subcommittee:
Thank you for the opportunity to testify today on behalf of the
American Federation of Government Employees, AFL-CIO (AFGE), the
exclusive representative of employees in the Veterans Benefits
Administration (VBA).
I currently serve VBA as a Decision Review Officer (DRO) at the
Cleveland VA Regional Office (RO). I joined VBA's workforce in
September of 2001. I worked with the Tiger Team Remand Unit that
resolved some of the oldest pending Board of Veterans Appeals remands
in the country and served as an operational model for VBA's Appeals
Management Center. I also served as a Specialized Rating Veterans
Service Representative (RVSR) in the Tiger Team and the Cleveland
Resource Unit of the Appeals Management Center. My duties with VBA have
also included temporary assignments to a Remand Quality Review Project
which was intended, at least in part, to identify common errors
occurring at the RO level which necessitate remand of cases to the RO
by the Board of Veterans Appeals. I have also served as an instructor
in Centralized Challenge Courses for newly hired RVSRs. I currently
serve as AFGE's representative on the VBA Design Committees for Basic
and Journey-level RVSR Certification Testing. Prior to my employment
with the VA, I was an attorney in private practice.
In reviewing past Congressional testimony by VBA, I was struck by
the consistent reference to increased complexity of claims and
continuing judicial refinement of the duty to assist veterans under the
Veterans Claims Assistance Act of 2000 (VCAA) as factors contributing
to the claims backlog. Yet, VBA's testimony never mentioned the obvious
impact of increasing claims complexity and duty to assist mandates on
employees' ability to develop and adjudicate claims, or review case
development and adjudications upon notice of disagreement from
claimants within the time constraints imposed by productivity
requirements set by management.
Training and Performance Management are Closely Linked
All who work in VBA claims processing agree that increased claims
complexity and additional duties imposed by law are perhaps the most
important factors slowing performance and reducing productivity.
Therefore, in addition to adequate hiring, effective training must be
an essential component of any VBA effort to increase the timeliness,
accuracy and consistency of claims processing. Ultimately, the veterans
seeking benefits are shortchanged by VBA's failure to make any
accommodation in performance standards to reflect growing claims
complexity and new legal requirements. VBA's insistence on holding
claims processors to production standards that do not allow adequate
time to develop, consider, and resolve complex claims in accordance
with the duties imposed by law is a disservice to veterans and
undermines the intent of the Veterans Claims Assistance Act.
GAO found in its May 2008 report that VBA's training program for
claims processors does not consistently track completion of training or
ensure that training requirements are uniformly adhered to at the RO
level. To the extent that GAO recommends that VBA be held accountable
for fully implementing and tracking the training it mandates, it is
laudable. However, I am very concerned that the report could be misread
to imply that individual claims processors are ultimately responsible
for ensuring that they complete their mandatory training requirements.
Ultimately, it is VBA management's responsibility to provide the
necessary time and resources for employees to complete well designed,
comprehensive, up-to-date training programs. No individual claims
processor can demand of local management that he or she receive
training when a determination is made that employee resources are
better devoted to other concerns, such as fulfilling a monthly or
fiscal year production goal. Management's failure to devote time to the
initial and continuing education of claims processors in favor of
fulfilling a short-term production goal is similar to VBA eating its
seed corn, since it deprives claims processors of the means to become
more efficient, accurate, and fulfill their ever changing and
increasingly complex duties.
Similarly, management's failure to ensure adequate training time
impairs the ability of VBA's certification testing program for VSRs and
RVSRs to be an objective measure of training effectiveness. This places
an unfair burden on individual claims processors who are thus less
likely to achieve a passing score on certification testing, and in
turn, advance their careers.
In order for training to be meaningful, management must afford the
participant time to read and analyze the material, and internalize its
meaning through cognitive effort and practical application. The
complexity of the claims process administered by VBA does not admit to
simple resolution by reference to checklists, decision trees or
presentation of information without elucidation. This complexity is
well illustrated by a videotape that was recently shown to VBA claims
processors on the application of GAF (global assessment of functioning)
scores in the context of assigning disability ratings for service
connected posttraumatic stress disorder. This 2 hour training video
focuses on one very discrete issue in rating a very specific type of
claim that is fraught with difficulties. It serves as a reminder that
even well-prepared training material presented by highly competent and
learned professionals can be of limited value in the absence of an
opportunity for meaningful interactive learning. Training materials
must be combined with the opportunity for trainees to receive timely
feedback from an individual who understands the subject and can provide
relevant, consistent, and immediate guidance to trainees struggling to
internalize the meaning of the material in the context of the duties
attendant to their positions.
Similarly, while materials such as Fast Letters, Training Letters,
and Decision Assessment Documents may be useful in drawing a claims
processor's attention to a particular nuance or change in the claims
process, they do not necessarily serve as useful training tools. The
usefulness of such materials is further diminished if claims processors
are not given adequate time to digest the materials VBA provides to
notify them of changes in interpretation or implementation of policies,
regulations, statutes, or case law.
GAO's favorable findings about the effectiveness of centralized new
hire training confirm the benefits of a curriculum that is developed,
designed and implemented through Central Office. Therefore, I urge this
Subcommittee to support greater centralization of mandatory continuing
training, specifically a nationally uniform curriculum taught by a
cadre of instructors at every RO who have completed the Instructor
Development Curriculum, and who are available to implement training
materials designed by Central Office. The presence of a team of
qualified instructors at every RO, charged with implementing a relevant
curriculum developed by Central Office staff and accountable to Central
Office, would help close the gap between VBA Central Office's
expectations for training and conflicting or incompatible goals of
local management.
The breakdown in VBA's training process seems to occur at two
critical junctures: first, at the RO where time devoted toward training
may be viewed as an unwarranted impediment to achieving immediate
production goals; and second, between Central Office and ROs because
Central Office does not adequately identify trends in errors that are
amenable to training or provide enough specific curricula for
continuing employee education.
VBA should apply lessons learned from its Remand Quality Review
Project to acquire data pertaining to common development errors by
reviewing a statistically relevant sample of deferred rating decisions
from a wide range of ROs. VBA should then use that data to tailor
training to common development errors, including improper deferrals of
rating decisions, and develop effective strategies to help employees
avoid those time consuming errors in the first instance. This type of
feedback loop already exists in the healthcare arena to prevent medical
errors, including the Veterans Health Administration that has long
played a leadership role in medical error reduction.
Performance Standards Must Reflect Claims Complexity and Legal
Requirements
VBA recognizes that increasingly complex claims and continuous
refinement of the legal requirements attendant to claims processing has
a detrimental effect on the size of VBA's pending workload inventory.
However, VBA has either failed or refused to recognize that those same
factors have a detrimental effect on the productivity of individual
claims processors insofar as additional time is needed to develop,
adjudicate, or review claims. By not adjusting individual productivity
standards to reflect the increasing complexity and difficulty of the
claims process, VBA may once again be failing to provide the service
its claimants deserve. The needs of claimants intersect with the
requirements imposed on claims adjudicators precisely in the
implementation of performance standards. If claims processors are
required to choose between developing, rating, or reviewing a case in
accordance with all legal requirements and fulfilling their production
requirements, a temptation is presented to make a decision in favor of
their own immediate interests. As an example, RVSRs are generally given
no production credit for identifying development deficiencies in a
given case, or directing action to correct those deficiencies via
deferred rating actions. Often, such deferred actions are time-
consuming and complex. Consequently, RVSRs are often met with a choice
between meeting their productivity requirements and ensuring that
decisions are rendered in accordance with all applicable duty to assist
requirements and are, in essence, given no meaningful credit for
ensuring that claims are adequately developed. One of the many
unfortunate side effects of this problem is that the VSRs charged with
developing claims oftentimes receive no meaningful feedback from RVSRs
concerning development deficiencies. Of course, the ultimate effect of
this system is felt by claimants, and reflected in an increasing appeal
rate. Such a system of measuring ``productivity'' is disrespectful to
claims processors and claimants alike. Productivity requirements that
do not take into account the increased time necessary to develop,
adjudicate, or review increasingly complex claims in an increasingly
stringent legal environment ultimately lead to bad service for
claimants, since productivity is often rewarded over quality.
As AFGE has urged this Subcommittee in the past, I cannot overstate
the importance of requiring thorough time-motion studies so that VBA
performance standards for claims processors may be informed and
adjusted by reference to valid and scientific data. Unless and until
VBA has valid empirical evidence concerning what claims adjudicators
can reasonably be expected to accomplish in a given period, any mandate
concerning what those employees must do in the same period will be
suspect. Moreover, insofar as such suspect standards are used to
project the need for additional human resources or trends concerning
future claims inventory levels, any projections upon which they are
based will also be suspect. The proposal in H.R. 5892 for a study of
the VBA work credit system will provide valuable data toward this goal.
Nor should VBA continue to set performance standards for claims
processors by fiat with reference to the ``experience'' of managers who
are years removed from any meaningful contact with the day-to-day
exigencies of claims processing, and expect to meaningfully address
workload trends or human capital requirements. These managers should be
required to devote a portion of their workday (e.g. 50 percent) to
developing or adjudicating claims in order to keep a fresh perspective
of what it takes to conform to an individual position description. Such
a requirement would also have the benefit of reducing the pending
claims inventory. Therefore, I also support the requirement in H.R.
5892 that managers pass the same certification tests as the employees
they supervise. The absence of such a requirement would permit non-
certified supervisors to be charged with critiquing the work of
certified employees, thereby seriously undermining the credibility of
the certification process.
Increased tracking and implementation of continuing training at VBA
is laudable, but only insofar as it will ultimately serve claimants. An
educated and well-trained workforce should be one of VBA's highest
goals. However, VBA's claims processing workforce must also be afforded
adequate time to perform their duties in accordance with their
training, and that consideration must be reflected in individual
performance standards.
Methods to Increase Accountability and Reduce Rating Variances
If continuing training of VBA claims processors is made consistent
by reference to a curriculum created by VBA centralized training staff
and implemented by a corps of instructors accountable to VBA's Central
Office, rating variances could reasonably be expected to decrease as
claims processors conform their individual activities to their uniform
training.
Conclusion
I view the GAO report not as an indictment on claims processors'
skills, abilities, or willingness to learn as much as a description of
VBA's failure to provide relevant, useful continuing training or
adequately track the efficacy of that training. Ultimately, training
deficiencies at VBA are not the product of nor should they be the
responsibility of individual claims processors.
The claims processing workforce at VBA is among of the finest and
most dedicated workforces in Government. Unfortunately, insofar as VBA
inadequately provides continuing training to claims processors and then
attempts to hold them responsible for deficiencies in quality and
productivity, claims processors may be disproportionately affected by
inadequate training. To illustrate that fact it might be instructive
for this Subcommittee to inquire of VBA whether they can recall a
single instance of any VBA manager who has ever been disciplined,
demoted, or formally reprimanded for failing to adequately train an
employee. In contrast, there are numerous instances of long term claims
processors with good work histories being disciplined, demoted,
formally reprimanded, or even discharged for failing to meet their
productivity or quality requirements. The crux of the problem is a
failure by VBA to provide the continuing training necessary for our
Members to be productive, efficient and accurate in fulfillment of
their duties. The onus of inadequate training should not be
disproportionately borne by employees charged with the day to day
processing of claims or the constituency they are honored to serve.
Thank you.
Prepared Statement of Kerry Baker, Assistant National Legislative
Director, Disabled American Veterans
Mr. Chairman and Members of the Subcommittee:
On behalf of the 1.3 million members of the Disabled American
Veterans (DAV), I am honored to appear before you today to discuss the
effectiveness of the Veterans Benefits Administration's (VBA's)
training, performance management, and accountability requirements. In
accordance with our congressional charter, the DAV's mission is to
``advance the interests, and work for the betterment, of all wounded,
injured, and disabled American veterans.''
TRAINING
VBA has a standard training curriculum for new claims processors
and an 80-hour annual training requirement for all claims processors.
The training program in VBA is basically a three-stage system. First,
VBA policy requires new staff to complete some orientation training,
which is provided in their home offices. Second, they are required to
attend a 2- to 3-week centralized training course that provides a basic
introduction to job responsibilities. Third, new staff are required to
spend several more months in training at their home offices, which
includes on-the-job training and/or instructor-led training that follow
a required curriculum via use of an online learning tool called the
Training and Performance Support System (TPSS). VBA policy states that
all claims processors are required to complete a minimum of 80 hours of
training annually. VA Regional Offices (ROs) have some discretion over
what training they provide to meet this requirement.
The VA's three-phased program for new claims processors is designed
to deliver standardized training, regardless of training location or
individual instructors. Topics included in the training program contain
a lesson plan with review exercises, student handouts, and copies of
slides used during the instructor's presentation. The VBA also has an
annual 80-hour training requirement for new and experienced rating
veterans' service representatives (RVSRs) and veteran service
representatives (VSRs).
The first phase of training for new RVSRs is prerequisite training.
It begins at new RVRSs' home regional offices when they begin working.
The prerequisite training is designed to lay the foundation for future
training by introducing new employees to topics such as the software
applications used to process and track claims, medical terminology, the
system for maintaining and filing a case folder, and the process for
requesting medical records. VBA specifies the topics that must be
covered during prerequisite training: however, ROs can choose the
format for the training and the timeframe. New VSRs and RVSRs typically
spend two to 3 weeks completing prerequisite training in their home
office before they begin the second program phase. Nonetheless, VA
personnel informed the DAV that many new employees are only allowed
approximately 7 days to complete the training.
The second phase of training is known as centralized training,
wherein new VSRs and RVSRs spend approximately 3 weeks in classroom
training. The 3-week timeframe is misleading because the first and last
portions of week one and three are utilized for travel. Therefore, the
actual training time is closer to 2 weeks.
Participants from multiple ROs are typically brought together in
centralized training sessions, which is usually at the Veterans
Benefits Academy in Baltimore, Maryland. Centralized training provides
an overview of the technical aspects of the VSR and RVSR positions.
These training classes usually have at least three instructors, but the
actual number can vary depending on the size of the group. VBA's goal
is to maintain a minimum ratio of instructors to students.
To practice processing different types of claims, VSRs work on
either real or hypothetical claims specifically designed for training.
Centralized training for new RVSRs focuses on topics such as: systems
of the human body; how to review medical records; and, how to interpret
medical exams. To provide instructors for centralized training, VBA
relies on RO staff who have received training on how to be an
instructor. Centralized training instructors may be VSRs, RVSRs,
supervisors, or other staff identified by RO managers as having the
capability to be effective instructors.
The VBA has increased the number of training sessions because of
the influx of new staff. In fiscal year 2007, VBA increased the
frequency of centralized training and its student capacity at the
Veterans Benefits Academy. During fiscal year 2007, VBA held 67
centralized training sessions for 1,458 new VSRs and RVSRs. Centralized
training sessions were conducted at 26 different ROs during fiscal year
2007, in addition to the Veterans Benefits Academy. By comparison,
during fiscal year 2006, VBA held 27 centralized training sessions for
678 new claims processors. Nonetheless, VBA has not run its benefits
academy near to full capacity in 2008, the reasons for which are
unclear.
When new VSRs and RVSRs return to their home office after
centralized training, they are required to begin their third phase of
training, which is supposed to include on-the-job, classroom, and
computer-based training modules that are part of VBA's Training and
Performance Support Systems (TPSS), all conducted by and at the RO. New
VSRs and RVSRs typically take about 6 to 12 months after they return
from centralized training to complete all the training requirements for
new staff.
In addition to the foregoing three-phase training program, VBA also
requires 80 hours of annual training for all VSRs and RVSRs. The
training is divided into two parts. At least 60 hours must come from a
list of core technical training topics identified by Compensation and
Pension (C&P) Service. VBA specifies more core topics than are
necessary to meet the 60-hour requirement, so regional offices can
choose those topics most relevant to their needs. They can also choose
the training method used to address each topic, such as classroom or
TPSS training. The RO managers decide the specificities of the
remaining 20 hours.
Analysis and Recommendations
The DAV has consistently maintained that VA should invest more in
training adjudicators and decision-makers, and that it should hold them
accountable for higher standards of accuracy. Nonetheless, such
training has not been a high priority in VBA. We have further
consistently stated that proper training leads to better quality, and
that quality is the key to timeliness. Timeliness follows from quality
because omissions in record development, failure to afford due process,
and erroneous decisions require duplicative work, which add to the load
of an already overburdened system. The VBA will only achieve such
quality when it devotes adequate resources to perform comprehensive and
ongoing training, devotes sufficient time to each case, and imposes and
enforces quality standards through effective quality assurance methods
and accountability mechanisms.
One of the most essential resources is experienced and
knowledgeable personnel devoted to training. More management devotion
to training and quality requires a break from the status quo of
production goals above all else. In a 2005 report from VA's Office of
Inspector General, VBA employees were quoted as stating: ``Although
management wants to meet quality goals, they are much more concerned
with quantity. An RVSR is much more likely to be disciplined for
failure to meet production standards than for failing to meet quality
standards,'' and that ``there is a lot of pressure to make your
production standard. In fact, your performance standard centers around
production and a lot of awards are based on it. Those who don't produce
could miss out on individual bonuses, etc.'' \1\ Little if anything has
changed since the Inspector General has issued this report.
---------------------------------------------------------------------------
\1\ Department of Veterans Affairs Office of Inspector General,
Rep. No. 05-00765-137, Review of State Variances in VA Disability
Compensation Payments 61 (May 19, 2005).
---------------------------------------------------------------------------
A review of VBA's training programs mentioned above reveals that
its problems caused by a lack of accountability do not begin in the
claims development and rating process--they begin in the training
program. Essentially, we can find little, if any, measurable
accountability in VBA's training program.
For example, despite VBA's program requirements for its new hires
to complete phase-one training before advancing to the phase-two
centralized training, some VA employees anonymously informed the DAV
that many candidates begin centralized training without having had the
opportunity to participate in and/or complete phase-one training.
Additionally, candidates are not held responsible via formal testing on
subjects taught during phase-one training. While oversight may or may
not exist for this portion of training, the DAV could find none.
Without resorting to a critique of the substance of VBA's subject
matter taught during phase-two training, or any other phase for that
matter, we limit our analysis again to accountability. As in phase one,
VBA refuses to test participants of phase-two training. The obvious
goal is to ensure employees attend the required course--ensuring that
employees achieve VBA's learning objectives appears to have no priority
whatsoever.
By now, a new employee has approximately 1 month of training and is
supposedly prepared for phase-three training. Keep in mind that during
phase three, new employees will work on real-world cases whose outcomes
affect the lives and livelihoods of disabled veterans and their
families. Real cases notwithstanding, again there is no accountability,
no testing, and no oversight outside that of which is provided locally;
again, that oversight is not measured nationally.
The result of such an unsupervised and unaccountable training
system is that no distinction exists between unsatisfactory performance
and outstanding performance. This lack of accountability during
training further reduces, or even eliminates, employee motivation to
excel. This institutional mindset is further epitomized in VBA's day-
to-day performance, where employees throughout VBA are reminded daily
that optimum work output is far more important quality performance and
accurate work.
The effect of VBA's lack of accountability in its training program
was demonstrated when it began offering skills certification tests to
support certain promotions. Beginning in late 2002, VSR job
announcements began identifying VSRs at the GS-11 level, contingent
upon successful completion of a certification test. The test consisted
of 100 multiple-choice questions, which were open-book. The VA allowed
participants to use online references and any other reference material,
including individually prepared notes in order to pass the test.
The first validation test was performed in August 2003. There were
298 participants in the first test. Of these, 75 passed for a pass rate
of 25 percent. VBA conducted a second test in April 2004. Out of 650
participants, 188 passed for a pass rate of 29 percent. Because of the
low pass rates on the first two tests, a 20-hour VSR ``readiness''
training curriculum was developed to prepare VSRs for the test. A third
test was administered on May 3, 2006, to 934 VSRs nationwide. Still,
the pass rate was only 42 percent. Keep in mind that these tests were
not for training, they were to determine promotions from GS-10 to GS-
11.
The VBA recently began similar testing with RVSRs. The DAV was
unable to obtain those tests result. VA employees nonetheless informed
us that VBA's test results for RVSRs were no better than test results
for VSRs.
These results reveal a certain irony, in that VBA will offer a
skills certification test for promotion purposes, but does not require
comprehensive testing throughout its training curriculum. The following
``accountability'' portion of this testimony further illustrates the
product of inadequate training.
Mandatory and comprehensive testing designed cumulatively from one
subject area to the next, for which VBA then holds trainees
accountable, should be the number one priority of any plan to improve
VBA's training program. Further, VBA should not allow trainees to
advance to subsequent stages of training until they have successfully
completed such testing.
To be fair, the DAV understands that VBA is not solely at fault on
this subject. The VA employees union has objected to the type of
testing mentioned herein. We do not expect such objections to cease. In
fact, we feel the only way to moot these objections is for Congress to
mandate such testing through statutory change. Section 105 of H.R. 5892
mandates some testing for claims processors and VBA managers, which is
an improvement over current practices; however, it does not mandate the
type of testing during the training process as explain herein.
Measurable improvement in the quality of and accountability for
training will not occur until such mandates exist.
ACCOUNTABILITY
The DAV has consistently stated that, in addition to training,
accountability is the key to quality, and therefore to timeliness as
well. As it currently stands, almost everything in VBA is production
driven. In addition to basing personnel awards on production, the DAV
strongly believes that quality should be awarded at least on parity
with production. However, in order for this to occur, VBA must
implement stronger accountability measures for quality assurance.
VA's quality assurance tool for compensation and pension claims is
the Systematic Technical Accuracy Review (STAR) program. Under the STAR
program, VA reviews a sampling of decisions from regional offices and
bases its national accuracy measures on the percentage with errors that
effect entitlement, benefit amount, and effective date.
Inconsistency signals outright arbitrariness in decision-making,
uneven or insufficient understanding of governing criteria, or rules
for decisions that are too vague or overly broad and allows them to be
applied according to the prevailing mindset of a particular group of
decision-makers. Obviously, VA must detect inconsistencies before the
cause or causes can be determined and remedied.
Simply put, there is a gap in quality assurance for purposes of
individual accountability in quality decision-making. In the STAR
program, a sample is drawn each month from a regional office workload
divided between rating, authorization, and fiduciary end-products. For
example, a monthly sample of ``rating''-related cases generally
requires a STAR review of 10 rating-related end products.\2\ Reviewing
10 rating-related cases per month for an average size regional office,
an office that would easily employee more than three times that number
of raters, is undeniable evidence of a total void in individual
accountability. If an average size regional office produced only 1,000
decisions per month, which we feel is quite conservative, the STAR
program would only review 1 percent of the total cases decided by that
regional office. Those figures leave no room for trend analysis, much
less personal accountability.
---------------------------------------------------------------------------
\2\ See M21-4, Ch. 3, Sec. 3.02.
---------------------------------------------------------------------------
To put this in better perspective, according to VA's 2007
performance and accountability report, the STAR program reviewed 11,056
compensation and pension (C&P) cases in 2006 for improper payments.
While this number appears significant, the total number of C&P cases
available for review was 1,540,211. Therefore, the percentage of cases
reviewed was approximately seven tenths of 1 percent, or 0.72 percent.
Another method of measuring the VA's need for more accountability
is an analysis of the Board's Summary of Remands, while keeping in mind
that its summary represents a statistically large and reliable sample
of certain measurable trends. The examples must be viewed in the
context of the VA (1) deciding 700,000 to 800,000 cases per year; (2)
receives over 100,000 NODs; and (3) submits 40,000 appeals to the
Board. The examples below are from October 2006 to October 2007.
Remands resulted in 998 cases because no ``notice'' under section
5103 was ever provided to the claimant. The remand rate was much higher
for inadequate or incorrect notice; however, considering the confusing
(and evolving) nature of the law concerning ``notice,'' we can only
fault the VA when it fails to provide any notice.
VA failed to make initial requests for SMRs in 667 cases and failed
to make initial requests for personnel records in 578 cases. The number
was higher for additional followup records requests following the first
request. This number is disturbing because initially requesting a
veteran's service records are the foundation to every compensation
claim. It is claims development 101.
The Board remanded 2,594 cases for initial requests for VA medical
records and 3,393 cases for additional requests for VA medical records.
The disturbing factor here is that a VA employee can usually obtain VA
medical records without ever leaving the confines of one's computer
screen.
Another 2,461 cases were remanded because the claimant had
requested a travel board hearing or video-conference hearing. Again,
there is a disturbing factor here. A checklist is utilized prior to
sending an appeal to the Board that contains a section that
specifically asked whether the claimant has asked for such a hearing.
The examples above totaled 7,298 cases or nearly 20 percent of
appeals reaching the Board, all of which cleared the local rating board
and the local appeals board with errors that are elementary in nature.
Yet they were either not detected or they were ignored. Many more cases
were returned for more complex errors. But for nearly a 20-percent
error rate on such basic elements in the claims process passing through
VBA's most senior of rating specialist is simply unacceptable.
The problem with the VA's current system of accountability is that
it does not matter if VBA employees ignored these errors because those
that commit such errors are usually not held responsible. They
therefore have no incentive to concern themselves with the quality of
their work. Above all else, these figures showing that the VA's quality
assurance and accountability systems require significant enhancement.
Recommendation
Congress should require the Secretary to report on how the
Department could establish a quality assurance and accountability
program that will detect, track, and hold responsible those VA
employees who commit egregious errors. Such report should be generated
in consultation with veterans' service organizations most experienced
in the VBA claims process.
The DAV believes that effective accountability can be engineered in
a manner that holds each VBA employee responsible for his/her work as a
claim moves through the system while at the same time holds all
employees responsible simultaneously. As errors are discovered
(definition of such errors to be determined, but specific to employee
responsibility), employees responsible for such errors must be held
accountable by forfeiture of work credit percentage.
For example, if a Decision Review Officer (DRO) reverses a decision
from a frontline rating specialist because of error, as opposed to
difference of opinion or receipt of new evidence, then the frontline
employee should be subject to forfeiture of a portion of work credit
that is normally used to track production standards applicable to
performance bonuses. In turn, if a case proceeds to the BVA and is
reversed or remanded on similar grounds, then both the frontline rater
as well as the DRO should forfeit work credit, and so one. The same
should apply to Veterans' Law Judges at BVA when the Court of Appeals
for Veterans Claims finds error in a BVA decision.
Such a cumulative accountability system would effectively eliminate
potential abuse of the system through the proverbial good-old-boy club.
One employee would be far less likely to cover for errors or look the
other way from errors committed by a fellow employee if they knew their
performance standards were equally at risk. This type of system would
ensure personal accountability at every stage in the claims process
without seriously disrupting or dismantling VBA's current performance
measurement system.
PERFORMANCE MEASUREMENT
VA's benefits delivery system has become particularly multifaceted,
especially when considering the various types of claims a beneficiary
may file, the various stages of development and decision-making within
each claim, and the potential changes that can occur at any particular
stage of the claim. Currently, VA utilizes over 50 pending end-product
codes\3\ for a multitude of actions. The number of end-product codes
may be further expanded by using ``modifiers'' that designate specific
``issues'' for types of claims within a certain broader category.
---------------------------------------------------------------------------
\3\ M21-4, App. A, Glossary of Terms and Definitions. Manpower
Control and Utilization in Adjudication Divisions (Pending End Product:
``A claim or issue on which final action has not been completed. The
classification code identified refers to the end product work unit to
be recorded when final disposition action has been taken.'').
---------------------------------------------------------------------------
The VA's end product codes are used in conjunction with its
productivity and work measurement system. The productivity system is
the basic system of work measurement used by C&P Service, but it is
also used for report and tracking. Additionally, VA's end-product codes
are also utilized in the STAR program. The program is also a tool used
for quantitative measurement, a tool utilized in preparing budget
forecasts, and in distributing available staffing.
Quantitative and productivity measurement are also tools used in
comparing and tracking employment of resources. Both productivity
measurement and work measurement are tools available to management for
this purpose. Quantitative measurement also allows Central Office and
Area Offices to compare stations and to track both local and national
trends. Productivity measurement and work measurement are complementary
measurement systems that each depend, in part, on VA's end product code
system. The end-product code system is further used in determining work
credit provided to VA's employees and is therefore vital in measuring
employee production goals and awarding performance awards. Changes
should not be mandated that would cause VA to lose the ability to
manage and track its day-to-day functions.
The DAV finds no measurable flaws in VBA's overall measurement
systems. In fact, our foregoing recommendations concerning improvements
in VBA's accountability would draw on the strengths in its measurement
systems, thereby allowing easier and less disruptive implementation of
stronger and more effective accountability. *** FORCED SO
BODONI DASH IS NOT AT TOP OF PAGE *** deg.
We hope the Subcommittee will review these recommendations and give
them consideration for inclusion in your legislative plans. Mr.
Chairman, thank you for inviting the DAV to testify before you today.
Prepared Statement of Ronald B. Abrams, Joint Executive Director,
National Veterans Legal Services Program
Mr. Chairman and Members of the Committee:
I am pleased to have the opportunity to submit this testimony on
behalf on behalf of the National Veterans Legal Services Program
(NVLSP). NVLSP is a nonprofit veterans service organization founded in
1980 that has been assisting veterans and their advocates for 28 years.
NVLSP has trained thousands of service officers and lawyers in veterans
benefits law, and has written educational publications that thousands
of veterans advocates regularly use as practice tools to assist them in
their representation of VA claimants. NVLSP also conducts quality
reviews of the VA regional offices on behalf of The American Legion.
NVLSP also represents veterans and their families on claims for
veterans benefits before VA, the U.S. Court of Appeals for Veterans
Claims (CAVC), and other Federal courts. Since its founding, NVLSP has
represented over 1,000 claimants before the Board of Veterans' Appeals
and the Court of Appeals for Veterans Claims (CAVC). NVLSP is one of
the four veterans service organizations that comprise the Veterans
Consortium Pro Bono Program, which recruits and trains volunteer
lawyers to represent veterans who have appealed a Board of Veterans'
Appeals decision to the CAVC without a representative.
We believe that the effectiveness of VBA training should be
measured by the quality of the work product produced by VA
adjudicators. Therefore, the quality and timeliness of VA adjudications
should reflect the effectiveness or lack thereof, of VA training.
Because the quality of VA adjudications is inadequate NVLSP must
conclude that VBA training is not effective or adequate.
In the experience of NVLSP (over 10 years of quality reviews, in
conjunction with The American Legion of approximately 40 different
VAROs combined with extensive NVLSP representation before the CAVC),
most of the most egregious VA errors are a result of premature
adjudications. Many VA managers emphasize quantity over quality. VA
managers and VA adjudicators have let us know that because production
is paramount, training is deemphasized because the time spent training
reduces the time available to produce decisions, and the training of VA
adjudicators regarding the procedures designed to protect the right of
claimants seeking VA benefits also reduces production.
In September 2008, courageous VA regional office employees filed a
grievance exposing this overemphasis on production. This grievance is
attached to my testimony. The grievance asserts that:
the regional office created and encouraged a culture in
which quantity is emphasized to the detriment of quality;
the VARO failed to properly implement training
initiatives to assure that those reviewing claims are sufficiently
trained in the relevant disciplines so as to reduce errors, improve the
quality of claims processing, and successfully complete newly
implemented certification requirements; and
the VARO failed to properly implement a fair and
impartial performance appraisal system that assures that quantitative
measures of performance are not emphasized to the detriment of measures
of the quality of that performance.
NVLSP believes that the quality of VARO adjudications is much worse
than what is reported by the VA. The remand and reversal statistics
produced by decisions issued by the Board of Veterans' Appeals (BVA or
Board) can be considered an independent review of the quality of
adjudications performed by the VAROs. BVA statistics provided by the
Veterans Appeals Control and Locator System (VACOLS) for FY 07 reveal
that Board decided over 40,000 appeals. The Board allowed 21.12 percent
(that is, granted the claim that the VA regional office had denied) and
remanded 35.36 percent of these appeals back to the VARO because the
VARO had wrongly failed to obtain all of the evidence it should have
attempted to obtain. Therefore, in 56.48 percent of the appeals decided
by the BVA, the BVA either reversed or remanded the VARO decision. This
56.48 percent statistic could be considered an error rate. Even if we
take into account the fact that new evidence can be added at the Board
and deduct 20 percent from the 56.48 percent, an error rate as high as
36 percent is not acceptable and does not verify the low error rate
claimed by VA in its VA Star Reports (close to a 90 percent ``accuracy
rate''). The reversal/remand rate thus far for FY 2008 is 59.4 percent.
The news gets worse. The BVA, in its rush to make final decisions
and to avoid remands also quite often prematurely denies claims that
should have been remanded. Of course, the error was originally
committed by the VARO, not the BVA. In September 2007, my fellow Joint
Executive Director, Bart Stichman, testified that ``[f]or more than a
decade, the Court's [Court of Appeals for Veterans Claims (Court or
CAVC)] annual report card of the BVA's performance has been remarkably
consistent. The 12 annual report cards issued over the last 12 years
yields the following startling fact: of the 16,550 Board decisions that
the Court individually assessed over that period (that is, from FY 1995
to FY 2006), the Court set aside a whopping 77.7 percent of them (that
is, 12,866 individual Board decisions). In each of these 12,866 cases,
the Court set aside the Board decision and either remanded the claim to
the Board for further proceedings or ordered the Board to award the
benefits it had previously denied. In the overwhelming majority of
these 12,866 cases, the Court took this action because it concluded
that the Board decision contained one or more specific legal errors
that prejudiced the rights of the VA claimant to a proper decision.''
How should a veteran seeking VA disability benefits feel? The Board
of Veterans' Appeals reverses and remands over 50 percent of all VARO
adjudications and the CAVC sets aside over 77 percent of the BVA
decisions that it reviews. These numbers do not inspire confidence in
the quality of VA adjudications.
It is clear that the adjudication culture at the VAROs needs to be
changed. Many VA managers act like they are producing widgets rather
than adjudicating claims filed by real people. Their goal should not be
just prompt adjudication; the goal should be a timely, accurate and
fair adjudication.
Even VA employees are frustrated and upset by their lack of
training and the overemphasis on production. The following is a quote
from the grievance attached to my testimony.
``As set forth in the Grievance, management has: (1)
established an environment in which ``there is an extreme
pressure to produce minimally acceptable work at any cost'';
(2) ``developed an employee culture where striving to achieve
unreasonable production criteria is paramount and . . .
actually doing difficult necessary work on cases is strongly
discouraged on penalty of removal''; and (3) [management]
commonly expresses sentiments such as, ``just decide the case
and let the veteran appeal.''
If the assertions in this grievance are true then we call upon VA
Central Office management to hold regional office management
accountable.
Thank you for permitting NVLSP to testify on such an important
issue.
__________
Kirkland and Ellis LLP
Chicago, IL
August 29, 2008
Via OVERNIGHT MAIL
Joyce A. Cange
Director, Cleveland Regional Office
Department of Veterans Affairs,
Veterans Benefits Administration
A.J. Celebrezze Federal Building
1240 East 9th Street
Cleveland, Ohio 44199
Re: April 2008 Grievance Of Local 2823
Dear Ms. Cange:
I write to advise that Kirkland & Ellis LLP represents Local 2823
of the American Federation of Government Employees (``Local 2823'') in
connection with the grievance that Local 2823 filed and amended in
April 2008 (the ``Grievance'' or ``GRV''), and I further write to
respond to an April 25, 2008 memorandum (the ``Memorandum'' or ``MEM'')
from your office that denied the Grievance for various reasons.\1\
---------------------------------------------------------------------------
\1\ Local 2823 does not address herein all the matters raised by
its Grievance or assertions made by your office's Memorandum in
response, as Local 2823 does not believe that such unaddressed matters
need to be resolved at this juncture to have a productive discussion of
the principal disputes between the parties. However, Local 2823 does
not waive and expressly reserves all rights and claims it has,
including those that are not expressly addressed herein but are
otherwise fairly encompassed by the Grievance.
---------------------------------------------------------------------------
1. The Grievance
Local 2823's Grievance demands resolution of several disputes that
Local 2823 believes are negatively affecting the processing of
veterans' benefits claims at the Cleveland VA Regional Office (the
``Cleveland VARO''). Specifically, the Grievance states that management
of the Cleveland VARO has, among other things:
1. created and encouraged a culture in which the quantity of
benefits claims rated is emphasized to the detriment of the quality of
the rating of those claims (see GRV at 3-4 para.para. (6)(b)-(d);
hereafter, the ``Culture Claim''); \2\
---------------------------------------------------------------------------
\2\ This may be one of the reasons why Ohio ranks second to last
nationally in the average compensation awarded to its disabled veterans
according to the VBA's Annual Benefits Report for FY2006. See, e.g.,
The Plain Dealer at A7 (Apr. 12, 2008) (summarizing disparate
disability compensation averages across states, with only Indiana
having a lower average than Ohio, but having fewer than half of the
number of disability recipients as Ohio); accord VBA Annual Benefits
Report--Fiscal Year 2006 at 103Sec. 1A0953 (state-by-state figures).
---------------------------------------------------------------------------
2. failed to properly, equitably and promptly implement training
initiatives to assure that those reviewing claims are sufficiently
trained in the relevant disciplines so as to reduce errors, improve the
quality of claims processing, and successfully complete newly
implemented certification requirements (see GRV at 2-3 para.para. 5(d)-
(e); hereafter the ``Training Claim''); \3\ and
---------------------------------------------------------------------------
\3\ See, e.g., 1997 Master Agreement between the Dep't of Veterans
Affairs and the Am. Fed'n of Gov't Employees [hereafter, ``MA''] at 142
(Art. 34) Sec. 1(A) (``The Department and the Union agree that the
training and development of employees is of critical importance in
carrying out the mission of the Department.''); id. Sec. 9
(``Procedures which ensure fair and equitable training opportunities
are appropriate subjects for local bargaining.''). See also 1/20/00
VBA-AFGE Mem. of Understanding [hereafter ``1/20/00 MOU''] para. 2
(``VBA commits to a standard of excellence in the quality and quantity
of training for all employees.''); id. para. 10 (``Local unions will be
given the opportunity to bargain over appropriate issues not otherwise
in conflict with this or other national level agreements, prior to
local implementation.''); 5/19/06 Mem. of Understanding; Interim Cert.
MOU for FY2006 [hereafter ``5/19/06 MOU''] at preamble (``The terms of
the original MOU on Certification dated 2000 are still in effect.'');
id. para. 15 (``The parties may negotiate locally on this subject
provided it does not conflict, interfere with, or impair the
implementation of this MOU and the Master Agreement.'').
---------------------------------------------------------------------------
3. failed to properly implement a fair and impartial performance
appraisal system that assures that quantitative measures of performance
are not emphasized to the detriment of measures of the quality of that
performance (see GRV at 4 para. 6(a); hereafter the ``Performance
Appraisal Claim'').\4\
---------------------------------------------------------------------------
\4\ See, e.g., Am. Art. 26 Sec. 1(D) (``The parties share an
interest in improving the performance of the Department's workforce'');
id. Sec. 1(H) (``The performance appraisal process as set forth in this
Article is intended to be innovative and evolutionary in nature. Its
effectiveness is critical to the Department achieving its mission.'').
While recent disputes between management and Local 2823 (and
certain of its officers) unfortunately have created an adversarial
atmosphere and tone, it is important to underscore and reaffirm that
Local 2823's fundamental reason for instituting this Grievance is to
improve the quality of the processing of benefits claims by assuring
that those responsible for reviewing such claims are (i) encouraged and
expected to perform high quality reviews (even where doing so may
reduce the quantity of claims that can be reviewed), (ii) sufficiently
trained to perform these high quality reviews, and (iii) appraised
based both on the quality as well as the quantity of their ratings--all
of which, Local 2823 believes, is consistent with and required by VA
policy, applicable agreements of the parties, and applicable law.
Local 2823 believes that the best way to achieve these objectives
is by working collaboratively and cooperatively with management to
achieve them.\5\ Accordingly, Local 2823 and the undersigned request
that within the next 2 weeks you provide them with a date and time when
we can sit down with you and your management team in the next 60 days
to see if there is a way that the parties can work together to
accomplish the aforementioned objectives and resolve their differences
by means of such ADR.\6\
---------------------------------------------------------------------------
\5\ See, e.g., MA at 4 (Preamble) Sec. 2 (``The Department and the
Union agree that a constructive and cooperative working relationship
between labor and management is essential to achieving the Department's
mission and to ensuring a quality work environment for all employees.
The parties recognize that this relationship must be built on a solid
foundation of trust, mutual respect, and a shared responsibility for
organizational success. [para.] Therefore, the parties agree to work
together in partnership and through this Master Agreement to identify
problems and craft solutions, enhance productivity, and deliver the
best quality of service to the Nation's veterans.'') (emphasis added).
See also id. at 8 (Art. 3) Sec. 2 (noting important ``Partnership''
principles that are to characterize parties' relationships, including
``Shared responsibility,'' ``Sharing of information,'' ``Reaching joint
agreements and making joint recommendations,'' ``Use of alternative
dispute resolution, interest-based, problem-solving techniques, and
facilitation,'' and ``Integration of interests''--all addressed to the
common goal of establishing and improving ``effective Partnerships
which are designed to ensure a quality work environment for employees,
more efficient administration of VA programs, and improved service to
veterans'') (emphasis added); id. Sec. 3 (``The scope of partnership
will include issues raised by either party regarding: A. Matters
involving personnel policies, practices, and working conditions.'');
id. at 15 (Art. 7) Sec. 1 (``Service to the veteran is the cornerstone
of the relationship between the Department and employees.''); id. at
177 (Art. 146) Sec. 1 (``The parties recognize that a new relationship
between the Union and the Department as full partners is essential for
reforming the Department into an organization that works more
efficiently and effectively and better serves customer needs,
employees, Union representatives and managers.'').
\6\ See, e.g., Master Agreement at 13 (Art. 6) Sec. 1 (``Union and
Management at all levels should be committed to use of ADR problem-
solving methods as a priority to resolve disputed matters.''); id.
Sec. 4(A) (``ADR is an appropriate subject matter for local
negotiations.''); id. at 14 (Art. 6) Sec. 4(D) (``ADR methods may be
used prior to or during a grievance/arbitration or statutory
appeal.''); id. at 165 (Art. 42) Sec. 6 (``The use of ADR is
encouraged. The parties agree that every effort will be made to settle
grievances at the lowest possible level.''); VA Dir. 5023 Sec. 2(a)
(``It is the policy of VA to recognize and deal with lawful labor
organizations on matters of concern to the employees they represent,
and to place primary reliance on informal settlement of any differences
or disputes at the earliest stage possible by discussion between VA
management and representatives of labor organizations.'').
---------------------------------------------------------------------------
If, however, management is unwilling to work collaboratively to
achieve these objectives, then Local 2823 has a responsibility to its
members (and to veterans) to prosecute the Grievance vigorously by
commencing arbitration proceedings, which Local 2823 will not hesitate
to do. Moreover, should the matter proceed to arbitration, for the
reasons set forth below, we are confident that the arbitrator will find
in Local 2823's favor and that your legal and other objections to the
Grievance that are recited in the Memorandum will not be sustained.
2. The Culture Claim
As set forth in the Grievance, management has: (1) established an
environment in which ``there is an extreme pressure to produce
minimally acceptable work at any cost''; (2) ``developed an employee
culture where striving to achieve unreasonable production criteria is
paramount and . . . actually doing difficult necessary work on cases is
strongly discouraged on penalty of removal''; and (3) commonly
expresses sentiments such as, ``just decide the case and let the
veteran appeal.'' (See GRV at 4 para. 6(a); id. para.para. 6(a), (b) &
(d).)
A. The Culture Claim I Sufficiently Particular
The Memorandum your office issued takes exception to the Grievance,
inter alia, on the grounds that it is insufficiently particular in that
it does not identify: (1) ``any specific acts that are being grieved'';
(2) ``any specific times, dates and places for any acts being
grieved''; and (3) ``any rationale supporting or even explaining how
VBA's alleged conduct violated any negotiated agreements or law.''
(Mem. at 1 para. 3.) The Memorandum, however, notably fails to identify
any agreement or law requiring that Local 2823's step-three grievance
(see GRV at 1 para. 1) be stated with greater particularity--because
there is none.
While the Master Agreement recites that a step-two grievance ``must
state, in detail, the basis for the grievance and the corrective action
desired'' (MA at 166 (Art. 42) Sec. 7 (emphasis added)), no such
requirement is prescribed for a step-three grievance like the one at
issue here. Regardless, even were the Grievance deemed a step-two
grievance (or deemed bound by a similar ``detailed basis''
requirement), the Grievance does ``state, in detail, the basis for the
grievance''--e.g., the Culture Claim, the Training Claim, and the
Performance Appraisal Claim. Moreover, the Master Agreement does not
state that ``all facts'' supporting a grievance must be recited in the
grievance in order to satisfy the ``detailed basis'' requirement of
Article 42 Sec. 7; it does not require that all ``acts'' encompassed by
the grievance be separately listed; it does not require specification
of times, dates, and places for all acts encompassed by a grievance;
and it does not require an explanation of each of the ways in which the
conduct at issue violated each applicable agreement and/or law. Nor
would it be reasonable to impose such a particularized pleading
requirement on Local 2823--before any documents have been turned over
by management relating to the claims pled as the basis for the
Grievance.
B. The Culture Claim Is Based on Violations of Applicable VA Policies,
Agreements, and/or Law
Actions of management giving rise to the Culture Claim violate VA
policy as well as applicable agreements and law.
First, VA Directive 5023 properly states and acknowledges that
``[t]he public interest demands the highest standards of employee
performance and the continued development and implementation of modern
and progressive work practices to facilitate and improve employee
performance and the efficient accomplishment of the operation of the
Government.'' See VA Dir. 5023 Sec. 2(b). Further, VA Directive 5023
provides that ``VA management shall carry out its duties in a manner
consistent with the terms and spirit of human resources policies,
principles and procedures that encourage the highest standard of
employee performance and the most efficient accomplishment of VA
operations.'' Id. Sec. 2(c) (emphasis added). Thus, the VBA violates
its own policy by promoting a working culture that discourages the
highest standard of employee performance so as to more expeditiously
process claims. Moreover, the VBA further violates its policy
respecting the efficient accomplishment of VA tasks by promoting a
culture that likely results in more claims decisions that are
vulnerable to reversal on appeal and re-processing on remand--a less
efficient manner of proceeding.
Second, as set forth in Article 7 Sec. 1 of the Master Agreement,
``[s]ervice to the veteran is the cornerstone of the relationship
between the Department and employees,'' and the ``parties recognize the
importance of a strong commitment to a comprehensive Total Quality
Improvement Program (TQI) in the Department.'' (Emphasis added). Thus,
discouraging quality reviews of benefits claims in the name of
processing a greater quantity of benefits claims violates the service-
to-the-veteran imperative that is to inform the work of both the
Department and its employees and ignores the importance of quality
considerations to the reviews undertaken--both of which violate the
letter and spirit of Article 7 Sec. 1 of the Master Agreement. (See
also Am. Art. 26 Sec. 1(A) (``The Department will strive for continuous
improvement in performance to fulfill the Department's commitment to
providing quality customer service.'') (emphasis added); Agreement
Between Dep't of Veterans' Affairs Cleveland Regional Office and AFGE
Local 2823 [hereafter ``Local Agreement'' or ``LA''] Sec. 1(A) (``The
mission of the Department of Veterans Affairs and this Regional Office
is to service the veteran and beneficiaries with timely and quality
service.'') (emphasis added).) Further, pursuant to Article 43
Sec. 3(B) of the Master Agreement, management has an obligation to
bargain locally with Local 2823 as to, inter alia, the ``methods and
means of performing work.'' Thus, by refusing to bargain over the
Culture Claim, which is squarely addressed to the Cleveland VARO's
methods and means of performing work, the Cleveland VARO is abrogating
this provision of the Master Agreement as well.
Third, an atmosphere that promotes quantity at the expense of
quality is fundamentally contrary to the Department's statutory
obligations to improve the quality of the claims rating process as well
as increase the quantity of claims processed. See, e.g., 38 U.S.C.
Sec. 7734(1)(E) (requiring an annual report to Congress that includes,
inter alia, ``actions taken to improve the quality of services provided
and the results obtained'') (emphasis added); id. Sec. 7734(2)
(requiring an annual report to Congress that includes, inter alia,
``information with respect to the accuracy of decisions, including
trends in that information.'').
C. Local 2823's Culture Claim Has Evidentiary Support
There is ample evidentiary support for Local 2823's Culture Claim,
both nationally and locally.
First, recently completed national studies and surveys have
confirmed the perception among many claims ratings personnel nationwide
that the Department has fostered an atmosphere in which quantitative
objectives are pursued to the detriment of qualitative objectives. For
example, in a recent report issued by the Veterans' Disability
Commission in October 2007, they noted the following:
In respect of the criticism concerning balancing quality and
quantity in employee performance, CNAC \7\ discovered there
exists a perception that VA emphasized quantity over quality.
In a national survey, 80 percent of raters said having enough
time to process a claim was one of their top three challenges.
They were also asked to rate the availability of time to decide
a claim, 54 percent of raters said availability of time was
fair or poor. It can be argued that this creates incentives for
RVSRs to make decisions that are not always fully backed by
evidence, which leads to more appeals, and remands, and
increases backlogs in the system. . . .
---------------------------------------------------------------------------
\7\ ``CNAC'' is a shorthand reference to the CNA Corporation, which
conducted scientifically valid and reliable surveys. See 10/07 VDBC
Report at 19 (``The Commission also examined the results of studies
undertaken on its behalf by the CNA Corporation (CNAC). . . .
Additionally, CNAC surveyed VA raters, service officers from veterans
service organizations, and disabled veterans and survivors. These
surveys were scientifically valid and reliable.'').
---------------------------------------------------------------------------
* * *
CNAC believes that the VA's training difficulties are made
exponentially worse because staff feel a need for more training
and that training seems to be sacrificed to meet work quotas.
This emphasis has encouraged a high staff turnover at VA. The
quality of claims is lessened since inexperienced individuals
are taking over for experienced raters.
Veterans Disability Benefits Comm'n, Honoring The Call To Duty:
Veterans' Disability Benefits In The 21st Century [hereafter, ``10/07
VDBC Report''] 342 (Oct. 2007) (emphasis added).\8\
---------------------------------------------------------------------------
\8\ Local 2823 further notes that it believes the failure of the
Cleveland VARO to share the results of the aforementioned study with
Local 2823 was a violation of Master Agreement Article 46 Sec. 8(C),
which on relevant part provides that ``[i]f a third party conducts a
survey and the results are distributed to the Department, the results
will be shared with the Union.'' See also Dep't of Veterans Affairs
(VA) FY2007 Performance and Accountability Report [hereafter, ``11/07
VA Report''] 78 (Nov. 15, 2007) (``The Veterans' Disability Benefits
Commission began work in May 2005 and issued its report in October
2007. VA will study the Commission's recommendations and begin taking
appropriate actions in 2008.'').
Moreover, in recent testimony to Congress, the Deputy Under
Secretary for Benefits, Michael Walcoff, acknowledged that current
quantitative measures of productivity, which do not take into account
the complexity of benefits claims and numbers of issues raised by a
---------------------------------------------------------------------------
claimant, are in need of revision:
To further enhance our ability to monitor performance, the
study team recommends the creation of a performance measurement
system focused on tracking the number of medical disabilities
or issues claimed. IBM believes that this issue-based
performance measurement system, in conjunction with the
existing claim-based performance measurement system, will
result in a more accurate and detailed measure of productivity
and workload. Under the current claim-based performance
measurement system, a regional office is given the same credit
for completing a claim with one issue as a claim with forty
issues. The study team believes that measuring work output by
both number of claims and number of issues at an organizational
level is a more accurate assessment of a regional office's
productivity. In addition, an issued-based performance
measurement at an individual level will provide more
specificity in the activities of staff and result in increased
accountability overall. [para.] VBA agrees with the idea of
adding an issue-based performance measurement system to our
current reporting structure.
February 18, 2008 Statement of Michael Walcoff Before The House
Committee On Veterans' Affairs Subcommittee On Disability Assistance
And Memorial Affairs, Examining The VA Claims Processing System
(emphasis added).
Second, in addition to such national evidence supporting Local
2823's Culture Claim, Local 2823 will be prepared at the arbitration
hearing to present evidence of specific instances of actions taken by
Cleveland VARO management contributing to the establishment and
promotion of a culture in which the quantity of ratings decisions is
valued to the detriment of quality. For example, VSR production
standards at the Cleveland VARO are set at 10 points per day, 25
percent higher than the national standard of 8 points per day. Also,
rating specialists at the Cleveland VARO only receive production credit
for cases they have decided, and not for reviewing cases that are sent
back to VSRs for further development. This means that RVSRs are
incentivized to decide cases without correcting errors in the file
instead of ``losing time'' while they rework cases that are
procedurally or developmentally flawed.
D. Local 2823's Clarified and Revised Information Requests Addressed to
the Culture Claim are Proper
In addition, as noted in the Grievance, Local 2823 requests
information from the Cleveland VARO that is normally maintained,
reasonably available, and necessary for Local 2823 to fulfill its
representational functions and responsibilities with respect to the
prosecution of the Culture Claim, and, to that end clarifies and
revises its requests for information, hereby requesting production of
the following information created on or after January 20, 2000:
Any communications between and among Cleveland VARO
management and supervisors respecting efforts to achieve quantitative
production requirements;
Any communications between and among Cleveland VARO
supervisors and employees respecting efforts to achieve quantitative
production requirements;
Documents sufficient to identify the numbers of employees
who have received awards, bonuses, and/or promotions for achieving or
exceeding quantitative production requirements;
Documents sufficient to identify the numbers of
supervisors who have received awards, bonuses, and/or promotions for
achieving or exceeding quantitative production requirements;
Documents sufficient to identify the numbers of employees
(if any) who have received awards, bonuses, and/or promotions for
achieving or exceeding qualitative performance standards;
Documents sufficient to identify the numbers of
supervisors (if any) who have received awards, bonuses, and/or
promotions for achieving or exceeding qualitative performance
standards;
Documents sufficient to identify the number of employees
who have received demotions, non-satisfactory ratings, or other
disciplinary actions or negative performance ratings for their failure
to achieve or exceed quantitative production requirements;
Documents sufficient to identify the number of
supervisors who have received demotions, non-satisfactory ratings, or
other disciplinary actions or negative performance ratings for their
failure to achieve or exceed quantitative production requirements;
Documents sufficient to identify the number of employees
(if any) who have received demotions, non-satisfactory ratings, or
other disciplinary actions or negative performance ratings for their
failure to achieve or exceed qualitative performance standards;
Documents sufficient to identify the number of
supervisors (if any) who have received demotions, non-satisfactory
ratings, or other disciplinary actions or negative performance ratings
for their failure to achieve or exceed qualitative performance
standards;
Any communications from employees in the Cleveland VARO
to supervisors or other Cleveland VARO management complaining about
quantitative production requirements and their effect on the quality of
the claims processing;
Any communications from supervisors in the Cleveland VARO
complaining about quantitative production requirements and their effect
on the quality of the claims processing;
Any statistical information tracking the quantitative
performance of the Cleveland VARO in terms of processing clams;
Any statistical information tracking the quality of the
Cleveland VARO in terms of processing claims; and
Any statistical information respecting the number and
rate of Cleveland VARO claims determinations that are reversed or
remanded on appeal.
Moreover, it is plain that such information respecting the
treatment of employees and supervisors with respect to the achievement
of quantitative production requirements as compared to the treatment of
employees and supervisors with respect to the achievement of
qualitative performance standards is both relevant and necessary to
understand and assess the working culture created by management
favoring quantitative attainments over qualitative achievements. (See
MA at 177 (Art. 46) Sec. 5 (``The Department agrees to provide the
Union, upon request, with information that is normally maintained,
reasonably available, and necessary for the Union to effectively
fulfill its representational functions and responsibilities. This
information will be provided to the Union within a reasonable time and
at no cost to the Union.'').). See also AFGE Local 1345 v. Fed'l Labor
Relations Auth., 793 F.2d 1360 (DC Cir. 1986) (Union entitled under 5
U.S.C. Sec. 7114 to obtain information regarding two employees who had
been dismissed from jobs within union's bargaining unit upon request
for information from employer, as union's status as bargaining
representative required it to have access to information to assess its
responsibility, including information regarding dismissal of unit
employees).\9\ Thus, Local 2823 expects to use the aforementioned
information to evidence a consistent and longstanding emphasis on and
encouragement of the attainment of quantitative production, with little
or no concomitant emphasis or regard for the attainment of qualitative
performance standards.
---------------------------------------------------------------------------
\9\ While the Memorandum asserts that such information requests
need be answered only insofar as the requirements of 5 U.S.C. Sec. 7114
are met, contending that Article 46 Sec. 5 of the Master Agreement
merely restates 5 U.S.C. Sec. 7114, section 5 does not so state, nor is
its text as limited as the provisions of 5 U.S.C. Sec. 7114 that are
addressed solely to data ``which is reasonably available and necessary
for full and proper discussion, understanding, and negotiation of
subjects within the scope of collective bargaining.'' Cf. NAGE Local
R14-143, 55 FLRA 317 (1999) (Chair Segal, Concurring & Dissenting In
Part) (dissenting from majority finding that Union conceded statutory
and contractual rights were the same, noting that ``such concession
appears particularly unlikely in view of the fact that Article 11 makes
no reference to, and does not otherwise restate, the Statute'') (noting
that where provision merely reiterates statute, then authority need
only take care to assure that contractual interpretation is not
inconsistent with statute). We further note that even if the Cleveland
VARO had legitimate grounds to withhold some of the information
requested, that does not mean it can withhold all information
requested. See generally AFGE Local 2263 v. Fed'l Labor Relations
Auth., 454 F.3d 1101 (10th Cir. 2006).
---------------------------------------------------------------------------
3. The Training Claim
As set forth in the Grievance, management has: (1) failed to honor
its commitment ``to a standard of excellence in the quality and
quantity of training for all employees'' (1/20/00 MOU para. 2), ``as
evidenced by the low pass rates of employees on previous tests and
great disparities between . . . veterans served by different regional
offices''; (2) failed ``to provide training that significantly furthers
the employee's knowledge, skills and abilities to serve veterans''; (3)
provided ``ongoing training that . . . bears little resemblance to the
training described in the January 20, 2000 MOU''; (3) failed to rotate
employees in a manner that would enable them ``to gain and maintain
proficiency in all aspects of their job''; (4) disadvantaged employees
seeking promotion by failing to adequately train them to process
claims, thereby preventing them from successfully competing for and
obtaining promotions, bonuses, awards, advances, and other merit-based
compensation and/or benefits. (GRV at 2-3 para.para. 5(d), (1), (2),
(3), 5(e).)
A. The Training Claim is Sufficiently Particular
For the same reasons that Local 2823's Culture Claim has been
stated with sufficient particularity in the Grievance, its Training
Claim has been stated in a sufficiently particular manner. Again, your
office's Memorandum does not identify any authority requiring a more
particularized statement of the facts underlying the Training Claim at
this juncture, and we are aware of none.
B. The Training Claim is Based on Violations of Applicable VA Policies
and/or Agreements
Like the Culture Claim, Local 2823's Training Claim is based on
violations of VA policies, applicable agreements, and applicable law.
First, as noted by Deputy Under Secretary for Benefits, Ronald R.
Aument, in a statement to Congress last October, training is important,
because ``[c]ritical to improving claims accuracy and consistency is
ensuring that our employees receive the essential guidance, materials,
and tools to meet the ever-changing and increasingly complex demands of
their decision-making responsibilities.'' October 16, 2007 Statement of
Ronald R. Aument, Deputy Under Secretary For Benefits, Before The House
Veterans' Affairs Subcommittee On Oversight And Investigations
[hereafter, 10/16/07 Aument Testimony]. As summarized by Under
Secretary Aument, the training regimen that the VA is supposed to
follow is intended to be centralized, standardized, and comprehensive:
[1] New hires receive comprehensive training and a consistent
foundation in claims processing principles through a national
centralized training program called ``Challenge.'' [2] After
the initial centralized training, employees follow a national
standardized training curriculum (full lesson plans, handouts,
student guides, instructor guides, and slides for classroom
instruction) available to all regional offices. Standardized
computer-based tools have been developed for training decision-
makers (71 courses completed and an additional 5 in
development). Training letters and satellite broadcasts on the
proper approach to rating complex issues are provided to the
field stations. [3] In addition, a mandatory cycle of training
for all Veterans Service Center employees has been developed
consisting of an 80-hour annual curriculum.
10/16/07 Aument Testimony. And the VA's most recent annual report again
reiterates that ``[t]raining remains a priority. . . . '' 11/07 VA
Report at 199. But notwithstanding these published policies and
statements, and as set forth below, Local 2823 has reason to believe
that the training provided by the Cleveland VARO: (1) does not provide
new hires with ``comprehensive training and a consistent foundation in
claims processing principles''; (2) does not assure that incumbent
employees follow a ``national standardized training curriculum'' that
is made equally and fully available to all within the Cleveland VARO;
and (3) does not assure that all employees receive the full cycle of
training and complete an 80-hour curriculum each year.
Second, the applicable agreements similarly make clear that
training is critical and the Department is responsible for providing it
on a fair and equitable basis. Specifically, Master Agreement Article
34 Sec. 1(A) provides as follows:
The Department and the Union agree that the training and
development of employees is of critical importance in carrying
out the mission of the Department. In recognition of this, the
Department will provide training and career development
opportunities to employees of the bargaining unit. The
Department is responsible for ensuring that all employees
receive the training necessary for the performance of the
employees' assigned duties.
(MA at 142 (Art. 34) Sec. 1(A) (emphasis added).) Moreover, the Master
Agreement requires fair and equitable administration of training among
employees. (See, e.g., MA at 143 (Art. 34) Sec. 3(C) (``When resources
for training are limited, approval for training funds will be based on
fair criteria that are equitably applied.'') (emphasis added).)
Further, the Master Agreement requires the Department to inform
employees, at least annually, about training opportunities, policies,
and nomination procedures. (See id. at 143 (Art. 34) Sec. 6(A) (``The
Department shall inform employees, at least annually, about Department
training opportunities, policies, and nomination procedures. Upon
request, the Department will advise individual employees of training
opportunities that meet identified educational or career
objectives.''). Finally, and perhaps most importantly, the Master
Agreement makes clear that such training matters are appropriate
subjects for local bargaining, stating: ``[p]rocedures which ensure
fair and equitable training opportunities are appropriate subjects for
local bargaining.'' (MA at 144 (Art. 34) Sec. 9 (emphasis added).\10\)
---------------------------------------------------------------------------
\10\ See also MA at 172 (Art. 44) Sec. 4(A) (``On all policies and
directives or other changes for which the Department meets its
bargaining obligation at the national level, appropriate local
bargaining shall take place at individual facilities and may include
substantive bargaining that does not conflict with negotiated national
policy and agreements.''); id. at 172-73 (Art. 44) Sec. 4(B)
(``Proposed changes in personnel policies, practices, or working
conditions affecting the interests of one local Union shall require
notice to the President of that local. Proposed changes in personnel
policies, practices, or working conditions affecting the interests of
two or more local Unions within a facility shall require notice to a
party designated by the NVAC President with a copy to the affected
local Unions. Proposed changes in personnel policies, practices, or
working conditions affecting the interests of more than one facility
shall require notice to a party designated by the national VA Council
President.''); id. at 169 (Art. 43) Sec. 1 (``Recognizing that the
Master Agreement cannot cover all aspects or provide definitive
language for local adaptability on each subject addressed, it is
understood that Local Supplements may include substantive bargaining on
all subjects covered in the Master Agreement so long as they do not
conflict, interfere with, or impair implementation of the Master
Agreement. However, matters that are excluded from Local Supplemental
bargaining will be identified within each Article.'') (emphasis added);
id. Sec. 2(A) (``The Local Supplemental Agreement may cover all
negotiable matters regarding conditions of employment insofar as they
do not conflict with the Master Agreement as defined in Section 1. . .
. Note: This is not intended to preclude local bargaining of items that
are not covered by the Master Agreement, i.e., policies, procedures and
directives initiated at the facility level or national level.'')
(emphasis added); id. Sec. 2(B) (``In the event either of the national
parties determines there exists a conflict with the Master Agreement,
they shall forward a written document to the respective local and the
other national party identifying the conflict for resolution at the
local level.''). These provisions of the Master Agreement and similar
provisions in the 1/20/00 MOU make clear that contrary to the
assertions in the Memorandum (see Mem. at 2-3 para.para. 5(a)-(c)), the
VA's belief that it has met its bargaining obligations at the national
level do not relieve it of the obligation to bargain locally as to the
matters put at issue by Local 2823's Grievance.
Yet once again, as set forth below, Local 2823 has reason to
believe that the Cleveland VARO is not honoring its training and
bargaining obligations under the Master Agreement, inasmuch as the
Cleveland VARO: (1) has not provided training and career development
opportunities to all employees; (2) has not ensured that all employees
have received the training necessary for the performance of their
assigned duties; (3) has not used fair criteria, equitably applied, to
assure that training is appropriately distributed among and between
employees; (4) has not advised individual employees who request such
information of training opportunities that meet identified educational
or career objectives, including, but not limited to, VSR and/or RVSR
certification; and (5) has refused to bargain with Local 2823 to ensure
that fair and equitable training opportunities are made available to
all employees.
Moreover, as regards training related to certification of VSR and
RVSR personnel, the parties 1/20/00 MOU provides that the Department
has a number of obligations, relating to training, reciting in relevant
part the following:
``VBA commits to a standard of excellence in the quality
and quantity of training for all employees. We will ensure training
programs, which are the core and prerequisite to certification, are
complete and sufficient to provide employees the necessary tools to
become certified. There will be a direct relationship between the
training program and certification.'' (1/20/00 MOU para. 2 (emphasis
added)).
``Employees will proceed through standardized training
such as the TPSS program, which may include pre and post tests for the
purpose of determining the efficacy of training. Training will include
mentoring, on-the-job-training and ongoing feedback.'' (1/20/00 MOU
para. 3 (emphasis added).)
``Where incumbent employees have not been performing the
full range of duties due to specialization or for other reasons, at the
employees request, we will ensure that they are provided training
sufficient for them to participate in the certification program.'' (1/
20/00 MOU para. 6 (emphasis added).)
``A copy of this MOU will be furnished to the Local
President of all VBA facilities represented by AFGE. Local unions will
be given the opportunity to bargain over appropriate issues not
otherwise in conflict with this or other national level agreements,
prior to local implementation.'' (1/20/00 MOU para. 10 (emphasis
added).)
However, once again, as set forth below the Cleveland VARO has
violated its obligations under the 1/20/00 MOU, because: (1) the
quality and quantity of training made available to employees does not
rise to the promised standard of excellence; (2) training programs are
not complete and sufficient to provide employees the necessary tools to
become certified; (3) employees have not proceeded through standardized
TPSS training, and the training generally has not included mentoring,
on-the-job training or ongoing feedback; (4) incumbent employees who
have not been performing the full range of duties due to specialization
or other reasons have not been provided training sufficient for them to
participate in the certification program; and (5) management has failed
and refused to bargain over these issues.
Finally, it is important to note that while your office's
Memorandum asserts that it is not obligated to bargain with Local 2823
regarding these training matters relating to VSR and RVSR certification
in light of certain alleged positions taken by the national Union in
negotiations with the Department, those assertions ignore the facts
that: (1) the matters for which Local 2823 has sought to bargain are
not the same as those put at issue by the dispute at the national level
(and the Memorandum makes no attempt to explain why they are); (2)
regardless of the resolution of VA's dispute with the national Union,
that will not resolve the training issues for which Local 2823 has
sought to bargain locally; (3) the Memorandum's assertions regarding
the exclusive recognition of the national Union ignore that the
Department agreed with the national Union that local bargaining would
still occur and the appropriate VARO would still be obligated to engage
in such local bargaining pursuant to the applicable provisions of the
Master Agreement.\11\
---------------------------------------------------------------------------
\11\ We further note in this regard that the Memorandum's citation
to U.S. Food & Drug Adm., Northeast & Mid-Atl. Regions v. AFGE Council
No. 242, 53 F.L.R.A. 1269 (1998) is misplaced and in fact supports
Local 2823's position here. Notably, in relevant part U.S. Food & Drug
states, ``[a] representative with a collective bargaining relationship
in a consolidated bargaining unit is not required to bargain locally
with individual components that make up the consolidated unit unless
such bargaining has been agreed to at the consolidated level.'' Id. at
1274 (emphasis added); id. (``Parties to a national, consolidated
bargaining unit may, and often do, authorize local components to
bargain supplemental and other agreements over particular subjects or
in particular circumstances.''). And here, it is plain that in the
Master Agreement and 1/20/00 MOU, the Department and national Union
agreed that local bargaining would continue to be available unless
expressly proscribed by the Master Agreement. See supra n.9.
---------------------------------------------------------------------------
C. The Training Claim Has Evidentiary Support
Once again, there is ample evidentiary support for Local 2823's
Training Claim, both nationally and locally.
First, both the Office of the Inspector General and third-party
studies have found that disparities in training across offices account
for at least some of the differences across offices with respect to the
average amount of compensation awarded to a claimant. For example, the
Deputy Inspector General, in a statement to Congress, represented that
``the degree of rater subjectivity can be influenced by . . . the
amount of training and rater experience.'' October 16, 2007 Statement
Of Jon A. Wooditch, Deputy Inspector General, Before The Subcommittee
On Oversight And Investigations Committee On Veterans' Affairs--U.S.
House Of Reps. Hrng On Disability Claims Ratings & Benefits Disparities
Within The VBA [hereafter, ``10/16/07 Wooditch Testimony'']. Moreover,
variations in training across offices also were identified by the
Institute for Defense Analyses (``IDA'') in its study, Analysis of
Differences in VA Disability Compensation (hereafter, the ``IDA
Study''), as a significant factor responsible for differences in
compensation awarded across offices. See, e.g., Hope Yen, AP,
Washington Post (July 19, 2007) (``But the study released to AP found
that roughly one-third of the problems could be blamed on poor VA
standards and inadequate training.'').\12\ Indeed, one of the
recommendations of the IDA Study was to ``[s]tandardize initial and on-
going training for rating specialists'' 10/16/07 Aument Testimony.
Thus, this national evidence suggests that the Cleveland VARO is not
receiving training commensurate with that made available to raters in
other offices.
---------------------------------------------------------------------------
\12\ The IDA study does not appear to be publicly available at this
juncture.
---------------------------------------------------------------------------
Moreover, the current insufficiency of training programs and
policies has been confirmed by the recent report of the Veterans'
Disability Benefits Commission, which states:
VBA regional office staff must receive adequate education and
training. Quality reviews should be performed to ensure these
frontline workers are well versed to rate claims. Adequate
resources must be appropriated to hire and train these workers
to achieve a manageable claims backlog.
(10/07 VDBC Report 338 (Recommendation 9.5).)
Second, as set forth in the Grievance, there are numerous instances
of the Cleveland VARO failing to implement equitably, promptly, and
appropriately training. For example, as a result of the Claims Process
Improvement teams that have been formed to focus on specific problem
areas, employees are not cycled through all areas and do not receive
on-the-job training in many areas before they are required to sit for
certification exams covering all areas. (See GRV at 2 Sec. 5(3).)
Moreover, training that has been provided has not been comprehensive,
but instead has largely been addressed to the specific tasks assigned
an employee at the time of hire. (See id.) The initial training for
VSRs in the Cleveland VARO is focused on the tasks performed by the
team to which the VSRs have been assigned, either pre-development or
post-determination. Then, 85 percent-90 percent of the training that
the VSRs receive on the job consists of overview courses that also
cover tasks they perform in their current jobs. This training regimen
leaves VSRs unprepared for their new duties when they switch teams and
the quality of their work (and their production) suffers. In addition
to these problems with VSR training, the Cleveland VARO also fails to
train RVSRs appropriately. At the time the grievance was filed, few
RVSRs in the office had completed the TPSS training modules that are
required for first-year rating specialists. Instead of addressing this
deficiency, the Cleveland VARO ignores it: three rating specialists
were promoted in October 2007, even though they had failed to complete
the basic compensation training module.
D. Local 2823's Clarified and Revised Information Requests Addressed to
the Training Claim are Proper
In addition, as noted in the Grievance, Local 2823 requests
information from the Cleveland VARO that is normally maintained,
reasonably available, and necessary for Local 2823 to fulfill its
representational functions and responsibilities with respect to the
prosecution of the Training Claim, and, to that end clarifies and
revises its requests for information, hereby requesting production of
the following information created on or after January 20, 2000 (except
as otherwise indicated):
Documents sufficient to show the number of employees
(including VSRs and RVSRs) who have received training in the Cleveland
VARO, the amount of training that they have received, and the types of
training that they have received in each of the fiscal years 2002,
2003, 2004, 2005, 2006, 2007, and 2008 (to date);
Documents sufficient to show all plans and programs for
training employees (including VSRs and RVSRs) at the Cleveland VARO or
elsewhere for the fiscal years 2002, 2003, 2004, 2005, 2006, 2007, and
2008 (to date);
All communications from employees complaining, praising
or otherwise evaluating the amount, content, type, availability, or
other aspects of training provided or made available to employees;
All studies, audits, and/or investigations by the VA and/
or third parties (including, but not limited to, the IDA Study)
addressing the adequacy of training made available to and/or required
of employees at the Cleveland VARO and nationally (including, but not
limited to, VSRs and RVSRs);
Documents sufficient to show all training that is
required and/or suggested for VSR and/or RVSR certification, including,
but not limited to, copies of instructions for RVSR and/or VSR national
training requirements;
Documents sufficient to show all on-the-job training
accomplished and its relationship to certification of VSRs and/or
RVSRs;
A copy of the curriculum formally identified as VSR
``Readiness Training'';
A copy of any training materials respecting ``the
Candidate Guide'';
A copy of any ``Boot Camp'' test or similar Cleveland
VARO practice test;
A copy of all materials found at the following website
address: http://cptraining.vba.va.gov/C&PTraining/VSR/VSRCerTng/
VSRCertCurriculum.htm;
A copy of the link to the VSR Certification Training
Guide that has been provided to each employee in the Cleveland VARO who
is eligible for certification; and
A copy of all materials found at the following website
address: http://cptraining.vba.va.gov/C&PTraining/VSR/VSRCertTng/
Documents/VSRCertTrainingGuide.pdf.
Further, it is plain that production of the requested training
information is relevant and necessary to Local 2823's prosecution of
its Training Claim, as the information will be used to establish the
specific departures of the Cleveland VARO from applicable training
requirements, agreements, and policies.
4. The Performance Appraisal Claim
In light of the foregoing, and as set forth in the Grievance,
``management has abrogated its duty to maintain a fair and impartial
performance appraisal system under multiple sections of Master
Agreement 26,'' and has otherwise abrogated VA policy and guidance in
its application of the existing performance appraisal system. (GRV at 4
para. 6(a).)
A. The Performance Appraisal Claim is Sufficiently Particular
For the same reasons that Local 2823's Culture and Training Claims
have been stated with sufficient particularity in the Grievance, Local
2823's Performance Appraisal Claim has been stated in a sufficiently
particular manner. Again, your office's Memorandum does not identify
any authority requiring a more particularized statement of the facts
underlying the Performance Appraisal Claim at this juncture, and we are
aware of none.
B. The Performance Appraisal Claim is Based on Violations of Applicable
VA Policies and/or Agreements
Local 2823's Performance Appraisal Claim is based on violations of
applicable VA policies and/or agreements.
First, as set forth in VA Directive 5023, ``[t]he public interest
demands the highest standards of employee performance and the continued
development and implementation of modern and progressive work practices
to facilitate and improve employee performance and the efficient
accomplishment of the operation of the Government.'' See VA Dir. 5023
Sec. 2(b). Further, as VA Directive 5023 also states, ``VA management
shall carry out its duties in a manner consistent with the terms and
spirit of human resources policies, principles and procedures that
encourage the highest standard of employee performance and the most
efficient accomplishment of VA operations.'' See id. Sec. 2(c)
(emphasis added).
Management has abrogated both of these VA imperatives, however, by
implementing a performance appraisal system that, for the reasons set
forth above and below: (1) does not promote the ``highest standards of
employee performance and the continued development and implementation
of modern and progressive work practices to facilitate and improve
employee performance and the efficient accomplishment of'' VA
operations; and (2) is fundamentally inconsistent ``with the terms and
spirit of human resources policies, principles and procedures that
encourage the highest standard of employee performance and the most
efficient accomplishment of VA operations.''
Second, as noted in the Grievance, management's implementation of
the existing performance appraisal system also contravenes applicable
provisions of Article 26 of the Master Agreement. Specifically, the
relevant provisions of Article 26 (as amended) that are abrogated by
management's implementation and application of the existing performance
appraisal system include the following:
``In its entirety and application, the performance
appraisal process will to the maximum extent feasible, be fair,
equitable, and strictly related to job performance as described by the
employee's job description.'' [Am. Art. 26 Sec. 3(A) (emphasis added).]
``Performance appraisals shall be fair and objective.''
[Id. Sec. 3(C) (emphasis added)]
``The union may provide input into any changes to
performance standards and/or establishment of new performance
standards.'' [Id. Sec. 5(A).]
``Performance standards and elements to the maximum
extent feasible shall be reasonable, realistic, attainable, and
sufficient under the circumstances to permit accurate measurement of an
employee's performance, and adequate to inform the employee of what is
necessary to achieve a `Fully Successful' level of achievement.'' [Id.
Sec. 5(C) (emphasis added).]
``The Union shall be given reasonable written advance
notice . . . when Management changes, adds to, or establishes new
elements and performance standards. Prior to implementation of the
above changes to performance standards, management shall meet all
bargaining obligations.'' [Id. Sec. 5(E).]
``Normally, elements are not weighted or assigned
different priorities. However, the Department will inform the employee,
at the time the elements and standards are communicated, whether
aspects of any job elements are to be accorded different priority. If
the elements, standards, or priority changes, that change(s) will be
communicated to the employee when it becomes effective.'' [Id.
Sec. 5(H).]
Notwithstanding these obligations, however, management has breached the
foregoing provisions of Article 26 of the Master Agreement by, inter
alia: (1) applying the existing performance appraisal system in a
manner that is unfair, inequitable, and almost exclusively related to
the achievement of quantitative production targets; (2) establishing
certification and related requirements that are now part of the
performance appraisal while refusing any input from Local 2823
respecting the content and implementation of the same; (3) failing to
give reasonable written advance notice to Local 2823 respecting the
implementation of newly established certification and related training
requirements; (4) weighting quantitative production measures more
highly than accuracy measures without formally apprising employees of
the same; (5) utilizing statistical production data to evaluate
individual performance in a manner that is unreliable, invalid, unfair
and inequitable because it fails to take into account the number of
issues raised by claims and the complexity of the claims process.
C. The Performance Appraisal Claim has Evidentiary Support
Like its other claims, Local 2823's Performance Appraisal Claim
also is amply supported both nationally and locally.
First, as adverted to in Deputy Under Secretary Wolcoff's recent
testimony to Congress this past February, even independent third-party
consultants have found problematic the existing performance appraisal
system's use of quantitative production criteria that fail to account
for the numbers of issues raised and complexity of claims. (See 2/14/08
Walcoff Testimony.) And, perhaps more importantly, in testifying before
Congress, the Veterans' Benefits Administration (``VBA'') has stated
that it ``agrees with the idea of adding an issue-based performance
measurement system to our current reporting structure.'' (Id.) Further,
as noted above, the Veterans' Disability Benefits Commission similarly
found that greater attention to quality should be paid in assessing the
performance of employees. (See 10/07 VDBC Report 338.)
Second, in terms of the Cleveland VARO, the existing performance
appraisal system applied by management again penalizes raters who take
the time to assure accurate ratings of multi-issue and complex claims
and favors those who are less accurate but meet or exceed applicable
production quotas. For example, a case with a minimal amount of
evidence, such as a well-documented knee injury, is given the same
production credit as a multi-volume multi-issue case for PTSD. Raters
are therefore tacitly (and sometimes overtly) encouraged to avoid
processing difficult claims. A different problem is the older case
whose record has been developed in several stages and requires follow-
up with several treating physicians. If an RVSR spends 1 hour with the
case and discovers that a private treatment record identified by the
veteran is missing, the RVSR should defer a decision on that case. The
RSVR, however, does not receive any production credit under the
performance plan for deferring this decision. Instead, the performance
plan gives the RVSR the stark choice of ignoring the missing evidence
and deciding the case or (correctly) deferring the case and absorbing
the entire time spent with that case as lost.
D. Local 2823's Information Requests Addressed to the Performance
Appraisal Claim are Proper
In addition, as noted in the Grievance, Local 2823 requests
information from the Cleveland VARO that is normally maintained,
reasonably available, and necessary for Local 2823 to fulfill its
representational functions and responsibilities with respect to the
prosecution of the Performance Appraisal Claim, which requests are
encompassed by the requests set forth above with respect to the Culture
and Training Claims. Again, the requested information is both relevant
and necessary, as it will be used by Local 2823 to assess the manner in
which the existing performance appraisal system has been applied by the
Cleveland VARO and to demonstrate the ways in which the current system
violates VA policy and provisions in Article 26 of the Master
Agreement.
5. The Veterans' Disability Benefits Claims Modernization Act of 2008,
H.R. 5892
Lastly, before closing, we wish to note that one of the reasons
that Local 2823 believes a meeting to discuss the parties' differences
is advisable at this juncture is that pending legislation may--at least
in Local 2823's view--moot many of these disputes.
In particular, the Veterans' Disability Benefits Claims
Modernization Act Of 2008, H.R. 5892, 110th Cong. (2d Sess. 2008) (the
``Proposed Legislation'') was referred to the House Committee on
Veterans' Affairs in April 2008, on July 29, 2008, an amended bill was
favorably reported out of Committee to the House floor, and on July 30,
2008, the amended bill was passed by the House on a roll-call vote of
429-0. See 154 Cong. Rec. H7256-H7263 (daily ed. July 29, 2008); 154
Cong. Rec. H7518 (daily ed. July 30, 2008). If enacted in its current
form, the Proposed Legislation would: (1) require the Department to
engage a third-party to annually assess the quality of claims
processing across the VAROs (see Proposed Legislation Sec. 106),
thereby potentially addressing concerns giving rise to Local 2823's
Culture Claim; (2) require the Department to study and develop new
certification standards and programs after obtaining appropriate input
from employees and their representatives (see id. Sec. 105), thereby
potentially addressing concerns giving rise to a portion of Local
2823's Training Claim; (3) require the Department to study and evaluate
training made available to employees (see id. Sec. 106(a)(3)), thereby
potentially addressing concerns giving rise to another portion of Local
2823's Training Claim; and (4) study and implement new performance
standards that would place greater emphasis on the quality of ratings
decisions and less emphasis on the quantity of them (see id. Sec. 103),
thereby potentially addressing the concerns giving rise to Local 2823's
Performance Appraisal Claim.\13\
---------------------------------------------------------------------------
\13\ Of course, the introduction of the Proposed Legislation merely
confirms that Local 2823's Culture, Training and Performance Appraisal
Claims are well-founded and raise issues of concern that are shared by
employees in other regional offices and Members of Congress. See, e.g.,
Statement of Congressman Hare In Support Of H.R. 5892, 154 Cong. Rec.
H7262 (daily ed. July 29, 2008) (``The largest factors contributing to
the claims backlog are the broken culture and processes at the VA.
There is a lack of accountability on raters, poor quality assurance
measures, a broken work credit system, virtually no training for the
VBA personnel, and an outdated information technology system. [para.]
H.R. 5892 squarely addresses these problems by creating a more
accountable and accurate system that rewards raters for the quality of
their work, and it holds them accountable for their mistakes, ensuring
that claims are processed correctly the very first time.'') (emphasis
added).
---------------------------------------------------------------------------
Accordingly, any information that management of the Cleveland VARO
could provide to Local 2823 respecting the Department's position with
respect to the Proposed Legislation--including whether the Department
opposes, supports, or partially opposes and partially supports the
Proposed Legislation--would be beneficial.
* * *
Thank you for your prompt attention to the foregoing matters, and I
look forward to a response from your office on or before September 15,
2008, so that the parties may quickly reach agreement on a date when
representatives of management and Local 2823 may sit down together in
Cleveland to further discuss (and hopefully resolve) the matters put at
issue by Local 2823's Grievance. Should I be able to answer any
questions in the interim, please do not hesitate to contact me.
Sincerely,
Jeffrey K. Lamb for Drew G.A. Peel
cc: John A. Limposte, Assistant Director
Prepared Statement of Patricia A. Keenan, Ph.D., Program Manager, Human
Resources Research Organization
Good afternoon. I am Patricia Keenan, a Program Manager at the
Human Resources Research Organization, known less formally as HumRRO.
HumRRO is a non-profit, 501(c)3 research and development organization,
established in 1951, that works with Government agencies and other
organizations to improve their effectiveness through improved human
capital development and management.
I will comment today about the Compensation and Pension (C&P)
Service's training program, as well as on methods to increase
accountability and reduce rating variance. I am the project leader for
HumRRO's work with VBA's Skills Certification program.
Skills Certification Testing Program
HumRRO's has worked closely with C&P Service on Skills
Certification program for Veterans Service Representatives (VSRs) and
Rating VSRs (RVSRs). We began assisting VBA with the program in 2001,
by developing the multiple-choice VSR test that is administered to GS-
10s seeking promotion to GS-11. In 2006 our work expanded to include
VSRs at Pension Maintenance Centers (PMC) and RVSRs who have just
completed training. Last year we began developing the Skills
Certification test for journey-level RVSRs.
The VSR test is completely operational with two administrations
planned per year. Development of the RVSR end-of-training test is
complete and the test is expected to become operational in December of
this year. Development of the PMC VSR test is also complete and is
expected to become operational next spring. In the future, all Skills
Certification tests will be administered via the Internet. This has the
advantage of allowing us to expand the types of items on the test to
include completion and short essay items.
The tests were developed using a content-validation strategy, which
requires that the tests reflect important job-related content. The
first step in creating that link was to conduct job analyses for each
of the four target positions. Subject matter experts linked important
tasks to the knowledges, skills and abilities required to perform them.
The number of tasks linked to a knowledge area is reflected in the
weight given to that knowledge area of the test blueprint and to the
number of items in that area that are on the test. To maintain the
content validity of a test, it is necessary that job analyses be
repeated periodically to ensure that the test still captures the
important job requirements. Because the original VSR job analysis was
conducted in 2001 and many characteristics of the job have changed
since then, we conducted a new job analysis this year. We are in the
final stages of revising the test blueprint for the VSR test.
C&P Training Programs
Since the Skills Certification program began in 2001, C&P Service
has initiated several training programs. Newly hired or promoted VSRs
and RVSRs are required to take centralized Challenge training. There is
an RVSR Challenge course and separate VSR Challenge courses for Pre-
Determination and Post-Determination. All three Challenge courses
include several Training Performance Support System (TPSS) modules.
These are web-based training modules and electronic job aids with
accompanying case-based performance practice and performance testing.
The content of the modules are content tailored to the specific
position. Experienced decision-makers also use the modules for
refresher training.
VSRs and RVSRs are also required to take 80 hours of refresher
training per year. Central Office decides the appropriate content for
60 percent of this training (48 hours); the other 40 percent (32 hours)
is determined by each Regional Office on an ``ad hoc'' basis. This mix
ensures that training addresses areas recognized as requiring
additional training both nationally and locally. Additional training
may be required for several reasons--to disseminate information about
new regulations or court decisions, to address areas that are commonly
appealed, or to correct problems identified during Systematic Technical
Accuracy Review (STAR) review. This refresher training is provided in
several ways--via classroom training, satellite broadcasts, net
meetings, or online courses. VSRs who are preparing to take the Skills
Certification Test also receive additional training time to prepare for
the test. Please note that HumRRO has no role in preparing or
delivering these training courses; our job is to help VBA by developing
an independent assessment of knowledge necessary for the job itself.
Training and Test Results
As of August, 2008, 1,227 participants have passed the VSR Skill
Certification test. The passing rates have risen steadily from 25
percent in the 2003 test to 49.58 percent in August of this year. These
rising passing rates for the VSR test indicates to us that training is
having a positive effect on test performance.
We asked participants in the May test which TPSS modules they had
completed or used as reference. The three modules with the highest
percentage of completion were Original Claim for Compensation, which
157 VSRs reported completing (33.12 percent), Original Claim for
Pension, which 127 respondents (26.7 percent), reported completing, and
Dependency Benefits, which 118 VSRs (24.9 percent) reported completing.
The other modules were completed by 21 percent of test takers or less.
The average number of modules completed was three. The general trend in
the data shows that those who completed the TPSS modules had the
highest scores, followed by those who did not use them at all (i.e.,
neither completed them nor used them as a reference), followed by those
who used them only as a reference. Newer employees were more likely to
have completed the TPSS modules whereas employees with longer tenure
were more likely to have used the TPSS modules as references. There was
no correlation between the number of TPSS modules completed and total
test score.
Similarly, we asked participants in this summer's RVSR end-of-
training test which TPSS modules they completed. Candidates reported
completing an average of 6.7 out of 10 modules. Eighty-seven percent of
respondents (n=334) said they had completed Rate an Original Claim for
Disability Compensation; 72.4 percent (n=278) reported completing Rate
an Original Claim for Disability Pension. Other modules were completed
by 71 percent of respondents or fewer. There correlation between the
number of TPSS modules completed and test score was significant
(r(384)=.17, p=.001). Slightly more than 92 percent of respondents
(n=354) reported completing Challenge training.
The field test for the PMC VSRs was conducted last week. The two
TPSS modules completed by the largest percentage of the 60 participants
were: Original Claim for Pension (n=24; 40 percent) and Income
Adjustments (n=19; 31.7 percent). We will not be able to further
analyze the data until scoring is completed in early October.
Impediments to Rating
As part of the job analyses required for test development, HumRRO
staff conducted a series of visits to regional offices to learn more
about the VSR and RVSR jobs. The main purpose of these site visits was
to identify the critical job elements for incumbents. However,
participants in focus groups and interviews also provided information
about other aspects of their jobs. We talked with incumbent RVSRs,
Decision Review Officers, coaches and Service Center Managers.
The rating process is cognitively complex, requiring the RVSR to
compare the facts and medical evidence presented in the claim folder to
the descriptions of level of disability found in 38 CFR, encompassing
two widely differing bodies of knowledge. When the RVSR has completely
reviewed the file, established that the veteran has a service-connected
condition and all pertinent evidence has been included, the rater
begins evaluation process. This is done by comparing the relevant facts
presented in the claims folder to the rating schedule, which is
organized around 15 body systems (e.g., endocrine, musculoskeletal,
respiratory, mental disorders). Participants in the job analysis site
visits cited being able to do this with relative ease is the major
difference between trainee RVSRs and their journey-level counterparts.
They indicated that trainee RVSRs often struggle with anatomy and
understanding medical terms at first, then realize that applying the
regulations, via the rating schedule, is actually the more challenging
part of the job.
The RVSR reviews the medical evidence for each separate condition
being claimed and matches the condition to a diagnostic code in the
rating schedule. These diagnostic codes, in turn, are associated with
descriptions for varying levels of severity of impairment. These levels
are either assigned percentages in increments of 10 on a scale from 0
to 100 or, for some disabilities such as a muscle injury, they are
evaluated on more general descriptions such as severe, moderately
severe, moderate, and slight. Thus the RVSR is first required to
understand the medical condition(s) (e.g., body system affected,
symptoms, severity, limitations thereof) and then to match that
information to the correct section(s) of the rating schedule to
determine the level of disability for each condition. In addition,
there are several factors that make the task even more demanding. These
fall broadly into two categories: workload and ambiguity.
Workload Factors
One of the recurring problems discussed by incumbents was that of
incomplete cases and files. When the RVSR begins the adjudication
process, the file should be ready for rating. However, they report that
often they find the case requires additional development, which the
RVSR can do or defer back to the VSR. The problems are varied and range
from an incomplete exam, missing justification for diagnosis, unclear
information from the veteran that must be clarified, to service records
that do not show sufficient information on which to establish service-
connection. While these problems can all be resolved, doing the
required additional development adds significantly to the time it takes
to process a case. The RVSR has spent time reviewing a case that was
not ready for a rating decision and the decision must wait until the
information can be collected.
A second problem is the sheer volume of cases awaiting
adjudication. The backlog of work has been growing for years and is
increasing more rapidly than ever with the influx of veterans from OEF/
OIF. Veterans today file claims at the time of separation and a large
proportion of cases contain multiple issues. A related factor is that
often a veteran files additional claims before the first has been
decided, with the result of holding up all of the veteran's claims
until they can be rated at once.
VBA has addressed the workload problem by hiring several thousand
VSRs and RVSRs in the past year. One result of this has been that newly
hired RVSRs (and VSRs) do not understand the development process well,
and often spend much of their time learning what makes a case ``ready
to rate.'' While they will eventually become proficient rating
specialists, they are less efficient than they might be due to lack of
understanding of this vital component of claims processing.
RVSRs also face conflicting demands for prioritizing their work.
OIF/OEF cases are given priority, as are old cases, and those in which
the veteran is facing financial hardship or a terminal illness.
Journey-level RVSRs often mentor less experienced RVSRs, reviewing
their work and providing feedback, an additional duty in addition to
their regular workload.
Ambiguity
Another common theme heard during the focus groups was that it is
critical for RVSRs to understand that what they are rating is most
often not black-and-white. There are gray areas in both the medical and
legal aspects of the job. A good deal of research is often required to
establish service-connection, verify stressors, and understand the
nature and severity of a medical condition. RVSRs have a large number
and variety of resources available to them to help gather this
information, which they then compare to the regulations. However, even
their best efforts often result in having to make evaluations based on
incomplete medical and legal information.
Inexperienced RVSRs usually take longer to rate a case than
experienced raters. They are less comfortable making decisions without
complete information, and comparing the medical information to the
regulations is not an exact science. RVSRs become more comfortable with
this ambiguity over time. They also become more familiar with the
rating schedule, so they are able to use it with more ease and become
more savvy in how to use the schedule. Comments from the focus group
respondents indicate that over time, RVSRs develop individual ``rules''
for how to match the medical evidence to the rating schedule, many
areas of which leave room for interpretation. This is likely one of the
factors that allows them to process cases more quickly; they do not
have to spend as much time deciding between evaluation levels. This is
also a source of rating variance.
VBA raters select the diagnostic code so the correctness of the
rating decision depends on the level of knowledge and understanding
each RVSR has about the medical descriptions in the rating schedule.
The rating schedule contains over 700 diagnostic codes representing
distinct physical and mental impairments that are grouped by body
systems or like symptoms. Although 700 diagnostic codes sounds like a
large number, compared to the several thousands of codes contained in
the International Classification of Diseases, Ninth Revision, Clinical
Modification (ICD-9-CM), used by the medical profession, it can be
assumed that the rating schedule is less detailed, and thus provides
less information to guide evaluations. Increased detail in the rating
schedule would likely reduce the amount of individual interpretation
that currently results in inconsistencies in rating decisions. It would
also require extensive training to learn the revised schedule and
additional job aids would need to be developed to improve use of the
revised schedule.
These factors make matching the medical evidence to the criteria
provided in the rating schedule a challenge. But an additional
challenge is that the rating schedule does not have diagnostic codes
for certain specific conditions, such as carpal tunnel syndrome,
Parkinson's disease, pacemakers, or pulmonary embolus. When a claim
includes an unlisted condition, the RVSR rates it by analogy to a
closely related disease or injury. By their nature, these ``analogous
codes'' lack criteria for rating, so raters have to research different
body systems to make the evaluation and exercise a wide range of
judgments to assign analogous codes.
A final opinion raised during the site visits was that many appeals
were the result of rating decisions that did not include sufficient
detail or explanation of why a claim was denied. As just described,
much individual judgment is required in the rating process. This makes
it even more important that all evidence be addressed in the rating
decision. The reasons and bases section should include all subjective
and objective evidence. It is important that veterans feel that their
case has been clearly understood and evaluated. The letter should tell
them what evidence was in their service records, what the VA medical
examination provided (or why a VA examination was not ordered), and all
medical evidence that was submitted (e.g., private medical records).
The letter has to establish the nexus between the medical evidence and
the regulations that determined the outcome, describing how service
connection was established, the regulations that applied to each issue,
and what evidence is needed to establish service-connection or to
receive a higher level of evaluation.
Reducing Impediments
The workload of RVSRs is not going to become lighter in the near
future, so easing the workload and reducing ambiguity could go a long
way toward improving ratings. We propose some suggestions to help
reduce existing impediments.
First, newly hired RVSRs should work Pre-Development for several
weeks to learn the system, why different types of evidence are needed,
and how to determine that a case is ready to be rated. This would have
at least two benefits. The trainee RVSR would not spend time working a
case that has insufficient evidence and the mentoring RVSR would not
have to do as much explanation about the types and need for different
types of evidence. The obvious drawback to this is that it would take
longer for new RVSRs to begin rating cases, but we believe having this
additional knowledge would pay off in the longer term.
Second, the rating schedule is being updated, so it is probably not
feasible to develop formal training for each new or revised diagnostic
code. But it is important to address the problem caused by individual
interpretation of diagnostic codes that are not detailed or specific
results. A job aid that includes more specific information about the
medical evidence would reduce the level of individual interpretation in
ratings. In particular, a job aid that included specific codes and
descriptions for disabilities that are relatively frequent and that
currently fall under analogous codes, would do a great deal for
improving accuracy in these ratings. It is much easier to identify the
appropriate evaluation level when the criteria describe specific levels
of disability (e.g., range of motion) rather than a more general
description such as mild, moderate, and severe. This would allow the
rater to match the code to the diagnosis provided in the medical
evidence, again reducing errors and variance in the award level.
One of the commonly cited reasons for appeals is that the reasons
and bases section of the decision does not provide sufficient
information or an easily understandable explanation that tells veterans
why a claim was denied and what they must do to have the decision
reconsidered. RVSRs use templates or scripts to outline the letter and
ensure that required information for each section is included. Merely
including all required information is not the same in terms of customer
service or meeting the spirit of VCAA as is a well-written letter, and
these templates cannot help with the complicated problem of presenting
technical information in a manner that is well organized and that is
easy for the veteran to understand. In this section of the letter, the
RVSR describes the material evidence received and the level of evidence
required to meet the legal standard as prescribed in the regulations.
The ability to understand, organize and clearly present all this
information is difficult to train. One remedy is to provide multiple
examples of well-written letters that RVSRs can use to guide their own
efforts. A more structured approach would be to take the ability to
synthesize information and present it in a well-structured, easily
comprehensible document into consideration when hiring or promoting
RVSRs.
A final overall recommendation to reduce variance in ratings is to
ensure that all RVSRs receive standardized training, both in content
and delivery. Some refresher training is delivered via online tools,
broadcasts, or in net meetings, which do provide standardization. A
good proportion of refresher training is determined by the local office
and is most often provided by Decision Review Officers. It is important
that they receive comprehensive training in the technical area being
addressed, but also understand how to deliver training; a train-the-
trainer workshop that teaches basic training principles as well as how
to work with adult learners should be required.
Summary
It has been HumRRO's pleasure to work with C&P Service for the past
7 years. We are honored to be even a small part of the valuable work
the Veterans Benefits Administration does for America's veterans. We
have watched both the Skills Certification program and C&P Services
Training grow over this time. The resources and effort devoted to
training have been reflected in steadily improving pass rates for the
VSR Skills Certification test and in the very good pass rate for the
RVSR end-of-training test.
The greatest impediments to rating accuracy are the pressure to
produce, the need for large amounts of medical knowledge and
understanding, and ambiguity in interpreting the legal requirements.
Being able to spend time in Pre-Development would increase a newly
hired RVSR's understanding of the overall claims process as well as the
variety and depth of development that is required to rate a case. Job
aids can neither reduce the ever increasing number of claims nor reduce
their complexity, but by providing increased detail they can make the
rating schedule easier to interpret and provide standardization that is
currently lacking. Writing and analytical ability were identified as
key attributes of good RVSRs, but there is at present no systematic
evaluation of these abilities when an individual is hired or promoted
to the RVSR position. Finally, we provide a reminder of the importance
of standardized training and delivery in ensuring that all rating
specialists have a common understanding and method of working.
Prepared Statement of Nicholas T. Bartzis, Cleveland, OH (Veteran)
Good morning, my name is Nicholas T. Bartzis. I'm here today as a
private citizen, concerned veteran, and employee of the Veterans
Benefits Administration. I have served in the position of Rating
Veterans Service Representative (RVSR) for approximately 8 years in the
Cleveland VARO. My relevant past experience included being a training
officer in the naval reserves. I possess a joint Law Degree and
Master's Degree in Public Administration. As an employee, I am
frustrated that I am forced to choose between (1) doing the right thing
for the veteran's claim before me by ordering all of the needed
intermediate work, or (2) meeting arbitrary production quotas imposed
by my supervisors.
First, I would like to thank each and every Member who voted for
H.R. 5892. I believe that enactment of this bill into law could go far
to correct many of the problems we now face. Thank you!
Rapid and accurate VA compensation is critical to the quick and
efficient transition of former servicemembers and spouses to civilian
life. In my opinion, VA managers and adjudicators can do a better job
adjudicating claims for VA disability benefits. In 2007, claims
processing time increased to an average 183 days. It is my experience
that most VA adjudicators endure a constant and mounting pressure to
increase their processing of claims in spite of the fact that initial
training and follow up training has not kept pace with the needs of VA
adjudicators. In general, I and my coworkers do not feel that more
decisions equal better individual decisions.
The problems with the VA adjudication process identified by
Congress, stakeholders and claimants seeking VA benefits are multi-
factored and not exclusively the fault of the Veterans Benefits
Administration (VBA) management. However, the problems created by the
adjudication climate fostered by the VBA are the primary problem. These
VBA issues are correctable, and they should be addressed immediately.
In my testimony I will focus on the following four topics:
VBA training, both qualitative and quantitative issues.
Performance management, and a culture that emphasizes
quantity of claims processed at the expense of the quality of the
decisions made.
Accountability, who should be held accountable and why.
Potential solutions, from the perspective of one
responsible for ratings claims.
Training is Insufficient in Quantity and Quality
I have been employed as an RVSR for 8 years. I was initially
promised formal, centralized training. I have never completed this
training because VA management stressed production and did not give
raters, such as myself, the time to complete this training. At each
juncture, case production requirements for the station trumped my
individual training needs. I want to stress that one reason cases are
not consistently decided by the over 50 VA regional offices (ROs) is
that, as I understand it (based on discussions with employees from
other ROs and published studies), training varies widely from RO to RO.
Therefore, benefits may be awarded to some veterans but are denied to
other veterans who are similarly situated.
Nationwide, for the period of January 1, 2007 through September 5,
2008, the Training and Performance Support System (``TPSS'') Basic
Rating Completion Report lists 2115 RVSR employees. Only 124 have
completed the TPSS portion of the training. This is a completion rate
of 17 percent. TPSS training is geared toward newly hired employees and
is seldom completed as assigned. Instead, what I see is that, depending
on the production demands of the station, those employees are quickly
placed in a production capacity. Meaning they are assigned to quickly
complete work for the station instead of undertaking further training
under penalty of removal should they insist on spending time training
instead of processing claims. These new employees are charged with
making a correct and timely decision on the claim before them without
first completing the proper training.
For example, VARO Cleveland has approximately 70 RVSRs. However, in
the period January 1, 2007 through September 5, 2008, the TPSS Basic
Ratings Completion Report listed 31 RVSRs scheduled to complete
training. Of those 31 identified, none have completed the assigned
training. This fact clearly shows a lack of commitment to training by
local management.
Poor initial training is only half of the problem. The job itself
requires frequent and detailed updates and discussion about various
changes in the law or court decisions. For example, the Veterans Claims
Assistance Act of 2000 (VCAA) spawned numerous lawsuits and
significantly changed the way VA operates. Also, upon issuance of a
court decision or law change, VA must provide rapid instructions to its
regional offices. Unfortunately, this is not the case. The U.S. Court
of Appeals for Veterans Claims (CAVC) decision in Vazquez v. Peake, 05-
0355, was decided in January 2008, but the VA did not provide
instructions to its regional offices respecting this decision until
June 2008.
Performance Management is Primarily Geared Toward Quantity of Claims
Processed and not Quality of Benefits Decisions
Performance management may be defined as: ``a system for relating
the individual performance on the part of the employee, to the
organizational objectives and performance of the agency.'' For VA
employees, their performance is controlled by two primary documents.
First, is the employee appraisal system, and second, is the employee's
performance plan.
The Performance Appraisal Program lists the general criteria for
how all Title 5 employees will be evaluated. A subset of the
Performance Appraisal Program is the individual's performance plan. The
individual's performance plan describes the work expected from
employees and measures that are expected of them. Presently for RVSRs,
there are five stated performance measures: productivity, timeliness,
customer service, quality of work, and organizational support. A
supervisor assigns values to each of these five stated performance
measures.
With the exception of productivity, I have seldom seen numerical
values assigned to any of the other measures for RVSRs. As such,
neither the employee nor any person who reviews their accomplishments
after the fact has an accurate description of how much work the
employee really did. In general, RVSRs do not obtain work credit for
work such as: deferring the rating for additional development by other
VA employees, instructional time for the VSR, or sufficient time for
reviewing a claims file and ordering a VA exam or reordering the VA
exam if it is insufficient. This fact is noted in recommendation number
two of the Institute for Defense Analysis study entitled Analysis of
Differences in Disability Compensation in the Department of Veterans
Affairs.
Good, knowledgeable RVSRs are, in a sense, punished when they do
work that is not credited by VA management. For example, an RVSR does
not get meaningful work credit for analyzing all the evidence in a case
and ordering a VA examination, or for reviewing the file and
determining that the present VA exam is inadequate, or for reviewing
the file and correctly asking a medical question. Careful review of a
VA claims file takes time. If, however, employees are not rewarded for
careful work and are instead rewarded for processing large volumes of
claims poorly, then inevitably claims will be improperly adjudicated
and veterans will suffer. For example, if some VA employees are behind
in their production, they may go through the file quickly (called top
sheeting) and hope that somehow the claim is rated correctly or any
error isn't caught. Supervisors, who focus on production, have, in my
experience, punished RVSRs who attempt to obtain additional needed
development. This is one way in which quantity is emphasized to the
detriment of quality.
Recently our post determination team had a 68.4 percent FYTD
production quantity failure rate for all journey level employees. This
means that 68.4 percent of all the journey level--the most experienced
employees in that team--have been unable to meet their cumulative
production quotas for that year. When brought to their attention, local
VA management was highly critical and dismissive of those who
identified this issue. Of course, while an individual employee may be
unfairly punished by management, the real loser is the veteran whose
claim is not adjudicated properly.
It is my understanding that VBA has not shared with representatives
or employees their method for determining how long each discrete step
for evaluating a claim should take.
More importantly, they have failed to quantify and provide fair
work credit for accomplishing critical intermediate work prior to a
claim being decided. Therefore what is needed is an accurate work
credit system, within the performance plan, that addresses each
discrete type of work and allows the employee sufficient time to
accomplish the work. Under the current system only one small portion of
the work is measured, the final product, and that has not worked.
Performance measures are intended to gauge the employee's
contribution to the agency's mission. Instead, what I see is that the
metrics as applied to the relevant VA employees do not adequately
address the time it takes to really do the work properly.
Accountability
I believe that Congress wants to know why and who is accountable
for our present dilemma of disparate decisions between states and
untimely decisions on claims nationwide. As I indicated above, the
problem has several causes. Many parties shoulder the burden of our
current dilemma. However, the primary burden for accountability for the
disparate decisions and long waiting times lies squarely with VBA.
Should VBA undertake and scientifically measure the discrete steps
it really takes to decide a claim correctly and seriously address the
problem with concerned stakeholders, much would be done to alleviate
the current problem. Instead, what I see is that the science not being
applied and the results shared. Without the science, the number of
employees needed to do the work is only an approximate guess based on
historical precedent. Without the science, how long it takes to
actually complete a typical claim and all its discrete steps is an
approximate guess.
I believe that most of the problems stem from one issue and one
issue only: Station performance goals that are not established based on
the time it takes to do the work. Instead what I see is that station
performance goals are set by the VA Central office using historical and
suspect data. A station's Director and service center manager will
sacrifice every activity that interferes with the employee deciding
cases, including training, in order to make stations' monthly
performance goals.
Potential Solutions
I believe that Congress is well on the way to correcting many of
the problems faced by veterans and the VBA employees who decide their
claims. While the process takes time, unnecessary delays pose real
hardships for veterans who are losing their houses or unable to feed
themselves and their families while solutions are devised and adopted.
With that in mind:
1. I strongly encourage the passage of H.R. 5892. This wide-
ranging bill is critical to addressing the root cause of the disparity
in disability compensation. Specifically, the study on employee work
credit and the agency work management system.
2. Increased oversight on VBA programs by Congress to insure
compliance with the letter and spirit of programs by the VA managers
assigned to accomplish the task.
3. Immediate implementation of a mandatory interim training
program for all employees. Within the next 12 months, each employee
must complete all identified but unaccomplished formal training per job
category.
4. Voluntary increased training by VBA to interested stakeholders
like County Veteran Service Officers and service organizations.
Thank you for your invitation to testify. I am available to answer
any questions of the Committee.
Prepared Statement of Michael Walcoff,
Deputy Under Secretary for Benefits, Veterans Benefits Administration,
U.S. Department of Veterans Affairs
Mr. Chairman and Members of the Committee:
Thank you for the opportunity today to speak on the important topic
of employee training within the Department of Veterans Affairs (VA),
Veterans Benefits Administration (VBA). I am pleased to be accompanied
by Ms. Dorothy Mackay, VBA's Director of Employee Development and
Training, and Mr. Brad Mayes, VBA's Director of Compensation and
Pension Service.
Today, my testimony will focus on initial and on-going training for
Compensation and Pension (C&P) employees. I also will describe employee
performance standards, certification requirements, training oversight,
and methods used to monitor and enhance the quality and consistency of
claims decisions.
It is critical that our employees receive the essential guidance,
materials, and tools to enable them to learn and develop the knowledge,
skills, and abilities required to be successful in their positions. To
that end, VBA has deployed training tools and centralized training
programs to provide a consistent approach to training.
New Employee Training
VBA has developed and implemented a standardized training
curriculum for new claims processing employees, referred to as the
Challenge Training Program. In fiscal year 2006, VBA provided Challenge
Training to 678 Veterans Service Representatives (VSRs) and Rating
Veterans Service Representatives (RVSRs). That number more than doubled
in fiscal year 2007 (VBA trained 1,447) and by the end of fiscal year
2008, VBA will have trained more than 2,150 new C&P employees.
In the Challenge Training Program, new hires receive consistent
instruction over the course of 2 years. Employees spend 3 weeks in
centralized training, generally at the Veterans Benefits Academy in
Baltimore, Maryland. The Academy can accommodate up to 240 students at
any one time. The centralized training brings together new hires from
different regional offices and provides consistent training and
networking opportunities. The shared learning experience enables
employees with diverse backgrounds who work in different regional
offices across the country to develop a shared sense of mission.
Instruction at the Academy is provided by experienced VBA staff who
are subject matter experts and, who have completed an instructor
development course. At the end of each daily session, trainees are
invited to complete an anonymous questionnaire to describe what value
they gained from the lessons and to evaluate the instructional methods.
This information is used to continually adjust and improve the quality
of training sessions.
When employees return to their home stations, they continue to
learn through additional on-the-job training. When classroom
instruction is used to develop knowledge on a particular topic, it is
generally followed by work on cases involving that topic. As the new
hires begin working on actual cases, they spend part of their day
developing skills through interactive use of the computer-based
Training and Performance Support System (TPSS). This system provides
topic modules with a mixed media approach to learning that includes
case studies and performance-based testing. It can be utilized by an
individual or accessed by a group of individuals to promote discussion
of a topic. Currently there are 11 TPSS modules for VSRs, and 13 TPSS
modules for RVSRs.
VBA is continually striving to enhance the quality of the Challenge
Training Program. An independent contractor, hired to evaluate the
program, surveyed 1,405 employees and 183 key personnel. We are
assessing the results from that evaluation and will use the information
to improve the Challenge Training Program.
Annual 80-Hour Training Requirement
All VSRs and RVSRs are required to complete 80 hours of training
each year, although new employees will complete more than this during
their first year. Admiral Cooper, the former Under Secretary for
Benefits, introduced this requirement. Its continuation is supported by
Admiral Dunne, the current Acting Under Secretary for Benefits, and the
rest of VBA leadership. The training has been mandated to ensure that
claims processing personnel are provided with the most current C&P
policies and procedures, as well as the latest decisions from the Court
of Appeals for Veterans Claims and precedent-setting opinions from the
VA General Counsel. This training also provides for refresher courses
in complex claims issues that are not seen on a regular basis, such as
special monthly compensation rating codes and benefits available for
permanently and totally disabled veterans.
Each regional office develops a general training plan for its
employees at the beginning of the fiscal year. At least 60 of the
required 80 training hours need to come from a list of core technical
training topics identified by the C&P Service. For the remaining 20
hours, regional offices are given flexibility to establish training
topics based on local needs. This allows regional offices to focus on
emerging issues or claims processing areas where errors have been
noted. Regional offices are also given flexibility in choosing the
instructional method used to deliver the total 80 hours of required
training. These methods may include classroom training using approved
lesson plans or use of the computer-based TPSS training modules. In
addition, regional offices can take advantage of nationwide training
programs offered on important and timely topics. Over the past year,
this training included topics such as researching stressors in claims
for PTSD, C&P medical examinations in claims for PTSD and traumatic
brain injury, and attorney representation of veterans in appealed
claims.
Employee Performance Standards and Accountability
Another VBA organizational cornerstone initiative to improve the
delivery of benefits and enhance accountability is our system of
individual performance assessment. All VSRs and RVSRs are subject to
national performance standards measuring the critical elements of
quality, productivity, customer service, and workload management, as
well as a non-critical element related to organizational support/
teamwork. Performance standards are commensurate with an employee's
experience level in the position. These standards are reviewed
periodically and amended as necessary in response to changes in
workload and claims processing procedures. Managers use an automated
tool, called ASPEN (Access Standardized Performance Elements
nationwide), to track work items completed and to measure VSR and RVSR
performance. Local accuracy reviews are conducted for all decision-
makers using the national quality criteria for the Systematic Technical
Accuracy Review (STAR) program.
Under VBA's performance appraisal program, employee performance is
evaluated annually and linked to the employee's performance standards.
An employee's level of achievement in each of the four critical
elements and one non-critical element is evaluated by a supervisor as
exceptional, fully successful, or less than fully successful. Based on
evaluations in each of the five areas, a combined overall performance
rating is given. The overall ratings include outstanding, excellent,
fully successful, minimally successful, and unsatisfactory.
In the event an employee's performance is not meeting the fully
successful level of achievement for a critical element, an employee
will be placed on a performance improvement plan (PIP). The PIP
identifies the employee's specific performance deficiencies, the
successful level of performance, the action(s) that must be taken by
the employee to improve to the successful level of performance, the
methods that will be employed to measure the improvement, and
provisions for counseling, additional and focused individualized
training, or other appropriate assistance. If the employee's
performance does not rise to at least the fully successful level for
his/her critical element(s), the employee will be removed from the
position.
In conjunction with the national performance standards, VBA has
developed a certification process to assess job proficiency. After
successfully demonstrating job proficiency through the certification
process, an employee is promoted to the journey level, thereby linking
job proficiency to pay grade.
Since 2002, the full performance level for a VSR has been the GS-11
level. Promotion to the GS-11 level is contingent upon successful
completion of certification testing. Through successfully passing the
certification test, VSRs demonstrate that they have the skills
necessary to perform the full range of VSR duties, including the
ability to work independently on the most complex cases and to review
and approve the work of others. Through the national certification
program, VBA is raising the skill levels of our core decision-makers
and producing greater consistency in claims decisions.
Training Oversight
Along with an expanded training agenda to accommodate the hiring
initiative, VBA has enhanced its training oversight methods to improve
accountability. Managers at all levels are held responsible for
ensuring that training goals are set and training requirements are met.
Each regional office is accountable for submitting a training plan at
the beginning of the fiscal year and following through on the plan. The
plan is based on an assessment of local needs and anticipates the
content and timing of training to fulfill the annual training
requirements for regional office employees. In addition, VBA recently
created the staff position of Training Manager for each regional
office. The Training Manager is responsible for local training reviews,
as well as analyzing performance indicators to determine local training
needs and implementing the training necessary to meet those needs.
The Training Manager is also the lead administrator for the
Learning Management System (LMS). LMS is a computerized learning system
that was implemented in 2008 that is designed to present training
sessions to individual employees and maintain a record of each
completed session. Hyperlinks are available through LMS to access
training course materials and curricula from the C&P Intranet training
Web site, as well as from TPSS training modules. LMS tracks learning
hours planned and completed, and is easily accessible by employees and
management.
On the national level, VBA has established an Employee Training and
Learning Board to establish training priorities, promote
accountability, and help ensure that training decisions are coordinated
and consistent with long range policy plans. The Board is chaired by
one of VBA's area Directors, and members include regional and central
office Directors.
Quality and Consistency
As part of the continued commitment to quality improvement, VBA is
expanding its quality assurance program. As part of the expansion, the
National Systematic Technical Accuracy Review (STAR) staff was
consolidated to Nashville, TN, and ten additional staff members were
hired.
The quality assurance program expansion supports an increase in the
annual case sample size for national accuracy reviews from 120 cases
per regional office to 246 cases per regional office. This represents a
more statistically sound sample size to measure regional office
accuracy levels.
In addition, the Quality Assurance Staff conducted several focused
case reviews this year. These included a special quality review of
radiation cases, an ongoing review of extraordinarily large benefit
awards, and a special review of cases completed by the Appeals
Management Center.
The Quality Assurance Staff is also responsible for conducting on-
going quarterly data analysis to identify the most frequently rated
disabilities or diagnostic codes; assessing the frequency of the
assignment or denial of service-connection for each code by regional
office; and assessing the most frequently assigned evaluation mode for
each code by regional office. Focused audit-style case reviews are
conducted at regional offices where rating results are found to be
significantly outside the established national averages in order to
identify causes of inconsistency. Through these regular reviews, VBA
expects to gain more consistent decision-making across regional
offices, as well as a better understanding of underlying causes for
variation across geographic boundaries.
These quality assurance programs are used to identify where
additional guidance and training are needed to improve accuracy and
consistency nationwide, as well as to drive VBA procedural and
regulatory changes.
Mr. Chairman, this concludes my testimony. I will be happy to
respond to any questions that you or other Members of the Committee may
have.
MATERIAL SUBMITTED FOR THE RECORD
U.S. Government Accountability Office
Report to the Chairman, Committee on Veterans' Affairs,
House of Representatives
VETERANS' BENEFITS: Increased Focus on Evaluation and
Accountability Would Enhance Training and
Performance Management for Claims Processors
May 2008
Report No. GAO-08-561
GAO Highlights
Why GAO Did This Study
Faced with an increase in disability claims, the Veterans Benefits
Administration (VBA) is hiring a large number of new claims processing
staff. We were asked to determine: (1) What training is provided to new
and experienced claims processors and how uniform is this training? (2)
To what extent has VBA planned this training strategically, and how
well is the training designed, implemented, and evaluated? and (3) To
what extent is the performance management system for claims processors
consistent with generally accepted practices? To answer the questions,
GAO reviewed documents including VBA policies and training curricula;
interviewed VBA central office officials; visited 4 of VBA's 57
regional offices, which were selected to achieve diversity in
geographic location, number of staff, and officewide accuracy in claims
processing; and compared VBA's training and performance management to
generally accepted practices identified by GAO.
What GAO Recommends
GAO is recommending that VBA collect feedback on training provided
by regional offices and use this feedback to further improve training,
and hold staff accountable for meeting their training requirement. GAO
also recommends that the VA assess and, if necessary, adjust its
process for placing staff in overall performance categories. In its
comments, VA agreed with GAO's conclusions and concurred with the
recommendations.
What GAO Found
VBA has a standardized training curriculum for new staff and a
training requirement for all staff, but does not hold staff accountable
for meeting this requirement. The curriculum for new staff includes
what is referred to as centralized training and training at their home
offices. All claims processors must complete 80 hours of training
annually, which may cover a mix of topics identified centrally and by
regional offices. Individual staff members face no consequences for
failing to meet the training requirement, however, and VBA has not
tracked training completion by individuals. It is implementing a new
system that should provide this capacity.
Although VBA has taken steps to plan its training strategically,
the agency does not adequately evaluate training and may be falling
short in training design and implementation. VBA has a training board
that assesses its overall training needs. However, the agency does not
consistently collect feedback on regional office training, and both new
and experienced staff GAO interviewed raised issues with their
training. Some new staff raised concerns about the consistency of
training provided by different instructors and about the usefulness of
an online learning tool. Some experienced staff believe that 80 hours
of training annually is not necessary, some training was not relevant
for them, and workload pressures impede training.
The performance management system for claims processors generally
conforms to GAO-identified key practices, but the formula for assigning
overall ratings may prevent managers from fully acknowledging and
rewarding staff for higher levels of performance. The system aligns
individual and organizational performance measures and requires that
staff be given feedback throughout the year. However, VBA officials
raised concerns about the formula used to assign overall ratings.
Almost all staff in the offices GAO visited were placed in only two of
five overall rating categories, although managers said greater
differentiation would more accurately reflect actual performance
differences. The Department of Veterans Affairs (VA) has not examined
the ratings distribution, but acknowledges a potential issue with its
formula and is considering changes.
Fiscal Year 2007 Appraisals for Four Offices
Were Concentrated in Two Categories
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: VBA regional offices.
__________
U.S. Government Accountability Office
Washington, DC
May 27, 2008
Hon. Bob Filner
Chairman
Committee on Veterans' Affairs
House of Representatives
Dear Mr. Chairman:
The Veterans Benefits Administration (VBA) is facing an increased
volume of claims for disability benefits related to the current
conflicts in Afghanistan and Iraq as well as the aging of veterans from
past conflicts. Between fiscal years 2000 and 2006, the number of
disability-related claims filed annually with VBA increased by almost
40 percent. As a result, VBA continues to experience challenges in
processing veterans' disability claims. As of fiscal year 2007, VBA had
approximately 392,000 disability claims pending benefit decisions, and
the average time these claims were pending was 132 days. According to
VBA, the current conflicts have also produced more claims related to
post-traumatic stress disorder and traumatic brain injury, conditions
few VBA staff have had much experience evaluating. To process the
increased volume of claims, in fiscal year 2007 the agency began hiring
a large number of new Veterans Service Representatives (VSR), who
collect evidence related to veterans' claims, and Rating Veterans
Service Representatives (RVSR), who evaluate claims and determine
benefit eligibility. It plans to add 3,100 new claims-processing staff
by the end of fiscal year 2008.
Given the increased volume of claims, the increased focus on
certain types of disabilities, and the large number of new hires,
training and performance management systems for VSRs and RVSRs now play
an especially critical role in enabling VBA to meet its organizational
claims processing goals for accuracy and productivity. Training that is
properly designed and implemented is vital both to help new staff learn
their jobs and experienced staff to update their knowledge and learn
about emerging issues. An effective performance management system would
also help VBA manage its staff on a day-to-day basis to achieve its
organizational goals. To provide Congress with information on the
training and performance management of claims processors, we were asked
to determine: (1) What training is provided to new and experienced
claims processors and how uniform is this training? (2) To what extent
has VBA developed a strategic approach to planning training for claims
processors and how well is their training designed, implemented, and
evaluated? And (3) To what extent is the performance management system
for claims processors consistent with generally accepted performance
management practices in the public sector?
To address these objectives, we collected documents and data from
VBA central office and interviewed central office staff. In addition,
GAO experts on training reviewed VBA documents related to training
curriculum, lesson plans, and course evaluations. We conducted site
visits to 4 of VBA's 57 regional offices--Atlanta, Baltimore,
Milwaukee, and Portland, Oregon. These offices were selected to achieve
diversity in geographical location, number of staff, and officewide
accuracy in claims processing. While we examined VBA-wide policies and
requirements, we primarily assessed how the training and performance
management systems are implemented at four sites. Therefore, our
results may not be representative of how these systems are implemented
across all regional offices. We assessed VBA's training and performance
management practices by comparing them to certain generally accepted
practices for Federal agencies in these areas that have been identified
by GAO.\1\ We conducted this performance audit from September 2007
through May 2008 in accordance with generally accepted Government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
(See app. I for more detailed information on our objectives, scope, and
methodology.)
---------------------------------------------------------------------------
\1\ These practices are laid out primarily in two GAO reports:
Human Capital: A Guide for Assessing Strategic Training and Development
Efforts in the Federal Government, GAO-04-546G (Washington, DC: March
2004) and Results Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success, GAO-03-488
(Washington, DC: Mar. 14, 2003).
---------------------------------------------------------------------------
Results in Brief
VBA has a standard training curriculum for new claims processors
and an 80-hour annual training requirement for all claims processors,
but staff are not held accountable for meeting this requirement. VBA's
three-stage training program for new staff is intended to deliver
training in a consistent manner. First, VBA policy states that new
staff are required to complete some orientation training, which is
provided in their home offices. Second, they are required to attend a
3-week standardized training session, referred to as centralized
training, that provides a basic introduction to their job
responsibilities. Third, new staff are required to spend several more
months in training at their home offices, which is supposed to include
on-the-job training, instructor-led training classes that follow a
required curriculum, and use of an online learning tool called the
Training and Performance Support System. VBA policy states that all
claims processors are required to complete a minimum of 80 hours of
training annually, and regional offices have some discretion over what
training they provide to meet this requirement. At least 60 hours must
be selected from a list of core topics identified by VBA central
office. Regional offices may choose the topics for the remaining 20
hours based on local needs, such as to prevent errors identified in
processing claims. Each regional office develops an annual training
plan listing the courses needed, and VBA central office periodically
reviews these plans and provides feedback to regional offices. Although
VBA has a training requirement for VSRs and RVSRs, it does not have a
policy outlining consequences for individual staff who do not complete
their required training. Further, VBA does not maintain data on the
training completed by individuals, but agency officials said they are
currently implementing a new, online learning management system that
should enable them to do so in the future.
VBA is taking steps to strategically plan its training, but does
not adequately evaluate its training and may be falling short in some
areas of training design and implementation. VBA appears to have
followed several accepted practices in planning its training, including
the establishment of a training board that assesses VBA's overall
training needs and makes recommendations to the Undersecretary for
Benefits. Also, VBA makes some effort to evaluate its centralized
training for new staff, soliciting feedback from students with forms
that are well-constructed and well-balanced. However, VBA does not
require regional offices to collect feedback on any of the training
they provide to new and experienced staff. In fact, claims processors
we interviewed raised some issues with the training they received. For
example, some new staff reported that different instructors in
centralized training sessions sometimes taught different ways of
performing the same procedure, and that one of VBA's online learning
tools--the Training and Performance Support System--is too theoretical
and often out-of-date. More experienced staff had mixed opinions as to
whether 80 hours of training annually is appropriate for all staff.
Also, many experienced staff indicated that training topics are
redundant from year to year, and some told us that courses available to
them are not always relevant for their position or experience level
because they are often adapted from courses for new staff. Some staff
said they struggle to meet the annual 80-hour training requirement
because of workload pressures.
The Department of Veterans Affairs' performance management system
for VSRs and RVSRs generally conforms to accepted practices, including
aligning individual and organizational performance measures, but the
system may not clearly differentiate among staff's overall performance
levels. Several elements of VSRs' and RVSRs' performance are evaluated,
and these elements are generally aligned with VBA's organizational
performance measures. For example, VSRs and RVSRs are evaluated on
their accuracy in claims processing, and one of VBA's organizational
performance measures is accuracy in claims processing. VBA's
performance management system is also consistent with other accepted
practices, such as providing performance feedback throughout the year
and emphasizing collaboration. However, the system may not clearly and
accurately differentiate among the overall performance levels of VSRs
and RVSRs. A VA-wide formula is used to translate an employee's ratings
on all individual elements into one of five overall rating categories.
Several VBA central and regional office managers raised concerns with
this formula, saying that it is difficult for staff to be placed in
certain overall performance categories, even if staff's performance
truly does fall within one of those categories. In fact, when we
reviewed the results of VSR and RVSR appraisals at the regional offices
we visited, almost all staff were placed in either the outstanding
(highest) or fully successful (middle) categories. To the extent that
the performance appraisals do not make meaningful distinctions in
performance, staff may lack the constructive feedback they need to
improve, and managers may lack the information they need to reward top
performers and address performance issues. Although VA acknowledged
this issue and indicated that it is considering changes to the system,
no formal actions have been taken to date.
We are recommending that VBA central office collect feedback on
training provided by the regional offices, to determine whether (1) 80
hours is the appropriate amount of annual training for all staff, (2)
regional offices are providing training that is relevant for all staff,
and (3) whether any changes are needed to improve the Training and
Performance Support System. We are also recommending that VBA central
office hold individual staff accountable for meeting their training
requirement and that VA assess and, if necessary, adjust its
performance rating system for staff to make it a more meaningful
management tool. In its comments, VA agreed with our conclusions and
concurred with our recommendations.
Background
In fiscal year 2007, the Department of Veterans Affairs (VA) paid
about $37\1/2\ billion in disability compensation and pension benefits
to more than 3.6 million veterans and their families. Through its
disability compensation program, the VBA pays monthly benefits to
veterans with service-connected disabilities (injuries or diseases
incurred or aggravated while on active military duty). Monthly benefit
amounts vary according to the severity of the disability. Through its
pension benefit program, VBA pays monthly benefits to wartime veterans
with low incomes who are either elderly or permanently and totally
disabled for reasons not service-connected. In addition, VBA pays
dependency and indemnity compensation to some deceased veterans'
spouses, children, and parents and to survivors of servicemembers who
died while on active duty.
When a veteran submits a benefits claim to any of VBA's 57 regional
offices, a Veterans Service Representative (VSR) is responsible for
obtaining the relevant evidence to evaluate the claim. For disability
compensation benefits, such evidence includes veterans' military
service records, medical examinations, and treatment records from VA
medical facilities and private providers. Once a claim is developed
(i.e., has all the necessary evidence), a Rating Veterans Service
Representative (RVSR) evaluates the claim, determines whether the
claimant is eligible for benefits, and assigns a disability rating
based on degree of impairment. The rating determines the amount of
benefits the veteran will receive. For the pension program, claims
processing staff review the veteran's military, financial, and other
records to determine eligibility. Eligible veterans receive monthly
pension benefit payments based on the difference between their
countable income, as determined by VA, and the maximum pension amounts
as updated annually by statute.\2\ In fiscal year 2007, VBA employed
over 4,100 VSRs and about 1,800 RVSRs to administer the disability
compensation and pension programs' caseload of almost 3.8 million
claims.
---------------------------------------------------------------------------
\2\ 38 U.S.C. Sec. 5312(a).
---------------------------------------------------------------------------
In 2001 the VA Claims Processing Task Force noted that VSRs were
responsible for understanding almost 11,000 separate benefit delivery
tasks, such as tasks in claims establishment, claims development,
public contacts, and appeals. To improve VBA's workload controls,
accuracy rates, and timeliness, the Task Force recommended that VA
divide these tasks among a number of claims processing teams with
defined functions. To that end, in fiscal year 2002, VBA developed the
Claims Processing Improvement model that created six claims processing
teams, based on phases of the claims process. (See table 1.)
Table 1--VBA's Disability Compensation and Pension Service's Claims Processing Teams
----------------------------------------------------------------------------------------------------------------
Team Summary of claims processing duties
----------------------------------------------------------------------------------------------------------------
Triage Team Establishes the regional office's tracking procedures for all mail as
well as processes claims that only require a brief review to determine
eligibility.
----------------------------------------------------------------------------------------------------------------
Pre-Determination Team Develops evidence for disability ratings and prepares administrative
decisions.
----------------------------------------------------------------------------------------------------------------
Rating Team Makes decisions on claims that require consideration of medical
evidence.
----------------------------------------------------------------------------------------------------------------
Post-Determination Team Develops evidence for non-rating issues, processes benefit awards, and
notifies veterans of rating decisions.
----------------------------------------------------------------------------------------------------------------
Public Contact Team Conducts personal interviews and handles telephone inquiries, including
calls from veterans.
----------------------------------------------------------------------------------------------------------------
Appeals Team Handles requests for reconsideration of claims where veterans have
formally disagreed with claim decisions.
----------------------------------------------------------------------------------------------------------------
Source: VBA.
Note: The Rating Team is made up of RVSRs, the Post-Determination and Public Contact teams are made up of VSRs,
and the Pre-Determination, Triage, and Appeals teams are made up of both RVSRs and VSRs.
According to one VA official, new claims processing staff generally
begin as VSRs and typically have a probationary period of about 1
year.\3\ After their probationary period ends, staff can either
continue to qualify to become senior VSRs or apply for RVSR
positions.\4\ VSRs are also given the option to rotate to other VSR
claim teams to gain a broader understanding of the claims process.
---------------------------------------------------------------------------
\3\ While new claims processors are on probation, 100 percent of
the claims work they perform is quality reviewed by a supervisor. After
their probationary period, only a small sample of their claims are
quality reviewed.
\4\ Typically, RVSRs are promoted VSRs, although in some instances,
VA hires RVSRs from outside of VA who have medical or legal experience.
---------------------------------------------------------------------------
VBA Has a Uniform Training Curriculum for New Claims Processors and an
Annual Training Requirement for All Claims Processors, but
Staff Are Not Held Accountable for Meeting This Requirement
VBA has established a standardized curriculum for training new VSRs
and RVSRs on how to process claims, and it has an 80-hour annual
training requirement for both new and experienced staff; however, it
does not hold individual staff accountable for meeting this
requirement. VBA has designed a uniform curriculum for training new
VSRs and RVSRs that is implemented in three phases--initial orientation
training, a 3-week training session referred to as centralized
training, and comprehensive on-the-job and classroom training after
centralizing training. It also requires all staff to meet an annual 80-
hour training requirement. To ensure that staff meet this requirement,
each regional office must develop an annual training plan, which can
contain a mix of training topics identified by VBA central office and
by the regional office. However, individual staff members are not held
accountable for meeting their training requirement.
Training for New Staff Is Conducted in Three Stages Using a Uniform
Curriculum
VBA has a highly structured, three-phased program for all new
claims processors designed to deliver standardized training, regardless
of training location or individual instructors. (See fig. 1.) For
example, each topic included in this training program contains a
detailed lesson plan with review exercises, student handouts, and
copies of slides used during the instructor's presentation. Each phase
in this program is designed to both introduce new material and
reinforce material from the previous phase, according to a VBA
official.
Figure 1--Phases of Training for New VSRs and RVSRs
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: GAO analysis.
According to VBA policy, the first phase of training for new VSRs
and RVSRs is prerequisite training. New VSRs and RVSRs begin
prerequisite training at their home regional office as soon as they
begin working. Prerequisite training lays the foundation for future
training by introducing new VSRs to topics such as the software
applications used to process and track claims, medical terminology, the
system for maintaining and filing a case folder, and the process for
requesting medical records. Although VBA specifies the topics that must
be covered during prerequisite training, regional offices can choose
the format for the training and the timeframe. New VSRs and RVSRs
typically spend 2 to 3 weeks completing prerequisite training in their
home office before they begin the second program phase, centralized
training.
During what is referred to as centralized training, new VSRs and
RVSRs spend 3 weeks in intensive classroom training. Participants from
multiple regional offices are typically brought together in centralized
training sessions, which may occur at their home regional office,
another regional office, or the Veterans Benefits Academy in Baltimore,
Maryland. According to VBA officials in three of the four offices we
visited, bringing together VSRs and RVSRs from different regional
offices helps to promote networking opportunities, while VBA officials
from two of these offices also stated that it provides a nationwide
perspective on VBA. Centralized training provides an overview of the
technical aspects of the VSR and RVSR positions. Training instructors
should follow the prescribed schedule and curriculum dictating when and
how material is taught. For example, for a particular topic, the
instructor's guide explains the length of the lesson, the instructional
method, and the materials required; lays out the information that must
be covered; and provides exercises to review the material. (See fig. 2
for a sample of an instructor's guide from the centralized training
curriculum.) Centralized training classes have at least three
instructors, but the actual number can vary depending on the size of
the group. VBA's goal is to maintain a minimum ratio of instructors to
students.
Figure 2--Phases of Training for New VSRs and RVSRs
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: VBA.
The first week of centralized training for VSRs focuses on key
concepts, such as security, privacy and records management;
terminology; and job tools, such as the policy manual and software
applications. The final 2 weeks of training focus on the different
roles and responsibilities of VSRs on the Pre-determination and Post-
determination teams in processing claims. To practice processing
different types of claims and processing claims from start to finish,
VSRs work on either real claims or hypothetical claims specifically
designed for training. Centralized training for new RVSRs--many of whom
have been promoted from the VSR position--focuses on topics such as
systems of the human body, how to review medical records, and how to
interpret a medical exam. According to staff in one site we visited,
RVSRs new to VBA also take VSR centralized training or its equivalent
to learn the overall procedures for processing claims.
To accommodate the influx of new staff it must train, in fiscal
year 2007 VBA substantially increased the frequency of centralized
training and is increasing student capacity at the Veterans Benefits
Academy. During fiscal year 2007, VBA held 67 centralized training
sessions for 1,458 new VSRs and RVSRs. Centralized training sessions
were conducted at 26 different regional offices during fiscal year
2007, in addition to the Veterans Benefits Academy. By comparison,
during fiscal year 2006, VBA held 27 centralized training sessions for
678 new claims processors.
To implement centralized training, VBA relies on qualified regional
office staff who have received training on how to be an instructor.
According to VBA officials, centralized training instructors may be
Senior VSRs, RVSRs, supervisors, or other staff identified by regional
office managers as having the capability and the right personality to
be effective instructors. Potential instructors have certain training
requirements. First, they must complete the week-long Instructor
Development Course, which covers the ways different adults learn, the
process for developing lesson plans, and the use of different training
methods and media. During this course, participants are videotaped and
given feedback on their presentation style. In addition, each time
instructors teach a centralized training session, they are supposed to
take the 2\1/2\ day Challenge Curriculum Course, designed to update
instructors on changes to the curriculum and general training issues.
Between October 2006 and February 2008, about 250 VSRs and RVSRs from
regional offices completed the Instructor Development Course, and VBA
officials reported that, given the influx of new VSRs and RVSRs, they
are increasing the number of times this course is offered in order to
train more instructors. Instructors can teach centralized training
sessions in their home office, another regional office, or the Veterans
Benefits Academy.\5\
---------------------------------------------------------------------------
\5\ Staff who teach classes other than centralized training are not
required to take the week-long Instructor Development Course, although
they may do so if openings exist. They can also take an 8-hour
condensed course for regional instructors.
---------------------------------------------------------------------------
When new VSRs and RVSRs return to their home office after
centralized training, they are required to begin their third phase of
training, which is supposed to include on-the-job, classroom, and
computer-based training, all conducted by and at their regional office.
In the regional offices we visited, managers indicated that new VSRs
and RVSRs typically take about 6 to 12 months after they return from
centralized training to complete all the training requirements for new
staff. During this final phase, new claims processing staff cover more
advanced topics, building on what they learned in centralized training.
Under the supervision of experienced claims processors, they work on
increasingly complex types of real claims. On-the-job training is
supplemented in the offices we visited by regular classroom training
that follows a required curriculum of courses developed by VBA's
Compensation and Pension Service, specifically for new VSRs and RVSRs.
For example, new VSRs might complete a class in processing burial
claims and then spend time actually processing such claims. The amount
of time spent working on each type of claim varies from a couple of
days to a few weeks, depending on the complexity of the claim. On-the-
job training is also supposed to be supplemented with modules from the
Training and Performance Support System (TPSS), an interactive online
system that can be used by staff individually or in a group.\6\ TPSS
modules provide detailed lessons, practice cases, and tests for VSRs
and RVSRs. Modules for new VSRs cover topics such as burial benefits
and medical terminology; RVSR modules cover topics such as the
musculoskeletal system, general medical terminology, and introduction
to post-traumatic stress disorder.
---------------------------------------------------------------------------
\6\ In 2001, we reported that VBA had spent or obligated about
$18.6 million of the estimated total TPSS program cost of $32 million.
See GAO, Veterans' Benefits: Training for Claims Processors Needs
Evaluation, GAO-01-601 (Washington, DC: May 31, 2001).
---------------------------------------------------------------------------
New and Experienced Staff Have an Annual Training Requirement, and
Regional Offices Develop Training Plans That Cover a Mix of
Topics Identified Centrally and Locally
A policy established by VBA's Compensation and Pension Service
requires both new and experienced VSRs and RVSRs to complete a minimum
of 80 hours of technical training annually, double the number VBA
requires of its employees in other technical positions.\7\ VBA
officials said this higher training requirement for VSRs and RVSRs is
justified because their jobs are particularly complex and they must
work with constantly changing policies and procedures.
---------------------------------------------------------------------------
\7\ VBA defines an experienced VSR or RVSR as one who has been in
that position for 1 year or more.
---------------------------------------------------------------------------
The 80-hour training requirement has two parts. At least 60 hours
must come from a list of core technical training topics identified by
the central office of the Compensation and Pension Service. For
example, core topics for VSRs in fiscal year 2007 included establishing
veteran status and asbestos claims development; topics for RVSRs
included due process provisions and eye-vision issues. VBA specifies
more core topics than are necessary to meet the 60-hour requirement, so
regional offices can choose those topics most relevant to their needs.
They can also choose the training method used to address each topic,
such as classroom or TPSS training. (See app. II for the list of core
technical training topics for fiscal year 2007.) Regional offices
determine the training topics that are used to meet the remaining 20
hours, based on local needs and input. Regional offices may select
topics from the list of core technical training topics or identify
other topics on their own.
The four regional offices we visited varied in the extent to which
they utilized their discretion to choose topics outside the core
technical training topics in fiscal year 2007. Two sites selected the
required 60 hours of training from the core requirements and identified
their own topics for the remaining 20 hours. In the other two sites,
almost all the training provided to staff in fiscal year 2007 was based
on topics from the list of core requirements. An official in one
regional office, for example, said that his office used its full 20
hours to provide training on new and emerging issues that are not
covered by the core technical training topics, as well as training to
address error prone areas. An official in another regional office said
the core requirements satisfied staff training needs in fiscal year
2007, possibly because this regional office had a large proportion of
new staff and the core topics are focused on the needs of new staff.
Regional offices must develop training plans each year that
indicate which courses will actually be provided to staff to enable
them to meet the 80-hour training requirement. The training plan is a
list of courses that the regional office plans to offer throughout the
year, as well as the expected length and number and types of
participants in each course. In the regional offices we visited, when
managers develop their training plans, they solicit input from
supervisors of VSRs and RVSRs and typically also consider national or
local error trend data. Regional offices must submit their plans to the
VBA central office at the beginning of each fiscal year for review and
feedback. Central office officials review the plans to determine
whether (1) the regional office will deliver at least 60 hours of
training on the required core topics, (2) the additional topics
identified by the regional office are appropriate, and (3) staff in
similar positions within an office receive the same level and type of
training. According to central office officials, they provide feedback
to the regional offices on their current plans as well as guidance on
what topics to include in the next year's training plans. Regional
offices can adjust their training plans throughout the year to address
shifting priorities and unexpected training needs. For example, a
regional office may add or remove courses from the plan in response to
changing trends in errors or policy changes resulting from legal
decisions. (See app. III for excerpts from the fiscal year 2007
training plans from the regional offices we visited.)
While regional offices have discretion over the methods they use to
provide training, the four offices we visited relied primarily on
classroom training in fiscal year 2007. In each of these offices, at
least 80 percent of the total fiscal year 2007 training hours completed
by all claims processors was in the form of classroom instruction (see
fig. 3). Officials in two of the regional offices we visited said they
used lesson plans provided by the Compensation and Pension Service and
adapted these plans to the needs of their staff; one regional office
developed its own courses. An official in one office said they
sometimes invite guest speakers, and an official in another regional
office said that classroom training is sometimes delivered as part of
team meetings. The offices we visited generally made little use of
other training methods. Only one office used TPSS for its training more
than 1 percent of the time. Two offices used self-instruction--such as
reading memos from VBA central office--for about 10 percent of their
training, and no office used videos for more than 1 percent of their
training. The central office usually communicates immediate policy and
regulatory changes through memos called Fast Letters, which may be
discussed in team meetings or may just be read by staff individually.
Figure 3--Most Fiscal Year 2007 Training Hours Completed by
Claims Processors in the Offices We Visited Were in the
Form of Classroom Instruction
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Staff Are Not Held Accountable for Meeting Their Training Requirement
Because the agency has no policy outlining consequences for
individual staff who do not complete their 80 hours of training per
year, individual staff are not held accountable for meeting their
annual training requirement, and at present, VBA central office lacks
the ability to track training completed by individual staff members.
According to VBA officials, however, the agency is in the process of
implementing an automated system that should allow it to track the
training each staff member completes. Officials reported that this
system is expected to be implemented during fiscal year 2008. VBA
officials reported that this system will be able to record the number
of training hours and the courses completed for each individual, staff
position, and regional office. One official said the central office and
regional office supervisors will have the ability to monitor training
completed by individual staff members, but that central office will
likely not monitor the training completed by each individual staff
member, even though it may monitor the training records for a sample of
staff members. Furthermore, despite the absence of a VBA-wide tracking
system, managers in two of the regional offices we visited reported
using locally developed tracking methods to determine the number of
training hours their staff had completed.
While individuals are not held accountable, VBA reported taking
some steps to ensure that staff complete the required number of
training hours. VBA central office periodically reviews the aggregated
number of training hours completed at each regional office to determine
whether the office is on track to meet the training requirement.\8\
According to a VBA official, managers in offices where the staff is not
on track to complete 80 hours of training during the year can be
reprimanded by a higher-level manager, and if their staff do not meet
the aggregate training hours at the end of the fiscal year, managers
could face negative consequences in their performance assessments.
---------------------------------------------------------------------------
\8\ To determine if VSRs and RVSRs in a regional office are
generally meeting their annual training requirement, the aggregate
number of training hours completed in a given year by all staff in that
office is divided by the number of staff in that office.
---------------------------------------------------------------------------
VBA Is Taking Steps To Strategically Plan Its Training for Staff, But
Does Not Adequately Evaluate Training and May Be Falling Short
in Design and Implementation
VBA is taking steps to strategically plan its training for VSRs and
RVSRs including the establishment of a training board to assess VBA's
training needs. VBA has also made some effort to evaluate its training
for new staff, but does not require regional offices to collect
feedback from staff on any of the training they provide. Although some
regional offices collect some training feedback, it is not shared with
VBA central office. Both new and experienced staff we interviewed did,
in fact, report some problems with their training. A number of new
staff raised issues with how consistently their training curriculum was
implemented. Experienced staff differed in their assessments of the
VBA's annual training requirement, with some indicating they struggle
to meet this requirement because of workload pressures or that training
topics are sometimes redundant or not relevant to their position.
VBA Is Taking Steps To Strategically Plan Its Training
VBA is taking steps to strategically plan its training for claims
processors, in accordance with generally accepted practices identified
by GAO. (See app. I for a detailed description of these generally
accepted practices.)
Aligning Training with the Agency's Mission and Goals
VBA has made an effort to align training with the agency's mission
and goals. According to VBA documents, in fiscal year 2004 an Employee
Training and Learning Board (board) was established to ensure that
training decisions within the VBA are coordinated; support the agency's
strategic and business plans, goals and objectives; and are in
accordance with the policy and vision of VBA.\9\ Some of the board's
responsibilities include establishing training priorities and reviewing
regional office and annual training plans.
---------------------------------------------------------------------------
\9\ According to VBA officials, the board is made up of a mix of
regional office and central office staff from different VBA business
lines including Employee Development and Training, Human Resources, the
Compensation and Pension Service, and the Insurance Service.
---------------------------------------------------------------------------
Identifying the Skills and Competencies Needed by the Workforce
VBA has identified the skills and competencies needed by VBA's
claims processing workforce. VBA developed a decision tree and task
analysis of the claims process, which GAO experts in the field of
training told us made it possible to understand and map both the claims
process and the decisions associated with it that supported the
development of VBA's training curriculum.
Determining the Appropriate Level of Investment in Training and
Prioritizing Funding
VBA is taking steps to determine the appropriate level of
investment in training and prioritize funding. According to VBA
documents, some of the board's responsibilities include developing
annual training budget recommendations and identifying and recommending
training initiatives to the Under Secretary of Benefits. VBA officials
also reported developing several documents that made a business case
for different aspects of VBA's training, such as VA's annual budget and
the task analysis of the VSR and RVSR job positions.
Considering Government Reforms and Initiatives
According to one VBA official, the agency identifies regulatory,
statutory, and administrative changes as well as any legal or judicial
decisions that affect how VBA does business and issues guidance
letters, or Fast Letters, which can be sent out several times a year,
to notify regional offices of these changes. Also, as a result of
Congress authorizing an increase in its number of full-time employees
and VBA's succession planning efforts, VBA has increased the number of
centralized training sessions for new staff and has also increased the
number of Instructor Development Courses offered to potential
centralized training instructors. As a result, VBA is taking steps to
consider Government reforms and initiatives to improve its management
and performance when planning its training.
VBA Collects Feedback on Centralized Training, but Regional Offices Do
Not Always Collect Feedback on the Training They Provide
According to accepted practices, Federal agencies should also
evaluate their training programs and demonstrate how these efforts help
employees, rather than just focusing on activities or processes (such
as number of training participants or hours of training). VBA has made
some efforts to evaluate its training for claims processors. During the
3-week centralized training session for new staff, VBA solicits daily
feedback from participants using forms that experts in the training
field consider well-constructed and well-balanced. According to one GAO
expert, the forms generally employ the correct principles to determine
the effectiveness of the training and ascertain whether the instructor
effectively presented the material (see fig. 4). VBA officials told us
that they have used this feedback to improve centralized training for
new staff. Management at one regional office cited the decision to
separate training curricula for VSRs on Pre-determination teams and
VSRs on Post-determination teams as an example of a change based on
this feedback.
Figure 4--Sample of VBA's Centralized Training Evaluation Form
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Although VBA evaluates centralized training, it does not require
regional offices to obtain feedback from participants on any of the
training they provide to new and experienced staff. In a previous GAO
report, VA staff told us that new training materials they develop are
evaluated before being implemented.\10\ However, none of the regional
offices we visited consistently collect feedback on the training they
conduct. Supervisors from three of the regional offices we visited told
us that they collect feedback on some of the training their office
conducts, but this feedback largely concerns the performance of the
instructor. Participants are generally not asked for feedback on course
content. Moreover, regional offices we visited that do, to some degree,
collect feedback do not share this information with VBA.
---------------------------------------------------------------------------
\10\ GAO, Veterans' Benefits: Improved Management Would Enhance
VA's Pension Program, GAO-08-112 (Washington, DC: Feb. 14, 2008).
---------------------------------------------------------------------------
VBA's Training Curriculum for New Staff Appears Generally Well
Designed, but Some Staff Raised Issues Concerning Its
Implementation
According to GAO experts in the training field, VBA's training
curriculum for new staff appears well designed. VBA's curriculum for
new staff conforms to adult learning principles, carefully defining all
pertinent terms and concepts, and providing abundant and realistic
examples of claims work. GAO experts also determined that VBA's
training for those who teach the curriculum for new staff was well
designed and would enable experienced claims processors to become
competent trainers because they are coached on teaching theory and have
multiple opportunities to practice their teaching skills and receive
feedback.
Many of the new staff at all four sites we visited reported that
centralized training provided them with a good foundation of knowledge
and prepared them for additional training conducted by their regional
office. Also, regional office managers from three offices we visited
told us that centralized training affords new staff the opportunity to
network with other new staff at different regional offices, which
imbues a sense of how their positions fit in the organization. However,
some staff reported that VBA's implementation of their centralized
training was not always consistent. A number of staff at three regional
offices reported that during their centralized training the instructors
sometimes taught different ways of performing the same procedures or
disagreed on claim procedures. Regional office officials told us that
while centralized training instructors attempt to teach consistently
through the use of standardized training materials, certain procedures
can be done differently in different regional offices while adhering to
VBA policy. For example, regional offices may differ on what to include
in veteran notification letters. VBA officials also told us that
centralized training conducted at the regional offices may not be as
consistent as centralized training conducted at the Veterans Benefits
Academy. According to these officials, unlike the regional offices, the
Veterans Benefits Academy has on-site training experts to guide and
ensure that instructors are teaching the curriculum consistently.
New staff also gave mixed assessments about how training was
conducted at their home office after they returned from centralized
training. While some staff at all of the regional offices we visited
told us that the additional training better prepared them to perform
their jobs, with on-the-job training identified as a useful learning
tool, others told us that the training could not always be completed in
a timely manner due to regional office priorities. Some management and
staff at two of the regional offices we visited reported that, because
of workload pressures, some of their RVSRs had to interrupt their
training to perform VSR duties. Also, a few new staff indicated that
VBA's TPSS was somewhat difficult to use.\11\ Although TPSS was
developed to provide consistent technical training designed to improve
the accuracy of claims ratings, a number of staff at all of the
regional offices we visited reported that TPSS was too theoretical. For
example, some staff said it provided too much information and no
practical exercises in applying the knowledge. Some staff also noted
that certain material in TPSS was out-of-date with policy changes such
as how to order medical examinations. Some staff at three of the
regional offices also reported that TPSS was not always useful in
training staff, in part, because TPSS does not use real cases. Three of
the regional offices reported using TPSS for less than 1 percent of
their training and VSRs at one regional office were unaware of what
TPSS was.
---------------------------------------------------------------------------
\11\ In 2001, GAO reported that VBA's TPSS may not fully achieve
its objectives of providing standardized training to new employees,
reducing the training period required for new employees, or improving
claims-processing accuracy and consistency. In the report, we
recommended actions the agency should consider in providing timely
standardized training and providing indicators of the impact of TPSS on
accuracy and consistency. In its technical comments on this report, VA
indicated it accomplished the first recommendation. See GAO-01-601.
---------------------------------------------------------------------------
Experienced Staff Expressed Mixed Views of the Design and
Implementation of Their Training
At all of the regional offices we visited, staff we spoke with
generally noted that training enables them to keep up-to-date on
changes in laws and regulations as well as provides opportunities for
obtaining refresher training on claims procedures they perform
infrequently. However, regional office staff we spoke with differed in
their assessment of the 80-hour requirement. Some regional office staff
said the number of training hours required was appropriate, while
others suggested that VBA adopt a graduated approach, with the most
experienced staff being required to complete fewer hours than new
staff. VBA officials told us that, in 2007, the Compensation and
Pension Service reviewed their annual training requirements and
determined the 80-hour annual training requirement was appropriate.
However, the officials we spoke with could not identify the criteria
that were used to make these determinations. Furthermore, VBA
management does not systematically collect feedback from staff
evaluating the usefulness of the training they must receive to meet
this requirement. Consequently, when determining the appropriateness of
the 80-hour requirement, VBA has not taken into account the views of
staff to gauge the effect the requirement has on them.
Experienced staff had mixed views on training provided by the
regional office. Staff at three regional offices said the core
technical training topics set by the Compensation and Pension Service
are really designed for newer staff and do not change much from year to
year, and therefore experienced staff end up repeating courses. Also, a
number of staff at all of the regional offices we visited told us some
regional office training was not relevant for those with more
experience. Conversely, other regional office staff note that although
training topics may be the same from year to year, a person can learn
something new each time the course is covered. Some VBA officials and
regional office managers also noted that some repetition of courses is
good for several reasons. Staff may not see a particular issue very
often in their day-to-day work and can benefit from refreshers. Also,
regional office managers at one office told us that the core technical
training topics could be modified to reflect changes in policy so that
courses are less repetitive for experienced staff.
Many experienced staff also reported having difficulty meeting the
80-hour annual training requirement due to workload pressures. Many of
the experienced staff we spoke with, at each of the regional offices we
visited, told us that there is a constant struggle between office
production goals and training goals. For example, office production
goals can affect the availability of the regional office's instructors.
A number of staff from one regional office noted that instructors were
unable to spend time teaching because of their heavy workloads and
because instructors' training preparation hours do not count toward the
80-hour training requirement. Staff at another regional office told us
that, due to workload pressures, staff may rush through training and
may not get as much out of it as they should.
Performance Management System for Claims Processors Generally Conforms
to Accepted Practices, but May Not Clearly Differentiate
Between Performance Levels
VA's performance management system for claims processors is
consistent with several accepted practices for effective performance
management systems in the public sector, but may not clearly
differentiate between staff's overall performance levels. VA's
performance management system aligns individual performance elements
with broader organizational performance measures, provides performance
feedback to staff throughout the year, and emphasizes collaboration.
However, the system may not clearly differentiate VSRs' and RVSRs'
varying levels of performance. While the system has five summary rating
categories for VSRs and RVSRs, several VBA managers told us that,
because of a problem with the formula used to convert ratings on
individual performance elements into an overall performance category,
it is more difficult for staff to be placed in certain categories than
others.
Performance Management System for Claims Processors Is Generally
Consistent With Accepted Practices
The elements used to evaluate individual VSRs' and RVSRs'
performance appear to be generally aligned with VBA's organizational
performance measures, something prior GAO work has identified as a
well-recognized practice for effective performance management systems
(see app. I). Aligning individual and organizational performance
measures helps staff see the connection between their daily work
activities and their organization's goals and the importance of their
roles and responsibilities in helping to achieve these goals. VSRs must
be evaluated on four critical elements: quality, productivity, workload
management, and customer service. RVSRs are evaluated on quality,
productivity, and customer service. In addition, VBA central office
requires regional offices to evaluate their staff on at least one non-
critical element. The central office has provided a non-critical
element called cooperation and organizational support, and although
regional offices are not required to use this particular element, all
four offices we visited did so (see table 2). For each element, there
are three defined levels of performance: exceptional, fully successful,
or less than fully successful.\12\ Table 2 refers only to the fully
successful level of performance for each element.
---------------------------------------------------------------------------
\12\ The central office has set a minimum performance level for
each element that defines the fully successful level of performance.
Regional offices may set higher fully successful levels for their
staff, and three of the offices we visited had set a higher level for
at least one element. Regional offices also have discretion to set the
level for exceptional performance in each element for their staff.
Table 2--Individual Performance Elements for VSRs and RVSRs
----------------------------------------------------------------------------------------------------------------
Standard for minimum Standard for minimum
fully successful fully successful
Performance element How Performance is evaluated performance (journey- performance (journey-
level VSR) level RVSR)
----------------------------------------------------------------------------------------------------------------
Critical
----------------------------------------------------------------------------------------------------------------
Quality A random selection of 5 cases or 85% accuracy 85% accuracy
phone calls per month is reviewed
for accuracy based on certain
criteria, for example whether all
necessary evidence was requested,
proper notifications were sent to
applicants, and accurate information
was provided in phone calls. Any
case or phone call with one or more
errors is counted as one inaccurate
case or call.
----------------------------------------------------------------------------------------------------------------
Productivity Number of weighted actions (VSRs) or 8 weighted actions per 3.5 weighted cases per
weighted cases (RVSRs) completed per day \a\ day \b\
day. VSRs receive different weights
for different actions, such as 0.125
for conducting a telephone interview
or 1.50 for developing the evidence
for a claim with a special issue
such as radiation. RVSRs receive
different levels of credit for
processing cases with different
numbers of issues to be evaluated.
----------------------------------------------------------------------------------------------------------------
Customer service Number of valid complaints about No more than 3 valid complaints or incidents
employee's behavior from external
customers or internal colleagues.
----------------------------------------------------------------------------------------------------------------
Workload management Completion of designated tasks in a Tasks are completed in Not applicable
timely manner, such as obtaining the timely manner 85
results of a medical exam within a percent of the time
specified period of time.
----------------------------------------------------------------------------------------------------------------
Non-critical \c\
----------------------------------------------------------------------------------------------------------------
Cooperation and Understanding of agency goals, Interacts with colleagues professionally.
organizational support interaction with colleagues, Follows directions and adheres to guidance
contribution to agency goals. conscientiously. Adjusts easily to different
working styles and perspectives.
----------------------------------------------------------------------------------------------------------------
Source: GAO analysis of VBA information.
Note: This table includes the levels set for journey-level VSRs and RVSRs, who are considered experienced and
fully trained in their positions. For some elements VBA sets different performance standards for entry-level
and experienced claims processors. For example, VSRs are typically promoted to the journey-level position
after about 2 years. VBA has separate, lower performance standards in the accuracy, productivity, and workload
management elements for VSRs who are not yet at the journey level. Also, regional offices have the option of
setting fully successful levels for their staff that are higher than the national minimum, but not lower. This
table indicates instances when the regional offices we visited have set thresholds that are higher than the
national minimum.
\a\ Milwaukee has set a fully successful level of 10 weighted actions per day.
\b\ Baltimore, Milwaukee, and Portland have set fully successful levels of, respectively, 4, 5, and 3.8 weighted
cases per day.
\c\ Regional offices are required to use at least one non-critical element. VBA central office provided regional
offices with the cooperation and organizational support element, but regional offices are not required to use
this element in particular.
Three critical elements in particular--quality, workload
management, and productivity--are aligned with VBA's organizational
performance measures (see table 3). According to VA's strategic plan,
one key organizational performance measure for VBA is overall accuracy
in rating disability claims. This organizational measure is aligned
with the quality element for VSRs and RVSRs, which is assessed by
measuring the accuracy of their claims-processing work. An individual
performance element designed to motivate staff to process claims
accurately should, in turn, help VBA meet its overall accuracy goal.
Two other key performance measures for VBA are the average number of
days that open disability claims have been pending and the average
number of days it takes to process disability claims. VSRs are
evaluated on their workload management, a measure of whether they
complete designated claims-related tasks within specific deadlines.
Individual staff performance in this element is linked to the agency's
ability to manage its claims workload and process claims within goal
timeframes. Finally, a performance measure that VBA uses to evaluate
the claims-processing divisions within its regional offices--and that,
according to VBA, relates to the organization's overall mission--is
production, or the number of compensation and pension claims processed
by each office in a given time period. Individual VSRs and RVSRs are
evaluated on their productivity, i.e., the number of claims-related
tasks they complete per day. Higher productivity by individual staff
should result in more claims being processed by each regional office
and by VBA overall.
Table 3--Performance Elements for VSRs and RVSRs and Corresponding Organizational Performance Measures for VBA
----------------------------------------------------------------------------------------------------------------
Performance element for VSRs and RVSRs Corresponding VBA performance measure(s)
----------------------------------------------------------------------------------------------------------------
Quality Accuracy rate for ratings of compensation claims
----------------------------------------------------------------------------------------------------------------
Productivity Number of compensation and pension claims completed by the claims-
processing division within a regional office in a given time period
----------------------------------------------------------------------------------------------------------------
Workload management \a\ Average days pending for compensation and pension claims
----------------------------------------------------------------------------------------------------------------
Source: VBA and GAO analysis.
\a\ Workload management element applies only to VSRs, not RVSRs.
The performance management system for VSRs and RVSRs also appears
to be consistent with several other accepted practices for performance
management systems in the public sector:
Providing and Routinely Using Performance Information to Track
Organizational Priorities
Providing objective performance information to individuals helps
show progress in achieving organizational goals and allows individuals
to manage their performance during the year by identifying performance
gaps and improvement opportunities. Regional offices are supposed to
use the critical and non-critical performance elements to evaluate and
provide feedback to their staff. Supervisors are required to provide at
least one progress review to their VSRs and RVSRs each year, indicating
how their performance on each element compares to the defined standards
for fully successful performance. In the offices we visited,
supervisors typically provide some feedback to staff on a monthly
basis. For example, VSRs in the Atlanta regional office receive a memo
on their performance each month showing their production in terms of
average weighted actions per day, their accuracy percentage based on a
review of a sample of cases, and how their performance compared to the
minimum requirements for production and accuracy. If staff members fall
below the fully successful level in a critical element at any time
during the year, a performance improvement plan must be implemented to
help the staff member improve.
Connecting Performance Expectations to Crosscutting Goals
Performance elements related to collaboration or teamwork can help
reinforce behaviors and actions that support crosscutting goals and
provide a consistent message to all employees about how they are
expected to achieve results. VSR and RVSR performance related to
customer service is evaluated partly based on whether any valid
complaints have been received about a staff member's interaction with
their colleagues. And performance related to the cooperation and
organizational support element is based on whether staff members'
interaction with their colleagues is professional and constructive.
Using Competencies to Provide a Fuller Assessment of Performance
Competencies, which define the skills and supporting behaviors that
individuals are expected to exhibit to carry out their work
effectively, can provide a fuller assessment of an individual's
performance. In addition to elements that are evaluated in purely
quantitative terms, VBA uses a cooperation and organizational support
element for VSRs and RVSRs that requires supervisors to assess whether
their staff are exhibiting a number of behaviors related to performing
well as a claims processor.
Involving Employees and Stakeholders to Gain Ownership of the
Performance Management System
Actively involving employees and stakeholders in developing the
performance management system and providing ongoing training on the
system helps increase their understanding and ownership of the
organizational goals and objectives. For example, VA worked with the
union representing claims processors to develop an agreement about its
basic policies regarding performance management. Also, VBA indicated
that it planned to pilot revisions to how productivity is measured for
VSRs in a few regional offices, partly so VSRs would have a chance to
provide feedback on the changes.
VA's System May Not Clearly Differentiate Between Performance Levels
Clear differentiation between staff performance levels is also an
accepted practice for effective performance management systems. Systems
that do not result in meaningful distinctions between different levels
of performance fail to give (1) employees the constructive feedback
they need to improve, and (2) managers the information they need to
reward top performers and address performance issues. GAO has
previously reported that, in order to provide meaningful distinctions
in performance for experienced staff, agencies should use performance
rating scales with at least three levels, and scales with four or five
levels are preferable because they allow for even greater
differentiation between performance levels.\13\ If staff members are
concentrated in just one or two of multiple performance levels,
however, the system may not be making meaningful distinctions in
performance.
---------------------------------------------------------------------------
\13\ See GAO, Human Capital: Preliminary Observations on the
Administration's Draft Proposed ``Working for America Act,'' GAO-06-
142T (Washington, DC: Oct. 5, 2005).
---------------------------------------------------------------------------
VA's performance appraisal system has the potential to clearly
differentiate between staff performance levels. Each fiscal year,
regional offices give their staff a rating on each critical and non-
critical performance element using a three-point scale--exceptional,
fully successful, or less than fully successful. Based on a VA-wide
formula, the combination of ratings across these elements is converted
into one of VA's five overall performance levels: outstanding,
excellent, fully successful, minimally satisfactory, and unsatisfactory
(see fig. 5). Regional offices may award financial bonuses to staff on
the basis of their end-of-year performance category.\14\ Prior to
fiscal year 2006, VA used two performance levels--successful and
unacceptable--to characterize each staff member's overall performance.
To better differentiate between the overall performance levels of
staff, VA abandoned this pass-fail system in that year, choosing
instead to use a five-level scale.
---------------------------------------------------------------------------
\14\ In three of the four offices we visited, staff members placed
in the outstanding and excellent categories receive bonuses, and in one
of these offices some staff in the fully successful category also
receive bonuses.
---------------------------------------------------------------------------
Figure 5--VA Overall Performance Appraisal Formula
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: GAO analysis of VBA information.
However, there is evidence to suggest that the performance
management system for VSRs and RVSRs may not clearly or accurately
differentiate among staff's performance. VBA central office officials
and managers in two of the four regional offices we visited raised
concerns with VA's formula for translating ratings on individual
performance elements into an overall performance rating.\15\ These
officials said that under this formula it is more difficult for staff
to be placed in certain overall performance categories than others,
even if staff's performance truly does fall within one of those
categories. Indeed, at least 90 percent of all claims processors in the
regional offices we visited were placed in either the outstanding or
the fully successful category in fiscal year 2007. (Fig. 6 shows the
distribution of overall performance ratings for claims processors in
each office.)\16\
---------------------------------------------------------------------------
\15\ Officials in the other two offices we visited reported no
problems with the performance appraisal formula. Officials in one of
these offices told us the current five-level system provides more
flexibility than the previous pass/fail system.
\16\ We asked VA for fiscal year 2007 performance appraisal data
for VSRs and RVSRs nationally to determine whether the distribution of
staff across overall performance categories is similar at the national
level. While VA indicated that it collects performance appraisal data
for regional office staff, the agency was unable to provide us with
appraisal data specifically for VSRs and RVSRs, as these positions are
part of a broader job series.
---------------------------------------------------------------------------
Figure 6--Fiscal Year 2007 Overall Performance Ratings for Claims
Processors in Four Regional Offices Were Concentrated
in the Outstanding and Fully Successful Categories
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Note: These data cover VSRs, RVSRs, and some other claims
processing staff.
Central and regional office managers noted that, in particular, it
is difficult for staff to receive an overall rating of excellent.
Managers in one office said there are staff whose performance is better
than fully successful but not quite outstanding, but under the formula
it is difficult for these staff to be placed in the excellent category
as the managers feel they should be. An excellent rating requires
exceptional ratings in all the critical elements and a fully successful
rating in at least one non-critical element. However, according to
staff we interviewed, virtually all staff who are exceptional in the
critical elements are also exceptional in all non-critical element(s),
so they appropriately end up in the outstanding category. On the other
hand, the overall rating for staff who receive a fully successful
rating on just one of the critical elements--even if they are rated
exceptional in all the other elements--drops down to fully successful.
Managers in one regional office commented that the system would produce
more accurate overall performance ratings if staff were given an
overall rating of excellent when they had, for example, exceptional
ratings on three of five overall elements and fully successful ratings
on the other two.
An official in VA's Office of Human Resources Management
acknowledged that there may be an issue with the agency's formula.
Although neither VBA nor VA central office officials have examined the
distribution of VSRs and RVSRs across the five overall performance
ratings, VA indicated it is considering changes to the system designed
to allow for greater differentiation in performance ratings. For
example, one possible change would be to use a five-point scale for
rating individual elements--probably mirroring the five overall
performance rating categories of outstanding, excellent, fully
successful, minimally satisfactory, and unsatisfactory--rather than the
current three-point scale. Under the proposed change, a staff member
who was generally performing at the excellent but not outstanding level
could get excellent ratings in all the elements and receive an overall
rating of excellent. This change must still be negotiated with several
stakeholder groups, according to the VA official we interviewed.
Conclusions
In many ways, VBA has developed a training program for its new
staff that is consistent with accepted training practices in the
Federal Government. However, because VBA does not centrally evaluate or
collect feedback on training provided by its regional offices, it lacks
the information needed to determine if training provided at regional
offices is useful and what improvements, if any, may be needed.
Ultimately, this information would help VBA determine if 80 hours of
training annually is the right amount, particularly for its experienced
staff, and whether experienced staff members are receiving training
that is relevant for their positions. Identifying the right amount of
training is crucial for the agency as it tries to address its claims
backlog. An overly burdensome training requirement needlessly may take
staff away from claims processing, while too little training could
contribute to processing inaccuracies. Also, without collecting
feedback on regional office training, VBA may not be aware of issues
with the implementation of its TPSS, the online training tool designed
to ensure consistency across offices in technical training. Setting
aside the issue of how many hours of training should be required, VBA
does not hold its staff accountable for fulfilling their training
requirement. As a result, VBA is missing an opportunity to clearly
convey to staff the importance of managing their time to meet training
requirements as well as production and accuracy goals. With the
implementation of its new learning management system, VBA should soon
have the ability to track training completed by individual staff
members, making it possible to hold them accountable for meeting the
training requirement.
As with its training program for VSRs and RVSRs, the VA is not
examining the performance management system for claims processors as
closely as it should. VBA is generally using the right elements to
evaluate its claims processors' performance, and the performance
appraisals have the potential to give managers information they can use
to recognize and reward higher levels of performance. However, evidence
suggests the formula used to place VSRs and RVSRs into overall
performance categories may not clearly and accurately differentiate
among staff's performance levels. Absent additional examination of the
distribution of claims processors among overall performance categories,
VA lacks a clear picture of whether its system is working as intended
and whether any adjustments are needed.
Recommendations for Executive Action
The Secretary of Veterans Affairs should direct VBA to:
Collect and review feedback from staff on the training
conducted at the regional offices to determine
if the 80-hour annual training requirement is
appropriate for all VSRs and RVSRs;
the extent to which regional offices provide training
that is relevant to VSRs' and RVSRs' work, given varying levels
of staff experience; and
whether regional offices find the TPSS a useful
learning tool and, if not, what adjustments are needed to make
it more useful; and
Use information from its new learning management system
to hold individual VSRs and RVSRs accountable for completing whatever
annual training requirement it determines is appropriate.
The Secretary of Veterans Affairs should also examine the
distribution of claims processing staff across overall performance
categories to determine if its performance appraisal system clearly
differentiates between overall performance levels, and if necessary
adjust its system to ensure that it makes clear distinctions.
Agency Comments
We provided a draft of this report to the Secretary of Veterans
Affairs for review and comment. In VA's written comments (see app. IV),
the agency agreed with our conclusions and concurred with our
recommendations. For example, VBA plans to consult with regional office
staff to evaluate its annual 80-hour training requirement and will
examine if staff performance ratings clearly differentiate between
overall performance levels. VA also provided technical comments that
were incorporated as appropriate.
We are sending copies of this report to the Secretary of Veterans
Affairs, relevant congressional Committees, and others who are
interested. We will also provide copies to others on request. The
report is also available at no charge on GAO's Web site at http://
www.gao.gov.
Please contact me on (202) 512-7215 if you or your staff have any
questions about this report. Contact points for the Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors are listed in appendix V.
Sincerely,
Daniel Bertoni
Director, Education, Workforce, and
Income Security Issues
__________
Appendix I: Objectives, Scope, and Methodology
We were asked to determine: (1) What training is provided to new
and experienced claims processors and how uniform is this training? (2)
To what extent has the Veterans Benefits Administration (VBA) developed
a strategic approach to planning training for claims processors and how
well is their training designed, implemented, and evaluated? And (3) To
what extent is the performance management system for claims processors
consistent with generally accepted performance management practices in
the public sector? To answer these questions, we reviewed documents and
data from the central office of the Department of Veterans Affairs'
Veterans Benefits Administration (VBA) and interviewed VBA central
office officials. We conducted site visits to and collected data from
four VBA regional offices, and visited the Veterans Benefits Academy.
We also interviewed officials from the American Federation of
Government Employees, the labor union that represents Veterans Service
Representatives (VSR) and Rating Veterans Service Representatives
(RVSR). We compared VBA's training and performance management systems
to accepted human capital principles and criteria compiled by GAO. We
conducted this performance audit from September 2007 through May 2008
in accordance with generally accepted Government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Regional Office Site Visits
We conducted site visits to 4 of VBA's 57 regional offices--
Atlanta; Baltimore; Milwaukee; and Portland, Oregon. We judgmentally
selected these offices to achieve some diversity in geographic
location, number of staff, and claims processing accuracy rates, and
what we report about these sites may not necessarily be representative
of any other regional offices or all regional offices (see fig. 7).\17\
---------------------------------------------------------------------------
\17\ To determine each office's accuracy performance in fiscal year
2007, we used data obtained from VBA's Systematic Technical Accuracy
Review (STAR) system. In an earlier GAO report, Veterans' Benefits:
Further Changes in VBA's Field Office Structure Could Help Improve
Disability Claims Processing, GAO-06-149 (Washington, DC: Dec. 9,
2005), we identified problems that affected the use of the STAR data to
make distinctions in accuracy among regional offices. However, for the
purposes of site selection for our current review, we judged the STAR
data to be sufficiently reliable. We made this determination based on a
sensitivity analysis we did on earlier year data that considered
sampled cases that were not sent in for STAR review. After this
analysis we found that even with the existing limitations in the STAR
data, Milwaukee and Baltimore had higher accuracy scores and Atlanta
and Portland had lower accuracy scores. Even though the sensitivity
analysis was done on earlier year data, the ranking of the four offices
was similar in fiscal year 2007, showing that the offices we deemed to
have higher accuracy scores in an earlier year still had higher
accuracy scores in fiscal year 2007 and the same remained true for the
offices with lower accuracy scores.
---------------------------------------------------------------------------
Figure 7--Regional Offices Selected for Site Visits
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
\a\ Full-time equivalents as of September 2007.
\b\ Rank among all 57 regional offices.
\c\ Claims-processing accuracy rate for the period of August 1,
2006 to July 31, 2007.
During our site visits, we interviewed regional office managers,
supervisors of VSRs and RVSRs, VSRs, and RVSRs about the training and
performance management practices in their offices. The VSRs and RVSRs
we interviewed at the four regional offices had varying levels of
experience at VBA. Regional office managers selected the staff we
interviewed. We also observed a demonstration of VBA's online learning
tool, the Training and Performance Support System (TPSS), and collected
data from the regional offices on, for example, the training they
provided during fiscal year 2007.\18\ In conjunction with our visit to
the Baltimore regional office, we also visited VBA's Veterans Benefits
Academy, where we observed classes for VSRs and RVSRs and interviewed
the Director of the Academy.
---------------------------------------------------------------------------
\18\ One question we asked the regional offices was whether each
course on their fiscal year 2007 training plan addressed a core
technical training topic. For three of the offices, the data we
received did not cover all training hours provided during the fiscal
year, but each office provided data on at least 99 percent of its
training hours.
---------------------------------------------------------------------------
Assessment of VBA's Training for Claims Processors
To determine whether VBA's training program is consistent with
accepted training practices in the public sector, we relied partly on a
guide developed by GAO that lays out principles that Federal agencies
should follow to ensure their training is effective.\19\ This guide was
developed in collaboration with Government officials and experts in the
private sector, academia, and nonprofit organizations; and in
conjunction with a review of laws, regulations and literature on
training and development issues, including previous GAO reports. The
guide lays out the four broad components of the training and
development process (see fig. 8).
---------------------------------------------------------------------------
\19\ GAO, Human Capital: A Guide for Assessing Strategic Training
and Development Efforts in the Federal Government, GAO-04-546G
(Washington, DC: March 2004).
---------------------------------------------------------------------------
Figure 8--Four Components of the Training and Development Process
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Source: GAO
Note: The evaluation component may include the use of participant
feedback to ensure continuous improvement, as well as an assessment of
the impact of training on organizational performance. We have reported
that higher-level evaluations that attempt to measure the return on
investment in a training program may not always be appropriate, given
the complexity and costs associated with efforts to directly link
training programs to improved individual and organizational
performance.
The guide also provides key questions for Federal agencies to
consider in assessing their performance in each component. (See table 4
for a sample of these questions.)
Table 4--Selected Key Questions to Consider in Assessing Agency's Training Program
----------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------
Planning/Front End Analysis Does the agency have training goals and related performance
measures that are consistent with its overall mission, goals, and
culture?
How does the agency identify the appropriate investment to
provide for training and development efforts and prioritize funding
so that the most important training needs are addressed first?
----------------------------------------------------------------------------------------------------------------
Design and Development What criteria does the agency use in determining whether to
design training and development programs in-house or obtain these
services from a contractor or other external source?
Does the agency use the most appropriate mix of centralized
and decentralized approaches for its training and development
programs?
----------------------------------------------------------------------------------------------------------------
Implementation What steps do agency leaders take to communicate the
importance of training and developing employees, and their
expectations for training and development programs to achieve
results?
How does the agency select employees to participate in
training and development efforts?
----------------------------------------------------------------------------------------------------------------
Evaluation To what extent does the agency systematically plan for and
evaluate the effectiveness of its training and development efforts?
How does the agency incorporate evaluation feedback into the
planning, design, and implementation of its training and development
efforts?
----------------------------------------------------------------------------------------------------------------
Source: GAO.
In addition, GAO training experts reviewed VBA materials, including
training curricula, lesson plans, and course evaluation forms, to
determine if these materials are consistent with accepted training
practices.
Assessment of VBA's Performance Management System for Claims Processors
In assessing the performance management system for VSRs and RVSRs,
we relied primarily on a set of accepted practices of effective public
sector performance management systems that has been compiled by
GAO.\20\ To identify these accepted practices, GAO reviewed its prior
reports on performance management that drew on the experiences of
public sector organizations both in the United States and abroad. For
the purpose of this review, we focused on the six accepted practices
most relevant for VBA's claims-processing workforce (see table 5).
---------------------------------------------------------------------------
\20\ GAO, Results-Oriented Cultures: Creating a Clear Linkage
between Individual Performance and Organizational Success, GAO-03-488
(Washington, DC: Mar. 14, 2003).
Table 5--Selected Accepted Practices for Effective Performance Management Systems
----------------------------------------------------------------------------------------------------------------
Practice Description
----------------------------------------------------------------------------------------------------------------
Aligning individual performance Explicitly aligning individuals' daily activities with broader
expectations with organizational goals results helps individuals see the connection between their work and
organizational goals and encourages individuals to focus on their
roles and responsibilities to help achieve those broader goals.
----------------------------------------------------------------------------------------------------------------
Connecting performance expectations to Fostering collaboration, interaction, and teamwork across
crosscutting goals organizational boundaries to achieve results strengthens
accountability for these results.
----------------------------------------------------------------------------------------------------------------
Providing and routinely using performance Providing objective performance information to both managers and
information to track organizational staff to show progress in achieving organizational results and other
priorities priorities helps them manage during the year, identify performance
gaps, and pinpoint improvement opportunities.
----------------------------------------------------------------------------------------------------------------
Using competencies to provide a fuller Using competencies, which define the skills and supporting behaviors
assessment of performance that individuals need to effectively contribute to organizational
results, can provide a fuller picture of an individual's
performance.
----------------------------------------------------------------------------------------------------------------
Making meaningful distinctions in Providing individuals with candid and constructive feedback helps
performance them maximize their contribution, and providing management with
objective and fact-based information that clearly differentiates
between different levels of performance enables it to reward top
performers and deal with poor performers.
----------------------------------------------------------------------------------------------------------------
Involving employees and stakeholders to Actively involving employees and stakeholders in developing the
gain ownership of performance management performance management system and providing ongoing training on the
systems system helps increase their understanding and ownership of the
organizational goals and objectives.
----------------------------------------------------------------------------------------------------------------
Source: GAO.
Appendix II: Fiscal Year 2007 Core Technical Training Requirements for
VSRs and RVSRs
------------------------------------------------------------------------
Position Course title or topic Training source
------------------------------------------------------------------------
Decision Review
Officers (DRO)
GS13/Rating
Veterans
Service
Representative
s (RVSR) GS7-
12
(Seasoned)
Required: 80
hours
Effective Dates C&P Training Website http://
Any DRO or RVSR Diabetes Mellitus cptraining.vba.va.gov/
who conducts a Leishmaniasis C&P_Training/RVSR/
training Original Compensation RVSR_Tng_Curr.htm
session will Ratings
also be given Original Pension
credit for Ratings
those training Original DIC Ratings
hours as part Rating re-opened
of their claims
training Claims for Increase
requirement. New and Material
Evidence
Re-opened DIC ratings
Routine Future Exams
3.105(e) reductions
Paragraph 28/29/30
ratings
Due Process Provisions
Clear and unmistakable
errors (3.105(a))
Ancillary Benefits
Accrued Ratings
Musculoskeletal issues
Eye-Vision Issues
Infectious Diseases
Ear-Hearing
Respiratory Disorders
Cardiovascular Issues
Digestive Issues
Genitourinary System
Gynecology
Hemic/Lymphatic
Endocrine (other than
DM)
Neurological
Mental Disorder (other
than PTSD)
PTSD
Special Monthly
Compensation (SMC)
The Appeals Process
Responsibilities of a
DRO
Hearings
Informal Conferences
Resolution of Claims
Certifying a case to
BVA
Processing Remands
Preparing a Statement
of the Case (SOC)
Preparing a
Supplemental
Statement of the Case
(SSOC)
Role of the Rating
Specialist
Benefit of the Doubt
Weighing Evidence
60 Hours of the
required 80 Hours
will be selected from
the suggested topics
above. The remaining
20 hours will be
selected at the
Stations discretion
based upon their own
individual quality
review.
(Training provided
from the above topics
can be focused on a
particular aspect of
the topic; i.e. Cold
Injuries and Rating
Hypertension from
Cardiovascular issues
could be separate
classes)
Participation in
Agency Advancement
Programs (i.e., LEAD,
LVA) does not
substitute for
Required training
requirements.
------------------------------------------------------------------------
Veteran Service
Representative
(VSR) GS 7-12
(Seasoned)
Required: 80
hours
Reference Materials:
Any Super Manual
Senior VSR, Training & WARMS
Senior VSR, or C&P Website
VSR, who Claims Folder
conducts a Maintenance
training Records Management
session will POA/Service Orgs.
also be given Original Compensation
credit for Claims
those training Re-opened Compensation
hours Claims
including VA Form 21-526
preparation Establishing Veteran
time as part Status
of their Claims Recognition
training Duty to Assist
requirement. Requesting VA Exams
Issue Specific Claims
Development
Asbestos Claims
Development
Herbicide Claims
Development
POW Claims Development
C&P Training Website
Intro. to Ratings
Paragraph 29 & 30
Ratings
Ratings & BDN
BDN 301 Interface
PCGL Award Letters
Dependents and the BDN
Compensation Offsets
Drill Pay Waivers
Pension Awards
Processing & BDN
Hospital Reductions
Burial Benefits
Death Pension
Accrued Benefits
Accrued Awards & the
BDN
Apportionments
Special Monthly
Pension
Helpless Child
Incompetency/Fiduciary
Arrangements
Claims Processing
Auto Allowance and
Adaptive Equipment
Special Adapted
Housing
Special Home
Adaptation Grants
Incarcerated Veterans
Processing Write Outs
FOIA/Privacy Act
Telephone & Interview
Techniques
Telephone Development
IRIS
Introduction to VACOLS
Education Benefits
Insurance Benefits
National Cemetery
VR&E Benefits
Loan Guaranty Benefits
General Benefits--FAQs
Suicidal Caller
Guidance
Non-Receipt of BDN
Payments
Mail Handling
Income & Net Worth
Determinations
Bootcamp test and
review of VSR
Readiness Guide (2 HRS
Required)
Reference Material
Training and
Navigation (1 HR
Required)
Appeals and Ancillary
Benefits
Ready to Rate
Development
Customer Service
FNOD Info and PMC
Process
Intro. to Appeals
Process
DRO Selection Letter
Income Adjustment
Materials
Income Adjustments
60 Hours of the
required 80 Hours
will be selected from
the suggested topics
above. The remaining
20 hours will be
selected at the
Stations discretion
based upon their own
individual quality
review.
------------------------------------------------------------------------
Veterans
Services
Representative
(VSR) GS 7-12
(New)
Required:
Entire
Curriculum
Curriculum is posted
(Follow C&P on C&P Training
Prescribed Intranet Site
Curriculum for
new VSRs, as
posted on
intranet.)
http://cptraining.vba.va.gov/
SHARE (BDN & CEST)
COVERS
PIES
Return with Honor
Video
MAPD
AMIE/CAPRI
Medical TPSS (Medical
Terminology)
Reader Focused Writing
Tools
--------------------------------------------------------
Pre-Determination Team
Training:
http://cptraining.vba.va.gov/
C&P Website
Claims Folder
Maintenance
Records Management
POA/Service
Organizations
Compensation
Original Compensation
Claims
Non-Original
Compensation Claims
VA Form 21-526, App.
For Compensation or
Pension
Establishing Veteran
Status
Claims Recognition
Duty to Assist
Selecting the Correct
Worksheet for VA
Exams
Issue Specific Claim
Development
Asbestos Claim
Development
Herbicide Claim
Development
POW Claim Development
Radiation Claim
Development
PTSD Claim Development
Undiagnosed Illness
Claim Development
Dependency
Contested Claims
Deemed Valid and
Common-law Marriage
Continuous
Cohabitation
Pension
Intro. To Disability
Pension
Overview of SHARE
(SSA)
Administrative
Decision Process
Character of Discharge
Line of Duty--Willful
Misconduct
Claims Development
Workload Management
Utilizing WIPP
--------------------------------------------------------
Post-Determination
Team Training:
http://cptraining.vba.va.gov/
Ratings & the BDN
BDN 301 Interface
Video
PCGL Award Letters
PCGL
Dependents & the BDN
Compensation Offsets
Drill Pay Waivers
Star Reporter
Pension Awards
Processing & the BDN
Hospital Reductions
Burial Benefits
Disallowance
Processing
DIC Benefits
Death Pension
Accrued Benefits
Accrued Awards & the
BDN
Apportionment
Special Monthly
Pension
Helpless Child
Incompetency/Fiduciary
Arrangements
Claims Processing
Automobile Allowance
and Adaptive
Equipment
Specially Adapted
Housing and Special
Home Adaptation
Grants
Incarceration
Processing Computer
Write Outs
DEA Training (req.
added 4/06)
--------------------------------------------------------
Public Contact Team
Training:
http://cptraining.vba.va.gov/
------------------------------------------------------------------------
Source: VBA.
Appendix III: Excerpts From Fiscal Year 2007 Training Plans for Four
Regional Offices
Each training plan we reviewed contained the same informational
categories, some of which were what courses were offered by the
regional office, whether or not the course was conducted, and how many
employees completed the training. Although the fiscal year 2007
training plans we reviewed include data on whether and when the course
was actually completed, the initial training plans submitted at the
beginning of the fiscal year of course do not have this information.
The lists provided below include the first 25 courses listed on each
plan alphabetically, a small sample of the courses that the regional
offices reported they completed for the fiscal year.
Table 6--Excerpt From Atlanta Regional Office Training Plan
------------------------------------------------------------------------
Number of Total hours
Course name employees of training
completed completed
------------------------------------------------------------------------
Accrued Benefits 15 150
------------------------------------------------------------------------
Accrued Ratings (2 sessions conducted) 47 80
------------------------------------------------------------------------
Administrative Decisions 15 60
------------------------------------------------------------------------
Ancillary Benefits 14 14
------------------------------------------------------------------------
Appeals and Ancillary Benefits (2 sessions 26 41
conducted)
------------------------------------------------------------------------
Apportionments (2 sessions conducted) 29 194
------------------------------------------------------------------------
Asbestos Claims Development 9 9
------------------------------------------------------------------------
Auto Allowance/Special Adapted Housing/ 15 30
Special Home Adaptation Grant
------------------------------------------------------------------------
Benefits Delivery Network 301 Interface (2 48 48
sessions conducted)
------------------------------------------------------------------------
Beneficiary Identification Records Locator 17 17
Subsystem Update
------------------------------------------------------------------------
Blast Injuries (2 sessions conducted) 20 20
------------------------------------------------------------------------
Burial Benefits (2 sessions conducted) 36 100
------------------------------------------------------------------------
Board of Veterans Appeals Examinations 46 69
------------------------------------------------------------------------
Compensation & Pension Website (2 sessions 108 270
conducted)
------------------------------------------------------------------------
Change of Address/Power of Attorney 17 34
Processing/No Record Mail
------------------------------------------------------------------------
Cardiovascular Issues 38 76
------------------------------------------------------------------------
Certifying a Case to Board of Veterans 12 12
Appeals
------------------------------------------------------------------------
Character of Discharge 78 78
------------------------------------------------------------------------
Claims Folder Maintenance (2 sessions 17 28
conducted)
------------------------------------------------------------------------
Claims for Direct Service Connection/ 34 34
Aggravation/Presumptive Service Connection
------------------------------------------------------------------------
Claims for Increase 29 58
------------------------------------------------------------------------
Claims Processing 139 69.5
------------------------------------------------------------------------
Claims Recognition 84 336
------------------------------------------------------------------------
Compensation Offsets (3 sessions conducted) 167 352.5
------------------------------------------------------------------------
Computer Security and LAN Procedures 6 6
------------------------------------------------------------------------
Source: VBA.
Note: Atlanta's training plan reported the regional office conducted a
total of 133 courses for fiscal year 2007.
Table 7--Excerpt From Baltimore Regional Office Training Plan
------------------------------------------------------------------------
Number of Total hours
Course name employees of training
completed completed
------------------------------------------------------------------------
Accrued Benefits 5 10
------------------------------------------------------------------------
Automated Medical Information Exchange/ 6 48
Compensation and Pension Record Interchange
------------------------------------------------------------------------
Appeals and Ancillary Benefits 3 3
------------------------------------------------------------------------
Asbestos Claims Development 3 3
------------------------------------------------------------------------
Access Standardized Performance Elements 2 2
Nationwide
------------------------------------------------------------------------
Auto Allowance and Adaptive Equipment (2 16 8
sessions conducted)
------------------------------------------------------------------------
Benefits Delivery at Discharge Development 14 21
------------------------------------------------------------------------
Benefits Delivery Network 301 Interface (2 5 7
sessions conducted)
------------------------------------------------------------------------
Benefit of the Doubt 3 12
------------------------------------------------------------------------
Burial Benefits (2 sessions conducted) 7 14
------------------------------------------------------------------------
Compensation & Pension Website (3 sessions 15 36.5
conducted)
------------------------------------------------------------------------
Certifying a Case to Board of Veterans 3 12
Appeals
------------------------------------------------------------------------
Character of Discharge 15 7.5
------------------------------------------------------------------------
Claims Folder Maintenance 7 14
------------------------------------------------------------------------
Claims Recognition 5 20
------------------------------------------------------------------------
Communication--Nonverbal Cues 3 1.5
------------------------------------------------------------------------
Computer Security and LAN Procedures 6 12
------------------------------------------------------------------------
Conducting a Field Exam 3 1.5
------------------------------------------------------------------------
Continuous Cohabitation (2 sessions 20 20
conducted)
------------------------------------------------------------------------
Core Values 5 5
------------------------------------------------------------------------
Control of Veterans Records System (3 10 12.5
sessions conducted)
------------------------------------------------------------------------
Customer Service (5 sessions conducted) 40 416
------------------------------------------------------------------------
Dealing with Difficult Payee Situations 3 3
------------------------------------------------------------------------
Deemed Valid and Common Law Marriages (2 20 12.5
sessions conducted)
------------------------------------------------------------------------
Dependency Issues (3 sessions conducted) 22 26.5
------------------------------------------------------------------------
Source: VBA.
Note: Baltimore's training plan reported the regional office conducted a
total 191 courses for fiscal year 2007.
Table 8--Excerpt From Milwaukee Regional Office Training Plan
------------------------------------------------------------------------
Number of Total hours
Course name employees of training
completed completed
------------------------------------------------------------------------
8824e 1 1
------------------------------------------------------------------------
Administrative Decisions 14 91
------------------------------------------------------------------------
Advanced Data Manipulation in Excel (VA 1 4
Learning Online)
------------------------------------------------------------------------
All--Litigation Hold Memo 130 32.5
------------------------------------------------------------------------
All-Encryption Training 1 0.5
------------------------------------------------------------------------
Ancillary Benefits 21 42
------------------------------------------------------------------------
Auto Allowance and Adaptive Equipment 28 28
------------------------------------------------------------------------
Blast Injuries (Video) 33 33
------------------------------------------------------------------------
Board of Veterans Appeals review 7 14
------------------------------------------------------------------------
Compensation & Pension Website 41 102.5
------------------------------------------------------------------------
Claims Assistant--Burials 4 4
------------------------------------------------------------------------
Claims Assistant/Program Support Clerk-- 24 24
Power of Attorney
------------------------------------------------------------------------
Claims Assistant/Program Support Clerk-- 21 178.5
Share and Cest
------------------------------------------------------------------------
Claims Assistant/Program Support Clerk-- 25 25
Veterans Appeals Control and Locator System
------------------------------------------------------------------------
Cardiovascular Issues 30 180
------------------------------------------------------------------------
Challenge 07-02 Centralized Training 6 720
------------------------------------------------------------------------
Challenge 07-02 Post Centralized Training 6 1,440
------------------------------------------------------------------------
Challenge 07-02 Pre-Req. 6 720
------------------------------------------------------------------------
Claims Folder Maintenance 41 82
------------------------------------------------------------------------
Claims Recognition 26 26
------------------------------------------------------------------------
Character of Discharge Determinations, Line 11 24.75
of Duty Determinations, and Administrative
Decisions.
------------------------------------------------------------------------
Compensation Offsets 27 94.5
------------------------------------------------------------------------
Core Values 2 3.5
------------------------------------------------------------------------
Control of Veterans Records System (2 2 3
sessions conducted)
------------------------------------------------------------------------
Compensation and Pension Examination Project 1 18
------------------------------------------------------------------------
Source: VBA.
Note: Milwaukee's training plan reported the regional office conducted a
total of 323 courses for fiscal year 2007.
Table 9--Excerpt From Portland Regional Office Training Plan
------------------------------------------------------------------------
Number of Total hours
Course name employees of training
completed completed
------------------------------------------------------------------------
020 Development 3 16.5
------------------------------------------------------------------------
2007 Veterans Service Center Management 1 26
Workshop
------------------------------------------------------------------------
3.105(e) Reductions 15 15
------------------------------------------------------------------------
38 CFR 3.14 & Pyramiding 2 0.5
------------------------------------------------------------------------
5-Tier Performance Evaluations 7 5.25
------------------------------------------------------------------------
8824 Preparation 1 5
------------------------------------------------------------------------
Absence & Leave Circular Training 13 13
------------------------------------------------------------------------
Account Analysis 3 6
------------------------------------------------------------------------
Account Audits 3 6
------------------------------------------------------------------------
Accrued Awards & the Benefits Delivery 2 2
Network
------------------------------------------------------------------------
Accrued Ratings 16 4
------------------------------------------------------------------------
Add Dependents/Verifying Service 18 9
------------------------------------------------------------------------
Admin Decisions/Rebuilt/Special Monthly 4 18
Compensation
------------------------------------------------------------------------
Administrative Decisions 5 2.5
------------------------------------------------------------------------
Agent Orange development 4 4
------------------------------------------------------------------------
Amputation Rule 2 0.5
------------------------------------------------------------------------
Ancillary Benefits 28 28
------------------------------------------------------------------------
Appeal Procedures--Refresher 3 5.25
------------------------------------------------------------------------
Appeals 33 33
------------------------------------------------------------------------
Appeals and Ancillary Benefits (3 sessions 34 13
conducted)
------------------------------------------------------------------------
Appeals--Training and Performance Support 1 16
System modules
------------------------------------------------------------------------
Application/eligibility 1 3.5
------------------------------------------------------------------------
Apportionments (2 sessions conducted) 4 14.5
------------------------------------------------------------------------
Asbestos Claims Development 23 23
------------------------------------------------------------------------
Access Standardized Performance Elements 6 6
Nationwide
------------------------------------------------------------------------
Source: VBA.
Note: Portland's training plan reported the regional office conducted a
total of 509 courses for fiscal year 2007.
Appendix IV: Comments From the Department of Veterans Affairs
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
Appendix V: GAO Contact and Staff Acknowledgments
GAO Contact
Daniel Bertoni (202) 512-7215 [email protected].
Staff Acknowledgments
In addition to the contact named above, Clarita Mrena, Assistant
Director; Lorin Obler, Analyst-in-Charge; Carolyn S. Blocker; and David
Forgosh made major contributions to this report; Margaret Braley, Peter
Del Toro, Chris Dionis, Janice Latimer, and Carol Willett provided
guidance; Walter Vance assisted with study design; Charles Willson
helped draft the report; and Roger Thomas provided legal advice.
Related GAO Products
Veterans' Benefits: Improved Management Would Enhance VA's Pension
Program. GAO-08-112. Washington, DC: February 14, 2008.
Veterans' Disability Benefits: Claims Processing Challenges
Persist, while VA Continues to Take Steps to Address Them. GAO-08-473T.
Washington, DC: February 14, 2008.
Disabled Veterans' Employment: Additional Planning, Monitoring, and
Data Collection Efforts Would Improve Assistance. GAO-07-1020.
Washington, DC: September 12, 2007.
Veterans' Benefits: Improvements Needed in the Reporting and Use of
Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.
Washington, DC: September 30, 2003.
Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government. GAO-03-893G. Washington,
DC: July 2003.
Results-Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success. GAO-03-488.
Washington, DC: March 14, 2003.
Major Management Challenges and Program Risks: Department of
Veterans Affairs. GAO-03-110. Washington, DC: January 1, 2003.
Veterans' Benefits: Claims Processing Timeliness Performance
Measures Could Be Improved. GAO-03-282. Washington, DC: December 19,
2002.
Veterans' Benefits: Quality Assurance for Disability Claims and
Appeals Processing Can Be Further Improved. GAO-02-806. Washington, DC:
August 16, 2002.
Veterans' Benefits: Training for Claims Processors Needs
Evaluation. GAO-01-601. Washington, DC: May 31, 2001.
Veterans' Benefits Claims: Further Improvements Needed in Claims-
Processing Accuracy. GAO/HEHS-99-35. Washington, DC: March 1, 1999.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 25, 2008
Michael Ratajczak
Decision Review Officer
Cleveland Veterans Affairs Regional Office
80 F Street, NW
Washington, DC 20001
Dear Mr. Ratajczak:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Response of Michael Ratajczak to:
Questions From the House Committee on Veteran's Affairs,
Subcommittee on Disability Assistance and Memorial Affairs
Question 1: How useful is online training as opposed to classroom
style? Which do employees prefer?
Response: The general experience of VBA employees is that online
training is less useful than classroom training. Online training is
extremely limited insofar as it is not easily conformed to the constant
changes permeating VBA's claims process. A computer based curriculum
developed only weeks prior to an employee participating in computer
based training might be inaccurate in some aspects due to changes in
law or policy. In addition, online training is not generally responsive
to questions not considered by its designer. So, a question may be
suggested to a VBA employee based upon their unique experiences and
some aspect of an online curriculum but that curriculum could not
provide any useful guidance on the issue presented.
In contrast, properly administered classroom training encourages
real-time interaction between the instructor and the participants, and
provides immediate feedback for questions presented. Moreover,
classroom training can easily incorporate any recent changes in law,
policy, or regulations into its curriculum, thereby providing more
relevant and up to date information. A classroom setting provides an
opportunity for participants to deviate from the prepared curriculum to
related topics having direct impact on their ability to perform their
duties, and provides a similar opportunity for instructors to identify
and correct common misconceptions among participants. Properly
administered classroom training provides immediate, consistent, useful
and relevant guidance based upon practical concerns expressed by
participants. It is the most efficient manner to ensure consistency in
processing and resolving complex claims, and ultimately provides the
best service to veterans.
Of course, the foregoing comments assume that the ``online
training'' in question is based upon a static format, as characterized
for example by VBA's TPSS program. As was suggested by some questions
posed during the September 18, 2008, hearing, some training can be
effectively presented electronically through the use of
teleconferencing technology. Such an approach has the advantage of
providing uniform presentation of the subject matter to a widely
dispersed audience by an authoritative instructing staff. However, the
utility of such training is limited by the number of participants and
the amount of available time. The danger of relying on teleconference
training to reach a large audience is that it cannot guarantee
effective feedback for the concerns of every participant.
The best approach to continuing training at VBA incorporates the
most useful aspects of computer based, classroom, and teleconference
instruction. Such an approach would dictate that VBA's Central Office
training staff create a standard curriculum and an electronic claims
file relevant to issues identified as requiring additional training.
Central Office training staff would then distribute the electronic
materials and curriculum to instructors at each Regional Office who are
certified by Central Office to conduct centralized training. After
soliciting comments from Regional Office instructors Central Office
staff could facilitate a teleconference among the Regional Office
instructors, addressing their disparate concerns and providing uniform
guidance as to how they should approach the materials and topics at
hand. Finally, Regional Office instructors could provide training to
adjudicators in a classroom setting at each Regional Office in
accordance with the guidance provided by Central Office staff during
the teleconference based upon Central Office's approved curriculum and
electronic claims file. Such an approach would ensure, to the extent
possible, that training throughout VBA was uniform in approach and
based upon a common experience and guidance from Central Office.
Ultimately, institutionalizing uniform training based upon shared
common experience will foster greater consistency in approach to
resolving veterans claims across Regional Offices, and eliminate ad hoc
approaches which result in disparity of entitlement determinations and
disability evaluations.
Questions 2: VA reinstituted the certification exam in August. What
was the response of employees who took the test? Did AFGE have input on
the new test?
Response: AFGE does not have any direct input on designing or
scoring any certification test promulgated by VBA. AFGE is afforded the
opportunity to designate a representative to witness the design of
certification tests and scoring workshops in accordance with a
provision of the Master Agreement between VA and the Union. AFGE
representatives to certification testing activities can informally
express concerns to VBA management during design and scoring
activities. However, in the absence of any concession by VBA management
of the validity of a concern raised by AFGE at a certification design
or scoring workshop, AFGE has no effective means of directly
influencing the certification process prior to its implementation. In
the absence of an immediate concession by VBA management to a point
raised by an AFGE representative in a certification design or scoring
activity, any information gathered by AFGE representatives through
their participation in the certification test design or scoring
activities can only be utilized to effect certification testing through
formal labor-management proceedings. AFGE members do serve as subject
matter experts who write the questions for certification tests, and
help determine essential knowledge and skills of positions that need to
be tested for successful certification. However, AFGE does not attempt
to influence its membership with regard to their duties as subject
matter experts, and does not actively encourage subject matter experts
to report to the Union concerning policies or directions expressed by
VBA management at any certification testing design or scoring activity.
Our experience is that employees who pass certification testing
express a general dissatisfaction regarding the relevance of the
material tested. Employees who do not pass certification testing
express similar dissatisfaction, and also express frustration that they
are not provided with any meaningful feedback regarding where they need
to improve their understanding in order to be certified. That
frustration is magnified when our members consider that they can be
otherwise fully successful in all aspects of their position, but
subject to adverse action or arrested career development solely because
if their inability to pass a certification test.
Regardless of their success in certification testing, frustration
arises among AFGE members when they contemplate that supervisors who
evaluate their job performance on a daily basis do not presently have
any similar certification requirement. The certification process, at
present, leaves our members with the curious potential to be evaluated
by supervisors who are not certified to have the same level of
knowledge and skill level as they do. The impact of that potential has
an obvious detrimental impact on the morale of our members.
Question 3: According to DAV testimony, the Union has objected to
the frequency and other requirements DAV suggests for testing at each
phase. What are the AFGE objections?
Response: Some of our concerns are expressed in the last two
paragraphs of our response to the preceding question.
While AFGE has no objection in principle to the concept of
certification testing, AFGE does have concerns about the practical
application of the certification process and the effects that flawed
certification testing can have on our members. As currently
constituted, certification testing is not designed to provide VBA
employees with any specific feedback concerning what knowledge or
skills they need to improve. This is a disservice both to employees who
have been successfully certified and want to improve their job
performance, and to employees who are attempting to be successfully
certified and want to improve their job performance. As currently
constituted, the certification testing program is not a useful learning
tool. Rather, it more akin to a hazing ritual which somewhat
arbitrarily determines whether an employee can continue in their
current position or ascend the career ladder. Some of our older members
with long histories of fully successful performance evaluations who
have many years of experience successfully serving veterans have found
it difficult to successfully complete their certification requirement
because of the nature and format of the test. In contrast, less
experienced members with recent experience in environments that
regularly utilize standardized testing (e.g. recent college graduates)
may perform adequately on certification testing, but not be as astute
in serving veterans due to their simple lack of experience. While VBA
management has recently expressed an interest in replacing older
employees with younger, more technologically savvy employees, AFGE does
not believe that interest is legitimate. Nor can AFGE, in service to
its full membership, accept that the design of certification tests
should favor an employee with the ability to access reference materials
quickly over an employee who understands the correct course of action
due to long experience. AFGE does not believe that any employee who can
and does successfully perform the duties of their position on a daily
basis should be subject to any adverse employment action solely because
he or she cannot successfully complete a single certification test. Nor
does AFGE believe the measure of an employee's timely, successful,
respectful and full service to veterans can be made by reference to a
single test.
Question 4: In your statement you suggest that the productivity
requirements are too high. On average, an experienced rater is
completing 2-3 claims a day. Are you suggesting that's too many? How
many cases a day should a rater be able to adjudicate?
Response: Rating Veterans Service Representative may, in fact
complete only 2-3 claims per day on average. If raters were only
required to complete 2 to 3 cases per day on average, no one could
object since that requirement would be based on an empirically verified
direct relationship between what can be accomplished and what is
required.
Unfortunately, VBA's National Performance Standard for Rating
Veterans Service Representatives requires that Journey-level RVSRs
(i.e. those with two or more years of experience) complete a minimum of
3\1/2\ weighted cases per day. This misfortune is amplified when one
considers that the 3\1/2\ weighted case per day requirement is a floor
and not a ceiling, and that individual Regional Offices are free to set
daily performance standards for RVSRs on an ad hoc basis which exceed
3\1/2\ weighted cases per day. So, for example, Tiger Team RVSRs are
required to complete 4 weighted cases per day. This misfortune becomes
a tragedy, both for claims adjudicators and for veterans, when RVSRs
who are afforded the ``privilege'' of participating in their negotiated
Flexiplace agreement and work from their homes are required to produce
even more cases in exchange for that ``privilege.'' This Flexiplace
tariff is usually an additional completed case per day. Hence, the
aforementioned Tiger Team employee would have to complete 5 cases per
day in days they work from home in order to maintain successful
performance and their eligibility to participate in the Flexiplace
program.
The misfortunes and tragedies described in the preceding paragraph
become absurd when one contemplates how the floor of 3\1/2\ cases per
day could be justified, let alone any upward deviation from that
productivity requirement. Regarding increased Flexiplace productivity
requirements, any such requirement is either tantamount to an admission
that daily supervision in the workplace is obtrusive and
counterproductive or, that RVSRs are required to work uncompensated
hours in addition to their scheduled tour of duty in order to fulfill
the requirements of their local Flexiplace agreements.
With regard to the floor requirement of 3\1/2\ cases per day, AFGE
believes that no valid empirical evidence has ever been collected to
suggest such a requirement is attainable with any acceptable level of
accuracy. A time-study analysis was referenced by a representative of
VBA management at the September 18, 2008, Hearing and a request was
made to provide a copy of that study to the Subcommittee. If the report
of any time-study analysis has been provided to the Subcommittee, AFGE
requests that we be provided with a copy of the document and an
opportunity to comment on it. With respect to how many cases a day an
RVSR should be able to adjudicate, two points are salient. First, no
one can dispute that an RVSR ought, or should be required, to
accurately resolve as many cases on any given day as he or she can. No
specific number can be assigned as a productivity requirement on any
given day given the disparate complexity of the claims inventory. Any
average daily productivity requirement must be calculated based upon
empirical evidence concerning what trained RVSRs actually produce with
an acceptable degree of accuracy over a 1 year period in order to be
valid.
Question 5: If training is not being offered and is not available
online when it can be completed at anytime employees should have a
mechanism to notify VA Central Office that such opportunity does not
exist at their RO. How can employees notify managers and senior leaders
that the training is not being provided or made available?
Response: Such a mechanism does exist for newly hired or promoted
employees who have recently completed the centralized portion of their
Challenge Training. Those employees are provided with access to a
website where they can provide feedback to Central Office's training
staff regarding the continuing progress of their Challenge Training
when they return to their Regional Offices. Such a mechanism could work
if all VBA training was centralized and tracked through Central Office
as has been suggested. The problem at present is there is no
accountability for Regional Office management who fail to ensure that
even the Challenge Training curriculum is implemented after completion
of the centralized portion of Challenge instruction. Any such
requirement would have to be made part of the Regional Office
Director's performance plan, and enforced on RO Directors as a
performance requirement by the Office of Field Operations. In the
absence of any such requirement, the temptation is for Regional Office
management to divert employee resources from completing their Central
Office approved training program to helping to achieve goals that are
measurements of Regional Office management's performance (e.g.
inventory reduction). The end result of this dysfunction is that newly
hired or promoted employees never complete a uniform course of
training, and their approach to claims processing activities is
determined by the experiences they receive at their particular Regional
Office as opposed to a common, centrally approved experience. The
ultimate detrimental consequent is that consistency and equality of
adjudication and evaluation of veterans' claims is not achieved.
As the situation now stands, employees can only notify their
supervisors that their training has been lacking when they are put on
notice that they are being subjected to a performance improvement plan
or some adverse employment action. At that point, the employee should
be given a meaningful opportunity to participate in identifying where
their weaknesses are and how those weaknesses can be addressed through
training. Unfortunately, the weakness most often identified by
management in such circumstances is a lack of productivity, and the
only means to increase productivity in the absence of a sound basic
understanding of the laws and regulations governing the administration
of VA benefits is through personal experience and effective training.
Employees do not strive to produce erroneous decisions or to produce
fewer decisions than they can. If an employee is making errors or
underproducing, it should be incumbent on the party that identifies
those traits to provide an effective remedial course of instruction,
tailored to the particular deficiencies identified.
Question 6: What would you recommend VA do to improve its training
techniques for newer employees?
Response: A good first step would be to mandate that all Challenge
Training requirements are fulfilled within a set period, and to
prohibit the attention of employees in Challenge Training from being
diverted to other tasks. In addition, it would be beneficial if newer
employees were assigned an individual mentor during their training and
for a period after they have completed their formal training. To the
extent possible, the mentor-trainee relationship should remain intact
so that trainees would not have to constantly adjust their approach to
the their work to conform with the idiosyncrasies of multiple mentors.
It would also be wise to segregate, to the extent possible,
centralized Challenge Training participants into classes composed of
trainees who have promoted internally (and thus have some knowledge and
experience with the VBA claims process) and those who are hired from
outside the VBA (and who require some remedial instruction regarding
the mundane aspects of claims processing--e.g. how a claim is routed
from the triage activity, to pre-development, to the rating board).
That action would allow instructors to devote greater attention to the
relative strengths and weakness of their classes, without running the
risk of being repetitious or boring some participants.
Also, after completion of Challenge Training, ongoing or remedial
training must be presented through a single and authoritative voice. As
I have suggested elsewhere, that voice should emanate from Central
Office, through instructors accountable to Central Office who are
assigned to each Regional Office.
Question 7: What about experienced raters, would you require them
to take the same 80 hours of training if their performance ratings were
``outstanding'' or ``excellent''?
Response: The protean nature of the legal requirements imposed on
the VA claims process mandates that training be continuous in nature.
Regardless of how well any RVSR is performing in any given period,
without a constant influx of relevant information in the form of
training performance will eventually suffer. Moreover, given the real
and perceived needs for increased productivity RVSRs are not
particularly well-situated to take time out of their day and study an
area where they have some confusion. More importantly, without relevant
guidance as to proper procedures and accurate information regarding
VBA's policies, RVSRs are not capable of identifying any error in their
approach to their duties. So, it is vital that RVSRs continue to devote
a discrete portion of their work year toward training. Having said
that, the 80 hour requirement may be somewhat excessive. Attorneys and
medical doctors in most states do not have that onerous a continuing
education requirement, and handle issues which are every bit as complex
and important as those in VBA's jurisdiction. It may be fruitful,
therefore, to reduce the time VBA spends on continuing training by
substituting relevant training developed by reference to common
adjudicatory errors for ``refresher'' training pertaining to the
general rating process. Once again, if such specific training is
centrally developed and administered, it should also foster consistency
in decision-making across Regional Offices.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 22, 2008
Kerry Baker
Associate National Legislative Director
Disabled American Veteran
807 Maine Avenue, SW
Washington, DC 20024
Dear Mr. Baker:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Post-Hearing Questions for Kerry Baker,
Assistant National Legislative Director of the Disabled American
Veterans
From the Committee on Veterans' Affairs,
Subcommittee on Disability Assistance and Memorial Affairs,
United State House of Representatives
September 18, 2008
Question 1: At the hearing we discussed the training requirements,
what do you think of the substance of the training?
Response: Although the Disabled American Veterans would like to
offer an opinion as the quality and substance of the VA's training
program, we are unable to do so as the VA continues not to provide
either this organization or any other Veterans Service Organization the
opportunity to review their training program.
Question 2: What do you think of VBA's STAR program? Is it
effective?
Response: The VA's quality assurance tool for compensation and
pension claims is the Systematic Technical Accuracy Review (STAR)
program. The DAV recommended in its testimony of September 18, 2008,
that Congress require the Secretary to report how the Department could
establish a quality assurance and accountability program that will
detect, track, and holds responsible those VA employees who commit
egregious errors. Such a report should be generated in consultation
with veterans' service organizations most experienced in the VBA claims
process.
Under the STAR program, the VA reviews a sampling of decisions from
regional offices and uses that sampling to base its national accuracy
measures regarding decisions affecting entitlements, benefit amounts,
and effective dates.
Unfortunately, there still exists a gap in quality assurance for
purposes of individual accountability in quality decision-making due to
the current small sampling size the VA chooses to use. Specifically, in
the STAR program, a sample is drawn each month from a regional office
workload divided between ratings, authorizations, and fiduciary end-
products. A monthly sample of ``rating''-related cases generally
requires a STAR review of 10 rating-related end products. Reviewing 10
rating-related cases per month for an average size regional office that
could easily employee more than three times that number of raters, is
undeniable evidence of a total void and lack of commitment in
individual accountability.
If an average size regional office produced only 1,000 decisions
per month, which we feel is quite conservative, the STAR program would
only review 1 percent of the total cases decided by that regional
office. Those figures leave no room for trend analysis, much less
personal accountability.
To put this in better perspective, according to VA's 2007
performance and accountability report, the STAR program reviewed 11,056
compensation and pension (C&P) cases in 2006 for improper payments
whereas, the total number of C&P cases actually available for review
was 1,540,211. This equals a percentage of cases reviewed of
approximately seven tenths of 1 percent, or 0.72 percent.
In closing, we find the STAR program does not fare well as an
effective management oversight tool for the VA. It offers only minute
sampling of cases compared to actual through-put and it has no
accountability mechanism. Effective accountability can be engineered in
a manner that holds each VBA employee responsible for his/her work as a
claim moves through the system while at the same time holding all
employees responsible simultaneously. As errors are discovered, the
responsible employees must be held accountable by forfeiture of work
credit percentage. One employee would be far less likely to cover for
errors or look the other way from errors committed by a fellow employee
if they knew their performance standards were equally at risk. This
type of system would ensure personal accountability at every stage in
the claims process, an essential element if STAR is to ever become an
effective management oversight system.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 25, 2008
Ronald B. Abrams
Joint Executive Director
National Veterans Legal Services Program
1600 K Street, NW, Suite 500
Washington, DC 20006-2833
Dear Mr. Abrams:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Response From Ronald Abrams for
Question From the House Committee on Veterans' Affairs,
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability''
September 18, 2008
Question 1: When NVLSP conducts its Quality Reviews along with The
American Legion, does it consistently look at the quality of training
as well. If so, what is your feedback on the training and what
suggestions have you made over the years.
Response: The answer is yes.
NVLSP attorneys are part of The American Legion (Legion) team that
conducts quality reviews at VA regional offices (ROs). Over the past
few years, NVLSP attorneys working with Legion Quality Review Teams
visited more than 40 VA regional offices for the purpose of assessing
overall operation. In general the quality reviews conducted by the
Legion/NVLSP team reveal that RO training suffers because at too many
regional offices there are too few experienced supervisors that could
provide trainee adjudicators and senior adjudicators proper mentoring
and quality assurance. Also we learned that many ROs postponed or
suspended training so that maximum effort could be expended on
production.
The Legion/NVLSP teams found that there is a general inconsistency
in how regional office employees are trained and how training is
implemented. For example, some stations have regular formalized or
structured training programs, while others have training programs that
are best described as more informal and sporadic. Some stations have
well established and structured training for new employees, but ongoing
training for experienced staff is very limited. These finding are
similar to a 2005 OIG study.
The Legion, in past testimony, recommended that VA reduce its
reliance on locally developed training materials and on-the-job
training by senior raters. The Legion suggested that the ROs develop
training packages based on errors noted the national STAR report and
from patterns of errors found by the Board of Veterans' Appeals.
Allowing the many regional offices to heavily rely on locally developed
training initiatives increases the likelihood of balkanization and
great variance in rating decisions. Unless regional offices (both
managers and individual adjudicators) learn from their mistakes and
take corrective action, there will continue to be a high rate of
improperly adjudicated claims, resulting in a consistently high appeals
rate and subsequent high BVA remand/reversal rate of regional office
decisions.
Below is a sample of our comments about training based on the
Legion/NVLSP quality reviews.
HOUSTON VA REGIONAL OFFICE
Interviews with RO employees identified several issues that may
adversely affect morale, and the quality of RO adjudications. . . .
Training is conducted on a weekly basis but several RO employees rated
the training as poor. They felt that the trainers were not qualified to
give the training, or not enthused about giving the training. A common
complaint was that most trainers merely read off overhead projectors or
from a manual. Some staff suggested that a ``Train the Trainer''
program would be beneficial.
BUFFALO VA REGIONAL OFFICE
Training is being completed on a regular basis, approximately two
to three times per month. The Team found the RO training plan to be
satisfactory but employees are often encouraged not to spend time on
training due to workload concerns.
RENO VA REGIONAL OFFICE
The station is in compliance with the 80 hours of required training
mandated by VA Central Office. Both formal and informal training
sessions are also conducted on a regular basis as well as continual
``on-the job-training.''
MUSKOGEE VA REGIONAL OFFICE
The station's training program is divided into two main categories:
initial training and continuous training. The bulk of training is for
new hires. After completing pre-requisite training, new hires are sent
to ``Challenge Training'' in nearby training hubs such as Milwaukee or
Chicago for 3 weeks. After completing ``Challenge Training,'' new hires
continue their education through web-based training modules and on-the-
job training. Continuous training is made up of both computer-based
web-based training and formal classroom training, which is often led by
the training coordinator, the Service Center Manager, Decision Review
Officers and rating specialists. This training is usually conducted
once or twice a month to meet the Central Office standard of 80 hours
of training a year.
PHOENIX VA REGIONAL OFFICE
In addition to a training program for new employees, there is also
ongoing training in place for experienced personnel. The DRO compiles,
for training purposes, detailed information on trends and reversals on
appealed cases. Also, when appealed decisions are reversed by the DRO,
the individual responsible for the original decision is informed of the
action taken by the DRO and provided specific reasons for such action.
This allows management to maintain accountability, identify problem
areas, direct training accordingly and, as a result, improve the
overall quality of decisions.
Prior to the establishment of the Appeals Management Center (AMC),
the DRO also prepared a complete list, used in conjunction with
training, summarizing reasons for remands broken down by individual
cases. Since the establishment of the AMC, the station has not been
able to compile detailed information on individual remands nor identify
trends for remands sent to the AMC. Although the appeals team receives
summaries of reasons for remand, they are only by region and there is
nothing specific for the individual stations. Not being able to
identify reasons for remand by individual station will adversely impact
the station's ability to identify remand trends and take corrective
action. Additionally, there is no mechanism in place at the station for
tracking BVA allowances. This not only makes it more difficult to
respond to or prevent delays in payment authorization, as previously
discussed, it negatively impacts training and, in turn, the quality of
decisions. By not tracking reasons for BVA allowances, which are
reversals of RO denials, the station is missing an important
opportunity to identify trends in erroneous denials and take necessary
action to avoid such errors in the future.
PITTSBURGH VA REGIONAL OFFICE
Training for DROs, RVSRs, and VSRs is conducted on a regular basis.
Training sessions are video taped for staff who couldn't attend and for
those that work at home. Station work performance standards do not
exceed standards established by VA Central Office. . . .
The station does not have a mechanism in place for tracking BVA
allowances. This negatively impacts training and, in turn, quality of
decisions. By not tracking reasons for BVA allowances, which are
reversals of RO denials, the station is missing an important
opportunity to identify trends in erroneous denials and take necessary
action to avoid such errors in the future. Additionally, as BVA
remands, in most instances, are being sent to the Appeals Management
Center (AMC) for development, the station is denied the opportunity to
review and track individual remands. Although the appeals team receives
summaries of reasons for remand by region, there is nothing specific
for individual stations. Not being able to identify reasons for remand
by individual station will adversely impact the station's ability to
identify remand trends, include in training and take corrective action.
ST. PETERSBURG VA REGIONAL OFFICE
. . . According to station management, current staffing is now
largely made up of individuals with limited training and limited
experience. We were informed that once basic training is completed, the
only training conducted is that which is specifically mandated by VBA.
There is no ongoing training program. With the current emphasis on
production quotas and reducing the claims backlog, management is
unwilling to take time away from production for training.
It was noted that there were too few managers/supervisors and many
of them lacked management training. It was also noted that some
inexperienced trainers were assigned to train new employees.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 22, 2008
Dr. Patricia Keenan
Program Manager
Human Resources Research Organization
66 Canal Center Plaza, Suite 400
Alexandria, VA 22314-1591
Washington, DC 20548
Dear Ms. Keenan:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Response to Questions From the House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability''
Patricia A Keenan, HumRRO
Question 1: You mentioned the focus groups that you conducted with
raters and noted that they develop their own ``individual rules.'' Can
you tell me more about this practice? Do you have examples of what
raters are doing when the medical evidence is ambiguous?
Response: I would be reluctant to characterize this as a
``practice,'' which makes the activity sound institutionalized. I see
it as a coping mechanism that helps individuals reduce the stress
associated with working in a situation that involves a high level of
decision-making under uncertainty. It probably contributes to the
variance in rating decisions as raters develop their own heuristics.
My response is based on what we heard in focus groups and what I
have learned about the RVSR job over the last few years as we have
developed the RVSR and journey-level RVSR skills certification tests.
We heard consistently that it takes several years for RVSRs to
become comfortable making rating decisions. There are several reasons
for this. These are very high stakes decisions, with a large impact on
the lives of veterans and their families. Raters would understandably
be very wary about making an error that could hurt these people. Many
inexperienced raters are afraid of making a mistake that results in the
case being appealed. Experienced raters have learned that everyone is
going to have a case appealed sooner or later, that there are a variety
of reasons for this happening (e.g., new evidence, error, differing
interpretation of the regulation), and it should not prevent a
decision. A third reason is that the criteria in some parts of the
Rating Schedule are not sufficiently detailed to allow the rater to
feel confident that an evaluation is exactly correct. For example, when
rating a complaint for a disorder of the ulnar nerve, the schedule
provides a detailed description of complete paralysis; however, no
description is given for lesser evaluations characterized as severe,
moderate or mild.
The first two problems are overcome through experience; the third
is more problematic. Currently, when there is no clear guidance in the
Rating Schedule, RVSRs either try to reason the problem through on
their own or solicit help from their peers, coaches or Decision Review
Officers (who serve as RVSR trainers). Reasoning through on their own
is mostly likely where developing individual decision rules is rooted.
Trainee RVSRs have a mentor (a journey-level RVSR) with whom they
discuss the evidence and compare it to the available guidance (e.g.,
the Rating Schedule, training materials, FAST letters). Discussing the
claim with others allows the RVSR to take advantage of prior
experiences of colleagues as well as their knowledge of the Rating
Schedule. Some Service Centers (e.g., San Diego) have a medical
professional (retired doctor or nurse) on site to help RVSRs with this
type of situation. These people are familiar with the rating process
and the Rating Schedule and can help the rater interpret the medical
information and match it to the appropriate section of the Rating
Schedule. Seeking input from others probably means that these
individual decision rules become shared within an office.
While the process is very understandable, it is important to
recognize that this is not solely a training problem. The deeper roots
are in the lack of detailed criteria in some areas of the Rating
Schedule and the time pressure raters work under in trying to keep up
with the ever increasing number of claims.
Question 2: The practice of using analogous codes has increasingly
permeated VA's rating process over the years and as you observed
increases subjectivity and variance. This seems very unfair to the
veteran. What would you suggest VA do instead of using these codes and
how could they better train employees to reduce this subjectivity?
Response: The use of analogous codes is not unfair, in and of
itself. There are legitimate reasons for using them. The first is that
maintenance of the Rating Schedule has not kept pace with medical
discoveries and advancements. Many of these conditions had not been
recognized at the time the Schedule was developed, so they were not
included. The result is that many relatively common conditions (e.g.,
Cohn's disease, erectile dysfunction, insomnia, shin splints) are not
currently included in the Rating Schedule. As the Schedule is updated,
the conditions will likely be added to the appropriate section(s).
Updating the Schedule, however, is likely to take years due to the
enormity of the effort. It might be more expedient for VA to conduct a
study to analyze the use of analogous codes and identify impairments
that occur often enough to deserve their own code or for which the
criteria in existing codes are not adequate.
Conducting such a study is not a simple proposition as the data are
not necessarily easy to interpret. For example, raters may use at least
two different diagnostic codes to rate Cohn's disease. This use of
multiple diagnostic codes makes it difficult to track the number of
Cohn's disease claims which, in turn, makes it difficult to recognize
when the number of cases would justify establishment of a separate
diagnostic code.
A second reason for using analogous codes is that there are some
symptoms that are not easily diagnosed. Indeed, the ``Persian Gulf War
Veterans' Benefits Act '' recognizes that many Gulf War (GW) veterans
suffer from chronic disabilities (e.g., fatigue, headache) resulting
from an undiagnosed illnesses. The Code of Federal Regulations (38 CFR,
Part II, Chapter 11, Subchapter 11, Sec. 1117) specifies the use of
analogous codes for evaluating these undiagnosed illnesses. Other
times, a medical opinion does not specify a particular condition. The
diagnosis may say that the symptoms are ``like'' a recognized,
diagnosable condition, but that condition is not exactly applicable to
the symptoms. In these cases, the rater should use a code that is
analogous to the recognized condition. It is likely that this
circumstance will never disappear, so analogous codes will always have
a place in the rating process.
Some studies have suggested using the same diagnostic categories
(International Classification of Diseases (ICD) and Diagnostic and
Statistical Manual for Mental Disorders (DSM)) used by VHA and other
health-care providers. While there are some advantages to doing so, to
do a wholesale change would cause a major upheaval in the rating
process. It would take a lot of time to develop new training materials/
classes and job aids. Conducting training would require that RVSRs take
a good deal of time away from their job to learn the new material. It
might be more feasible to add parts of the ICD to the existing Rating
Schedule, making the transition over time.
For both situations, C&P Services could update the Rating Schedule
and/or develop training and job aids for RVSRs to learn the new
diagnostic codes. The problem of working with analogous codes properly
is not entirely a training matter, although that is certainly one
avenue to pursue. C&P Service could review existing training materials
to make sure they provide adequate guidance for raters. It might be
informative to gather input from raters as to the type of information
they think would be most useful to them. C&P could also develop
additional guidelines for identifying analogous codes. For example,
raters might find it useful to take the time (or feel they had the
time) to do a quick search of the Internet to give them information
they might use to find the appropriate diagnostic code.
Question 3: You outlined 3 recommendations in your testimony on
September 18, 2008. Have you previously presented these ideas to VA and
what was their response?
Response: No, I had not discussed these ideas with VA. As a
contractor working on the skills certification program, it was not my
role to advise VA on areas outside the scope of that work.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 22, 2008
Michael Walcoff
Deputy Under Secretary for Benefits
Veterans Benefits Administration
U.S. Department of Veterans Affairs
810 Vermont Avenue, NW
Washington, DC 20420
Dear Mr. Walcoff:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Questions for the Record
The Honorable John J. Hall, Chairman,
Subcommittee on Disability Assistance and Memorial Affairs,
House Committee on Veterans' Affairs
September 18, 2008
Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability
Question 1: What does VBA do about tying training to individual
employee performance?
Response: Supervisors are required to conduct oversight of employee
training plans and are held responsible for ensuring that employees
meet their training requirements. Employee training is tracked through
the learning management system (LMS). Through LMS, employees complete a
detailed 80-hour training plan each year, which contains curricula
deemed appropriate to achieve successful performance in their position.
In addition to the required course work, employees can add courses to
their learning plan. In fiscal 2008, the Veterans Benefits
Administration (VBA) hired training managers to provide additional
oversight of the scheduling, and completion of training.
In the event an employee's performance is not meeting the fully
successful level of achievement for a critical element, the employee
will be placed on a performance improvement plan (PIP). The PIP
identifies the employee's specific performance deficiencies; the
action(s) that must be taken by the employee to improve to the
successful level of performance; the methods that will be employed to
measure the improvement; and assistance to be provided such as
counseling, or focused individualized training.
Question 2: In your testimony you noted that VBA contracted with an
independent consultant to evaluate the Challenge Training. Although, we
discussed this briefly, can the Subcommittee staff get a copy of those
results?
Response: As part of its national level training evaluation
program, VBA contracted for an evaluation of the challenge program. The
evaluation sought to answer two questions:
Does challenge training, as implemented, provide value to
VBA?
What are the opportunities for improving challenge
training?
The report indicates that much value has been realized by VBA's
efforts to provide quality, timely training to its new claims
processing employees. It also noted areas where VBA could improve the
overall management and curriculum of challenge. Efforts are already
underway to address the recommendations. A copy of the report is
attached.
[The report is being retained in the Committee files.]
Question 3: How does the information gathered from the STAR review
get integrated with the information from the Compensation and Pension
Evaluation Program (CPEP) reviews regarding the quality of the exams
conducted in cooperation with the Veterans Health Administration?
Response: There is no mechanism for integration of systematic
technical accuracy review (STAR) and CPEP reviews. However, the STAR
questions and the CPEP key indicator questions are similar and
essentially cover the appropriateness of the exam request. CPEP reviews
are focused on the quality of the examination and STAR reviews are
based upon the accuracy of the resulting decisions from those
examinations.
Question 4: At the hearing, the Chairman asked about the time in
motion study, can you please share those results with the Subcommittee
as well?
Response: VBA contracted with SRA International, Inc., to conduct a
work measurement study in March 2007 at 15 regional offices. VBA's
pension maintenance centers, benefits delivery at discharge rating
activity sites, and foreign claims processing offices were among the
regional offices selected to participate in the study. The electronic
work measurement application final work rate standards report,
submitted by SRA International, is attached.
[The report is being retained in the Committee files.]
The work rates standards provided in SRA's final report contained
some unexpected results, such as longer processing times for original
disability compensation claims with 7 issues or less than for claims
with 8 issues or more, which prompted a thorough review of the
methodology used in the study. Critical flaws were uncovered that
produced inaccurate results. The primary errors included:
Some completed claims were double counted, resulting in
artificially low work-rate standards.
Brokered work (those claims completed by select regional
offices for other regional offices) was not included in SRA's
computations. Not including brokered work inflated the work-rate
standard at the regional office completing the work and lowered the
work-rate standard at the office of original jurisdiction. It also
influenced the overall work-rate standard since only a fraction of
regional offices were included in the study.
Several formula errors were uncovered in the work-rate
standard spreadsheet compiled by SRA that affected the outcome of the
study. The errors causing the greatest impact on the work-rate standard
calculations were observation count computations.
Revisions were made by VBA to the work-rate standards spreadsheet
in an attempt to remedy the above issues. The resulting data was then
re-analyzed and compared to prior work-measurement study findings. The
changes from prior findings were substantial and found to be associated
with the factors below:
Inconsistent adherence to guidance among participating
regional offices on clearing appeals-related claims. A reliable number
of completed appeals-related claims were unavailable; therefore, no
valid conclusion could be drawn from the work-measurement results.
Lack of record of the types of brokered claims completed.
Without a record of the exact types of brokered claims completed,
completed claims counts could not be properly credited or debited,
leading to results not reflective of true workloads.
Pension-related claim results were questionable. Prior
studies ran for 4 months to account for the cyclical nature of the
pension workload. The 2007 study was conducted for only 1 month,
providing a narrower view of the pension workload cycle.
The sample size for several claim types was too small or
nonexistent. Because of the small to nonexistent sample size, the
results were unreliable. This again was due to the fact that the study
only ran for 30 days.
The above issues contributed to a wide variation of results between
participating regional offices. This disparity led to the conclusion
that the results could not be representative of the time to process
claims. It was therefore decided to not use the results of the work
measurement study.
The recently mandated study of the work credit system and work
management system will measure and manage the work production of VBA
employees who handle claims for compensation and pension benefits and
will also evaluate more effective means of improving performance.
Lessons learned from the 2007 study will be used to ensure a valid
methodology.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 22, 2008
Bradley Mayes
Director
Compensation and Pension Service
Veterans Benefits Administration
U.S. Department of Veterans Affairs
810 Vermont Avenue, NW
Washington, DC 20420
Dear Mr. Mayes:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Questions for the Record
The Honorable John J. Hall, Chairman,
Subcommittee on Disability Assistance and Memorial Affairs,
House Committee on Veterans' Affairs
September 18, 2008
Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability
Question 1: When the ROs report their 80 hours of training, do you
know what materials they trained on or just that they completed the
required hours?
Response: Employees are required to complete 60 hours of core
technical training. The topics for the core training are identified by
Compensation and Pension (C&P) service. The remaining 20 hours are
determined by the Regional Office (RO) based on local needs. The topics
and numbers of hours each employee completes are recorded in the
learning management system.
Question 2: What level of supervision does C&P Service provide to
the RO regarding training?
Response: C&P service reviews the training accomplishments of each
RO at the end of the fiscal year to ensure training requirements were
met. Feedback from the reviews is provided to the regional offices.
Committee on Veterans' Affairs
Subcommittee on Disability Assistance
and Memorial Affairs
Washington, DC.
September 25, 2008
Dorothy Mackay
Director
Employee Development and Training
Veterans Benefits Administration
U.S. Department of Veterans Affairs
810 Vermont Avenue, NW
Washington, DC 20420
Dear Ms. Mackay:
In reference to our House Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs Hearing on
``Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability'' on September
18, 2008, I would appreciate it if you could answer the enclosed
hearing questions as soon as possible.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Sincerely,
John J. Hall
Chairman
__________
Questions for the Record
The Honorable John J. Hall, Chairman,
Subcommittee on Disability Assistance and Memorial Affairs,
House Committee on Veterans' Affairs
September 18, 2008
Examining the Effectiveness of Veterans Benefits Administration
Training and Performance Management and Accountability
Question 1: When RO's develop their training strategic plan, how
are the gaps identified and incorporated into future training?
Response: In fiscal 2008, the Veterans Benefit Administration (VBA)
created a training manager position at every regional office (RO) to
improve the ability to identify gaps and plan training. The training
manager is responsible for local training reviews, analyzing
performance indicators to determine local training needs, and
implementing the training necessary to meet those needs. For example,
local accuracy reviews are conducted by the Compensation and Pension
(C&P) service's systematic technical accuracy review (STAR) staff on
claims decisions using national quality criteria. The results of the
reviews are available to the RO training manager for trend analysis and
formulation of remedial training as necessary.
Question 1(a): How does joint data collection and root cause
analyses with the VBA on cases that have been remanded inform training?
Response: The C&P program review staff conducts monthly remand
reviews of all pre-certification Board of Veterans Appeals (BVA)
remands. The review results are available to all ROs on the C&P
Intranet and are used to provide timely feedback on pre-certification
remands in support of efforts to reduce avoidable remands. The results
are also used for training purposes. As an example, BVA travel board
attorneys use the results when they provide training at the regional
offices.
Recently, analysis was conducted that compared the primary STAR
error for Veterans Claims Assistance Act (VCAA) compliant development
to the corresponding pre-certification error categories on the remand
reasons report. The results are under review.
Detailed remand reason reports are run quarterly and are also
posted on the C&P Intranet Web site by RO, area, and nationwide.
Question 2: The Instructors Training manual is five volumes. That
doesn't seem like an effective tool. Have you consulted with private
sector organizations that specialize in developing training protocols
for Federal and private sector agencies?
Response: The instructors' training manual includes training
materials for the three challenge training classes, each given over the
course of 3 weeks. The three challenge curriculums include rating
veterans service representative (RVSR), pre-determination veterans
service representative (VSR) and post-determination VSR. The manual is
split into five volumes, two for RVSR training, one for pre-
determination VSR training, and two for post-determination VSR
training. Each volume includes instructor and student guides,
presentations, and reference materials, including copies of fast
letters, court decisions, flow charts, and sample letters.
VBA is currently exploring ways to reduce the size of the training
manuals as the curriculum is revised. All training materials are
reviewed by internal certified instructional design specialists. VBA
also works with a contractor, Camber Corporation, on the materials.