[House Hearing, 111 Congress]
[From the U.S. Government Publishing Office]
EXAMINATION OF THE U.S. DEPARTMENT OF
VETERANS AFFAIRS REGIONAL OFFICE
DISABILITY CLAIMS QUALITY REVIEW METHODS
=======================================================================
HEARING
before the
SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS
of the
COMMITTEE ON VETERANS' AFFAIRS
U.S. HOUSE OF REPRESENTATIVES
ONE HUNDRED ELEVENTH CONGRESS
SECOND SESSION
__________
March 24, 2010
__________
Serial No. 111-68
__________
Printed for the use of the Committee on Veterans' Affairs
----------
U.S. GOVERNMENT PRINTING OFFICE
57-012 PDF WASHINGTON : 2010
For sale by the Superintendent of Documents, U.S. Government Printing
Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800;
DC area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC,
Washington, DC 20402-0001
COMMITTEE ON VETERANS' AFFAIRS
BOB FILNER, California, Chairman
CORRINE BROWN, Florida STEVE BUYER, Indiana, Ranking
VIC SNYDER, Arkansas CLIFF STEARNS, Florida
MICHAEL H. MICHAUD, Maine JERRY MORAN, Kansas
STEPHANIE HERSETH SANDLIN, South HENRY E. BROWN, Jr., South
Dakota Carolina
HARRY E. MITCHELL, Arizona JEFF MILLER, Florida
JOHN J. HALL, New York JOHN BOOZMAN, Arkansas
DEBORAH L. HALVORSON, Illinois BRIAN P. BILBRAY, California
THOMAS S.P. PERRIELLO, Virginia DOUG LAMBORN, Colorado
HARRY TEAGUE, New Mexico GUS M. BILIRAKIS, Florida
CIRO D. RODRIGUEZ, Texas VERN BUCHANAN, Florida
JOE DONNELLY, Indiana DAVID P. ROE, Tennessee
JERRY McNERNEY, California
ZACHARY T. SPACE, Ohio
TIMOTHY J. WALZ, Minnesota
JOHN H. ADLER, New Jersey
ANN KIRKPATRICK, Arizona
GLENN C. NYE, Virginia
Malcom A. Shorter, Staff Director
______
SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS
JOHN J. HALL, New York, Chairman
DEBORAH L. HALVORSON, Illinois DOUG LAMBORN, Colorado, Ranking
JOE DONNELLY, Indiana JEFF MILLER, Florida
CIRO D. RODRIGUEZ, Texas BRIAN P. BILBRAY, California
ANN KIRKPATRICK, Arizona
Pursuant to clause 2(e)(4) of Rule XI of the Rules of the House, public
hearing records of the Committee on Veterans' Affairs are also
published in electronic form. The printed hearing record remains the
official version. Because electronic submissions are used to prepare
both printed and electronic versions of the hearing record, the process
of converting between various electronic formats may introduce
unintentional errors or omissions. Such occurrences are inherent in the
current publication process and should diminish as the process is
further refined.
C O N T E N T S
__________
March 24, 2010
Page
Examination of the U.S. Department of Veterans Affairs Regional
Office Disability Claims Quality Review Methods................ 1
OPENING STATEMENTS
Chairman John J. Hall............................................ 1
Prepared statement of Chairman Hall.......................... 30
Hon. Doug Lamborn, Ranking Republican Member..................... 3
Prepared statement of Congressman Lamborn.................... 31
WITNESSES
U.S. Department of Veterans Affairs:
Belinda J. Finn, Assistant Inspector General for Audits and
Evaluations, Office of Inspector General..................... 4
Prepared statement of Ms. Finn............................. 32
Bradley G. Mayes, Director, Compensation and Pension Service,
Veterans Benefits Administration............................. 24
Prepared statement of Mr. Mayes............................ 51
U.S. Government Accountability Office, Daniel Bertoni,
Director, Education Workforce, and Income Security........... 6
Prepared statement of Mr. Bertoni.......................... 36
______
American Legion, Ian C. de Planque, Assistant Director, Veterans
Affairs and Rehabilitation Commission.......................... 16
Prepared statement of Mr. de Planque......................... 48
American Veterans (AMVETS), Raymond C. Kelley, National
Legislative Director........................................... 15
Prepared statement of Mr. Kelley............................. 48
Disabled American Veterans, John L. Wilson, Assistant National
Legislative Director........................................... 14
Prepared statement of Mr. Wilson............................. 45
National Veterans Legal Services Program, Ronald B. Abrams, Joint
Executive Director............................................. 13
Prepared statement of Mr. Abrams............................. 42
MATERIAL SUBMITTED FOR THE RECORD
Follow-up Letter:
Hon. George J. Opfer, Inspector General, U.S. Department of
Veterans Affairs, to Hon. John Hall, Chairman,
Subcommittee on Disability Assistance and Memorial
Affairs, Committee on Veterans' Affairs, letter dated
April 29, 2010........................................... 55
Post-Hearing Questions and Responses for the Record:
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Daniel Bertoni, Director, Education,
Workforce, and Income Security, U.S. Government
Accountability Office, letter dated April 5, 2010, and
response letter dated May 4, 2010........................ 55
Hon. John J. Hall, Chairman, Subcommittee on Disability
Assistance and Memorial Affairs, Committee on Veterans'
Affairs, to Bradley Mayes, Director, Compensation and
Pension Service, Veterans Benefits Administration, U.S.
Department of Veterans Affairs, letter dated April 5,
2010, and VA responses................................... 57
EXAMINATION OF THE U.S. DEPARTMENT OF
VETERANS AFFAIRS REGIONAL OFFICE
DISABILITY CLAIMS QUALITY REVIEW METHODS
----------
WEDNESDAY, MARCH 24, 2010
U.S. House of Representatives,
Committee on Veterans' Affairs,
Subcommittee on Disability Assistance and Memorial Affairs,
Washington, DC.
The Subcommittee met, pursuant to notice, at 2:28 p.m., in
Room 334, Cannon House Office Building, Hon. John Hall
[Chairman of the Subcommittee] presiding.
Present: Representatives Hall, Donnelly, and Lamborn.
Mr. Hall. Good afternoon, everybody. Thank you for your
patience.
Would we all please rise for the Pledge of Allegiance.
[Pledge of Allegiance.]
OPENING STATEMENT OF CHAIRMAN HALL
Mr. Hall. Welcome to the Subcommittee on Disability
Assistance and Memorial Affairs' hearing entitled, Examination
of the U.S. Department of Veterans Affairs (VA) Regional Office
Disability Claims Quality Review Methods--Is the Veterans
Benefits Administration's (VBA's) Systematic Technical Accuracy
Review or STAR Making the Grade?
We are going to try to make abbreviated opening statements
by myself and Ranking Member Lamborn as we understand votes
will be called any time.
That said, I welcome you all here in what has been a
profoundly historic and important week for the Nation and for
our veterans. Over the last 7 days, the full Committee convened
a successful Summit, which brought many of you and dozens of
other top veteran stakeholders together to aid us in fixing the
VA compensation and pension claims process.
From the Summit came a lot of very useful information which
we welcome and look forward to using to solve the problems that
VA faces in processing disability claims of our Nation's
veterans.
Next in a rare Sunday session, Congress passed and the
President signed a sweeping health care reform package. And I
am pleased that Secretary Shinseki as well as the Chairman of
the full VA Committee and the Armed Services Committee have
signed a letter and sent it to the veterans service
organizations (VSOs) stating unequivocally that TRICARE and VA
care will not be adversely affected by the national health care
reforms.
Also in the past few days, we passed the End Veteran
Homelessness Act of 2010 to provide funding to help Secretary
Shinseki's goal of ending homelessness for America's warriors,
the Help Heroes Keep Their Homes Act, the COLA, the cost of
living increase for veterans, and the National Guard Employment
Protection Act.
This particular hearing will be about the Systematic
Technical Accuracy Review (STAR) technical review system and we
will look at the accuracy of assessing disability compensation
and pension claims rating and the disparity between accuracy in
the different regional offices (ROs).
Using this quality control tool, VBA should be able to
focus attention on poorly-performing ROs and help the agency
direct additional staff and training to the problem offices and
at the same time look at those who are the highest-performing
ROs and find out what they are doing right.
The STAR system was implemented in October of 1998. Since
fiscal year 2007, VBA has set for itself a goal of completing
compensation claims ratings without error 90 percent of the
time.
Its long-term strategy goal is 98 percent. Unfortunately,
we are still far from achieving that goal. And until the STAR
system provides an accurate accounting of the error rate at
VBA, it is difficult to envision a path to meeting this goal.
So we are honored to have today the Office of the Inspector
General (OIG), the U.S. Government Accountability Office (GAO),
both of which have produced studies revealing issues that may
be those impeding the efficiency and consistency of the STAR
system. We are looking forward to hearing their testimony.
I am personally troubled by GAO's finding that the VBA
claims processing accuracy rate is particularly low in cases
pertaining to post-traumatic stress disorder (PTSD) and
traumatic brain injury (TBI). Today we hope to analyze these
studies and hear testimony regarding those issues from OIG and
GAO.
There have been improvements made by the VBA to the claims
rating system and the STAR system. We look forward to hearing
about these improvements and how we, the Subcommittee, the full
Committee, and Congress, can help.
To fully understand STAR, it is important to review the
compensation and pension (C&P) rating system itself. To assess
a claim, the application for disability assistance must be
developed, a process that involves obtaining all necessary
evidence to support the veteran's claim. After development, the
claims go to a rating veterans service representative (RVSR).
The RVSRs determine if that disability is service-connected
and assigns a percentage rating that is intended to represent
the average earning reduction a veteran with that condition
would experience in a civilian occupation. The veteran is then
notified of that decision.
For reopened claims, the assigned diagnostic codes affect
the veteran's overall combined percentage of disability. The
ROs, regional offices personnel applying the appropriate
diagnostic code percentages to determine the combined level of
disability.
Once that claim is completed, it is declared an end product
and the result of that claim is cleared and a work credit is
given to the regional office. So a completed claim and
corresponding cleared end product is then subject to review by
STAR reviewers.
In the 110th Congress, I introduced H.R. 5892, which sought
to improve VBA's quality control measures. This bill was
incorporated into an omnibus veterans' package, which was
signed into law as Public Law 110-389. The Veterans Disability
Benefits Claims Modernization Act of 2008, H.R. 5892, was part
of that. So our hearing should also provide a chance to gauge
how well these quality control measures are working.
I thank you for being here, in advance for your testimony,
and now I would like to recognize Ranking Member Lamborn for
his opening statement.
[The prepared statement of Chairman Hall appears on p. 30.]
OPENING STATEMENT OF HON. DOUG LAMBORN
Mr. Lamborn. Well, thank you, Mr. Chairman.
And thank you for waiting, although I would have been happy
for you to go ahead and start. But I know as a courtesy, you in
a spirit of bipartisanship, you wanted to wait. So I thank you
for that.
And I do apologize. There was a misunderstanding between my
staff and I. I thought that we were going to start after this
first series of votes because I specifically asked that. And
once they found the mistake, they notified me immediately, but
I was two buildings away. So I got here as quickly as I could.
But I know everyone's time here is extremely valuable and
so I apologize. It is totally the fault of me and my office.
Mr. Chairman, I want to, like you, welcome everyone to this
hearing on the Department of Veterans Affairs' STAR Program.
Throughout my tenure on this Committee, my fellow Members and I
have called for stronger accountability within the VA claims
system. For too long, the primary focus has been on production
and this has led to an error rate that is unacceptable.
I believe that the VA's greatest challenge, the claims
backlog, is largely attributable to hasty decisions made
without proper regard for accuracy. The ramifications of this
approach can be seen throughout the entire system.
Therefore, VA employee performance awards cannot be based
entirely on production. There must also be a valid measure of
quality.
Under the STAR Program, a statistically valid sample of
rating decisions from various regional offices is reviewed for
accuracy. While this method may be useful from a macro
perspective, it is not sufficient for ensuring individual
accountability.
VA must be able to identify employees in need of
individualized remedial training. Without this essential
component of the quality assurance process, VA will have
perpetual problems in its claims system.
In the 110th Congress, this Committee passed a provision
that was included in the Veterans Benefits Improvement Act of
2008 that required VA to conduct a study on the effectiveness
of the current employee work credit system.
I believe the upcoming report, along with the testimony
that we will hear today, will provide valuable feedback for the
Department to improve its Quality Assurance and Accountability
Program.
I look forward to hearing from our witnesses today, and I
thank you all for your participation.
And, Mr. Chairman, I thank you also and I yield back.
[The prepared statement of Congressman Lamborn appears on
p. 31.]
Mr. Hall. Thank you, Mr. Lamborn.
We are moving at lightning speed now. We would like to ask
our first panel to join us at the witness table. We have
Belinda J. Finn, Assistant Inspector General for Audits and
Evaluations, Office of Inspector General, U.S. Department of
Veterans Affairs, and Daniel Bertoni, Director of Education,
Workforce, and Income Security with the Government
Accountability Office.
Welcome to you both, and your full written statements are
entered in the record. So we will recognize you for
5 minutes each for however much of that you would like to give
to us directly.
Ms. Finn, welcome, and you are now recognized for 5
minutes.
STATEMENTS OF BELINDA J. FINN, ASSISTANT INSPECTOR GENERAL FOR
AUDITS AND EVALUATIONS, OFFICE OF INSPECTOR GENERAL, U.S.
DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY LARRY
REINKEMEYER, DIRECTOR, KANSAS CITY AUDIT OPERATIONS DIVISION,
OFFICE OF INSPECTOR GENERAL, U.S. DEPARTMENT OF VETERANS
AFFAIRS; AND DANIEL BERTONI, DIRECTOR, EDUCATION, WORKFORCE,
AND INCOME SECURITY, U.S. GOVERNMENT ACCOUNTABILITY OFFICE
STATEMENT OF BELINDA J. FINN
Ms. Finn. Thank you, Chairman Hall.
I am fighting some allergies that sometimes cause coughing
fits, so I am trying to avoid that.
Mr. Hall. You are excused.
Ms. Finn. Thank you for having us here today. I am pleased
to be here to discuss our review of VBA's Quality Assurance
Program and how VBA can improve the programs to better serve
our veterans.
I am joined today by Larry Reinkemeyer, who is the Director
of the Kansas City Audit Operations Division.
The OIG is committed to proactively reviewing the
Department's key internal controls to identify weaknesses
before they escalate into significant problems.
Over the past 2 years, my office has audited three
components of VBA's Quality Assurance Program, rating accuracy,
rating consistency, and VBA's Site Visit Program.
Although our written statement covers our work in all three
areas, my comments today are only addressing our audit of the
Systematic Technical Accuracy Review or STAR Program.
In March 2009, we issued the audit of VBA's compensation
rating accuracy and consistency reviews. We concluded that
VBA's STAR process did not effectively identify and report
errors in compensation claim rating decisions. We projected
that about 77 percent of claims were accurate as opposed to
VBA's reported accuracy rate of 87 percent.
This equated to approximately 88,000 additional claims
where veterans' monthly benefits may be incorrect.
We identified five areas where VBA needed to improve the
STAR Program. VBA agreed with all of our recommendations and
just recently reported that all corrective actions were
complete.
First, the STAR reviewers did not identify some errors
because they either did not thoroughly review available
evidence or they inappropriately misclassified benefit
entitlement errors as comments that did not count against the
accuracy rate.
Second, STAR management required regional offices to report
quarterly on actions taken to correct errors but did not
follow-up to ensure that these offices actually took the
corrective actions on comments.
Third, VBA excluded brokered claims from STAR reviews. The
officials told us that the STAR Program's primary focus was to
assess and report rating accuracy for each of the individual
regional offices. Since two regional offices are involved in
brokered claims, these officials believed it would be difficult
to assign responsibility for rating accuracy on a brokered
claim.
Fourth, the STAR reviewers did not ensure regional offices
submitted all of the selected compensation claim ratings for
review. In fact, the offices did not submit about seven percent
of the requested ratings for the 12-month period ending
February 2008. We reviewed a sample of these unsubmitted claims
and identified a benefit entitlement error rate of
approximately 22 percent.
Last, the STAR reviewers were not required to complete
formal training on an annual basis. In contrast, the regional
office staff that prepare and complete ratings are required to
complete 80 hours of training per year to stay current on laws,
policies, and processes that affect rating claims.
The STAR management relies on the regional offices to take
corrective actions on the issues identified by the STAR team.
Since April 2009, the OIG Benefits Inspection Division has
issued eight reports where we looked at regional office
procedures to ensure the accurate and timely correction of
errors identified by STAR.
Our analysis of 148 errors found that regional office staff
had not corrected 27 percent of these and in some cases had
erroneously reported to STAR that the errors had been
corrected.
In closing, VBA is under tremendous pressure to process
claims and reduce the growing backlog. Without an effective and
reliable Quality Assurance Program, VBA leadership cannot
adequately monitor performance to make necessary program
improvements and ensure veterans receive accurate and
consistent ratings.
Mr. Chairman, thank you again for the opportunity to be
here today. Mr. Reinkemeyer and I would be pleased to answer
any questions you may have.
[The prepared statement of Ms. Finn appears on p. 32.]
Mr. Hall. Thank you, Ms. Finn. You had 4 seconds remaining.
Good job.
Ms. Finn. Thank you.
Mr. Hall. Mr. Bertoni, welcome. You are now recognized for
5 minutes.
STATEMENT OF DANIEL BERTONI
Mr. Bertoni. Mr. Chairman, Members of the Subcommittee,
good afternoon. I am pleased to be here to discuss the
Department of Veterans Affairs' efforts to improve the quality
of disability decisions.
For years, we have noted that VA's claims processing
challenges not only include making quicker decisions in
reducing its claims backlog but also improving accuracy and
consistency.
My statement today focuses on steps VA has taken in
response to recommendations from us and others to enhance its
quality assurance tools, namely the Systematic Technical
Accuracy Review or STAR Program, as well as other programs
designed to address decisional consistency.
Since STAR was first implemented, we have made numerous
recommendations for improvement. For example, very early on, we
noted that STAR reviewers lacked organizational independence
because they also had claims processing duties and reported
directly to regional managers whose claims they may review.
Per our recommendation, VA moved to require
organizationally independent STAR reviewers who are precluded
from making claims decisions.
In subsequent work, we found that STAR sampling was
insufficient to ensure the accuracy of disability pension
decisions.
VA addressed our findings by consolidating pension
processing into three locations and establishing a separate
STAR review for pension claims.
More recently we reported that VA is not using STAR to
separately assess the accuracy of the benefits delivery at
discharge and quick-start claims, alternative processes for
fast tracking VA disability compensation claims for active-duty
servicemembers.
To date, the Agency has opted not to evaluate the extent to
which staff are accurately developing and rating these claims,
although such information could better inform training and
focus program monitoring efforts.
VA's Office of Inspector General has also recommended
changes to the STAR Program which VA has begun to address,
including establishing minimum annual training requirements for
reviewers, mandating an additional supervisory review of STAR
reports, sampling brokered claims for accuracy, and
implementing more stringent procedures for conducting STAR
reviews.
Finally, the Agency has begun to take steps to address
deficiencies that both we and VA's Inspector General have
identified with its consistency review programs, which assess
the extent to which regional offices and individual raters make
consistent decisions on the same claim.
We recommended that VA conduct systematic studies of
impairments identified as having potentially inconsistent
decisions. And in fiscal year 2008, VA did initiate this
effort.
However, last year, the Inspector General reported that VA
had not followed through on its plans to conduct additional
reviews, which was attributed in part to insufficient STAR
staffing resources.
The Agency has since developed a consistency review
strategy and is in the process of conducting fiscal year 2010
reviews. However, these efforts have only recently begun and it
is too early to assess their impact.
And despite various recommendations for improvement and
actions to address them, VA has struggled over the years to
improve the accuracy rate for disability compensation decisions
which was 84 percent in fiscal year 2009 and well short of VA's
stated goal of 90 percent.
VA has attributed its inability to meet accuracy goals in
part to the large numbers of newly-hired personnel conducting
claims development work and their general lack of training and
experience. We have also noted that human capital challenges
associated with providing training to help new staff become
more proficient will likely continue into the near future and
could impact quality.
Thus, it is important that VA continue to improve and
maintain a robust quality assurance framework that not only
supports staff in their understanding of the very complex
business of making timely, accurate, and consistent disability
decisions but also ensures that all veterans receive the
benefits they are legally entitled to.
Over time, VA's newly-implemented quality assurance
initiatives have the potential to improve decisional accuracy
and consistency if compliance with enhanced protocols,
procedures, and standards is sustained.
However, it is imperative that VA remain proactive in its
quality assurance efforts going forward, especially as aging
veterans and more veterans from current conflicts add to VA's
already substantial claims workloads.
And, Mr. Chairman, this does conclude my statement. I am
happy to answer any questions you may have. Thank you.
[The prepared statement of Mr. Bertoni appears on p. 36.]
Mr. Hall. Thank you, Mr. Bertoni.
I am just going to yield to Mr. Lamborn to make a comment
about his questions.
Mr. Lamborn. Mr. Chairman, I am going to waive my questions
for the sake of time and just follow-up in writing to the
degree that we need further explanations.
So thank you.
Mr. Hall. Thank you, Mr. Lamborn.
And I will try to squeeze a few questions in here before we
have to go across the street to vote.
Ms. Finn, your OIG report indicates that there are about
203,000 claims where a veteran is receiving incorrect monthly
benefits.
Does this number reflect only veterans who are being
underpaid? I ask because it is important to know whether there
is a systematic bias toward underpaying rather than overpaying
veterans. What does your data suggest.
Ms. Finn. No. We found errors are reported both for
overpayments and underpayments.
Mr. Hall. Can you give us a ratio or is that roughly equal.
Mr. Reinkemeyer. No. I think most of the errors are
underpayments where the claims examiner did not identify all
the issues. A veteran may have filed his claim and had eight or
nine issues and a couple of them were omitted in the rating
decision. So I would say most, but we have no numbers on that.
And the 203,000 is the projected number based on our sample.
Mr. Hall. Okay. Well, that would be a good thing if you can
cull that from your data. That would be good for us to know if
it is tilted toward underpayment and how much.
Mr. Reinkemeyer. Sure.
[The VA Inspector General George Opfer, subsequently
followed up in a letter dated April 29, 2010, which appears on
p. 55.]
Mr. Hall. Ms. Finn, your testimony also indicates that VA
reported that it has completed actions to implement the
recommendations of the OIG report of March 2009.
Are there plans at OIG to follow-up on this?
Ms. Finn. We do not have immediate plans for follow-up
right now. We do over time, though, follow-up on selected
reports to actually assess the effectiveness of corrective
actions.
We always follow-up with management to ensure that they can
report to us what they have done.
Mr. Hall. Thank you.
Your study observed that STAR reviewers do not identify
some errors because they either do not thoroughly review
available evidence, they fail to identify the absence of
necessary medical information, and sometimes they misclassify
errors in a way that resulted in errors not being counted
against the accuracy rate.
What do you believe can cure these deficiencies? Is it a
training issue alone?
Ms. Finn. I believe our recommendation in the report
related to increased supervisory reviews on those issues.
Certainly to ensure that comments that should be corrected
should be counted as errors were not counted.
Mr. Hall. Thank you.
And you noted that in your testimony that STAR management
required that regional offices report quarterly on actions
taken to correct benefit entitlement errors, but they did not
require or follow-up to ensure regional offices were actually
taking corrective actions on the comments made by the STAR
reviewers.
Do you have any indication that VA has remedied this
concern and what assurances the corrective action at VA as
purported has actually been implemented in a satisfactory
manner?
Ms. Finn. The only assurance we have is VBA's response to
our recommendation. And as I alluded to in my oral and written
testimony, we look at this issue when we go on our regional
office inspections and we do not always find that STAR errors
have been actually corrected.
Mr. Hall. And lastly for you, Ms. Finn, your review
indicates that brokered claims are experiencing a 69 percent
accuracy rate.
What impact do you believe brokered claims will have on the
overall VBA claims accuracy rate in the short term and long
term or can you tell yet?
Ms. Finn. No, I cannot tell you. We are conducting a
national audit looking at the brokering or redistribution of
claims program. And we are assessing the impact on the
timeliness of claims processing and also how this could
possibly impact accuracy or how the brokering program is being
implemented in respect to accuracy.
So I do not at this point have any real solid projections.
Our sampling that we did a year ago was based on a sample of
the brokered claims to determine our error rate at that time.
Mr. Hall. I would ask you, Ms. Finn, and then Mr. Bertoni
also, the following question.
In the list of the 2009 STAR accuracy ratings for all
regional offices, that we were given, the most accurate RO was
Des Moines with an accuracy rating of 92.34 percent, ranging
down to Baltimore with a 69.34 percent accuracy rate. The
Washington Regional Office is not rated but we understand its
quality level may be lower than Baltimore. Have you already or
do you have plans to try to identify what it is that the Des
Moines office is doing that others maybe should be emulating or
what it is that Baltimore is doing that others should avoid or
is it not that simple? Are there other factor?
Ms. Finn. I do not know that it is that simple. We have not
yet visited the Des Moines office on our benefit inspection. We
have, however, gone to Baltimore. And we do a selected review
of claims. We are looking at specifically claims for PTSD, TBI.
We have looked at diabetes and also we look at brokered claims.
So our review of claims is not consistent necessarily with
the STAR methodology and, therefore, the numbers are not
directly comparable.
We did find when we looked at those selected claims in
Baltimore that they had about 38 percent inaccuracy on those
types of selected claims. We also found a number of management
issues in Baltimore in that they had had leadership vacancies
and they also had had staff removed from the regular regional
office operations to work on a disability pilot.
Mr. Hall. Right.
Ms. Finn. So we felt that impacted their ability to do
quality work.
Mr. Hall. So the virtual RO pilot project and DES pilot may
have been hurting, at least temporarily, the overall accuracy
rating?
Ms. Finn. We felt it was an issue that needed to be
addressed. We could not directly quantify the impact.
Mr. Hall. Okay. Well, thank you very much.
Mr. Bertoni, would you care to address that same question.
Mr. Bertoni. Yes. In terms of the variation in quality, I
think the OIG testimony bears out some interesting issues with
the accuracy rate itself.
I think one question might be variation might be due
perhaps to quality of the review across the regions. I mean, we
see lack of training, folks who may not know completely the
rules of the game who may be reviewing these claims. And in
some regions, they may be doing a more thorough job. In other
regions, they may not.
The other part is that there is some blame to lay on the
regions in terms of quality. I think you cannot underrate
regional management. There are good managers out there who
really embrace quality and it is ingrained in the staff. They
are doing innovative things in terms of training.
And we have a report we are about to issue to this
Committee in regard to training and some actually best practice
type things that are going on out there.
And also the ability to hire staff and keep staff. And I
think it is more difficult in certain areas than others,
especially some of the larger urban areas, and supervisors.
So I think it is a range of things. It is a combination of
things that sort of lead to this variance.
But I would also, again, hearken back to the quality of the
reviews. Is it true that these two regions are entirely
different? It may be reviewers are making mistakes and it might
lead to some of this disparity.
Mr. Hall. Mr. Bertoni, in our last hearing examining the
benefit delivery at discharge (BDD) and quick-start programs,
you noted that VA lacks sufficient and specific performance
measures for assessing the accuracy of decisions on BDD claims.
And you recommended that VA consider options for separately
estimating the accuracy of such decisions.
However, VA has asserted that the cost of sampling pre-
discharge claims as part of STAR would outweigh the benefits.
If VA subjected BDD and quick-start claims to STAR review,
could we make these programs better, even better than they
apparently are? If so, what staffing, training, or other
resources do you think would be needed?
Mr. Bertoni. I think the answer is yes. There were 250,000
initial claims that year and BDD represented a little over
50,000. So that is about 20 percent of the new claims coming
into the pipeline.
So if you can make a difference there, you can make a
difference. And the goal is to make BDD claims an even larger
part of the initial claims.
The idea that it is not cost effective, I do not know if I
have seen enough to agree to that. VA did do some analysis for
us, but it was only on the cost side, full-time equivalents,
space, modifications to STAR. They never pulled a sample of BDD
claims to say what did we get from this additional review. So
what is the return on investment on the other side is not
clear.
And I continue to believe that VA continues to process more
claims outside the traditional process. They are reengineering
processes. They are reengineering, you know, how they do work.
And I think they are using in many cases sort of STAR as more
of a blunt instrument rather than using it to really drill down
into these specialized new business processes to see how they
are doing.
And I think at the end of the day, they need to do more of
that to find out where they are getting the return on
investment and where they need to walk away from things.
Mr. Hall. You asserted in your testimony that the accuracy
rate was 86 percent in fiscal year 2008 and 84 percent for
fiscal year 2009, well short of VBA's goal of 90 percent.
So at this point, what additional recommendations do you
have for improving the STAR Program.
Mr. Bertoni. I think there is a mixture here again. There
are program design issues that they have to address, but there
is also management and oversight. And I think the OIG really
hit on many of those.
I was extremely surprised to hear that you have such an
important program and had no training requirements. I was very
surprised. The fact that they have not done that yet is
something that they really need to move on.
A supervisory review, you know, having another look at
these cases before they are blessed or deemed to be correct is
very important. It is part of their quality assurance process.
They need to build in some management controls and certainly
training.
We have a lot of new staff and integrating those staff and
putting them in a position to be able to accurately assess
claims is going to take good management, good supervisory
review to sort of teach people as to, you know, what needs to
be done. So I think that is very important.
Mr. Hall. Thank you, Mr. Bertoni.
You also note that due to increased Congressional support,
VBA has increased its compensation and pension staffing.
Specifically you point out that VBA more than doubled the size
of the quality assurance staff, allowing it to increase the
scope of quality assurance reviews.
Do you believe this additional staff has made a difference
or are you of the opinion that staffing alone is insufficient
to address deficiencies? I think you started to answer this
question, but----
Mr. Bertoni. Staffing never hurts. And if you staff up a
program and you give folks the tools that they need to do the
job they need to do, it could make a difference. And that means
again training them up and putting them in a position to make a
difference.
So the numbers will help. But, again, it is going to take
time. There is a learning curve. But over time, I think it will
be effective, but you also have to build in the management
controls to make sure that the reviews do have the authority
and the oversight teeth that they should have.
Mr. Hall. Thank you.
Some VSOs point to the high rate of remand ordered by the
Board of Veterans' Appeals (BVA) or the U.S. Court of Appeals
for Veterans Claims (CAVC) as indications that the accuracy
rate is a lot lower than reported by VBA.
What is your assessment of how the errors found by the
Board and the Court impact VBA's reported claims processing
accuracy rate?
And maybe, Ms. Finn, if you could comment on that as well.
Ms. Finn. Mr. Chairman, we have not done any specific work
on the appeals process, so I would prefer not to comment.
I would note, however, that remands involve other issues
other than just whether the rating decision was accurate. There
could be procedural reasons for a remand. But other than that,
I will leave that one to Mr. Bertoni.
Mr. Bertoni. We had worked in that area not very recently,
but I would say again there are a range of reasons why a case
would be remanded back. Sometimes it is a matter of it has been
so long that the impairments change, new impairments are
introduced, other evidence has not been considered. So there
are a lot of reasons.
But I think overall in hearing what I have heard today with
the OIG report, what we know and work that we have done in the
past in terms of missing claims and how that could affect error
rates, I would say the error rate--I do not have a whole lot of
faith in the error rate or, I am sorry, the accuracy rate of 84
percent. It is probably some other lower figure, but I do not
know what that is.
Mr. Hall. And, lastly, Mr. Bertoni, before the changes in
the STAR Program were mandated by Public Law 110-389, only ten
cases per month for each RO were being reviewed by the STAR
system. Today VA staff advises us that twice as many cases are
being reviewed, 240 cases per RO per year.
How many total cases were decided by the regional offices
last year and is a sample of 20 cases per month per RO
sufficient or should that sample be larger, particularly for
ROs to process a larger number of claims?
Mr. Bertoni. Unfortunately, I do not have the number of
total cases. But any time you can ramp up the numbers is going
to make the projectability of what you do more rigorous.
Our concern again with the current system is STAR tends to
be used more as a blunt instrument to account for accuracy of
all claims regardless of varied business processes.
More and more, we see these claims being processed through
these alternative or re-engineered processes in order to
expedite things. And the Agency could be using STAR more
effectively to assess how they are doing with these alternative
processes. They are not. And BDD again is a good example of
that.
Every re-engineered process should include some kind of
quality assurance whether it is continuous or a snapshot or a
limited investment. Without doing that, you waste a lot of
time. You do not know what you are getting for your money and
you might end up where you do not want to be at the end of the
day.
I think failure to build any regular or limited investment
quality assurance into any of these alternative processes is a
failure of management. And we have noted along the way these
varied again re-engineered or alternative means by which cases
are being decided where the quality assurance, the STAR review
was not sufficient in our view to get a good sense of how
accurate and consistent many claims are.
Mr. Hall. Well, thank you, Mr. Bertoni and Ms. Finn and Mr.
Reinkemeyer. I am grateful for your testimony. And we may have
further questions for you that we will send you in writing.
We now have a series of votes called on the floor,
approximately 30 minutes we are told, so we will ask our next
panels to be patient, please, as you are used to, I am sure, by
now.
And the first panel is excused. Thank you again.
The Subcommittee is recessed for votes.
[Recess.]
Mr. Hall. Welcome again, and thank you for your patience.
The Subcommittee on Disability Assistance and Memorial Affairs
will now resume its hearing on the STAR Review, Making the
Grade. And our second panel, thank you for joining us again.
Ronald B. Abrams, Joint Executive Director, National Veterans
Legal Services Program (NVLSP); John L. Wilson, Assistant
National Legislative Director, Disabled American Veterans
(DAV); Raymond C. Kelley, National Legislative Director of
AMVETS; and Ian C. de Planque, Assistant Director, Veterans
Affairs and Rehabilitation Commission of the American Legion.
Gentlemen, thank you so much for being here today. And your
full statement, written statement, of course is entered into
the record. So we will give you each 5 minutes to expound upon
it, and starting with Mr. Abrams. You are now recognized.
STATEMENTS OF RONALD B. ABRAMS, JOINT EXECUTIVE DIRECTOR,
NATIONAL VETERANS LEGAL SERVICES PROGRAM; JOHN L. WILSON,
ASSISTANT NATIONAL LEGISLATIVE DIRECTOR, DISABLED AMERICAN
VETERANS; RAYMOND C. KELLEY, NATIONAL LEGISLATIVE DIRECTOR,
AMERICAN VETERANS (AMVETS); AND IAN C. DE PLANQUE, ASSISTANT
DIRECTOR, VETERANS AFFAIRS AND REHABILITATION COMMISSION,
AMERICAN LEGION
STATEMENT OF RONALD B. ABRAMS
Mr. Abrams. Thank you, Mr. Chairman. I have some good news.
We have just met with hundreds of service officers for Legion
and Purple Heart. And I am happy to tell you that when they
talked to me they said STAR does catch errors, they do order
the VA to fix them, and there is some training based on the
errors caught by STAR. However, based on the same comments of
these service officers, and our own personal experience, we
find that the training based on the errors caught by STAR is
not generally doing the job. The errors still continue and the
emphasis is still on production in the regional offices.
NVLSP believes that the quality of the regional office
adjudications is much worse than what is reported by the VA.
And all you have to do if you want to look at it in an
independent check is look at the statistics produced by the
Board and the Court of Appeals for Veterans Claims. And I can
tell you as far as consistency goes, I defy almost anyone to
look at 20 evaluations of mental conditions and make rhyme or
reason out of them. The rating schedule is not applied on a
consistent basis.
I am not going to over the BVA statistics. You all have
them. I will say that the results of what we do with the
American Legion, and we have done over forty quality checks in
regional offices, we find that the quality is much worse. In
many instances claims are denied in a premature manner or
benefits are not paid at the proper rate because the regional
office was more concerned about claiming work credit and
reducing the backlog than taking the time to develop and
analyze the claim properly. STAR shows about a 16 percent error
rate. We are finding over a 30 percent error rate, in general.
Here are some of the errors that we have found that you
should know about. Assignment of erroneous low ratings for
service-connected mental conditions. Erroneous denial of claims
for service-connection for mental conditions. Failure to
consider 38 USC 1154(b), the Combat Veteran Statute. Erroneous
denials of claims for individual unemployability. Failure to
consider presumptive service-connection. And inadequate
requests for medical opinions. Sometimes doctors are not asked
the right questions and that leads to a litany of problems.
Based on that we ask that the VA work measurement system be
altered so that quality, production, and timeliness are
concepts that really drive the system. That is the most
important thing. You need to change that to make any effective
change. We would also like a quality control with teeth and an
independent quality check.
I will leave it at that except to say that what the VA is
doing now is obviously not working. We need to have things
changed. Thank you.
[The prepared statement of Mr. Abrams appears on p. 42.]
Mr. Hall. Thank you, Mr. Abrams. Mr. Wilson, you are now
recognized.
STATEMENT OF JOHN L. WILSON
Mr. Wilson. Thank you, sir. Mr. Chairman and Members of the
Subcommittee, I am pleased to appear before you on behalf of
Disabled American Veterans to address VBA's Systematic
Technical Accuracy Review program.
Successfully reforming the veterans benefits claims process
will require training and accountability, two elements central
to producing quality results for veterans. VBA continues to
struggle with the quantity of claims, with quality, training,
and accountability taking a backseat. VBA's primary quality
assurance program is the STAR program. The STAR program can
identify three types of errors: benefit entitlement, decision
documentation and notification, and administrative.
The STAR program was evaluated by the VA OIG in the March
2009 report, which determined the program did not provide a
complete assessment of rating accuracy. The OIG found STAR
processors did not effectively identify and report all errors
in compensation claim rating decisions. While VBA stated STAR
reviewers achieved an 87 percent technical accuracy rate, the
OIG projected an accuracy rate of only 78 percent based on its
review of STAR reviewed cases. The OIG determined that this
equates to approximately 203,000 claims in that 1 year alone
with veterans monthly benefits may be therefore incorrect. The
OIG determined the STAR reviewers did not identify some of the
missed errors because they either did not thoroughly review
available medical and non-medical evidence, did not identify
the absence of necessary medical information, or
inappropriately misclassified benefit entitlement errors in the
comments section. These findings point to the need for greater
management oversight and an effective formal training program
for the STAR reviewers.
We had heard reports that while STAR reviewers could
benefit from formal training current workload requirements do
not allow for the necessary training time. This common theme
for VBA underlies even STAR reviews; quantity over quality.
Even in the area that is supposed to ensure quality of at least
a technical nature.
The need for a quality control program as an adjunct to the
STAR quality assurance program can also be seen when considered
through a review of the Board of Veterans' Appeals summary of
remands. Nineteen thousand one hundred cases, or 34 percent of
appeals reaching the BVA in fiscal year 2009, were due to
remands for notice not being provided to claimants, failed
requests for service medical records and personnel records, or
ignored travel Board requests. These elementary errors were
either undetected or ignored. A 34 percent error rate on such
basic elements in the claims process is simply unacceptable.
With no incentive to prevent such errors, and a constant
focus on production, quality will continue to decline. DAV
agrees with the VA OIG recommendations to improve the STAR
program. In addition, we recommend VBA establish a quality
control program that looks at claims in process in order to
determine not just whether a proper decision was made but how
it was arrived at in order to identify ways to improve the
system. Combining results from such quality control reviews
with STAR's quality assurance results, and the data from
remands from the Board of Veterans' Appeals, and the Court of
Appeals for Veterans Claims, could yield valuable information
on trends and causes of errors. If this data could be
incorporated into a robust IT system, proper analysis of such
data would provide management and employees insights into
processes and decisions. With a modern IT system, VBA would be
able to do quality control in real time, not just after the
fact. This in turn would lead to quicker and more accurate
decisions on benefits claims and more importantly to the
delivery of all benefits earned by the veteran, particularly
disabled veterans, in a timely manner.
That concludes my testimony. I would be happy to answer any
questions.
[The prepared statement of Mr. Wilson appears on p. 45.]
Mr. Hall. Thank you, Mr. Wilson. Mr. Kelley. You are
recognized.
STATEMENT OF RAYMOND C. KELLEY
Mr. Kelley. Thank you for giving AMVETS the opportunity to
present our views on the STAR program. AMVETS agrees with VA's
Office of Inspector General's March, 2009 report that
identified eight issues that will improve the process of
reviewing claims for errors, and AMVETS is please to see VBA is
taking action to correct these issues. AMVETS is concerned,
however, with what is done with the information that is gleaned
from STAR.
For the STAR program to truly be effective AMVETS believes
three things must be done. First, STAR must be enhanced so
trends and errors can be easily identified by regional offices.
With this information, VBA must hold ROs accountable for
failures in accuracy and insist that the ROs develop
improvement strategies and include training for these accuracy
issues.
Second, VBA must change its culture of timeliness and
strive for a culture of accuracy. Whether or not STAR is
completely accurate in its review is important but not nearly
as important as what is done to ensure the same mistakes are
not made again.
Third, OIG must conduct periodic reviews of the STAR
program to ensure that its accuracy ratings are within a 3
percent error margin.
Even though AMVETS believes the STAR program is effective,
we believe it could be expanded to ensure that specific
programs such as BDD and specific conditions can be tracked for
anomalies that occur so improvement strategies and specific
training can be implemented. After the Veterans Claims
Assistance Act (VCAA) was introduced in the first quarter of
fiscal year 2002, nearly half of all errors were VCAA related.
VBA had the ROs retrain their claims processors and in the last
8 months of the fiscal year these types of errors were reduced
by one-third. This type of accountability needs to be the rule
and not the exception.
As you know, the STAR program identifies errors in claims
decisions each month through random sampling of each regional
office. The results of the reviews are sent to the ROs. Errors
that are found are to be corrected by the RO who made the
decision. The corrections are made, but all too often the ROs
do not implement strategies to ensure that claims processors do
not continually repeat the same mistakes.
AMVETS believes the reason these strategies are not
developed is that the culture within the regional offices is
one of timeliness and not one of accuracy. STAR has
consistently found that nearly 20 percent of claims are in
error over the past decade, but VBA has done little to ensure
that mistakes that are made in the regional offices are
understood and that strategies for improvements are put in
place. On the other hand, VBA does require ROs to develop
corrective action plans if they do not reach strategic goals
for production, inventory, and timeliness. This paradigm must
be flipped.
The March, 2009 OIG report clearly defines the gaps in the
STAR program that have caused the 10 percent disparity in
compensation and pension rating accuracy. AMVETS believes VBA
is taking action to close these gaps, however we believe it is
important to have this accuracy fall within a 3 percent margin
of error. Therefore, AMVETS requests that OIG conduct a follow
up to the 2009 report to ensure VBA's gap solutions are
productive and that OIG continue to conduct periodic reviews of
STAR to be sure the program reaches and maintains that 3
percent margin of error.
Mr. Chairman, this concludes my remarks and I will be happy
to answer any questions that you have.
[The prepared statement of Mr. Kelley appears on p. 48.]
Mr. Hall. Thank you, Mr. Kelley. Mr. de Planque.
STATEMENT OF IAN C. DE PLANQUE
Mr. De Planque. Thank you Mr. Chairman and Members of the
Committee. On behalf of the American Legion I would like to say
that the STAR program is a potentially effective tool that if
used in a more effective manner could really help VA deal with
the accuracy issues. It is refreshing to see that in the OIG
report from March 2009, that the issues that OIG identified VA
concurred with and has recently reported that they are making
the changes to correct those.
The American Legion would recommend three points which
could help VA enhance the efficiency of STAR and the ability
for it to be effective. The first point is to create an
aggregate record of all of the errors that are reported through
STAR so that it can be analyzed. The second point would be to
use the collected data to develop a targeted training program.
And the third point would be to have regular, outside,
impartial oversight of the process.
Going back to elaborate on the first part. The errors are
being reported back to the ROs. However, to the best of our
knowledge there is no one consolidated effort to aggregate
these mistakes, common errors, and deficiencies that are being
noted. If you combine this with common errors and deficiencies
noted from the Board of Veterans' Appeals, the Appeals
Management Center (AMC), and the Court of Appeals for Veterans
Claims, you can develop an overall picture both on a regional
office level, perhaps even on a team level, but more
importantly on a national level of what the common errors
within VA are. And you can use that to determine where to
devote the resources to improve accuracy within VA.
That leads directly into point number two, targeted
training. We have reports and when we have spoken to VA
employees on the American Legion quality review visits to
regional offices, it is noted and VA notes that when STAR
identifies problems they are reported back to the teams and the
coaches for correction involved. However, if you can notice a
national trend, if you can notice that consistently VA is
experiencing problems with, say, rating mental health
disorders, or improperly asking for exams, then this can be set
into a targeted training program so that VA is getting the most
use possible out of their 80 to 85 hours of mandated training,
that it is going to correct the areas that you need corrected.
If you take a math test and you realize that you are having
problems with binomial equations, then you need to go back and
do some extra work on binomial equations so that the next time
you take that math test you get it right. This is exactly the
same sort of thing that VA could use this aggregate data to
accomplish.
And the third point that I want to make is about outside
oversight, third party oversight. In the recent report this
morning that was published in the Federal Register on the OIG
investigation of Togus, Maine, one of the things that was
pointed out, 25 percent of the STAR errors were not corrected
in accordance with VBA policy. And two examples that are listed
here, STAR instructed the regional office to inform a widow
that her child could be entitled to Dependency and Indemnity
Compensation (DIC) benefits. There is no evidence in the claims
folder showing staff informed the widow of this potential
entitlement. Furthermore, the RO staff erroneously informed
STAR that they corrected the error. Secondly, STAR instructed
the RO to send a notification letter for a burial claim to the
proper claimant. While the staff informed STAR that they
corrected the error they did not send the revised error to the
proper claimant.
So clearly, and this is not new that we have seen in the
reports of the OIG of the various specific regional offices,
even if STAR is capturing the errors they are not necessarily
being followed up on, which is why you need third party
oversight. You need somebody to go in and double check that
they are crossing the I's, dotting the T's, and correcting
these errors.
VA has potentially a very effective tool here and we want
them to be able to use it effectively. And with the three
points that we believe will be effective to create an aggregate
of that information that can be searched for trends, to use
those trends to target the training, and to add outside
oversight, we believe that this can be an effective tool for
VA.
I thank you very much, Mr. Chairman and the Committee, and
we would be happy to answer any questions.
[The prepared statement of Mr. de Planque appears on p.
48.]
Mr. Hall. Thank you, is it Mr. de Planque or de Planque,
just----
Mr. De Planque. It is de Planque, technically.
Mr. Hall. De Planque, okay, thank you. Just like to try to
get the correct pronunciation. You mentioned that training
should be more specialized. Who do you think should oversee
this training? Should it be standardized at either the RO or
national levels, or individualized for each local VA office?
Mr. De Planque. What I am proposing here, what the American
Legion is proposing here, is to have some national training
that is targeted to national areas of deficiency. However,
within the individual ROs you also, there are different
training staff within the individual ROs. And so when they
notice something that is within their particular office
obviously they have some initiative to direct the training on
their own.
However, what we are talking about here is, when you are
identifying a national problem that a consistent number, or
even a regional problem, that ROs in a certain area are
deficient in performance in a certain aspect, such as applying
the mental health disability schedule, or properly sending out
VCAA notice, then it is important to have the national training
specifically craft training that will address the issues that
are noted. And that is why it is important to actually have a
method for capturing that that takes into account not only STAR
but also, as I said, there needs to be a method for capturing
the errors that are being seen at the Appeals Management
Center, that are being seen at the Board of Veterans' Appeals,
and that are being seen at the courts. And if you combine those
four elements, STAR, the AMC, the BVA, and the CAVC, then you
actually have a big picture, the picture from, you know, 3,000
feet of what is going on. And you can create the targeted
training that is going to be more effective.
Mr. Hall. Thank you. And just one more question. Sir, you
suggested incentives for bonuses in the management of the ROs
are driven by quantity rather than quality. How would you
change the incentive structure to help encourage the accuracy
and quality of claims?
Mr. De Planque. There is technically one point that
addresses accuracy. However, it does not substantially impact
the bonus structure. The vast majority of it is driven by
timeliness or quantity of claims. And what the American Legion
has maintained for a long time is that the work credit system
needs to be addressed. We have proposed, although we are open
to other proposals, the idea that work credit not be counted
until a claim is finally adjudicated so that you can determine
that it was correctly done every step of the way. That may be a
more complicated system. However, some system that takes into
account the accuracy of the claim in counting the credit for
that work rather than simply that the claim was passed along to
another desk needs to be implemented. As the system stands
right now where you are counting people on three and a half
claims a day rather than three and a half correctly done claims
a day, that is where you are running into the problem. If you
start counting them on accurately and effectively completed
claims, then that is when VA is going to be in a situation
where the incentive is to get the claims done right and not to
get a large number of claims done.
Mr. Hall. Thank you. Mr. Kelley, in your testimony AMVETS
asserts that the STAR program should be expanded to ensure that
specific programs such as BDD and specific conditions can be
tracked for anomalies that occur so that improvement strategies
and specific training can be implemented. VA has claimed that
the benefits of such changes do not surpass the cost of these
measures. Do you concur with that position, and if not why?
Mr. Kelley. I do not agree with the, that it is not worth
the cost. These are veterans who are waiting for a benefit, an
accurate benefit. And I do not think there should be a cost
analysis against what veterans are waiting and deserve to have
done accurately.
Mr. Hall. You point out that not only is training for VBA
raters needed but sometimes retraining is needed to reteach key
lessons based on updated procedures. In the 10 years since
STAR's establishment, the VA has yet to surpass a 90 percent
rating accuracy. With that in mind, do you think that the STAR
review team needs to be retrained to make sure that they are
correctly employing the STAR quality control system?
Mr. Kelley. I have not really put any thought into that. It
is a good concept. I think just having enough people in place
within the STAR, I think they had 24 billets and they were
using eight to process these, you are putting a lot of pressure
on individuals again to do a lot of work in a little bit of
time. I think making sure that they have the right resources
and that they are trained properly, and retrained. If you start
seeing deficiencies it is absolutely time to go back and
retrain.
Mr. Hall. Thank you, Mr. Kelley. I just wanted to comment
that we had three veterans who were from, members of AMVETS,
with their service dogs in our office last week. And it was a
remarkable experience. My dogs were very interested when I got
home. But it is a great program and I want to thank you for the
organization's work with service dogs, and trying to make them
more available to all veterans who need them.
Mr. Abrams, you suggested VBA should increase its claim
processing accuracy by employing a system of awards and
disincentives. What would the system look like in your mind?
Mr. ABRAMS. Well, if you adjudicate claims from the point of
view of the veteran as opposed to VA, and the disincentive is
if you do not do it right and the appellate period extends, the
managers in the regional office cannot claim work credit, that
is a disincentive. If you give the managers the incentive to do
it right because you are going to reward them with work credit,
they are going to encourage their workers to do it right. VA
workers will work to what their managers want them to do, as do
postal workers, and patent office workers. And you have read
about the problems with those groups. Set the system up so that
what the managers want is what we want, quality adjudications.
Then we could also reward people who do good work and
plenty of it. But instead of just plenty of it work, that is
not enough. That is a good way to make people do the right
thing, and get benefits promptly to our veterans.
Mr. Hall. Which was what this Committee intended in the
legislation we passed in 2008.
Mr. Abrams. We certainly support that. We were happy that
we did it, and we anxiously await the response.
Mr. Hall. As do we. You suggest in your testimony that VBA
staff, or some VBA staff, take an adversarial attitude toward
claims filed by veterans seeking benefits for mental conditions
such as PTSD and TBI. What examples do you have of such bias
that you can come up with off the top of your head?
Mr. Abrams. If you want, we have plenty of cases where not
only the regional office but the Board of Veterans' Appeals
denied claims even though the veteran met the requirements in
the diagnostic code to get the higher evaluation. In fact we
had one, I just trained on it yesterday, where we screened it.
It was a BVA denial, and the VA General Counsel did not even
let us brief the case. They said let us award the benefit. They
completely conceded. There is a reason. There is something
going on where the VA does not want to evaluate properly. And I
am not saying every case. We are talking about a certain
percentage of cases that need to be corrected. But we are
finding a high level there. And we have other instances, but I
would have to go back through files and go over them, and if
they are public I can share them.
Mr. Hall. I would appreciate that and I am sure the
Committee would. Mr. Wilson, in your testimony you indicate
that the primary problem with the current accountability system
is that VBA employees are not usually held responsible for
their errors. In what specific way would you propose that VBA
hold its employees accountable? Are Mr. Abrams' suggestions
similar to what you would suggest?
Mr. Wilson. We view the STAR program as a quality assurance
program. It does what it does rather well generally speaking,
although it certainly has an error rate that is nonetheless
unacceptable. But it does measure the process itself. It tells
you what the results of that process will provide. It does
that. What it does not provide, however, is quality control.
You do not have a quality control program where you can
determine if rating decisions are correct based on the merits
of the case. There is no effective program in place beyond each
review to determine whether this process is giving the correct
rating decisions. What we need is a program that will provide
quality inspections and deliverable quality peer reviews of the
of the work done at the end of the day. This work, currently by
coaches, is done only on an occasional basis, unfortunately.
What is needed is a program that would aggregate this quality
data, roll it up and provide VBA a national level, trend
analysis providing opportunities for recognition and training,
focused on various regional offices or individual employees.
That data could also be tied into the kinds of compensation
packages and appraisals that would be provided to not only the
senior supervisors, but to all individuals in the VBA.
Mr. Hall. Your testimony alludes to the benefits of
creating a new, more robust IT system that would incorporate
many different measurements of accuracy. We heard from our IT
witness previously that at least temporarily in Baltimore there
may be a lengthening of the time or an increase in inaccuracy,
a decrease in accuracy, due to the fact that they have siphoned
people off to work on the virtual RO who otherwise would be
working on the normal, old-fashioned way of processing claims.
So I am just curious what your thoughts are about this move
that is being attempted into the virtual world and increased
use of IT? Are there less costly alternatives? Or should the VA
pursue this and keep going down the road that they are
currently on.
Mr. Wilson. It is an important initiative and I would like
to respond for the record to the question.
[Mr. Wilson subsequently provided the following
information:]
VA must continue moving forward on the path towards a fully
electronic IT system--the Veterans Benefits Management System (VBMS)--
which will become the backbone of the claims processing system.
Although there may be temporary losses in production, and potentially
accuracy as well, while new methods and technology are developed and
implemented, VA must remain focused on long term reform. Over the next
few years, VA should seek to minimize such transitory losses of
production and quality through increased resources and personnel.
If VA were simply focused on reducing the backlog quickly they
could probably achieve some short term reductions through non-
technological means. However that would neither be sustainable nor
conducive to creating a claims processing system that ensures veterans
receive accurate and timely benefits.
We also continue to have concerns about whether VA's focus on
production could come at the expense of quality and accuracy, and how
that is influencing the development of the VBMS. Given the enormous
pressure on VA to reduce the backlog, we believe that an independent,
outside expert review of the VBMS development program would provide
Congress with greater certainty that VA is on the right path, as well
as provide early warnings if changes to the VBMS program are required.
Mr. Hall. Okay. I have more questions for all of you that
we are going to submit in writing because of the possibility of
votes being called, and we still have another panel to hear
testimony from. We will send you some questions in writing in
addition to the ones that have been asked already. But one last
one maybe each of you could respond to. According to VA's
assessment of accuracy we range from a high in Des Moines of
92.34 percent accuracy to a low in Baltimore of 69.33 percent,
and Washington lower than that, although we do not have a
number. I am sorry to say that New York, my home State, is
close to the bottom at 76 percent, according to VA's estimate
which the OIG suggests may be actually optimistic by as much as
ten percentage points. But I, maybe starting with Mr. Abrams
you could tell what you suggest that the VA do to take, to
learn from those who are getting the best results? Mr. Abrams,
do you want to----
Mr. Abrams. We have been to Des Moines to do a quality
check, and outside of the fact that we could not get back to DC
from there, which is a long story, and an ice storm somewhere
involved, too, we did not find such a high quality of work. It
was better than some offices, but not at 92 percent. In my
experience, I do not really trust VA statistics except to note
that they are generally worse than what are reported. I say
that because I have seen that over 36 years.
However, some offices in rural areas attract a higher level
of worker. And we do not talk about that a lot, but working for
the VA in New York with the kicker to get a little more money
does not give you the same quality of life as working in Des
Moines. So one of the problems is attracting a higher level of
worker in the regional office. Also, a smaller office tends to
be on top of things a little more. The worst office we found
was St. Petersburg, the largest one. It was a disaster, in my
view.
Mr. Hall. Right. And they are about two-thirds of the way
down the list here, but not all the way at the bottom.
Mr. Abrams. Well, I am only looking at what we actually
found. And since I did a lot of those, you know, cases I can
talk to it personally.
Mr. Hall. Sure. Mr. Wilson.
Mr. Wilson. This is a response for the record on that as
well, sir.
[Mr. Wilson subsequently provided the following
information:]
Rather than focusing on the disparities in performance at various
Regional Offices, we would emphasize that quality and accuracy must be
prioritized in every location, as well as at level of leadership and
management. While VA's STAR program provides a standard assessment of
technical accuracy of claims work, it fails to provide true quality
control at every stage of the process. Furthermore, the IG's review of
the STAR program found that it failed to properly measure even
technical accuracy, with about an additional 10 percent error rate
going undetected.
The only solution is for VA to foster a management culture that
measures and rewards quality of results, not just quantity, and
provides sufficient training of VA's management and workforce in order
to achieve this outcome. Employees at all levels of the organization
must be encouraged to provide constructive feedback at each step in the
claims process to ensure that optimum performance is achieved. Creating
and maintaining such an atmosphere will benefit employees, management
and veterans. In the long run, the only way that VA will ever reduce
and eliminate the backlog of overdue claims is by building a quality
system that `gets it right the first time.'
Mr. Hall. Thank you. Mr. Kelley.
Mr. Kelley. I think if you look at the ROs there is a
personality that comes out in each one of these ROs. And in
that there is a culture. And we need to figure out how can we
get everybody to look at things the same way. Localized
training on certain issues that have been identified as a true
problem for that one area need to be identified, but there
needs to be national, standardized training throughout to
ensure that everybody is thinking from the same book and
everybody is moving forward with the same thought in mind.
Mr. Hall. Thank you. Mr. de Planque.
Mr. De Planque. Well the simple answer that you would want
to do in a case like that is figure out a way to clone Iowa, if
that is your top performing thing, and replicate it everywhere
else. But as Mr. Abrams pointed out, you do not necessarily
have the same base, the same resources, the same workload at
every regional office. And so there are going to be
fluctuations.
We agree completely on what you have to address in the pool
of workers that you can attract with the salaries that are
there for VA. Obviously, any employment is great in a difficult
economy. But people want to be competitive, and they want to be
in a position to do that. And in some areas where the cost of
living is much higher it is less attractive and so that is
difficult. And we do not talk about that a lot, but it is
something that is there, and it is there to consider.
The nationally standardizing training and having good,
solid means of national analysis to see, ``Hey, how do we
replicate the Iowa results in other offices?'' Is there a way
to condense that, put it into a training package and get that
to everyplace else? That is what you are dealing with. But also
in each regional office there is different staff, there is
different management. They are all trying to do the same task,
but you have a different climate in each one. The accuracy
rates that we have seen have been fairly consistently, when the
American Legion does quality reviews, have been fairly
consistently in the 25 percent to 33 percent range of error,
which is, and that is an across the board sort of average.
Which is below obviously what the STAR averages are finding,
and if you look across the board, then something like 92
percent may be optimistic, although you should still consider
that that is better than what a number of others are doing.
So in addition to finding out where you are seeing the
deficiencies are, also look for where you are seeing the
strengths are and use that in your training programs to, again,
like we said, create targeted training. Create training that is
targeting what you know is both positive and negative about
your organization and to correct those errors. Play to your
strengths, and beef up your weaknesses.
Mr. Hall. Thank you, Mr. de Planque, and Mr. Kelley, Mr.
Wilson, Mr. Abrams, for your written and oral testimony, and
for your patience while we wait for votes to happen today. It
is always good to hear from you and to see you. So I thank you
for your service to our veterans and to our country. And the
second panel may now be excused.
And we will ask our third panel to join us. Bradley G.
Mayes, the Director of Compensation and Pension Service for the
Veterans Benefits Administration, U.S. Department of Veterans
Affairs; accompanied by Edna MacDonald, Acting Deputy Director
of Compensation and Pension Service of VBA; and Terence Meehan,
Director of Employee Development and Training of the VBA. Thank
you all for being here. Thank you for your patience. As you
know, your full written statement is already made a part of the
hearing record so feel free to abridge it or add to it, or
whatever you would like to do in 5 minutes.
Mr. Mayes.
STATEMENT OF BRADLEY G. MAYES, DIRECTOR, COMPENSATION AND
PENSION SERVICE, VETERANS BENEFITS ADMINISTRATION, U.S.
DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY EDNA MACDONALD,
ACTING DEPUTY DIRECTOR, COMPENSATION AND PENSION SERVICE FOR
POLICY AND PROCEDURES, VETERANS BENEFITS ADMINISTRATION, U.S.
DEPARTMENT OF VETERANS AFFAIRS; AND TERENCE MEEHAN, DIRECTOR OF
EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS
ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS
Mr. Mayes. Mr. Chairman, thank you for providing me this
opportunity to discuss the Veterans Benefits Administration's
quality assurance program and the positive effect it has on the
processing of veterans disability claims. And joining me today,
as you said, are Edna MacDonald to my right, Acting Deputy
Director of Compensation and Pension Service for Policy and
Procedures; and to my left, Terence Meehan, Director of VBA
Employee Development and Training.
Mr. Chairman, I have prepared remarks. But let me just say
before I get into that, that I agree completely with virtually
every panelist that has been up here, that quality must be at
the core of our philosophy for serving veterans. There is no
question about that.
The Subcommittee has indicated a special interest in our
Systematic Technical Accuracy Review program. However, I want
to emphasize that STAR is only one of four tiers of our
multifaceted national quality assurance program. The STAR
component focuses on claims processing accuracy, while the
other three components address regional office oversight,
rating consistency, and special focus reviews. Along with the
STAR program, these components collaborate to ensure high
quality and consistent decisions for veterans.
STAR is the quality assurance component that focuses on
claims processing accuracy. STAR reviews are focused on
outcomes for veterans rather than specific processes by which
the outcomes are reached. STAR reviews evaluate the quality of
the rating decision product that VBA provides for veterans. It
is from the veteran's perspective that there is an expectation
that we understand the claim, evaluate it accurately and
fairly, and provide proper compensation under the law.
The purpose of STAR reviews is to ensure that rating
decision outcomes meet these expectations. The STAR program
incorporates review of three types of work, claims that usually
require a rating decision, authorization work that does not
usually require a rating decision, and fiduciary work. A
determination of benefit entitlement accuracy for rating and
authorization work utilizes a structured checklist to ensure
all issues were addressed, claims assistance was provided, and
the decision was correct. Accuracy results are calculated on
the results of the benefit entitlement reviews. STAR findings
provide statistically valid accuracy results at both the
regional office and the national level.
VBA continues to focus on expanding and improving the STAR
program. In 2008, the STAR staff was consolidated to Nashville,
which provided space for expansion and allowed aggressive
recruitment for more staff. Since then, STAR has completed an
extensive expansion effort, more than doubling the staff and
increasing the sample size to obtain a statistically valid
sample at the regional office level. In 2010, quality review of
fiduciary cases was also transferred to Nashville.
During fiscal year 2009, a little over 20,000 cases were
reviewed under the program. The targeted number of cases for
review in 2010 is 37,932. The STAR sample was expanded in 2009
to include a review of brokered work completed by VBA's
resource centers and Tiger Team. Sampling was increased for the
pension management centers to allow measurement of pension
entitlement decisions. Ongoing reviews of the disability
evaluation system cases, and Appeals Management Center cases
became part of the monthly compensation quality sample in
fiscal year 2009.
The VBA quality assurance program has undergone significant
change over the past several years, and has become more
comprehensive by expanding the type and breadth of cases
reviewed. But we acknowledge this is a journey that we are on,
and we look forward to working with our stakeholders to
continuously improve our quality assurance program.
Mr. Chairman, thank you for the opportunity to provide you
an update on our accomplishments at this hearing. And with
that, I would be glad to answer any questions that you may
have.
[The prepared statement of Mr. Mayes appears on p. 51.]
Mr. Hall. Thank you, Mr. Mayes. I appreciate the challenge
that you face, and the need to evaluate things even as numbers
are growing, numbers of staff are growing, numbers of claims
being brought in by the new diseases that are service-connected
automatically, not to mention the returning Operation Iraqi
Freedom/Operation Enduring Freedom (OIF/OEF) soldiers and
veterans, and the aging of our older veterans population. But I
am curious just, you know, for instance what action you have,
or have taken, or are considering taking regarding for instance
Baltimore or Washington given the disparity between, you know,
their accuracy rate under your evaluation and the more
successful. I mean, I would just guess that maybe Nashville is
rated near the top because STAR is based there? Is that a
correct assumption? And, but what, is there something underway
already to try to bring Baltimore and Washington up.
Mr. Mayes. Well first, I will tackle that last one. I do
not think it hurts that we have our quality assurance operation
in Nashville, although it is not actually in the regional
office. But in Nashville you had a director who I understand is
going to be retiring soon, Brian Corley, who came up through
the system understanding the C&P program and the adjudication
process, and was very dedicated, I know personally, in training
and focusing on developing his workforce. He knew the value of
developing that workforce because that in the end would reap a
long-term reward. While you may have to pay the price early on
to train these employees, down the road they would become more
effective. He preached that, and he enforced that.
Now, I think one of the earlier panelists mentioned that
this has to be a philosophy, and I echoed that. I am not
suggesting that in Baltimore they do not have that philosophy
or they do. What I am saying, though, is what the service has
to do, is help shine the light on areas where there are
deficiencies. STAR helps do that. With STAR, you have that
array of the quality by regional offices. But we also do site
surveys. We have gone into Baltimore on a site visit, and we
have highlighted some areas that they need to focus their
attention on. We work closely with our operations element
through our Office of Field Operations. We share this
information with them. And then the expectation is that the
management team will execute on these recommendations. That is
how this process works. We have provided that information, and
I believe that they have set about executing a plan for
addressing these deficiencies that we have identified. It has
resulted, I believe, in a higher quality rating.
Mr. Hall. Does the VA keep records of how performances
changed or improved with new leadership at the ROs? For
instance, in New York, I know, as it serves my constituents in
the Hudson Valley and other New York regional veterans,
recently had a new director appointed. And I am curious if,
well currently in this ranking has the fifth lowest accuracy
rating. Is that changing? Have you had time yet to ascertain
whether the quality, the accuracy rate has started to improve
since the new director has taken over there.
Mr. Mayes. Mr. Chairman, I want to be clear. I am beginning
to get out of my lane a little bit. The way we are organized,
with me being the head of the policy component in VBA, our
program goes in and takes a look and identifies these gaps, and
then we work closely with that operation's component to then
come back and rectify those deficiencies.
My understanding is the Director just reported there. So I
think it might be too early for us to tell whether some of the
countermeasures that she has put in place are beginning to reap
some benefit.
Mr. Hall. Do you have accuracy ratings for the previous
year, 2008?
Mr. Mayes. We do. It is a 12-month cumulative accuracy
score, so it is looking back at the 12-month cumulative number.
We have those numbers going back historically, yes.
Mr. Hall. Thank you, Mr. Mayes. Could you provide the
Committee, please, with any changes in the directors, the
different ROs within the last 2 years? And any correlation that
you see in terms of the accuracy before and after the new
management took their office? That is something, just homework
for you.
[The VA subsequently provided the information in the answer
to Question 5 of the Post-Hearing Questions and Responses for
the Record, which appears on p. 58.]
Mr. Mayes. All right, we can provide that.
Mr. Hall. I think it is worth looking at. Would you address
the question that the previous panel was talking about, in
terms of incentives and disincentives, to try to encourage the
higher accuracy performers and maybe disincentivize those who
are not producing the same rate of accuracy?
Mr. Mayes. Well one thing I would say is that accuracy is
in the performance plan for every individual that is involved
in the claims process, from the director right down to the
claims examiner who is developing the claim, to the RVSR who is
rating the claim. Every one of those decision makers, and the
managers in that process, every one of those employees has a
critical component related to accuracy. So the assertion that
we somehow do not value accuracy is not borne out by how we
measure individuals' performance. It is production as well, but
there is equal weight in the performance plan for all of those
employees for accuracy.
Mr. Hall. I appreciate that. And I note that it is a
complicated thing that is not explained by any one factor. For
instance, you do have areas where it may be easier to attract,
due to the cost of living and the standard of living one can
achieve on a given salary. Just as with air traffic
controllers, or with doctors in the private medical field and,
you know, other areas. There is obviously in Des Moines, as was
pointed out before, a chance to do better on a given salary
than in New York or Washington.
But there has been conversation in the veterans community,
in the public press, and here in Congress and in this Committee
about the evaluations, year end evaluations and bonuses that
are given to people at different, in different positions, at
VBA. And, you know, it seems to me that that is something that
I would look at if I were going to try to get some teeth into
this quickly. We need to try to motivate people to try to get
up to where Des Moines is. In fact, we need to get more Des
Moines here. So are you looking at or have you already made any
changes in the way bonuses are set?
Mr. Mayes. Well Mr. Chairman, I am making the assumption
that you are talking about the senior leadership level. I am
not involved in the process by which the bonuses are
determined.
Mr. Hall. But I believe it goes down below that, also. Are
not bonuses given out to career VBA employees?
Mr. Mayes. Yes. We can certainly give performance awards to
employees, up and down the line, across the spectrum of
employees. I was saying, quality, even for the journey-level
employee that is processing this work, is a core component of
their performance plan. I know when I was in Cleveland as the
Director, or previously in Milwaukee, that if somebody was
failing to achieve their performance plan target for quality,
then that counted against them in the mix with respect to who
was going to get performance awards, because we valued quality.
Mr. Hall. So if we looked at offices that are not
performing as well we would see lower year end bonuses or
performance awards?
Mr. Mayes. Mr. Chairman, because I have not been involved
in that process. That is one I have to take back for the
record.
[The VA subsequently provided the information in the answer
to Question 6 of the Post-Hearing Questions and Responses for
the Record, which appears on p. 58.]
Mr. Hall. Okay. We will try to find out, and if you could
find out for us that would be good. Is it safe to conclude
since Atlanta, Chicago, New York, L.A., and Baltimore are all
near the bottom of the list that accuracy levels are driven
down by volume?
Mr. Mayes. I think that Mr. Abrams made some points about
smaller stations. It is easier to control your workload. I
think that is a valid observation. But there is evidence that
larger stations can successfully manage that workload. It can
be more challenging because you have a lot more work coming
through there; you have a lot more employees that you have to
manage. Even though the span of control at the first line
supervisor level may be the same, there certainly are some
complexities when you have a large operation. Edna, do you want
to add to that?
Ms. MacDonald. I believe that some of the other panels
already talked about the challenges in hiring and retaining a
good workforce. As you point out, most of those are
significantly urban cities. We do recognize that is a challenge
for us to compete and to retain a workforce. Not that we cannot
do it, but it is sometimes easier in your smaller stations like
Muskogee, which is a very large office for us, but we are able
to retain a very strong workforce.
Mr. Mayes. That is precisely why we built out our quality
assurance operation in Nashville because we were having a
challenge in recruiting and retaining employees at that grade
level here in the Washington, DC, area. I know the OIG report
mentioned our site survey staff and the difficulty that we have
had in getting employees to come in and stay and do that work.
We have made the decision we are going to move that part of our
operation to Nashville as well because we are achieving more
success there.
Mr. Hall. And when is that going to happen?
Mr. Mayes. We have an announcement on the street right now
for employees for part of that operation in Nashville. So as
soon as we can through the hiring process, we will begin that
transition, just like we did for the reviewers. We moved that
from Washington to Nashville.
Mr. Hall. Could you please provide us with, if you have it,
the incentives or inducements that the Department is offering
to hire people in the cities that you are having a harder time
retaining or hiring staff in? Just so we have some idea of what
the range is. As I said, there are other branches of government
that are doing the same thing for the same reasons. And it
would be helpful for us to know that.
[The VA subsequently provided the following information:]
VBA offers relocation incentive bonuses on a case-by-case basis for
management positions in geographic areas where recruitment and
retention of a skilled and qualified workforce are challenging. These
challenges historically occur in areas where:
Employers outside of the Federal Government pay
higher salaries for similar positions;
Local candidates do not possess the unique
competencies required for VBA positions; or,
The geographic location, position, or duties are
undesirable to qualified candidates.
All relocation incentive bonus requests are pre-approved by the
Associate Deputy Under Secretary for Field Operations and the Under
Secretary for Benefits prior to announcing the position. Relocation
incentive amounts typically range from $5,000 to $15,000 depending on
the position, location, and past recruitment efforts.
OIG reports that the STAR quality control system was
hindered by VBA's exclusion of brokered claims from STAR
reviews. Specifically, based on OIG's 2009 review it found a 69
percent accuracy rate. While VBA initially, as I understand it,
resisted the idea of reviewing brokered claims there is an
indication that recently you have embraced this recommendation
and begun to review those claims. What impact do you believe a
review of brokered claims will have on the overall accuracy
rate?
Mr. Mayes. Well, we are reporting it now. We started with
cases worked in April. Right now the national accuracy rate for
brokered work is 77 percent. For the Tiger Team it is 78
percent. When you factor that in to the national numbers, I
think it stays the same at 83 percent. Is that correct, Edna?
Right now given the sample sizes, it is not impacting the
overall nationwide accuracy rate.
But I think the fact that we are looking at this work now
is important, because it means that folks that are promulgating
decisions out of our resource centers know that there is the
possibility that you are going to have your case selected for
review at that national level.
Mr. Hall. How do you explain the difference between STAR
comments on a claim and issues identified as errors? What can
you do, or what are you doing, to ensure that all issues are
correctly identified as errors or comments.
Mr. Mayes. Well, when we select our sample, we are sampling
claims that have been completed over a 1-month period previous
to the month that we are doing the reviews. The sample is
driven by the end product control that is associated with the
work that was done. The way we set our system up, we are doing
the review for the claim that was at issue associated with that
control. But sometimes during the course of a review we might
find where we missed something, but it is not associated with
the claim that is subject to the review. So we will annotate
that as a comment, rather than call an error on the claim that
is subject to the review in this particular review. So what the
OIG commented on was that for error calls where we are saying,
``You made a benefit entitlement error on that particular
claim,'' we were requiring a second level review. But in those
reviews where there was no error associated with that claim,
but we noted a comment, we were not requiring a second level
review. We concurred with the OIG said that we need to have the
second level review not just on those cases where we call a
benefit entitlement error but also in those situations where we
found something in the record that warranted a comment. We have
implemented that. So that was something that we agreed with the
Inspector General on.
Mr. Hall. Thank you, Mr. Mayes. Thank you, Ms. MacDonald
and Mr. Meehan. I am grateful to you for the work you are doing
and for coming here again, yet again, to tell us about it. I
will also ask you to put yourselves in the shoes of those
veterans in Washington and Baltimore, New York, and in areas
served by the other RO regions which according to the OIG, have
quality levels maybe as much as 10 percent below what your
numbers show. And I realize you are dealing with a very
complicated situation. But each veteran is also dealing with a
very complicated situation that they may not have the resources
to deal with. So I know you understand the seriousness of this,
and the Committee does also.
You and all the witnesses will have, and Members of the
Committee will have, 5 days to extend or revise remarks. And
thank you to all of our panels. The hearing is now adjourned.
[Whereupon, at 4:55 p.m., the Subcommittee was adjourned.]
A P P E N D I X
----------
Prepared Statement Hon. John J. Hall, Chairman, Subcommittee on
Disability Assistance and Memorial Affairs
Good Afternoon.
Would everyone please rise for the Pledge of Allegiance?
Flags are located at the front and back of the room.
Ladies and gentlemen, I welcome you all to the halls of Congress in
what has been a profoundly historic and important week for the nation
and for our veterans. Over the past seven days the Full Committee
convened a successful Claims Summit which brought together dozens of
the nation's top veterans' stakeholders including VA officials, GAO
representatives, Veterans Service Organizations (VSOs), veterans'
advocates, and academia and even industry leaders in this area. The big
news from the Summit is that VA admitted what many of us have known for
a while, the system for processing compensation and pension (C&P)
claims is broken beyond repair, and must be replaced. We welcome this
acknowledgement and look forward to working with VA in developing this
new system.
Of course, on a rare Sunday session, Congress passed, and
yesterday, the President signed part one of the sweeping package to
reform the nation's health care system. I for one look forward to the
second part of health reform reaching the President's desk.
Finally, this week, we have passed a number of bills in the full
House that will significantly help our veterans. These include the End
Veteran Homelessness Act of 2010 to provide $200 million in support of
Sec. Shinseki's goal of ending homelessness for America's warriors. We
also passed the Helping Heroes Keep Their Homes Act of 2009, sponsored
by our colleague Tom Perriello, which would protect home mortgages of
veterans and servicemembers. We also passed the disability compensation
COLA bill, that would allow a Cost of living increase by the end of the
year, also sponsored by Mr. Perriello. The National Guard Employment
Protection Act of 2010 which aims to help preserve jobs of soldiers
ordered to full time deployments with the National Guard is set to be
voted on today. And I remain committed to helping Chairman Filner push
legislation to help our veterans who were harmed by Agent Orange
exposure.
In this afternoon's hearing entitled: Examination of VA Regional
Office Disability Quality Review Methods: Is VBA's Systematic Technical
Review System (STAR) Making the Grade?, we will examine the primary
quality review tool employed by the Department of Veteran Affairs (VA)
to assess the accuracy of disability compensation and pension claims
processing. The STAR system can help VBA monitor the quality and
accuracy of ratings at its regional offices (ROs). Through this quality
control tool, VBA can focus attention on poorly performing ROs and help
the agency direct additional staff and training to problem offices. At
the end of the day the goal is for VA to get the claim right the first
time so that veterans are not improperly denied the benefits they
deserve or faced with lengthy appeals cycles.
The STAR system was implemented by VBA in October 1998 to improve
the measurement of the accuracy of claims processing. Since FY 2007,
VBA has set for itself a performance goal of completing compensation
claim ratings without error, 90% of the time. Its long term strategic
accuracy goal is 98%. Unfortunately, the VA is far from achieving this
goal. Until the STAR system provides an accurate accounting of the
error rate at the VBA, it is difficult to envision a path for meeting
this goal.
VA's Office of Inspector General (OIG) and the Government
Accountability Office (GAO) produced studies which revealed several
issues that impede the efficiency and consistency of the STAR system.
Specifically, OIG suggests that any claims ratings accuracy numbers
reported by VA should be discounted by as much as 10% to yield an
accurate performance measurement. I'm also personally troubled by GAO's
finding that VBA claim processing accuracy rate is particularly low in
cases filed by veterans seeking benefits based upon Post Traumatic
Stress Disorder (PTSD) and/or Traumatic Brain Injury (TBI). Today, we
intend to analyze these studies through hearing testimony from
representatives of VA's OIG and GAO.
VBA has made some improvements to its claims rating system since
the implementation of STAR system. We look forward to hearing from them
about these improvements and how we can help them with any areas of
concern.
To fully understand the STAR quality assurance program, it's
important to review the C&P rating system itself. Through its
disability compensation program, VBA pays monthly benefits to veterans
for injuries or diseases incurred or aggravated while on active
military duty. To access a claim, the application for disability
benefits must be ``developed,'' a process that involves obtaining all
necessary evidence of the veteran's service and disability to support
the claim. After development, claims go to a Rating Veterans Service
Representative (RVSR) for a decision. RVSRs determine if a veteran's
disability is service connected and assigns a percentage rating
(ranging from 0 to 100 percent) that is intended to represent the
average earning reduction a veteran with that condition would
experience in civilian occupations. The veteran is then notified of the
decision.
For reopened claims, the veteran's previously assigned diagnostic
codes with evaluations also affect the veteran's overall combined
percentage of disability. Regional offices use previously assigned
diagnostic codes, along with their percentages, in combination with
current assigned diagnostic code percentages to determine the combined
level of disability. Once a claim is completed, the result of that
claim or ``end product'' is cleared, and work credit is given to the
regional office.
A completed claim and corresponding cleared end product is then
subject to review by STAR reviewers based on a statistical sample of
all completed rating end products.
In the 110th Congress, I introduced H.R. 5892, which outlined a
series of steps to improve the quality of VA's claims processing
system. These recommendations formed the core of directives which were
codified by P.L. 110-389, the Veterans Disability Benefits Claims
Modernization Act of 2008. Today's hearing also provide us a chance to
gauge how these quality control measures have been implemented by VA.
With that, I look forward to the informed testimony of our
witnesses and insightful comments and questions from my colleagues on
the Subcommittee.
I now recognize Ranking Member Lamborn for his Opening Statement.
Prepared Statement of Hon. Doug Lamborn, Ranking Republican Member,
Subcommittee on Disability Assistance and Memorial Affairs
Thank you Mr. Chairman,
And welcome everyone, to this hearing on the Department of Veterans
Affairs STAR program.
STAR is an acronym for Systematic Technical Accuracy Review, which
is VA's program for quality assurance.
Throughout my tenure on this Committee, my fellow members and I
have called for stronger accountability within the VA claims system.
For too long, the primary focus has been on production and this has
led to an error rate that is unacceptable.
I believe that the VA's greatest challenge, the claims backlog, is
largely attributable to hasty decisions made without proper regard for
accuracy.
The ramifications of this approach can be seen throughout the
entire system.
Therefore, VA employee performance awards cannot be based entirely
on production, there must also be a valid measure of quality.
Under the STAR program, a statistically valid sample of rating
decisions from various regional offices is reviewed for accuracy.
While this method may be useful from a macro perspective, it is not
sufficient for ensuring individual accountability.
VA must be able to identify employees in need of individualized
remedial training.
Without this essential component of the quality assurance process,
VA will have perpetual problems in its claims system.
I also have other concerns based on the written statements of
today's participants.
There seems to be some contradiction as to whether brokered claims
are excluded from STAR review, or whether STAR requires follow-up to
ensure that VA Regional Offices provide corrective action or remedial
training.
Perhaps these were former deficiencies that have since been
corrected.
I am encouraged by GAO's report that VA has made a number of
improvements, but I also agree that challenges remain.
In the 110th Congress this committee passed a provision that was
included in the Veterans' Benefits Improvement Act of 2008 that
required VA to conduct a study on the effectiveness of the current
employee work-credit system.
I believe the upcoming report, along with the testimony that we
will hear today, will provide valuable feedback for the Department to
improve its quality assurance and accountability program.
I look forward to hearing from our witnesses today, and I thank you
all for your participation
Thank you, I yield back.
Prepared Statement of Belinda J. Finn, Assistant Inspector
General for Audits and Evaluations, Office of Inspector General,
U.S. Department of Veterans Affairs
INTRODUCTION
Mr. Chairman and Members of the Subcommittee, thank you for the
opportunity to discuss the Office of Inspector General's (OIG) work
regarding the Veterans Benefits Administration's Systematic Technical
Accuracy Review (STAR) program. I am accompanied by Mr. Larry
Reinkemeyer, Director of the OIG's Kansas City Audit Operations
Division.
The OIG is committed to proactively reviewing the effectiveness of
key management controls to assure the accomplishment of mission
critical service responsibilities to veterans, such as the delivery of
accurate and timely disability benefits. Further, we strive to focus
our efforts on identifying control weaknesses before they escalate into
significant problems. Over the past 2 years, we issued audit reports
covering aspects of VBA's quality assurance process--the STAR, Rating
Consistency Review, and Site Visit programs. In March 2009, we issued
the Audit of Veterans Benefits Administration Compensation Rating
Accuracy and Consistency Reviews (Report No. 08-02073-96). In May 2009,
we issued the Audit of Veterans Benefits Administration Compensation
and Pension Site Visit Program (Report No. 08-02436-126).
Also in fiscal year (FY) 2009, we established a Benefits Inspection
Division to provide recurring oversight of regional offices by focusing
on disability compensation claims processing and performance of
Veterans Service Center (VSC) operations.
STAR PROGRAM
Improving the quality of rating decisions, which includes accuracy
and consistency of disability compensation claim rating decisions, is
among VBA's highest priorities. The STAR program is a key mechanism for
evaluating regional office performance in processing accurate benefit
claims for veterans and beneficiaries. The STAR program provides a
comprehensive review and analysis of compensation rating processing
associated with specific claims or issues. VBA's FY 2008 and FY 2009
goal for compensation claim ratings was a 90 percent accuracy rate.
STAR reviewers select a number of claims to review from VA regional
offices nationwide and use a checklist designed to facilitate a
consistent, structured review and classify errors into three
categories: benefit entitlement errors, decision documentation/
notification errors, and administrative errors.
Results
During the 12-month period ending February 2008, STAR reviewers
found that VBA staff accurately rated about 87 percent of the claims
(approximately 767,000 claims of 882,000 claims reviewed). We reviewed
a random sample of STAR reviewed claims and found additional errors
affecting veterans' benefits that STAR reviewers did not identify. As a
result, we projected VBA's accuracy rate for claims reviewed was only
78 percent.
The STAR program does not provide a complete assessment of
compensation claim rating accuracy. We found that VBA officials
excluded brokered claims from STAR reviews. Brokered claims are those
assigned to one regional office but sent to another office to be rated.
When we combined the results of our review of brokered claims with our
review of STAR reviewed claims, we projected that about 77 percent
(approximately 679,000 claims) of compensation claims were accurate and
VBA's reported error rate was understated by approximately 10
percentage points. This equates to approximately 88,000 additional, or
about 203,000 total, claims where veterans' monthly benefits may be
incorrect. This difference occurred because STAR reviewers did not
identify all errors, and VBA officials excluded a significant number of
compensation claim ratings from review. Additionally, VBA officials had
not implemented an effective formal training program for the STAR
reviewers.
We identified five areas in our review of the STAR program where
VBA needed to take action to improve its quality assurance and
oversight efforts.
STAR reviewers did not identify some errors because
they either did not thoroughly review available medical and
non-medical evidence or identify the absence of necessary
medical information. STAR reviewers also misclassified some
errors in a way that resulted in the error not being counted
against the regional office's accuracy rate. In these cases,
they recorded a ``comment'' instead of an ``error'' although
the errors clearly affected veterans' benefits entitlements and
should have been counted as errors.
STAR management required regional offices to report
quarterly on actions taken to correct benefit entitlement
errors but they did not require or follow up to ensure regional
offices took corrective actions on comments made by STAR
reviewers. Ultimately, they relied on the regional office to
take corrective action on all issues identified whether the
STAR reviewer identified an error or a comment. From our
sample, we identified 33 compensation claim ratings where STAR
reviewers made comments instead of reporting issues as errors.
At least six of the comments related to issues that could
affect the veterans' benefits. We found that regional office
staff had not corrected any of the six comments potentially
affecting the veteran's benefits.
VBA officials excluded brokered claims from STAR
reviews. VBA officials told us STAR reviewers do not review
brokered claims because the STAR program's primary focus is on
assessing and reporting rating accuracy for each of the
regional offices. Since two or more regional offices are
involved in brokered work, VBA officials stated it would be
difficult to assign responsibility for the rating accuracy to
one specific regional office. Thus we found that STAR
management was replacing brokered claims selected for review
with non-brokered claims. We reviewed a sample of brokered
claims that were not evaluated and found a 69 percent accuracy
rate.
The STAR reviewers did not ensure regional offices
submitted all of the selected compensation claim ratings to the
STAR program for review. Regional offices did not submit about
600 (7 percent) of the approximately 9,000 requested claim
ratings for the 12-month period ending February 2008. We
reviewed 54 of the 600 pending requests and identified 12 (22
percent) benefit entitlement errors. STAR management relies on
regional office staff to submit the requested claims and only
follows up with the regional offices that do not submit any
requested claims for a given month. A STAR manager stated they
did not have sufficient resources to follow up on individual
claims that regional office staff do not submit. Therefore,
regional office staff can cherry pick claims because STAR
reviewers do not reconcile the claim requests. This control
weakness provided opportunities for regional office staff to
withhold claims if they suspect the claims to have errors.
STAR reviewers are not required to complete formal
training on an annual basis. The reviewers met infrequently to
discuss issues, and had no set formal training schedule or
requirements. Regional office staffs that prepare and complete
ratings and awards for compensation claims are required to
achieve 80 hours of training per year to stay competent on
laws, policies, and processes on rating-related issues.
However, the STAR program manager stated their program workload
requirements currently do not allow for the amount of training
time necessary, yet agreed the program staff could benefit from
formal training.
We recommended that VBA:
Ensure STAR reviewers evaluate all documentation
related to the claim selected for review.
Establish a requirement that all STAR reviewer
comments receive a second review to make sure the comment was
not a benefit entitlement error.
Establish procedures to review brokered claims as
part of the STAR program.
Enforce procedures requiring regional offices to
submit all requested claims to the STAR program office for
their review or submit written justification to the STAR
program's office requesting to exclude the claim from the
review.
Establish minimum annual training requirements for
STAR reviewers that are comparable to regional office rating
staff training requirements.
VBA agreed with all five recommendations and reported that it had
completed actions to implement the recommendations in our March 2009
report.
RATING CONSISTENCY PROGRAM
To address variances in compensation rating decisions within
individual VA regional offices and across the Nation, VBA developed a
rating consistency review plan that included metrics to monitor rating
consistency and a method to identify variances in compensation claim
ratings. VBA's plan would identify unusual patterns of variance in
claims, and then review selected variances to assess the level of
decision consistency within and between regional offices. However, as
of March 2009, VBA had not fully implemented its rating consistency
review plan.
Results
In FY 2008, VBA officials identified 61 diagnostic codes where
offices appeared to be making inconsistent decisions in the evaluation
of a granted claim or whether offices granted or denied the claim. VBA
officials planned to conduct 22 of the 61 reviews in FY 2008 consisting
of 20 grant/denial rate and 2 evaluation reviews. However, they only
initiated two grant/denial rate reviews and did not complete either
review until December 2008. Additionally, VBA did not initiate either
of the evaluation reviews designed to reduce variances in compensation
claim ratings. In March 2010, VBA informed us that insufficient
staffing prevented them from completing any consistency reviews in FY
2009. However, they have now hired the necessary employees. The first
FY 2010 review was completed in January 2010; a second review started
in February 2010.
We identified three areas that impaired VBA's ability to fully
implement its rating consistency review plan.
VBA did not have an accurate universe of claims to
review. From April through May 2008, VBA officials encountered
a delay in completing planned consistency reviews because the
universe of claims included completed claims with diagnostic
codes outside the scope for the requested period. VBA officials
notified the Office of Performance, Accountability, and
Integrity of the data integrity issue and worked with
Compensation and Pension (C&P) Service staff to identify an
appropriate universe by June 2008.
Data captured by STAR reviewers contained veterans'
personally identifiable information (PII). In July 2008, VBA
officials stopped all consistency reviews until they could take
action to secure PII in STAR's electronic records used to
capture and review data and analyze the results. VBA officials
took the necessary actions to correct this condition and
secured the database in December 2008.
VBA officials did not assign a sufficient number of
staff to accomplish consistency reviews. The STAR program
office was authorized 26 reviewers for rating accuracy and
consistency reviews, and had 18 reviewers on board. However,
only 8 of the 18 reviewers conducted consistency reviews. The
eight reviewers tasked with completing reviews of consistency
were not sufficient to complete the assigned work. A STAR
manager estimated that approximately 12 reviewers were needed
to complete the 22 planned reviews. However, in addition to the
consistency reviews, STAR management also assigned these same
eight reviewers to conduct at least seven special focus reviews
involving thousands of compensation claim ratings. We concluded
that the assigned staffing was insufficient to complete this
planned work.
As part of future rating consistency review plans, inter-rater
reliability reviews (IRRR) should be included, along with the annual
rating consistency reviews. In July 2008, C&P Service staff conducted
an IRRR and found that 76 (31 percent) of the 246 participants
incorrectly rated a relatively simple back strain claim. In August
2008, C&P Service staff conducted another IRRR and found that 30 (13
percent) of the 247 participants incorrectly rated a Post Traumatic
Stress Disorder compensation claim. According to VBA officials, the
most common errors were incorrect evaluations due to misinterpretation
of the appropriate facts and criteria. C&P Service managers used the
results of the IRRRs to plan focused training efforts, and they plan to
conduct follow-up IRRRs to evaluate the effectiveness of the training.
The IRRRs allow VBA officials to target a single rating issue to ensure
the consistent application of policies and procedures nationally.
We recommended VBA develop an annual rating consistency review
schedule and complete all planned reviews as scheduled, dedicate
sufficient staff to conduct consistency reviews in order to complete
planned workload and reviews, and include inter-rater reliability
reviews as a permanent component of their consistency review program.
VBA agreed with these recommendations and reported that it had
completed actions to implement the recommendations.
Site Visit Program
The C&P Service's Site Visit program was established to ensure
centralized oversight and provide technical assistance to VBA's 57
regional offices. Site Visit teams monitor compliance with policies and
procedures and identify best practices to assist in achieving high
performance. This includes determining if regional offices are
correcting errors identified by STAR teams.
Results
The Site Visit program lacks an adequate infrastructure and
management strategy to meet its mission and goals. We identified three
areas that VBA needed to address to leverage the benefits of their Site
Visit program.
The Site Visit team experienced significant turnover
during FY 2006 and FY 2008 and has never been fully staffed to
the allotted eight full-time equivalent positions. Program
officials stated that they have been unable to maintain
adequate staffing of the Site Visit program because of
difficulties in recruiting from field offices qualified
candidates who are willing to relocate to the Washington, DC,
area. In addition, we found that C&P Service cannot ensure that
onsite evaluations are performed in compliance with generally
applicable governmental standards for independence or that
sufficient independence exists between the Site Visit program's
employees and VSCs reviewed.
C&P Service did not review all 57 VSCs in any 3-year
period, and 7 (12 percent) of 57 VSCs were only visited once
from FY 2001 to FY 2008. Because the planned 3-year cycle of
review coverage has not been met, potentially low-performing
VSCs who could most benefit from a Site Visit program
evaluation may not be visited frequently enough. In addition,
C&P Service does not have formal policies and procedures to
ensure site survey protocols are modified to reflect emerging
C&P issues and systemic deficiencies identified during site
visits.
C&P Service has not established procedures and
guidelines to identify and disseminate best practices. Also,
C&P Service has not developed reports that adequately develop
the causes of errors identified, and a follow-up process to
ensure that action items are resolved. In addition, C&P Service
does not adequately identify and report system-wide trends to
senior VBA managers, thus missing out on opportunities to
proactively address issues and concerns found during individual
site visits nationwide.
We recommended C&P Service:
Develop a staffing plan to ensure that sufficient
resources are made available to complete VSC reviews on a 3-
year cycle.
Comply with generally applicable government standards
for independence when performing site visits.
Develop a procedure to continuously monitor and
update protocols to address systemic issues identified during
Site Visit reviews, management concerns and priorities, and
changes in program operations.
Develop a process for the identification of best
practices and resolution of inconsistencies in the application
of policies and procedures.
Develop and implement policies, procedures, and
performance measures to strengthen follow-up on corrective
action plans developed by regional offices on issues identified
during onsite evaluations.
Ensure Site Visit reports issued for VSC operations
to more fully identify the root cause of issues affecting VSC
performance. VBA agreed with all six recommendations and
reported that it had completed actions to implement the
recommendations.
VBA agreed with these recommendations and reported that it had
completed actions to implement the recommendations.
OIG REGIONAL OFFICE INSPECTIONS RESULTS
STAR management relies on the regional office managers to take
corrective action on all issues identified by the STAR team. Since
April 2009, the OIG's Benefits Inspection Division has issued eight
reports that include a review of regional office procedures to ensure
the accurate and timely correction of errors identified by the VBA's
STAR program. We found that regional offices did not have formal
procedures established to ensure employees took corrective actions on
the identified errors and as a result, five of the eight regional
offices had not corrected all of the errors identified by the STAR
team. Our analysis of 145 errors identified by STAR found that regional
office staff did not correct 40 (28 percent) of the errors. Further,
regional office staff erroneously reported to STAR that 21 (53 percent)
of those 40 errors were corrected, although no corrections were made.
We will continue to review and report on regional offices'
performance in correcting errors identified during STAR reviews in
future OIG benefit inspections and to report on other issues affecting
accuracy and timeliness of claims processing. We will also confirm
during these inspections whether the actions taken by VBA to implement
our recommendations to improve the C&P Site Visit and Rating
Consistency programs were effective. Currently, we plan on conducting
up to 18 inspections in FY 2010, which means that each regional office
will be inspected on a 3-year basis. Once we have sufficient data, we
plan to issue roll-up reports that identify trends affecting accuracy
and timeliness and include recommendations for improvement across the
VBA system.
CONCLUSION
VBA is under tremendous pressure to process claims and reduce the
growing backlog. Without an effective and reliable quality assurance
program, VBA leadership cannot adequately monitor performance to make
necessary program improvements and ensure veterans receive accurate and
consistent ratings.
Mr. Chairman, thank you for the opportunity to discuss these
important issues. We would be pleased to answer any questions that you
or other members of the Subcommittee may have.
Prepared Statement of Daniel Bertoni, Director,
Education, Workforce, and Income Security,
U.S. Government Accountability Office
VETERANS' DISABILITY BENEFITS: VA Has Improved Its Programs for
Measuring Accuracy and Consistency, but Challenges Remain
GAO Highlights
Why GAO Did This Study
For years, in addition to experiencing challenges in making
disability claims decisions more quickly and reducing its claims
backlog, the Department of Veterans Affairs (VA) has faced challenges
in improving the accuracy and consistency of its decisions.
GAO was asked to discuss issues surrounding VA's Systematic
Technical Accuracy Review (STAR) program, a disability compensation and
pension quality assurance program, and possible ways, if any, this
program could be improved.
This statement focuses on actions VA has taken; including those in
response to past GAO recommendations, to (1) address identified
weaknesses with STAR and (2) improve efforts to monitor the consistency
of claims decisions. This statement is based on GAO's prior work, which
examined several aspects of STAR, as well as VA's consistency review
activities, and on updated information GAO obtained from VA on quality
assurance issues that GAO and VA's Office of Inspector General (OIG)
have identified. GAO also reviewed VA's OIG March 2009 report on STAR.
GAO is not making any new recommendations.
What GAO Found
Over the past several years, GAO has identified several
deficiencies with the Veterans Benefit Administration's (VBA) STAR
program, and although VBA has taken actions to address these issues, it
continues to face challenges in improving claims accuracy. For example,
GAO found that STAR reviewers lacked organizational independence, a
basic internal control principle. In response to our finding, VA began
utilizing organizationally independent reviewers that do not make
claims decisions. GAO also found that sample sizes for pension claims
were insufficient to provide assurance about decision accuracy. In
response to GAO's recommendation, in fiscal year 2009, VA began
increasing the number of pension claims decisions it reviews annually
at each of its offices that process pension decisions. VA has also
taken a number of other steps to address weaknesses that VA's OIG found
in the STAR program, including (1) establishing minimum annual training
requirements for reviewers and (2) requiring additional supervisory
review of STAR reviewers' work. Although it has made or has started
making these improvements, VBA remains challenged to improve its
decision accuracy for disability compensation decisions, and it has not
met its stated accuracy goal of 90 percent. VBA's performance has
remained about the same over the past several fiscal years.
In addition, VA has taken steps to address deficiencies that GAO
and the VA's OIG have identified with consistency reviews--assessments
of the extent to which individual raters make consistent decisions on
the same claims. For example, in prior work, GAO reported that VA did
not conduct systematic studies of impairments that it had identified as
having potentially inconsistent decisions. In response to GAO's
recommendation, in fiscal year 2008, VBA's quality assurance staff
began conducting studies to monitor the extent to which veterans with
similar disabilities receive consistent ratings across regional offices
and individual raters. However, last year, VA's OIG reported that VA
had not followed through on its plans to conduct such reviews. In
response to this and other OIG findings and recommendations, VA took a
number of actions, including developing an annual consistency review
schedule and hiring additional quality assurance staff. However, VBA
has only recently begun these programs to improve consistency, and it
is too early to assess the effectiveness of their actions.
__________
Mr. Chairman and Members of the Subcommittee:
I am pleased to have the opportunity to comment on the Department
of Veterans Affairs' (VA) efforts to improve the accuracy and
consistency of its disability compensation and pension benefit
decisions. As we and other organizations have reported over the last
decade, VA's claims processing challenges are not limited to making
decisions more quickly and reducing its claims backlog; but also
includes improving the accuracy and consistency of its decisions. The
number of veterans awaiting decisions could grow as servicemembers
returning from ongoing conflicts and aging veterans submit claims.
According to VA, about 35 percent of veterans from ongoing hostilities
file claims. It is important not only that decisions be timely, but
also accurate. Accurate initial claims decisions can help ensure that
VA is paying cash disability benefits to those entitled to such
benefits and also help prevent lengthy appeals. Meanwhile, consistent
decisions help ensure that comparable medical conditions of veterans
are rated the same, regardless of which VA regional benefits office
processes the claim.
You asked us to discuss issues surrounding VA's disability
compensation and pension quality assurance programs; particularly, the
Systematic Technical Accuracy Review (STAR) program. My statement
focuses on STAR, which deals with accuracy, and two other VA quality
assurance activities that focus on consistency.\1\ More specifically,
my remarks will focus on actions VA has taken to (1) address
deficiencies identified with STAR and (2) improve efforts to monitor
the consistency of claim decisions. This statement is based on our
prior work, which examined several aspects of STAR, as well as VA's
consistency review programs, and on updated information we obtained
from VA on quality assurance vulnerabilities that we and VA's Office of
Inspector General (OIG) have identified. We also reviewed VA OIG's
March 2009 report on STAR and consistency reviews.\2\ Our work was
conducted in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
---------------------------------------------------------------------------
\1\ These are (1) reviews of consistency of claims decisions across
VA's Veterans Benefits Administration, which is responsible for
administering VA's disability compensation and pension programs, by
type of disabling condition; and (2) inter-rater reliability reviews,
which examine the consistency of raters when evaluating the same
condition based on a comparable body of evidence.
\2\ Office of Inspector General, Department of Veterans Affairs,
Audit of Veterans Benefits Administration Compensation Accuracy and
Consistency Reviews (Washington, D.C.: Mar. 12, 2009).
---------------------------------------------------------------------------
Background
Through its disability compensation program, VA pays monthly
benefits to veterans with service-connected disabilities.\3\ Under its
disability pension program, VA pays monthly benefits to low-income
veterans who have disabilities not related to their military service or
are age 65 or older. VA also pays compensation to the survivors of
certain veterans who had service-connected disabilities and of
servicemembers who died while on active duty.
---------------------------------------------------------------------------
\3\ The amount of disability compensation depends largely on the
severity of the disability, which VA measures in 10 percent increments
on a scale of 0 percent to 100 percent. In 2010, basic monthly payments
for veterans range from $123 for 10 percent disability to $2,673 for
100 percent disability.
---------------------------------------------------------------------------
Veterans and their survivors claim benefits at one of the Veterans
Benefits Administration's (VBA) 57 regional offices. Once the claim is
received, a service representative assists the veteran in gathering the
relevant evidence to evaluate the claim. Such evidence includes the
veteran's military service records, medical examinations, and treatment
records from VA medical facilities and private medical service
providers. Also, if necessary for reaching a decision on a claim, the
regional office arranges for the veteran to receive a medical
examination. Once all necessary evidence has been collected, a rating
specialist evaluates the claim and determines whether the claimant is
eligible for benefits. If so, the rating specialist assigns a
percentage rating. Veterans with multiple disabilities receive a single
composite rating. Since 2001, VBA has created 15 resource centers that
are staffed exclusively to process claims or appeals from backlogged
regional offices. Most of these centers focus either on making rating
decisions, or on developing the information needed to evaluate claims.
In addition to the traditional claims process, any member of the
armed forces who has seen active duty--including those in the National
Guard or Reserves--is eligible to apply for VA disability benefits
prior to leaving military service through VA's Benefits Delivery at
Discharge (BDD) program or the related Quick Start program.\4\ In 2006,
VA completed its consolidation of BDD rating activity into its Salt
Lake City, Utah, and Winston-Salem, North Carolina, regional offices,
to increase the consistency of BDD claims decisions. Also, under the
Department of Defense (DoD)--VA disability evaluation system pilot
program, servicemembers undergoing disability evaluations, if found
medically unfit for duty, receive VA disability ratings. This rating
covers both the unfitting conditions identified by the military service
and conditions identified by the servicemember during the process. The
rating is used by both DoD and VA to determine entitlement for
disability benefits.\5\
---------------------------------------------------------------------------
\4\ In order to be eligible for the BDD program, servicemembers
must meet several requirements, which include filing a VA claim 60 to
180 days prior to an honorable discharge and completing a medical
examination. Under BDD, the examination also serves as Department of
Defense's separation physical examination. Quick Start is for those
servicemembers--primarily members of the National Guard and Reserve--
who cannot meet the BDD time frame.
\5\ For our review of the DoD--VA disability evaluation system
pilot program, see GAO, Military Disability System: Increased Supports
for Servicemembers and Better Pilot Planning Could Improve the
Disability Evaluation Process, GAO-08-1137 (Washington, D.C.: Sept. 24,
2008).
---------------------------------------------------------------------------
Enacted in October 2008, the Veterans' Benefits Improvement Act of
2008 required VA to contract for an independent, 3-year review of VBA's
quality assurance program.\6\ This review is to include, among other
items, assessments of the accuracy of disability ratings and their
consistency across VA regional offices. VA contracted with the
Institute for Defense Analyses (IDA) to conduct this study. According
to VA, IDA will provide preliminary findings in the Summer of 2010, and
VA is scheduled to report to the Congress in October 2011.
---------------------------------------------------------------------------
\6\ Pub. L. No. 110-389, Sec. 224; 38 U.S.C. Sec. 7731(c).
---------------------------------------------------------------------------
STAR Program
Under the STAR program, which was implemented in fiscal year 1999,
VBA selects a random sample of completed claims decisions each month
from each of its regional offices to review for accuracy. STAR
reviewers assess decision accuracy using a standard checklist. For
decisions affecting benefit entitlement, this review includes an
assessment of whether (1) all issues in the claim were addressed; (2)
assistance was provided to the claimant, as required by the Veterans
Claims Assistance Act of 2000; and (3) the benefit entitlement decision
was correct. If a claim has any error, VBA counts the entire claim as
incorrect for accuracy rate computation purposes. The STAR reviewer
then returns the case file and the results of the review to the
regional office that made the decision. If an error was found, the
regional office is required to either correct it or request
reconsideration of the error determination. VBA uses the national
accuracy rate from STAR reviews of compensation entitlement decisions
as one of its key claims processing performance measures. VA also uses
STAR data to estimate improper compensation and pension benefit
payments.
Consistency Review Activities
One VA consistency review activity involves conducting studies of
regional offices' decisions on specific conditions such as post-
traumatic stress disorder where VBA found differences, such as in
benefit grant rates, across regional offices through comparative
statistical analysis. VBA uses the results of these reviews to identify
root causes of inconsistencies and to target training. Under another VA
consistency review activity, called inter-rater reliability reviews,
VBA provides rating specialists a sample case file to assess how well
raters from various regional offices agree on an eligibility
determination when reviewing the same body of evidence. These reviews
allow VBA officials to target a single rating issue and take remedial
action to ensure the consistent application of policies and procedures
nationally.
VA Has Implemented Procedures to Address Deficiencies Identified with
the STAR Program, but Continues to Face Challenges in Improving
Accuracy
Over the past decade, VBA has taken several actions to improve its
STAR program and to address deficiencies identified by both GAO and
VA's OIG. For example, in March 1999, we found that STAR review staff
lacked sufficient organizational independence because they were also
responsible for making claims decisions and reported to regional office
managers responsible for claims processing.\7\ In response to our
findings, VBA took steps to address this by utilizing reviewers who do
not process claims and who do not report to managers responsible for
claims processing. More recently, in February 2008, we found that STAR
was not sampling enough initial pension claims to ensure the accuracy
of pension claims decisions.\8\ Because initial pension claims
constituted only about 11 percent of the combined compensation and
pension caseload subject to accuracy review, few were likely to be
included in the STAR review sample. We recommended that VBA take steps
to improve its quality assurance review of initial claims, which could
include reviewing a larger sample of pension claims. According to VBA,
it has addressed this issue by consolidating pension claims processing
in its three Pension Management Centers \9\ and establishing a separate
STAR sample for pension claims. During fiscal year 2009, VBA began
reviewing more pension claim decisions and reported that, for fiscal
year 2009, its pension entitlement accuracy was 95 percent, exceeding
its goal.
---------------------------------------------------------------------------
\7\ GAO, Veterans' Benefits Claims: Further Improvements Needed in
Claims-Processing Accuracy, GAO/HEHS-99-35 (Washington, D.C.: Mar. 1,
1999).
\8\ GAO, Veterans' Benefits: Improved Management Would Enhance VA's
Pension Program, GAO-08-112 (Washington, D.C.: Feb. 14, 2008).
\9\ The Pension Management Centers are located in St. Paul,
Minnesota; Philadelphia, Pennsylvania; and Milwaukee, Wisconsin.
---------------------------------------------------------------------------
In a September 2008 report, we noted that VA lacked sufficient and
specific performance measures for assessing the accuracy of decisions
on BDD claims and recommended that VA consider options for separately
estimating the accuracy of such claims decisions.\10\ VA conducted an
analysis of the costs of sampling pre-discharge claims as part of STAR
and concluded that the costs would outweigh possible, unquantifiable
benefits. VA also noted that the two sites that rate BDD claims
surpassed the national average in accuracy for claims overall.\11\
While generally responsive to our recommendation, VA's analysis did not
specifically review the accuracy of BDD claims relative to traditional
claims. Moreover, because BDD claims do not comprise all claims
reviewed at the two rating sites, we continue to believe VA's analysis
was not sufficient to estimate the relative accuracy of BDD claims at
these sites. While we agree that the benefits of reviewing accuracy are
difficult to measure, if VA had better information on the accuracy of
BDD claims, VA could use such information to inform training and focus
its monitoring efforts. In contrast, VA currently performs STAR reviews
that target rating decisions made by its Baltimore and Seattle offices
under the DoD--VA disability evaluation system pilot program. Such a
targeted review could also be conducted for BDD claims.
---------------------------------------------------------------------------
\10\ GAO, Veterans' Disability Benefits: Better Accountability and
Access Would Improve the Benefits Delivery at Discharge Program, GAO-
08-901 (Washington, D.C.: Sept. 9, 2008).
\11\ BDD claims are rated at the regional offices in Winston-Salem,
North Carolina, and Salt Lake City, Utah.
---------------------------------------------------------------------------
In its March 2009 report, VA's OIG also identified several
deficiencies in the STAR program and recommended corrective actions.
The OIG found that (1) regional offices did not always submit all
requested sample cases for review, (2) reviewers did not evaluate all
documentation in sample files, and (3) reviewers were not properly
recording some errors. The OIG also found that VBA was not conducting
STAR reviews of redistributed cases (for example, claims assigned to
resource centers for rating). The OIG reviewed a sample of
redistributed claims and found that 69 percent had accurate entitlement
decisions, well below VBA's reported rate of 87 percent for the 12-
month period ending in February 2008. Further, the OIG found that VBA
did not have minimum training requirements for STAR reviewers.
As of March 2010, VBA had taken actions to respond to all of the
OIG's recommendations related to STAR, including (1) implementing
procedures to follow up on cases not submitted by regional offices; (2)
adding a mechanism to the STAR database to remind reviewers of key
decision points; (3) requiring a second-level review of STAR reviewers'
work; and (4) establishing a requirement that STAR reviewers receive 80
hours of training per year. In addition, during fiscal year 2009, based
in part on the OIG's recommendation, VBA also began monitoring the
accuracy of claims decided by rating resource centers as it does for
regional offices. As we noted in our January 2010 report, VBA has
significantly expanded its practice of redistributing regional offices'
disability claims workloads in recent years,\12\ and gathering
timeliness and accuracy data on redistributed claims could help VBA
assess the effectiveness of workload redistribution.
---------------------------------------------------------------------------
\12\ VBA refers to the practice of redistributing claims as
``brokering.''
---------------------------------------------------------------------------
In addition, as the Congress has provided more resources to VBA to
increase compensation and pension staffing, VBA has devoted more
resources to quality review. In fiscal year 2008, VBA more than doubled
the size of the quality assurance staff, allowing it to increase the
scope of quality assurance reviews. VA states that in the 12-month
period ending in May 2009, STAR staff reviewed over 14,000 compensation
and pension benefit entitlement decisions.
Although VBA has taken steps to address deficiencies in the STAR
program, the accuracy of its benefit entitlement decisions has not
improved. The accuracy rate was 86 percent in fiscal year 2008 and 84
percent in fiscal year 2009, well short of VBA's fiscal year 2009 goal
of 90 percent.\13\ VA attributed this performance to the relatively
large number of newly hired personnel conducting claims development
work and a general lack of training and experience. Human capital
challenges associated with providing the needed training and acquiring
the experience these new claims processors need to become proficient at
their jobs will likely continue in the near future. According to VBA
officials, it can take 3 to 5 years for rating specialists to become
proficient.
---------------------------------------------------------------------------
\13\ This rating-related accuracy measure includes original and
reopened claims for disability compensation and dependency and
indemnity (survivor) compensation benefits. Reopened claims include
cases where a veteran seeks a higher rating for a disability or seeks
compensation for an additional condition.
VA Has Taken Actions to Strengthen Efforts to Monitor Consistency of
---------------------------------------------------------------------------
Claims Decisions
VA has taken actions to address deficiencies identified with its
consistency review programs, but it is still too early to determine
whether these actions will be effective. In prior work, we reported
that VBA did not systematically assess the consistency of decision-
making for any specific impairments included in veterans' disability
claims. We noted that if rating data identified indications of decision
inconsistency, VA should systematically study and determine the extent
and causes of such inconsistencies and identify ways to reduce
unacceptable levels of variations among regional offices. Based on our
recommendation, VBA's quality assurance staff began conducting studies
to monitor the extent to which veterans with similar disabilities
receive consistent ratings across regional offices and individual
raters.\14\ VBA began these studies in fiscal year 2008. VBA identified
61 types of impairments for consistency review and conducted at least
two inter-rater reliability reviews, which found significant error
rates.
---------------------------------------------------------------------------
\14\ GAO, Veterans' Benefits: Quality Assurance for Disability
Claims and Appeals Processing Can Be Further Improved, GAO-02-806
(Washington, D.C.: Aug. 16, 2002).
---------------------------------------------------------------------------
In its March 2009 report, the OIG noted that, while VBA had
developed an adequate rating consistency review plan, including metrics
to monitor rating consistency and a method to identify variances in
compensation claim ratings, it had not performed these reviews as
scheduled. In fact, VBA had initiated only 2 of 22 planned consistency
reviews in fiscal year 2008. The OIG reported that VBA had not
conducted these reviews because STAR staffing resources were not
sufficient to perform all of their assigned responsibilities and noted
that VBA's quality review office had not staffed all of its authorized
positions. In addition, the OIG found that inter-rater reliability
reviews were not included in VBA's quality assurance plan. The OIG
recommended that VBA (1) develop an annual rating consistency review
schedule and complete all planned reviews as scheduled; (2) dedicate
sufficient staff to conduct consistency reviews in order to complete
planned workload and reviews; and (3) include inter-rater reliability
reviews as a permanent component of its consistency review program.
VBA reported that it has developed an annual consistency review
schedule and is in the process of conducting scheduled fiscal year 2010
reviews. As of January 2010, VBA also added six staff members to
perform quality assurance reviews. Further, VBA incorporated inter-
rater reliability reviews into its fiscal year 2009 quality assurance
plan. Because VBA has only recently implemented these initiatives, it
is too early to determine their impact on the consistency of claims
decisions.
Conclusion
Over the years, VA has been challenged in its efforts to ensure
that veterans get the correct decisions on disability claims the first
time they apply for them, regardless of where the claims are decided.
Making accurate, consistent, and timely disability decisions is not
easy, but it is important. Our veterans deserve timely service and
accurate decisions regardless of where their claims for disability
benefits are processed. To fulfill its commitment to quality service,
it is imperative that VA continue to be vigilant in its quality
assurance efforts, as this challenge will likely become even more
difficult as aging veterans and veterans returning from ongoing
conflicts add to VA's workload.
Mr. Chairman, this concludes my prepared statement. I would be
pleased to respond to any questions you or Members of the Subcommittee
may have at this time.
GAO Contact and Staff Acknowledgments
For further information about this testimony, please contact Daniel
Bertoni at (202) 512-7215 or [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this testimony. In addition to the contact named
above, key contributors to this statement include Shelia Drake, Jessica
Orr, Martin Scire, and Greg Whitney.
Related GAO Products
Veterans' Disability Benefits: Further Evaluation of Ongoing
Initiatives Could Help Identify Effective Approaches for Improving
Claims Processing. GAO-10-213. Washington, D.C.: January 29, 2010.
Veterans' Disability Benefits: Preliminary Findings on Claims
Processing Trends and Improvement Efforts. GAO-09-910T. Washington,
D.C.: July 29, 2009.
Military Disability System: Increased Supports for Servicemembers
and Better Pilot Planning Could Improve the Disability Evaluation
Process. GAO-08-1137. Washington, D.C.: September 24, 2008.
Veterans' Disability Benefits: Better Accountability and Access
Would Improve the Benefits Delivery at Discharge Program. GAO-08-901.
Washington, D.C.: September 9, 2008.
Veterans' Benefits: Improved Management Would Enhance VA's Pension
Program. GAO-08-112. Washington, D.C.: February 14, 2008.
Veterans' Benefits: Further Changes in VBA's Field Office Structure
Could Help Improve Disability Claims Processing. GAO-06-149.
Washington, D.C.: December 9, 2005.
Veterans Benefits: VA Needs Plan for Assessing Consistency of
Decisions. GAO-05-99. Washington, D.C.: November 19, 2004.
VA Disability Benefits: Routine Monitoring of Disability Decisions
Could Improve Consistency. GAO-06-120T. Washington, D.C.: October 20,
2005.
Veterans' Benefits: Improvements Needed in the Reporting and Use of
Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.
Washington, D.C.: September 30, 2003.
Veterans' Benefits: Quality Assurance for Disability Claims and
Appeals Processing Can Be Further Improved. GAO-02-806. Washington,
D.C.: August 16, 2002.
Veterans' Benefits: Quality Assurance for Disability Claims
Processing. GAO-01-930R. Washington, D.C.: August 23, 2001.
Veterans' Benefits Claims: Further Improvements Needed in Claims-
Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.
Prepared Statement of Ronald B. Abrams, Joint Executive
Director, National Veterans Legal Services Program
Mr. Chairman and Members of the Committee:
I am pleased to have the opportunity to submit this testimony on
behalf of the National Veterans Legal Services Program (NVLSP). NVLSP
is a nonprofit veterans service organization founded in 1980 that has
been assisting veterans and their advocates for thirty years. We
publish numerous advocacy materials, recruit and train volunteer
attorneys, train service officers from such veterans service
organizations as The American Legion and Military Order of the Purple
Heart in veterans benefits law, and conduct quality reviews of the VA
regional offices on behalf of The American Legion. NVLSP also
represents veterans and their families on claims for veterans benefits
before VA, the U.S. Court of Appeals for Veterans Claims (CAVC), and
other federal courts. Since its founding, NVLSP has represented over
2,000 claimants before the Board of Veterans' Appeals and the Court of
Appeals for Veterans Claims (CAVC). NVLSP is one of the four veterans
service organizations that comprise the Veterans Consortium Pro Bono
Program, which recruits and trains volunteer lawyers to represent
veterans who have appealed a Board of Veterans' Appeals decision to the
CAVC without a representative. In addition to its activities with the
Pro Bono Program, NVLSP has trained thousands of veterans service
officers and lawyers in veterans benefits law, and has written
educational publications that thousands of veterans advocates regularly
use as practice tools to assist them in their representation of VA
claimants.
It is clear that the quality of VA adjudications is not
satisfactory and is a major contributor to the size of the backlog.
Because many claims are improperly denied, because many VA adjudicators
are inadequately trained, because many VA regional offices are
improperly managed, because many VA regional offices are inadequately
staffed, and because VA Central Office management has not acted to fix
these problems in any meaningful way, many veterans and other claimants
for VA benefits have to file unnecessary appeals, wait several years
for a BVA remand, and wait for the VA to obtain evidence that should
have been requested during the original adjudication of the claim.
These appeals clog the system and create unneeded work for the VA. Of
course, it would have been better for the VARO to do the work correctly
the first time.
NVLSP believes that the quality of VARO adjudications is much worse
than what is reported by the VA. Valid indicators of the poor quality
of adjudications performed by the VAROs are the remand and reversal
statistics produced by decisions issued by the Board of Veterans'
Appeals (BVA or Board) and decisions issued by the U.S. Court of
Appeals for Veterans Claims (CAVC).
BVA statistics provided by the Board for FY 2009 reveal that Board
decided over 48,000 appeals. The Board granted additional benefits in
24 percent of the appeals and remanded 37.3 percent of these appeals
back to the VAROs. Therefore, 61.3 percent of the VARO decisions that
were appealed and decided by the BVA were either reversed or remanded.
These statistics are about 4 percent worse than the statistics I
reported to you in 2008.
The news gets worse. The BVA, in its rush to make final decisions
and to avoid remands quite often prematurely denies claims that should
have been remanded. The CAVC in a large number of appeals finds fault
with how the VA processed the claim and remands the case back to the VA
to correct claims processing errors. The CAVC reports that of the 3,270
merits decisions it issued in fiscal year 2009, it affirmed the BVA
decision in less than 20 percent of the cases. In over 80 percent of
all the BVA decisions appealed to the CAVC and decided on the merits,
the CAVC either reversed or remanded the BVA decision
Even Chief Justice Roberts was startled when he learned that in
litigating with veterans before the CAVC, the government more often
than not takes a position that is substantially unjustified. These
statistics are shocking and revealing. The VA is required to conduct
ex-parte, non-adversarial claims adjudications in a veteran friendly
environment. (38 U.S.C. 5107(b), 38 C.F.R. 3.102) and provide veteran
claimant's with a non-adversarial adjudication system.
The results of the Legion/NVLSP quality reviews continue to be
discomforting. The American Legion/NVLSP team usually spends a week in
a VARO reviewing the quality of recently adjudicated claims where The
American Legion represented the veteran. The results of these quality
reviews reveal that in many instances claims are improperly denied or
benefits are not paid at the proper rate because the RO was more
concerned about claiming work credit and reducing the VARO backlog than
taking the time to develop and analyze the claim properly. The Legion
team generally finds a much higher error rate than the 12 percent
generally reported by STAR.
The good news is that most of the VA service center: managers,
coaches, decision review officers, and raters that we have interviewed
on these many quality reviews are sincere when they mention the need
for quality adjudications. We have, however, met many VA regional
office officials who are free to admit that their main focus is on
production, not quality.
INDEPENDENT QUALITY REVIEWS
During the past several years the National Veterans Legal Services
Program (NVLSP) has performed, on behalf of The American Legion and
several state departments of veterans' affairs, quality reviews of
decisions issued by several VA Regional Offices (ROs). Our conclusion,
based on these reviews and on information received at our service
officer trainings, is that although the VA is to be commended for
initiatives to stop blatant work measurement (end-product) cheating and
to emphasize quality, the most needed change--full and fair
adjudication of veterans' claims--has not become a reality.
Essentially, while NVLSP commends VBA for its quality initiatives,
we are forced to conclude that these initiatives combined with the STAR
program have not achieved the desired result.
Premature Adjudications Resulting in Adverse Decisions
The most important and pervasive problem facing veterans seeking VA
disability benefits is the eagerness of some ROs to adjudicate claims
before all necessary evidence has been obtained. For example, some ROs
prematurely deny claims based on inadequate VA examinations. In some
cases, even where the VA examiner clearly fails to respond to a
specific question asked by the RO, the examination report is not
returned as inadequate. Instead, the claim is adjudicated and denied on
the basis of the inadequate report. In other instances, claims are
denied before all service medical records are received. Other claims
are sometimes denied before the veteran has a fair opportunity to
submit independent medical evidence. These all-too-frequent cases of
premature denial result from an over-emphasis on timeliness and a lack
of accountability.
We certainly believe that claims for VA disability benefits should
be accurately adjudicated in a timely manner. However, because of a
management emphasis on timeliness, or a perceived emphasis on
timeliness, some VA adjudicators appear to believe that they are
pressured to make premature final decisions. In most instances, we have
discovered that a decision made prematurely is likely to take the form
of a denial of benefits rather than an award of benefits.
Let us make something very clear: The timeliness of VA adjudication
is but one factor in the overall assessment of the VA disability claims
adjudication system. We realize that the overall timeliness statistics
provided by the VBA show that VBA has not met its goal to reduce the
time it takes to adjudicate claims for disability benefits. Even though
the VA has not met its goal in this respect, we urge that you not
overemphasize timeliness to the detriment of quality. It does veterans
little good to have their claims promptly, but inaccurately, denied.
The errors found by STAR and the subsequent trainings based on STAR
findings have not significantly improved this situation.
One may wonder why VA adjudicators would want to prematurely deny
claims. The answer lies in the VA work measurement system. When a claim
for VA benefits is prematurely and inaccurately denied, many veterans
submit new evidence to reopen their claim. The VA considers the new
evidence a second claim and the employee earns double work credit.
Adjudication officers, now called service center managers, have
informed us off-the-record that they feel pressured to prematurely
adjudicate claims because they expect other ROs will do the same, and
they want to show that their productivity and timeliness is as good as
other ROs.
The VA work measurement system should encourage a timely and
accurate adjudication, not just a timely adjudication. Section 2 of
H.R. 3047 would change when VA regional offices (VAROs) can claim work
credit, and was a good bill that would have helped to accomplish this
goal. NVLSP looks forward to reviewing the overdue VA report that was
mandated by PL 110-389.
Adversarial Attitude
Our quality review has identified a systemic attitude problem in
some ROs, which may take one of several forms. One example is that
despite the general tendency to deny prematurely, some ROs ``develop to
deny.'' That is, these ROs consistently seek to develop negative
evidence in cases where all the evidence of record before the RO,
without further development, would reasonably support the grant of
benefits.
Another attitude problem is that some ROs have biases against
certain types of VA claims for benefits. For example, veterans seeking
service connection for mental conditions, entitlement to individual
unemployability benefits, or entitlement to compensation based upon
secondary service connection, in some instances, have to jump over a
higher bar than those who file other types of claims.
In addition, some ROs either refuse to consider or are unaware of
beneficial statutes in Title 38, United States Code. For example our
quality reviews have found that 38 U.S.C. Sec. 1154(b), which provides
in most cases that the statement of a combat veteran about an injury
that occurred during combat will be accepted as true even though there
is no official record of the injury, is sometimes conspicuously
disregarded.
Communication Problems
In many cases, the VA's communication with its veteran-claimants
causes real problems. For example, VA notifications often fail to
provide an adequate explanation of the reasons and bases for the
adverse VA determination. Other communication problems noted by NVLSP
are:
Inadequate development letters (development letters
are sent by the VA to the veteran and his or her
representative, asking for further information or evidence)
that do not comply with VA's guidance that letters should
clearly tell the claimant what evidence is needed and what
exactly has to be done to establish entitlement to the benefit
sought (see M21-1, Part III, para. 1.04a.); and
Telephone communication with the veteran that is not
monitored or sanctioned by the veteran's representative (the VA
does not even inform the representative that it is about to
contact the representative's client).
Widespread Errors
The following is a list of a systemic pattern of errors that we
have noticed during our quality review checks. These errors are:
Assignment of erroneously low disability ratings for
service-connected mental conditions;
Erroneous denial of claims for service connection for
mental conditions;
Failure to consider 38 U.S.C. Sec. 1154(b);
Erroneous denial of claims of individual
unemployability;
Inadequate requests for medical opinions (for
example, the standard of proof in the VA claims process is
rarely explained to VA doctors, and in many instances
conclusions regarding critical facts are not communicated to
doctors who are asked to provide medical opinions); and
Non-responsive VA examination reports (for example,
some VA examiners do not comply with the AMIE protocol, and
other examiners fail to respond to specific questions), coupled
with the acceptance of these inadequate examination reports by
ROs.
In general, there is a lack of coordinated local (RO) quality
control and a subsequent failure to act on recognized patterns of
errors.
NVLSP Recommendations
Based on the foregoing observations, NVLSP makes the following
suggestions:
VA's work measurement system should be altered so
that quality as well as timeliness are twin concepts that
together drive the system.
To provide VA quality control with ``teeth'' and
prevent end-product and work measurement abuses, an aggressive
independent quality control should be performed.
VBA should conduct regular meetings with its
stakeholders to inform them of any actions VBA has taken to
correct systemic adjudication problems. The stakeholders should
be informed about the patterns of errors identified nationally,
the ROs where there are significant problems, VBA's plans to
correct these problems, changes in management, progress reports
on previous initiatives, and an invitation for the stakeholders
to participate and coordinate in the correction of problems.
VA should institute a system of awards and
disincentives for managers and adjudicators. VA managers and
adjudicators who perform accurate and timely work should be
rewarded. Managers who do not perform adequately should be
appropriately chastised.
VA employees who do a good job should be paid a
reasonable salary, receive bonuses and be promoted.
VA management should more clearly communicate with
its employees what it wants from them. If management focuses on
quality as well as efficient work, veterans will be better off.
NVLSP acknowledges that the adjudication of claims for VA benefits
is very complicated. However, we believe the stakeholders want to help
correct adjudication problems. We would be happy to meet regularly with
the VA to talk about the problems we have identified and suggested
solutions.
We would like to commend VBA managers for initiatives in reducing
outright end-product and work measurement dishonesty and efforts to
emphasize quality. While these efforts are commendable, it is time to
see results.
Our experience has taught us that VA managers are reasonable people
who want to do the right thing. These managers care about veterans and
know that the claims adjudication system is not working properly. To
help these managers we ask you to encourage the VA to make at least the
most necessary changes, alter VA's work measurement system, institute
an aggressive quality control program, and support its efforts to
coordinate with its stakeholders.
We appreciate the opportunity to provide the subcommittee with this
testimony.
Thank you.
Prepared Statement of John L. Wilson, Assistant
National Legislative Director, Disabled American Veterans
Mr. Chairman and Members of the Committee:
The Disabled American Veterans (DAV) has consistently stated that
the keys to successfully reforming the veterans' benefits claims
process are training and accountability, two elements central to
producing quality results for veterans. However, today the Veterans
Benefits Administration (VBA) remains production driven from the Monday
morning workload reports to personnel awards.
The DAV strongly believes that quality should be rewarded at least
on parity with production. However, in order for this to occur, VBA
must first implement and then inculcate a comprehensive quality control
program to complement its quality assurance program.
VBA's primary quality assurance program is the Systematic Technical
Accuracy Review (STAR) program. The STAR program can identify three
types of errors--benefit entitlement, decision documentation/
notification, and administrative. STAR looks at whether a proper VCAA
pre-decision ``notice'' was provided and whether the rating decision
was merited based on the available evidence.
Under the STAR program, VA reviews a sampling of decisions from
regional offices and bases its national accuracy measures on the
percentage with errors that affect entitlement, benefit amount, and
effective date. This is a technical review to ensure a variety of
transactions are properly carried out. Inconsistency in technical
accuracy may signal several types of processing problems, including:
uneven or insufficient understanding of governing criteria, operating
rules that are too vague and allow an overly broad interpretation
which, in turn, leads to significant variance, or outright
arbitrariness in decision-making. Obviously, VA must detect
inconsistencies before the cause or causes can be determined and
remedied.
The STAR program was implemented in the late 1990s. It was intended
to be a national quality assurance program that would assist VBA in
identifying processing vulnerabilities and error trends. It was
designed to draw statistically significant case samples from all
regional offices, using stratified random sampling algorithms. Using
this tool, VBA could review a statistically valid case sample each
year, and calculate its national error rates. Using the STAR program
was also intended to identify major national error trends, so the
Compensation and Pension (C&P) program could initiate corrective
measures. Such corrective measures could include training, improved
procedural guidance, or automated system improvements.
The STAR program was not designed to provide evaluative data at the
working unit or individual level. There are two major reasons for this.
First, the sample sizes used in STAR are small. While totals may be
significant at the national level, breaking out the data to Regional
Offices may not provide numbers significant enough to ascertain a trend
with an individual employee. Second, the STAR program essentially
assesses the outcome, not the process of getting there. So, a claim
that took two years to process because of piece-meal development would
not have an error called if the resulting decision was correct and all
pre- and post-notification requirements were met. Such processing
delays would fall under the purview of quality control in-process
reviews and not quality assurance programs such as STAR.
Quality control findings from local in-process reviews would assist
a station in assessing overall performance towards achieving the goal
of timely and accurate decisions on veterans' claims. VBA recognized
the importance of such quality control when the STAR program was
created to replace the ineffective Statistical Quality Control program.
At that time, VBA requested funding to implement a local C&P quality
control program. That program--called Systematic Individual Performance
Assessment (SIPA)--was announced in 2000 as a new initiative to monitor
individual performance. Under this program, the VA would review an
annual sample of 100 decisions for each adjudicator to identify
individual deficiencies, ensure maintenance of skills, promote accuracy
and consistency of claims adjudication, and restore credibility to the
system. The reviewers would have performed related administrative
functions, such as providing feedback on reviews, maintaining reports,
and playing a role in employee development and ongoing training.
Unfortunately, the VA abandoned this initiative during 2002, and
proficiency is now apparently subjectively assessed by supervisors
based on their day-to-day perceptions of employee performance. The SIPA
program may have been abandoned due to inadequate resources. Without
any quality assurance review on the individual level, the VA is
unlikely to impose effective accountability down to the individual
adjudicator level, where it must go if optimum quality is expected.
VBA elected instead to install a much reduced quality control
procedure, where coaches' review several cases per month for Veterans
Service Representatives (VSR) and Rating Veterans Service
Representatives (RVSR), as their quality control mechanism. This hybrid
process has not provided adequate individual accountability, or
sufficiently robust data to identify local process improvements.
Coaches typically do not have the time to manage day-to-day operations
and pull case files for ad hoc reviews of employees.
With significant attention on the C&P claims backlog, it is
understandable that VBA wants to maximize its personnel and resources
to direct claim decision-making. However, technical accuracy is
arguably the most important component of the C&P claims process.
Pushing cases faster will not help if significant numbers of cases are
done wrong.
VBA needs to elevate quality to the highest priority. This means
they should dedicate adequate resources to both quality assurance and
quality control programs. The VSRs, RVSRs and local management teams
need to understand that high quality work will be recognized and
rewarded. Further, they need to understand that there will be clear
thresholds for individual quality, repeated errors will be identified
and associated with their processor, and that there will be appropriate
consequences for insufficient accuracy rates.
The STAR program was evaluated by the VA Office of Inspector
General as part of its review of compensation rating accuracy in March
2009, in the report titled ``Audit of Veterans Benefits Administration
Compensation Rating Accuracy and Consistency Reviews.'' The OIG
determined that VBA's STAR program does not provide a complete
assessment of rating accuracy.
During the 12-month period ending in February 2008, VBA's STAR
process did not effectively identify and report all errors in
compensation claim rating decisions. VBA identified a national
compensation claim rating accuracy of 87 percent. Of the approximately
882,000 compensation claims measured by STAR reviewers, VBA estimated
that about 87 percent were technically accurate. The OIG, on the other
hand, reviewed a random sampling of cases that had also been reviewed
by STAR reviewers and found additional errors. They projected an
accuracy rate of only 78 percent. They also audited brokered cases. Of
that sampling, they found an accuracy rate of 69 percent. Combining the
audit of brokered claims with those STAR reviewed claims, results in a
projected accuracy rate of about 77 percent of claims. The OIG
determined that this equates to approximately 203,000 claims in that
one year alone where veterans' monthly benefits may be incorrect.
The OIG found that STAR reviewers did not identify some of the
missed errors because they either did not thoroughly review available
medical and non-medical evidence, did not identify the absence of
necessary medical information, or inappropriately misclassified benefit
entitlement errors as comments.
These findings are, on their surface, a result of STAR reviewers
not finding all the errors in the cases they reviewed. They also point
to the need for greater management oversight and an effective formal
training program for the STAR reviewers. STAR reviewers could benefit
from formal training; however STAR managers have said that current
workload requirements do not allow for the amount of training time
necessary. This is a common theme for VBA that underlies even STAR
reviews--quantity over quality--even in the area that is supposed to
ensure quality of at least a technical nature.
The need for a quality control program as an adjunct to the STAR
program can also be seen when considered through a review of the Board
of Appeals for Veterans Claims' Summary of Remands. The summary
represents a statistically large and reliable sample of certain
measurable trends. The examples must be viewed in the context of the VA
(1) deciding over 880,000 cases per year; (2) receiving over 133,000
Notice of Disagreements; and (3) over 49,000 appeals to the Board. The
examples below are from fiscal year (FY) 2009:
1. Remands resulted in 801 cases because no ``notice'' under
section 5103 was ever provided to the claimant. In addition,
there were 4,048 remanded for inadequate or incorrect notice,
some of which may result from the current generic notice
letters sent VBA. The DAV continues to call for changes to
these letters to include more specific information, which could
help lower the incidence of this error.
2. VA failed to request for Service Medical Records in 1,739
cases and failed to request for personnel records in 1,511
cases. These numbers are disturbing because initially
requesting a veteran's service records is the foundation to
every compensation claim.
3. The Board remanded 7,814 cases for failure to request VA
medical records. The disturbing factor here is that a VA
employee can usually obtain VA medical records without ever
leaving the confines of one's computer screen.
4. Another 3,187 cases were remanded because the claimant had
requested a travel board hearing or video-conference hearing.
Again, there is a disturbing factor here. A checklist is
utilized prior to sending an appeal to the Board that contains
a section that specifically asked whether the claimant has
asked for such a hearing.
The examples above totaled 19,100 cases or 34 percent of appeals
reaching the Board, all of which cleared the local rating board and the
local appeals board with errors that are elementary in nature. Yet they
were either not detected or they were ignored. Many more cases were
returned for more complex errors. Regardless, a 34 percent error rate
on such basic elements in the claims process involving VBA's most
senior rating specialists is simply unacceptable.
The problem with the current accountability system is that VBA
employees who commit such errors are usually not held responsible. With
no incentive to prevent such errors and a constant focus on production,
quality will continue to decline.
DAV agrees with the VA OIG that VBA could improve the STAR program
by establishing: a mechanism to ensure STAR reviewers evaluate all
documentation related to the claim selected for review; a requirement
that all STAR reviewer comments receive a second review to make sure
the reviewer appropriately recorded the comment instead of a benefit
entitlement error; procedures to review brokered claims as part of the
STAR program; and minimum annual training requirements for each STAR
reviewer that are comparable to regional office rating staff training
requirements.
In addition, DAV recommends that VBA establish a quality control
program that looks at claims in-process in order to determine not just
whether a proper decision was made, but how it was arrived at in order
to identify ways to improve the system. Combining results from such
quality control reviews with STAR's quality assurance results and the
data from remands from the Board of Veterans' Appeals and the Court of
Appeals for Veterans Claims could yield valuable information on trends
and cause of errors. If the data from all such reviews could be
incorporated into a robust IT system, proper analysis of such data
would provide management and employees important insights into
processes and decisions. This in turn would lead to quicker and more
accurate decisions on benefits claims, and most importantly, to the
delivery of all earned benefits to veterans, particularly disabled
veterans, in a timely manner
That concludes my testimony. I would be pleased to answer any
questions the Committee may have.
Prepared Statement of Raymond C. Kelley, National
Legislative Director, American Veterans (AMVETS)
Chairman Hall, Ranking Member Lamborn, and members of the
subcommittee, thank you for the opportunity to appear before you today
to provide AMVETS' views regarding VBA's Systematic Technical Accuracy
Review (STAR) Program.
The Systematic Technical Accuracy Review (STAR) Program is VBA's
compensation and pension accuracy measurement system. AMVETS agrees
with the VA's Office of Inspector General's March 12, 2009 report that
identified eight issues that will improve the process of reviewing
claims for errors. And AMVETS is pleased to see VBA is taking action to
correct these issues. AMVETS is concerned though, with what is done
with the information that is gleaned from STAR.
For the STAR program to truly be effective, AMVETS believes three
things must be done. First, STAR must be enhanced so trends in errors
can be easily identified by the regional offices. With this
information, VBA must hold ROs accountable for failures in accuracy and
insist that ROs develop improvement strategies that include training
for all accuracy issues. Second, VBA must change its culture of
timeliness and strive for a culture of accuracy. Wither or not STAR is
completely accurate in its reviews is important but not nearly as
important as what is done to ensure the same mistakes are not repeated.
Third, OIG must conduct periodic review of the STAR program to ensure
that STAR's accuracy ratings are within a 3 percent margin of error.
Even though AMVETS believes the STAR program is effective, we
believe it should be expanded to ensure that specific programs, such as
the Benefits Delivery at Discharge (BDD) program and specific
conditions can be tracked for anomalies that occur, so improvement
strategies and specific training can be implemented. When the Veterans
Claims Assistance Act (VCAA) was introduced in the first quarter of FY
2002, nearly half of all errors were VCAA related. VBA had the ROs re-
train their claims processors and in the last 8 months of the FY, these
types of errors were reduced by one-third. This type of accountability
needs to be the rule and not the exception.
As you know, the STAR program identifies errors in claims decisions
each month through random sampling of each Regional Office (RO). The
results of the reviews are sent to the ROs. Errors that are found are
to be corrected by the RO that made the decision. The corrections are
made, but all too often, the ROs don't implement strategies to ensure
that claims processors don't continually repeat the same mistakes.
AMVETS believes the reason these strategies are not developed is that
the culture within the regional offices is one of timeliness and not
one of accuracy. STAR has consistently found that 20 to 25 percent of
claims are in error over the past decade, but VBA has done little to
ensure that mistakes that are made in the regional offices are
understood and that strategies for improvements are put in place. On
the other hand, VBA does require ROs to develop corrective action plans
if they do not reach strategic goals for production, inventory and
timeliness. This paradigm must be flipped.
The March, 2009 OIG Report clearly defined the gaps in the STAR
program that have caused the 10 percent disparity in compensation and
pension rating accuracy. AMVETS believes VBA is taking action to close
these gaps. However, AMVETS believes it is important to have this
accuracy fall within a 3 percent margin of error. Therefore, AMVETS
requests that OIG conduct a follow-up to the 2009 report to ensure
VBA's gap solutions are productive, and that OIG continue to conduct
periodic reviews of STAR to ensure the program reaches and maintains a
3 percent margin of error.
Mr. Chairman, thank you again for providing AMVETS the opportunity
to present our views on the STAR program. This concludes my testimony
and I will be happy to answer any questions you may have.
Prepared Statement of Ian C. de Planque, Assistant Director,
Veterans Affairs and Rehabilitation Commission, American Legion
Mr. Chairman and Members of the Subcommittee:
I appreciate this opportunity to express the views of the nearly 3
million members of The American Legion on the Department of Veterans
Affairs' (VA's) Systemic Technical Accuracy Review (STAR). VA
maintains, as stated by Acting Under Secretary for Benefits Michael
Walcoff in a January 3, 2010 appearance on 60 Minutes, that: ``We
stress over and over again to our employees that quality is our number
one indicator, that's absolutely a requirement for successful
performance.'' STAR has the potential to be an effective tool to
achieve this end; however, VA is currently falling short in their
effective implementation of this tool. This failure can, and should be
corrected. The American Legion recommends VA consider taking focused
actions to reverse this current situation.
On March 12, 2009, VA's Office of the Inspector General (VAOIG)
issued Report No. 08-02073-96, Audit of Veterans Benefits
Administration Compensation Rating Accuracy and Consistency Reviews.
This report detailed numerous flaws in the current implementation of
the STAR system. Although VA acknowledged some of those criticisms, The
American Legion is unaware of any significant corrective actions by VA
leadership.
The VAOIG audit was conducted to evaluate the Veterans Benefits
Administration's (VBA's) quality assurance program, of which STAR is a
component. Some of the more troubling findings included:
VBA's STAR process did not effectively identify and report
errors in compensation claim rating decisions. VBA identified a
national compensation claim rating accuracy of 87 percent for
the 12-month period ending February 2008. We projected that VBA
officials understated the error rate by about 10 percent, which
equates to approximately 88,000 additional claims where
veterans' monthly benefits may be incorrect. In total, we
projected about 77 percent (approximately 679,000) of the
almost 882,000 claims completed were accurate for the 12-month
period ending February 2008. The 87 percent rate is not
comparable to the accuracy rate VBA reports in the Performance
and Accountability Report because that rate includes pension
claims. Further, this accuracy rate only included errors that
affected a veteran's benefits. STAR identifies, but does not
report, other errors related to the rating decision's
documentation, notification, and administration.
That 88,000 claims are potentially incorrect is certainly troubling
and clearly unacceptable to The American Legion. This possible trend
will undoubtedly add to the current claims backlog challenge being
aggressively addressed by VA Secretary Eric Shinseki. Further
problematic is the inaccuracy of VA's system of reporting its own
errors and isolating problem areas and/or responsible individuals in
the claims rating process. Without external observation, such as this
audit by VAOIG, such errors may never have come to light and an
inaccurate picture of the overall flaws in the disability system may
never have been recorded. More importantly, corrective actions cannot
be taken in a timely manner.
Furthermore, the VAOIG audit reports:
VBA officials planned to conduct 22 reviews in FY 2008
consisting of 20 grant/denial rate and 2 evaluation reviews.
However, they only initiated two grant/denial rate reviews and
these were not completed until December 2008. Furthermore, VBA
officials did not initiate either of the two planned evaluation
reviews to analyze and improve the consistency of disability
compensation ratings and to reduce the variances between
states.
Even where problem areas or potential problem areas are identified,
VA is not conducting followup or analysis on their projected plans.
Effective as STAR may be, if it is not implemented as intended, it
cannot help correct problems. Aggressive congressional oversight would
seem essential to assuring timely application of these procedures.
VAOIG concluded in this instance:
Without an effective and reliable quality assurance program,
VBA leadership cannot adequately monitor performance to make
necessary program improvements and ensure veterans receive
accurate ratings. Further, without implementing an effective
rating consistency program, VBA officials cannot successfully
assess or prioritize the improvements needed for claim rating
consistency.
If this was the only problem discovered, it would be enough to call
into question the effectiveness of the system as currently configured.
However, with further investigation it becomes clear that the current
use of STAR does not accurately assess the claims benefits operational
picture.
The report states:
In addition, VBA officials excluded brokered claims from STAR
reviews. We reviewed a sample of brokered claims and found an
accuracy rate of 69 percent.
The brokering of claims is increasingly becoming an integral part
of the way VBA conducts operations today. Brokering is a system
utilized by VA to shift claims from Regional Offices with a larger
workload to Regional Offices with less substantial workloads to
increase the ability to process a greater number of claims overall.
Brokering claims also raises other serious issues that merit further
investigation. Accountability is one of the major concerns. The
American Legion believes that without STAR analysis of the brokered
claims there is a lack of accountability for these claims, which is
deeply troubling.
How effective is the actual STAR analysis? This quote from the
report raises additional unsettling issues for The American Legion:
STAR reviewers did not identify some of the missed errors
because they either did not thoroughly review available medical
and non-medical evidence (or identify the absence of necessary
medical information), or they inappropriately misclassified
benefit entitlement errors as comments.
The American Legion asks how can a system intended to assess the
quality and accuracy of actions by VBA be effective, if evaluators do
not have access to the full file information required to make the
decision.
Even with these errors and flaws in the current system, such a
potentially powerful tool for self correction should not be abandoned
completely. Perhaps a better solution is to look for ways, both
internal and external, in which the present system could be adapted to
effectively improve the use of the existing components.
In order to rectify existing problems within STAR, The American
Legion suggests VA could make improvements by implementing a three-step
process for change.
1. Compile results nationwide of rating errors identified by
STAR evaluations. Currently there appears to be no systemic
method to track errors. This data could be critical to identify
patterns, whether in an individual sense, on a Regional Office
(RO) level, or nationally across the scope of VA operations. If
this information is currently gathered, it does not appear to
be used for analysis to detect trends which could indicate
systemic problems within VA. This data, coupled with data
gathered on errors at lower levels from the Board of Veterans
Appeals (BVA) and the Appeals Management Center (AMC), would be
an excellent tool to assess VA's quality overall by supplying
details that could indicate problem areas.
It is not enough to know what VA's accuracy rate across the
system is. VA must also know where are their greatest areas of
concern in order to determine areas that could be addressed and
provide the most efficient and effective use of resources.
2. Utilize data and patterns gathered from STAR to plan and
implement a program of training. Adequate and effective
training is a key concern noted often in the adjudication of
claims. This data could show specific topics in which employees
need additional training to improve accuracy of ratings. Such
information could help VA leadership craft an effective
training schedule to maximize the training resources. Future
training would not be generalized, but rather targeted to fix
specifically identified problems. This focused approach would
assure that training resources would be used to the greatest
possible impact.
3. Augment STAR for accuracy with outside oversight to ensure
the effectiveness of the program. One obvious complaint about
the current implementation of STAR is a lack of adequate
followup. The American Legion believes third-party oversight
offers the opportunity to provide impartial and critical
followup to assure compliance. The American Legion strongly
advocates the use of external oversight for validation.
The American Legion recommends VA should closely monitor and record
common errors from previous BVA and AMC remands and grants. Such status
monitoring and documentation could help VA identify errors consistently
made in a Regional Office or by specific individuals that are
eventually recognized and corrected later in the process. The American
Legion believes this would help isolate trends needing immediate
corrective action. Unless there is a mechanism for identifying,
compiling, reporting and correcting those errors, the office(s) or
individual(s) making repeated mistakes continue. This concept also
applies to the systemic review nationwide of claims by STAR. The
American Legion believes if the error reporting of all three entities
is combined, it would constitute an even more effective pool of data to
enhance VA's analysis of internal problems.
As Acting Under Secretary Walcoff stated, that quality of
processing and not quantity of processing is the primary concern of VA,
then an overarching and thorough assessment of every aspect of claims
rating in which quality falls short is essential to rectifying the
system. The American Legion believes there is plenty of data available
to be harvested that would reveal exactly where VA is making its most
common mistakes. There is no question that there are issues where VA
struggles more than others. The American Legion recommends VA should
effectively use this data to focus available resources where VA could
improve accuracy. VBA should analyze where the process is weakest and
use the analysis to develop meaningful training to improve performance.
The most obvious solution to improve accuracy is meaningful and
timely training of employees. Utilizing the wealth of information on
weak points that can be generated by compiling data on common errors,
VA can create a targeted training plan focused on weakness most in need
of improvement. If focused and meaningful, VA's continuing education
and training can maximize effectiveness of performance.
When The American Legion conducts its quality assessment visits to
Regional Offices around the country, one of the most common complaints
from VA personnel, who actually process the claims, is the lack of
useful training. An all too often made complaint from VA personnel is
that the training is repetitive, unclear and not in areas that they
struggle with most. Employees identify segments of the rating schedule
that they find:
most confusing,
raise questions about compliance with notification
due to the Veterans' Claims Assistance Act (VCAA), or
questions on how to better communicate with examining
physicians to achieve exams that provide enough information to
adequately rate a claim.
STAR review, in conjunction with data gathered from the BVA, AMC
and perhaps employee review can find where VA employees are struggling
most and get them the training they need to get the job done right the
first time. The American Legion is confident that VA employees want to
get the claims right, but struggle under the dual burdens of inadequate
training in a complex system and time pressures that place a greater
value on production rather than on quality. VA recently increased the
required training time annually from 80 hours to 85 hours. More of this
training will be standardized. By reforming training with regard to
common errors identified through internal review processes such as
STAR, VA can build on its commitment to more effective and meaningful
training and provide its employees all the skill sets and knowledge
they need to do their jobs right--the first time.
The fact that VA has been inconsistent following up on corrections
identified by STAR highlights the need for outside oversight. Even when
problems have been identified, VA has fallen short at the goal of
rectifying those problems. VA states a target of 90 percent accuracy,
yet VAOIG has pointed out that VA's internal numbers are artificially
inflated by inaccurate self-reporting and still fall short of those
numbers. Incentives for bonuses in the management of Regional Offices
are all numbers-based and driven by workload measurement (quantity not
quality). Consequently, there appears to be little to no incentive
within VBA to accurately report numbers that would work counter to
achieving these bonus goals. The American Legion recommends that
external audits and congressional oversight would greatly help redirect
emphasis on quality over quantity.
Under the provisions of Public Law 110-389, the Veterans' Benefits
Improvement Act of 2008, the VA Secretary is to contract with an
independent entity to conduct, over a three-year period, an assessment
of the VA quality assurance program. In addition, the VA Secretary is
to report the entity's findings and conclusions to Congress. The
American Legion looks forward to this report, but would encourage VA to
aggressively strive for marked improvements in accuracy before, during
and after this mandated assessment. The American Legion is confident
that VA is capable of internal improvement. There is sufficient
information now to begin making STAR work the way it should for VA
employees and veterans.
The American Legion stands ready to answer any questions of this
Subcommittee and thank you again for this opportunity to provide
testimony.
Prepared Statement of Bradley G. Mayes, Director,
Compensation and Pension Service, Veterans Benefits
Administration, U.S. Department of Veterans Affairs
Mr. Chairman and Members of the Subcommittee, thank you for
providing me with this opportunity to discuss the Veterans Benefits
Administration's (VBA) Quality Assurance Program and the positive
effect it has on the processing of Veterans' disability claims. Joining
me today are Edna MacDonald, Acting Deputy Director of Compensation &
Pension Service for Policy and Procedures, and Terence Meehan, Director
of VBA Employee Development and Training.
The Subcommittee has indicated a special interest in our Systemic
Technical Accuracy Review (STAR) Program. However, I want to emphasize
that STAR is only one of four tiers of our multi-faceted national
Quality Assurance Program. The STAR component focuses on claims
processing accuracy, while the other three components address regional
office oversight, rating consistency, and special focus reviews. Along
with the STAR Program, these components collaborate to ensure high
quality and consistent decisions for Veterans. Before discussing the
STAR program, I will briefly discuss the three other facets of our
Quality Assurance Program, which endeavors to ensure that compensation
and pension benefits are provided in a timely, accurate, and consistent
manner.
Quality Assurance Program
Oversight
The oversight component involves compliance oversight visits to
regional offices by members of Central Office site survey teams. Each
regional office is visited on a three-year rotating basis. All
operations of the visited regional office are reviewed and evaluated,
with recommendations provided for improving claims processing
efficiency, accuracy, and timeliness. Additionally, the site visit team
assesses whether the regional office is in compliance with VBA policy
and procedures and consistent with national standards. A Web site has
recently been created to share the ``best practices'' identified during
site visits.
Consistency
The consistency component is based on mining of source data from
the VBA corporate database that houses information from all regional
office rating decisions. Given the possibility for variation in
disability decisions, it is incumbent on VA to ensure program integrity
by having a credible system for identifying indications of
inconsistency among its regional offices and then remedying any
inconsistencies found to be unreasonable. The two key pieces of
information obtained are the grant rate and the evaluation
distribution.
Data analysis of recently completed rating decisions identifies the
most frequently rated diagnostic codes, and plots both the grant/denial
rate and evaluation assigned across all regional offices. This
information focuses on rating decisions for specific disabilities and
provides a method to evaluate consistency and accuracy on a regional
office level. A focused review of files from the regional offices that
are above or below the national average is conducted with the goal of
identifying causes for the statistical anomaly. Once root causes are
identified, the regional offices are notified of any recommendations,
which may include training to correct problems or improper procedures
identified during the review.
Special Focus Reviews
Special focus reviews address topics of special interest to VBA or
other stakeholders where accuracy and consistency are an issue and are
conducted as needed in support of VA's mission and needs. They address
a specified purpose or type of claim and may involve a nationwide
review or a review of work assigned to a specific regional office. The
ad hoc reviews can be one-time or recurring in nature. For example,
consolidation of the processing of radiation claims began on October
16, 2006. In 2008 the STAR staff conducted a special focused review of
radiation claims completed between October 2006 and October 2007. The
findings from this review provided a means for assessing the
consolidation effort. The review found the overall accuracy and
processing timeliness of radiation claims improved following
consolidation.
STAR
STAR is the quality assurance component that focuses on claims
processing accuracy. STAR reviews are focused on outcomes for Veterans
rather than specific processes by which the outcomes are reached. STAR
reviews evaluate the quality of the rating decision product that VBA
provides for Veterans. From the Veteran's perspective, there is an
expectation that we understand the claim, evaluate it accurately and
fairly, and provide proper compensation under the law. The purpose of
STAR reviews is to ensure that rating decision outcomes meet these
expectations.
The STAR system includes review of three types of work: claims that
usually require a rating decision; authorization work that does not
generally require a rating decision; and fiduciary work. The focus on
rating-related decisions and authorization actions is a benefit
entitlement review using a structured checklist to ensure all issues
were addressed, claims assistance was provided, and the decision was
correct (to include a correct effective date). Accuracy results are
calculated on the results of the benefit entitlement review. STAR
findings provide statistically valid accuracy results at both the
regional office and national level. In addition, quality reviews of the
accuracy of VBA examination requests and VHA examination reports are
conducted in collaboration with the Compensation and Pension
Examination Program (CPEP) office.
VBA continues to focus on expanding and improving the STAR Program.
In 2008 the STAR staff was consolidated to Nashville, which provided
space for expansion and allowed aggressive recruitment for more STAR
staff Since then, STAR has completed an extensive expansion effort,
more than doubling the staff and increasing the sample size to obtain a
statistically valid sample at the regional office level. In 2010,
quality review of fiduciary cases was transferred to Nashville. During
fiscal year (FY) 2009, 24,747 cases were reviewed for rating and
authorization accuracy, and 3,671 cases were reviewed for fiduciary
accuracy. The targeted number of cases for STAR review in FY 2011 is
37,932. The STAR sample was expanded in 2009 to include a review of
brokered work completed by VBA's 12 Resource Centers and one Tiger Team
(a Cleveland-based team focus on processing a unique subset of claims),
and sampling was increased for the Pension Management Centers to allow
measurement of pension entitlement decisions. Ongoing reviews of the
Disability Evaluation System cases and Appeals Management Center cases
became part of the monthly compensation quality sample in FY 2009.
STAR error trends are identified and used as training topics.
Training continues to be a priority and is conducted using a variety of
methods, including a monthly national Quality Call, where Compensation
& Pension Service's training, policy, and procedures staffs collaborate
with the STAR staff to address national error trends identified in STAR
assessments.
To assure accuracy of STAR findings, a second-level peer review of
all comments was implemented in FY 2009. Regional offices are provided
explanations on all error calls, and they are required to take
corrective action. On a quarterly basis, regional offices are required
to certify to VBA headquarters the corrective action taken for all
errors identified by STAR. The reported actions are validated during
the oversight visits conducted by the site survey teams.
Section 224 of the Veterans' Benefits Improvement Act of 2008 (PL
110-389) required VA to contract with a third-party entity to conduct
an assessment of the quality assurance program, evaluate a sample of
employees' work, measure the performance of VA regional offices and
accuracy of rating, assess employee and manager performance, and
produce data to help identify trends. VBA contracted with the Institute
for Defense Analyses (IDA) to conduct this assessment. IDA furnished
preliminary findings concerning its evaluation of the national accuracy
of work, the regional office accuracy of work, the accuracy of
disability ratings, and the consistency of disability ratings. Its
preliminary findings include an assessment that the current STAR
accuracy review program is adequately designed to estimate, within a 5
percent margin of error, the percentage of claims completed that
contain errors both nationally and by regional office. IDA is
continuing to determine options for identifying the accuracy and
consistency of disability rating decisions.
VBA anticipates that further improvements in the STAR Program, as
well as other components of the Quality Assurance Program, will result
from recommendations from the IDA study. We are looking forward to the
recommendations from IDA and working collaboratively with them to
further improve our Quality Assurance Program.
Quality Assurance and Training
Training Improvements
VBA is committed to using the error trends and accuracy findings to
improve overall quality. VBA uses nationwide error patterns identified
by STAR reviews, as well as information from other components of the
Quality Assurance Program, to adjust and develop the employee training
curricula.
All employees, regardless of training level, must receive 80 hours
of instruction annually. Instructional methods may include Training
Performance Support System (TPSS) modules, lectures, or practical
application exercises. For intermediate and journey-level employees,
the 85 hours must include 40 Core Technical Training Requirement (CTTR)
hours. These involve standardized training curricula of essential
topics and information. Employees must complete an additional 20 hours
of training from a list of standardized topics provided by VBA. The
final 20 hours may be used by regional offices to train on local issues
and areas of concern. This approach ensures that new and experienced
employees are grounded in standardized claims processing fundamentals.
Data from STAR reviews, consistency reviews, special focus reviews,
and regional office site visits, are used to develop training for our
new hires, as well as our intermediate and journey-level employees.
Claims processing personnel are timely informed of errors and
inconsistency trends and provided with constructive feedback, including
instructions on how to avoid such errors in the future. The error
trends identified in STAR reviews provide us the information we need to
assess the effectiveness of our training programs and make necessary
adjustments. This promotes our goal of providing accurate, fair, and
consistent claims processing.
Office of Employee Development and Training
To ensure that our training is up to date and incorporates the
lessons learned from the Quality Assurance Program, VBA has a robust
training evaluation program conducted by the Technical Training and
Evaluation section of the Office of Employee Development and Training.
With the assistance of professionally qualified outside evaluators,
this office has undertaken two formal evaluations of the centralized
Challenge program for newly hired claims personnel. The first formal
evaluation, conducted during 2007-2008, included visits to 16 regional
offices and surveys with 1,405 respondent trainees. This led to the
following three recommendations, which were adopted in reformulating
the Challenge curricula:
Re-sequencing the content to eliminate redundancy and
make better use of resources,
Providing standardized hands-on claim processing
during the centralized training portion, and
Creating more formal requirements for delivery and
compliance.
The second evaluation of Challenge training began at the end of
2009. Preliminary findings show that the changes made to Challenge
significantly improved its effectiveness, as demonstrated by new
trainees now being able to correctly process simple cases immediately
after completing the centralized portion of the training. TPSS training
was separately evaluated during 2005-2006 and 2006-2007, and then
subsumed into the Challenge evaluations because of its high usage
during the initial training phase.
In total, the Office of Employee Development and Training visited
34 regional offices during the programmatic evaluations of Challenge
and collected more than 3,200 surveys. This is in addition to the
hundreds of individual participant Challenge surveys done in the course
of the three phases of Challenge classes.
Performance support training tools allied to TPSS modules continue
to show high and increasing usage, reflecting their utility to the
field. For example, the Medical Electronic Performance Support System
provides computerized visual images of the various human body systems.
It was developed with STAR review input to assist with identifying the
appropriate rating codes associated with different body systems and to
facilitate medical examination requests. This tool had 633,585 unique
user sessions in FY 2009, a 32 percent increase from FY 2008. Another
training tool, the Veterans Service Representative Assistant, was
designed to assist with claims development and had 34,696 unique
sessions in FY 2009, a 62 percent increase from FY 2008. The increased
use of these performance support tools may be attributable to a growing
population of new claims processors who have learned about them during
Challenge training, where their use is integrated into practical
applications.
VBA continues to improve its assessment of training program quality
and effectiveness by collecting more timely feedback from participants.
Improvements in the last two years, for instance, include adding
surveys during the home-station segments of Challenge training and
promulgating an official requirement to complete surveys for both
Challenge and CTTR training.
Conclusion
The VBA Quality Assurance program has undergone significant change
over the past several years and has become more comprehensive by
expanding the type and breadth of cases reviewed. This is a journey
that we are on, and we look forward to learning from the IDA review and
working with our stakeholders to continuously improve our Quality
Assurance Program.
Thank you for the opportunity to provide you an update on our
accomplishments.
MATERIAL SUBMITTED FOR THE RECORD
U.S. Department of Veterans Affairs
Office of Inspector General
Washington, DC.
April 29, 2010
The Honorable John Hall
Chairman, Subcommittee on
Disability Assistance and Memorial Affairs
Committee on Veterans' Affairs
United States House of Representatives
Washington, DC 20515
Dear Mr. Chairman:
At the March 24, 2010, hearing before the Subcommittee on
Examination of VA Regional Office Disability Claims Quality Review
Methods--Is VBA's Systematic Technical Accuracy Review (STAR) Making
the Grade?, you requested additional information on the results of file
reviews associated with our March 2009 report, Audit of Veterans
Benefits Administration Compensation Rating Accuracy and Consistency
Reviews. Specifically, you asked for the ratio of underpayments and
overpayments in files that contained errors. We did not collect data
from the VA regional offices during our audit to determine the ratio.
The Veterans Benefits Administration may be able to determine the ratio
from other data sources.
Thank you for your interest in the Department of Veterans Affairs.
Sincerely,
/S/Joanne M. Moffett for
GEORGE J. OPFER
Inspector General
Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs
Washington, DC.
April 5, 2010
Daniel Bertoni
Director, Education, Workforce, and Security
U.S. Government Accountability Office
441 G Street, NW
Washington, DC 20548
Dear Mr. Bertoni:
Thank you for testifying at the House Committee on Veterans'
Affairs' Subcommittee on Disability Assistance and Memorial Affairs'
oversight hearing on the ``Examination of VA Regional Office Disability
Claims Quality Review Methods--Is VBA's Systematic Technical Accuracy
Review (STAR) Making the Grade?,'' held on March 24, 2010. I would
greatly appreciate if you would provide answers to the enclosed follow-
up hearing questions by Thursday, May 6, 2010.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Due to the delay in receiving mail, please provide your responses
to Ms. Megan Williams by fax at (202) 225-2034. If you have any
questions, please call (202) 225-3608.
Sincerely,
John J. Hall
Chairman
__________
United States Government Accountability Office
Washington, DC.
May 4, 2010
The Honorable John J. Hall
Chairman
Subcommittee on Disability Assistance and Memorial Affairs
Committee on Veterans' Affairs
House of Representatives
Subject: VA Has Improved Its Programs for Measuring Accuracy and
Consistency, but Challenges Remain
This letter responds to your April 5, 2010, request that we answer
questions related to our testimony on March 24, 2010.\1\ During that
hearing, we discussed the progress the Department of Veterans Affairs
(VA) has made in improving its ability to measure the accuracy and
consistency of its decisions on disability benefit claims; and
remaining challenges to improving accuracy and consistency. Your
questions, along with our responses, follow.
---------------------------------------------------------------------------
\1\ Veterans' Disability Benefits: VA Has Improved Its Programs for
Measuring Accuracy and Consistency, but Challenges Remain, GAO-10-530T
(Washington, D.C.: March 24, 2010).
1. In your testimony, you observed that while the STAR system
was used to review the accuracy of some Veterans Benefits
Administration (VBA) claims programs, Benefits Delivery at
Discharge (BDD) and Quick Start pre-discharge claims were not
assessed for quality by the STAR program. Are there other
programs that are not being assessed by the STAR system, which
should be, in your opinion? If there are such programs, what
are the benefits of using STAR to examine the quality of these
VBA programs? Could STAR reviews for all VBA ratings programs
help improve the agency's ability to more timely deliver
---------------------------------------------------------------------------
benefits to deserving veterans.
In our September 2008 report on BDD,\2\ we noted that VBA lacked
measures that would allow it to assess whether decisions on BDD claims
were more or less accurate than decisions under its traditional claims
process, or whether BDD locations faced challenges in processing
claims. As noted in my statement, improved accuracy would improve VBA's
ability to provide veterans and their survivors the benefits to which
they are entitled. Also, improved accuracy helps to ensure that
decisions are correct the first time, which can prevent delays in VA's
lengthy appeals process. In addition, VBA's consistency reviews have
the potential to improve VBA's ability to provide comparable benefits
to comparable veterans across its 57 regional offices. We have not
identified any other specific types of claims where we have recommended
targeted reviews under the Systematic Technical Accuracy Review (STAR)
program.
---------------------------------------------------------------------------
\2\ GAO, Veterans' Disability Benefits: Better Accountability and
Access Would Improve the Benefits Delivery at Discharge Program, GAO-
08-901 (Washington, D.C.: Sept. 9, 2008).
2. Detail all the VBA programs that are currently being
assessed by the STAR system, and provide the annual accuracy
---------------------------------------------------------------------------
rates for those programs for 2009.
VA provided information on several types of claims where it was
conducting STAR reviews, with fiscal year 2009 accuracy rates.
Table 1: Accuracy rates of VBA programs assessed by STAR, Fiscal Year
2009
------------------------------------------------------------------------
Accuracy rate (percent)
------------------------------------------------------------------------
Compensation entitlement (key 84
performance measure)
------------------------------------------------------------------------
Pension entitlement 97
------------------------------------------------------------------------
Redistributed (brokered) cases 77 \a\
------------------------------------------------------------------------
Disability Evaluation System pilot cases 88
\b\
------------------------------------------------------------------------
Appeals Management Center \c\ 83
------------------------------------------------------------------------
\a\ Benefit entitlement decisions made by VBA's 9 rating resource
centers. VBA also reported a 78 percent accuracy rate for ratings by
the Tiger Team at the Cleveland Regional Office. According to VBA, it
began STAR reviews of these cases in July 2009.
\b\ According to VBA, it began STAR reviews of Disability Evaluation
System pilot ratings by the Baltimore and Seattle regional offices in
June 2009.
\c\ According to VBA, it began reviewing Appeals Management Center
decisions in July 2009.
Source: VBA.
3. You both observed that STAR review staff lacked the
manpower and training to fully utilize the quality control
mechanisms offered by the STAR program. With these issues in
mind, how many Full Time Equivalent (FTE) employees would be
ideal, and what training for those employees would you
recommend?
In its March 2009 report, VA's Office of Inspector General noted
that VBA lacked sufficient staff to conduct planned consistency
reviews. Since then, VBA has hired additional quality assurance staff.
We have not done the work needed to estimate the number of Full Time
Equivalent (FTE) staff the quality assurance programs need to conduct
its planned quality assurance reviews.
In responding to these questions, we relied on information we
collected in preparing our March 24, 2010, written statement. For
further information, please contact Daniel Bertoni at (202) 512-7215 or
[email protected].
Daniel Bertoni
Director, Education, Workforce, and Income Security
Committee on Veterans' Affairs
Subcommittee on Disability Assistance and Memorial Affairs
Washington, DC.
April 5, 2010
Bradley Mayes
Director, Compensation and Pension Service
Veterans Benefits Administration
U.S. Department of Veterans Affairs
810 Vermont Avenue, NW
Washington, DC 20240
Dear Mr. Mayes:
Thank you for testifying at the House Committee on Veterans'
Affairs' Subcommittee on Disability Assistance and Memorial Affairs'
oversight hearing on the ``Examination of VA Regional Office Disability
Claims Quality Review Methods--Is VBA's Systematic Technical Accuracy
Review (STAR) Making the Grade?,'' held on March 24, 2010. I would
greatly appreciate if you would provide answers to the enclosed follow-
up hearing questions by Thursday, May 6, 2010.
In an effort to reduce printing costs, the Committee on Veterans'
Affairs, in cooperation with the Joint Committee on Printing, is
implementing some formatting changes for material for all full
committee and subcommittee hearings. Therefore, it would be appreciated
if you could provide your answers consecutively on letter size paper,
single-spaced. In addition, please restate the question in its entirety
before the answer.
Due to the delay in receiving mail, please provide your responses
to Ms. Megan Williams by fax at (202) 225-2034. If you have any
questions, please call (202) 225-3608.
Sincerely,
John J. Hall
Chairman
__________
Questions for the Record from the
HVAC DAMA Subcommittee Hearing on
Examination of VA Regional Office
Disability Claims Quality Review Methods
Is VBA's Systematic Technical Accuracy Review (STAR) Making the Grade?
March 24, 2010
Question 1. Does VA know why the Des Moines Regional Office (RO) is
the highest performing RO with an accuracy rate reported by STAR of 92
percent?
Response: Regional office performance varies as a result of a
number of factors including workforce experience, local economic and
employment environment, and staff turnover. Regional offices that
consistently perform well are in areas where VA is an employer of
choice. In these locations, VA is able to recruit and retain high-
performing employees. The Des Moines RO has a stable and well-trained
workforce, allowing the RO to achieve a high accuracy rate.
Question 2. Does the VA know why the Des Moines RO is performing at
such a higher rate than the Baltimore RO (69 percent)?
Response: ROs that have difficulty in meeting performance targets
are predominantly in high-cost metropolitan areas with high employee
turnover. The Baltimore RO is located in a densely populated area and
faces significant competition in recruiting and retaining a highly
skilled workforce.
Question 3. Please detail the plans that VA has in place to take
lessons learned from what is being done right in the Des Moines RO and
apply them to lower performing ROs like Baltimore.
Response: VA aggressively monitors regional office performance and
develops specific action plans to improve problem areas. Performance is
evaluated against both national and individual targets that are
established at the beginning of each fiscal year. Oversight is provided
through site visits conducted by both the Compensation and Pension
Service and the Area Directors. Lessons learned and specific examples
of ``best practices'' from these visits are provided to assist ROs
increase their performance.
Question 4: Please provide the Subcommittee with data about the
breakdown of claims processing errors reported by STAR and the VA OIG
report. What percentages of these errors were Veterans being under
compensated, and what percentage were Veterans being overcompensated?
Response: STAR performed 21,747 national quality reviews in FY 2009
and cited 2,676 benefit entitlement errors. Of the benefit entitlement
errors, 18 percent were potential Veteran/Beneficiary overpayments and
24 percent were potential Veteran/Beneficiary underpayments.
Question 5. Please provide the Subcommittee with data for outgoing
and new RO directors for the past three years, along with RO error data
for the same time. Does the VA find any correlation between new
leadership and a change in the accuracy of an RO?
Response: The RO director is a critical component in the
performance of a regional office. However, VBA does not believe that
the STAR accuracy rate, be it positive or negative, can be attributed
to just one factor or one individual. Data requested is attached
(Attachment 1).
Question 6. Please provide the list of all the names of the VA
Regional Office Directors, the date on which they were hired, their
current salaries, last bonus, and the 2009 accuracy rate per STAR
review.
Response: See spreadsheet. [Attachment 2, which lists the current
salaries and bonuses of the VA Regional Directors is being retained in
the Committee files. ]
Question 7: Please provide the name and location of the STAR team
director.
Response: The STAR team director is Edna MacDonald, Assistant
Director for Quality Assurance. She is located in VA Central Office,
Washington DC.
Question 8: Please provide the number of STAR reviewers.
Response: There are 38 STAR team reviewers, which is an increase of
66 percent from FY 2009.
Attachment 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Time in STAR Rating STAR Rating STAR Rating
Station Area Director Effective Date of Position Previous Director Accuracy Accuracy Accuracy
Appointment (Yrs) Rate (FY09) Rate (FY08) Rate (FY07)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Philadelphia Eastern Lastowka, Thomas 7/1/1990 19.8 Cary, Robert 79.9% 87.4% 91.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Columbia Southern Hawkins, Carl 10/1/1998 11.6 Sloan, Stedman 88.1% 87.6% 88.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Boise Western Vance, James 1/30/2000 10.2 Barker, Barry 82.4% 88.2% 88.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Waco Central Lowe, Carl 3/12/2000 10.1 McRae, Jerry 84.5% 81.7% 90.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Baltimore Eastern Wolohojian, George 12/15/2002 7.4 Quinton, Newell 67.9% 82.7% 88.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Phoenix Western Flint, Sandy 6/1/2003 6.9 Nappi, Patrick 86.1% 86.8% 91.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
St. Louis Central Unterwagner, Dave 9/21/2003 6.6 Graves, Kim 84.0% 84.0% 93.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
New Orleans Central Christian, Rowland 2/8/2004 6.2 Jackson, Barry 81.9% 88.4% 93.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
San Diego Western Fetzer, Lily 8/8/2004 5.7 Dusenbery, Michael 83.5% 82.2% 86.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Buffalo Eastern Terrell, Donna 10/31/2004 5.5 McCoy, Jack 82.9% 87.5% 90.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Montgomery Southern Randle, Ric 6/26/2005 4.8 Watson, Montgomery 81.6% 84.4% 84.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Muskogee Central Jarvis, Sam 10/2/2005 4.6 Burks, Larry 89.2% 86.9% 94.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Manila Western Skelly, Jon 1/8/2006 4.3 Barker, Barry 89.7% 89.7% 95.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Louisville Southern Thompson, Keith 4/16/2006 4.0 Wardle, Jimmy 87.6% 86.5% 89.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Manchester Eastern Woolford, Charles 8/20/2006 3.7 Cully, Maribeth 76.0% 81.0% 85.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
White River Junction Eastern Woolford, Charles 8/20/2006 3.7 Cully, Maribeth 79.1% 83.3% 81.6%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Los Angeles Western Witty, Kerrie 9/3/2006 3.6 Liff, Stewart 75.5% 81.5% 81.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Denver Western Jacobs, Janice 11/26/2006 3.4 Alger, Jeffrey 86.0% 89.7% 90.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Detroit Eastern Walker, Linda 12/24/2006 3.3 Thompson, Keith 78.1% 76.1% 90.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Roanoke Southern Nicholas, William 2/4/2007 3.2 Smith, John 85.5% 89.5% 90.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Cleveland Eastern Cange, Joyce 7/8/2007 2.8 Mayes, Brad 88.6% 84.9% 95.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Oakland Western Flint, Lynn 7/8/2007 2.8 Smith, Catherine 82.3% 89.6% 91.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hartford Eastern Leonard, Dave 7/22/2007 2.8 Walker, Linda 83.1% 87.4% 86.1%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Little Rock Central Rawls, Cheryl 8/5/2007 2.7 Nicholas, William 87.9% 86.9% 86.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Providence Eastern Navaratnasingam, Pritz 9/2/2007 2.6 Witty, Kerrie 86.0% 90.3% 91.6%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Huntington Southern Commens, Zita 3/30/2008 2.1 Rawls, Cheryl 82.9% 86.0% 94.1%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Atlanta Southern Bocchicchio, Al 4/27/2008 2.0 Burks, Larry 79.0% 77.5% 84.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Winston-Salem Southern Umlauf, Dan 6/22/2008 1.8 Montgomery, John 86.0% 90.6% 89.6%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Salt Lake City Western Bilosz, Mark 7/20/2008 1.8 Wadsworth, Douglas 80.4% 88.2% 88.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Lincoln Central Miller, Loren 8/3/2008 1.7 Smith, Peter 92.9% 94.9% 93.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Wichita Central Waller, Antione 10/12/2008 1.5 Umlauf, Dan 79.4% 84.7% 86.7%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Chicago Central Honeycutt, Duane 11/9/2008 1.4 Olson, Michael 81.6% 86.5% 88.7%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Indianapolis Eastern Stephens, Michael 1/4/2009 1.3 Kuewa, Dennis 84.6% 86.0% 88.1%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
St. Paul Central Fillman, Carol 1/18/2009 1.3 Alger, Jeffrey 86.1% 91.7% 90.9%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Togus Eastern Karczewski, Scott 2/25/2009 1.2 Bilosz, Mark 90.4% 92.9% 90.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Des Moines Central Davis, Dave 3/15/2009 1.1 Braley, Richard 92.7% 91.5% 93.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Honolulu Western Betts, Tracy 3/29/2009 1.1 Reed, Greg 75.9% 83.7% 78.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Jackson Southern Moore, Craig 7/19/2009 0.8 Adair, Joseph 81.8% 82.3% 87.1%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
San Juan Southern Murphy, Thomas 8/2/2009 0.7 Moreno, Sonia 72.1% 75.7% 82.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Pittsburgh Eastern McCoy, Beth 10/11/2009 0.5 Cully, Maribeth 80.6% 88.7% 92.4%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Albuquerque Western Singleton, Grant 10/25/2009 0.5 Moore, Craig 79.8% 90.9% 93.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
New York Eastern Malley, Sue 10/25/2009 0.5 Amberg-Blyskal, Patricia 75.9% 76.0% 83.0%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Seattle Western Prieb, Patrick 10/25/2009 0.5 Fillman, Carol 81.6% 83.5% 88.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Reno Western Russell, Ed 11/8/2009 0.5 Iddings, Donald 90.7% 89.2% 89.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Portland Western Marshall, Chris 11/22/2009 0.4 McClellan, 86.6% 85.8% 87.0%
Christopher
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sioux Falls Central Brubaker, James 1/4/2010 0.3 Smith, John 86.9% 94.5% 90.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Washington Southern Reyes-Maggio, Judy 10/31/2004 5.5 Wilson, Keith N/A 59.7% 75.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Appeals Mgmt Cntr OFO Burke, Ronald 2/1/2009 1.2 Russo, Arnold N/A N/A N/A
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
St. Louis RMC Central Hawthorne, Corey 3/28/2010 0.1 Prieb, Patrick N/A N/A N/A
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Anchorage Western Remotely managed by Salt 86.5% 84.9% 86.3%
Lake City Director
(Bilosz, Mark)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Boston Eastern VACANT Braley, Richard 77.5% 81.3% 81.1%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Cheyenne Western Remotely managed by the N/A N/A N/A
Denver
Director (Jacobs,
Janice)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Fargo Central Remotely managed by 91.6% 91.3% 86.7%
Sioux Falls
Director (Brubaker,
James)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Fort Harrison Western Remotely managed by Salt 91.4% 91.0% 92.6%
Lake City Director
(Bilosz, Mark)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Houston Central VACANT Henderson, Ursula 82.4% 82.5% 84.3%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Milwaukee Central VACANT Olson, Michael 92.7% 93.0% 92.6%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Nashville Southern VACANT Corley, Brian 88.2% 92.6% 93.2%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Newark Eastern VACANT McCourt, John 81.0% 83.7% 82.8%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
St. Petersburg Southern VACANT Barker, Barry 84.7% 87.0% 91.7%
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Wilmington Eastern Remotely managed by the 82.4% 83.6% 83.6%
Philadelphia Director
(Lastowka, Thomas)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------