[House Hearing, 111 Congress] [From the U.S. Government Publishing Office] EXAMINATION OF THE U.S. DEPARTMENT OF VETERANS AFFAIRS REGIONAL OFFICE DISABILITY CLAIMS QUALITY REVIEW METHODS ======================================================================= HEARING before the SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS of the COMMITTEE ON VETERANS' AFFAIRS U.S. HOUSE OF REPRESENTATIVES ONE HUNDRED ELEVENTH CONGRESS SECOND SESSION __________ March 24, 2010 __________ Serial No. 111-68 __________ Printed for the use of the Committee on Veterans' Affairs ---------- U.S. GOVERNMENT PRINTING OFFICE 57-012 PDF WASHINGTON : 2010 For sale by the Superintendent of Documents, U.S. Government Printing Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC, Washington, DC 20402-0001 COMMITTEE ON VETERANS' AFFAIRS BOB FILNER, California, Chairman CORRINE BROWN, Florida STEVE BUYER, Indiana, Ranking VIC SNYDER, Arkansas CLIFF STEARNS, Florida MICHAEL H. MICHAUD, Maine JERRY MORAN, Kansas STEPHANIE HERSETH SANDLIN, South HENRY E. BROWN, Jr., South Dakota Carolina HARRY E. MITCHELL, Arizona JEFF MILLER, Florida JOHN J. HALL, New York JOHN BOOZMAN, Arkansas DEBORAH L. HALVORSON, Illinois BRIAN P. BILBRAY, California THOMAS S.P. PERRIELLO, Virginia DOUG LAMBORN, Colorado HARRY TEAGUE, New Mexico GUS M. BILIRAKIS, Florida CIRO D. RODRIGUEZ, Texas VERN BUCHANAN, Florida JOE DONNELLY, Indiana DAVID P. ROE, Tennessee JERRY McNERNEY, California ZACHARY T. SPACE, Ohio TIMOTHY J. WALZ, Minnesota JOHN H. ADLER, New Jersey ANN KIRKPATRICK, Arizona GLENN C. NYE, Virginia Malcom A. Shorter, Staff Director ______ SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS JOHN J. HALL, New York, Chairman DEBORAH L. HALVORSON, Illinois DOUG LAMBORN, Colorado, Ranking JOE DONNELLY, Indiana JEFF MILLER, Florida CIRO D. RODRIGUEZ, Texas BRIAN P. BILBRAY, California ANN KIRKPATRICK, Arizona Pursuant to clause 2(e)(4) of Rule XI of the Rules of the House, public hearing records of the Committee on Veterans' Affairs are also published in electronic form. The printed hearing record remains the official version. Because electronic submissions are used to prepare both printed and electronic versions of the hearing record, the process of converting between various electronic formats may introduce unintentional errors or omissions. Such occurrences are inherent in the current publication process and should diminish as the process is further refined. C O N T E N T S __________ March 24, 2010 Page Examination of the U.S. Department of Veterans Affairs Regional Office Disability Claims Quality Review Methods................ 1 OPENING STATEMENTS Chairman John J. Hall............................................ 1 Prepared statement of Chairman Hall.......................... 30 Hon. Doug Lamborn, Ranking Republican Member..................... 3 Prepared statement of Congressman Lamborn.................... 31 WITNESSES U.S. Department of Veterans Affairs: Belinda J. Finn, Assistant Inspector General for Audits and Evaluations, Office of Inspector General..................... 4 Prepared statement of Ms. Finn............................. 32 Bradley G. Mayes, Director, Compensation and Pension Service, Veterans Benefits Administration............................. 24 Prepared statement of Mr. Mayes............................ 51 U.S. Government Accountability Office, Daniel Bertoni, Director, Education Workforce, and Income Security........... 6 Prepared statement of Mr. Bertoni.......................... 36 ______ American Legion, Ian C. de Planque, Assistant Director, Veterans Affairs and Rehabilitation Commission.......................... 16 Prepared statement of Mr. de Planque......................... 48 American Veterans (AMVETS), Raymond C. Kelley, National Legislative Director........................................... 15 Prepared statement of Mr. Kelley............................. 48 Disabled American Veterans, John L. Wilson, Assistant National Legislative Director........................................... 14 Prepared statement of Mr. Wilson............................. 45 National Veterans Legal Services Program, Ronald B. Abrams, Joint Executive Director............................................. 13 Prepared statement of Mr. Abrams............................. 42 MATERIAL SUBMITTED FOR THE RECORD Follow-up Letter: Hon. George J. Opfer, Inspector General, U.S. Department of Veterans Affairs, to Hon. John Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, letter dated April 29, 2010........................................... 55 Post-Hearing Questions and Responses for the Record: Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Daniel Bertoni, Director, Education, Workforce, and Income Security, U.S. Government Accountability Office, letter dated April 5, 2010, and response letter dated May 4, 2010........................ 55 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Bradley Mayes, Director, Compensation and Pension Service, Veterans Benefits Administration, U.S. Department of Veterans Affairs, letter dated April 5, 2010, and VA responses................................... 57 EXAMINATION OF THE U.S. DEPARTMENT OF VETERANS AFFAIRS REGIONAL OFFICE DISABILITY CLAIMS QUALITY REVIEW METHODS ---------- WEDNESDAY, MARCH 24, 2010 U.S. House of Representatives, Committee on Veterans' Affairs, Subcommittee on Disability Assistance and Memorial Affairs, Washington, DC. The Subcommittee met, pursuant to notice, at 2:28 p.m., in Room 334, Cannon House Office Building, Hon. John Hall [Chairman of the Subcommittee] presiding. Present: Representatives Hall, Donnelly, and Lamborn. Mr. Hall. Good afternoon, everybody. Thank you for your patience. Would we all please rise for the Pledge of Allegiance. [Pledge of Allegiance.] OPENING STATEMENT OF CHAIRMAN HALL Mr. Hall. Welcome to the Subcommittee on Disability Assistance and Memorial Affairs' hearing entitled, Examination of the U.S. Department of Veterans Affairs (VA) Regional Office Disability Claims Quality Review Methods--Is the Veterans Benefits Administration's (VBA's) Systematic Technical Accuracy Review or STAR Making the Grade? We are going to try to make abbreviated opening statements by myself and Ranking Member Lamborn as we understand votes will be called any time. That said, I welcome you all here in what has been a profoundly historic and important week for the Nation and for our veterans. Over the last 7 days, the full Committee convened a successful Summit, which brought many of you and dozens of other top veteran stakeholders together to aid us in fixing the VA compensation and pension claims process. From the Summit came a lot of very useful information which we welcome and look forward to using to solve the problems that VA faces in processing disability claims of our Nation's veterans. Next in a rare Sunday session, Congress passed and the President signed a sweeping health care reform package. And I am pleased that Secretary Shinseki as well as the Chairman of the full VA Committee and the Armed Services Committee have signed a letter and sent it to the veterans service organizations (VSOs) stating unequivocally that TRICARE and VA care will not be adversely affected by the national health care reforms. Also in the past few days, we passed the End Veteran Homelessness Act of 2010 to provide funding to help Secretary Shinseki's goal of ending homelessness for America's warriors, the Help Heroes Keep Their Homes Act, the COLA, the cost of living increase for veterans, and the National Guard Employment Protection Act. This particular hearing will be about the Systematic Technical Accuracy Review (STAR) technical review system and we will look at the accuracy of assessing disability compensation and pension claims rating and the disparity between accuracy in the different regional offices (ROs). Using this quality control tool, VBA should be able to focus attention on poorly-performing ROs and help the agency direct additional staff and training to the problem offices and at the same time look at those who are the highest-performing ROs and find out what they are doing right. The STAR system was implemented in October of 1998. Since fiscal year 2007, VBA has set for itself a goal of completing compensation claims ratings without error 90 percent of the time. Its long-term strategy goal is 98 percent. Unfortunately, we are still far from achieving that goal. And until the STAR system provides an accurate accounting of the error rate at VBA, it is difficult to envision a path to meeting this goal. So we are honored to have today the Office of the Inspector General (OIG), the U.S. Government Accountability Office (GAO), both of which have produced studies revealing issues that may be those impeding the efficiency and consistency of the STAR system. We are looking forward to hearing their testimony. I am personally troubled by GAO's finding that the VBA claims processing accuracy rate is particularly low in cases pertaining to post-traumatic stress disorder (PTSD) and traumatic brain injury (TBI). Today we hope to analyze these studies and hear testimony regarding those issues from OIG and GAO. There have been improvements made by the VBA to the claims rating system and the STAR system. We look forward to hearing about these improvements and how we, the Subcommittee, the full Committee, and Congress, can help. To fully understand STAR, it is important to review the compensation and pension (C&P) rating system itself. To assess a claim, the application for disability assistance must be developed, a process that involves obtaining all necessary evidence to support the veteran's claim. After development, the claims go to a rating veterans service representative (RVSR). The RVSRs determine if that disability is service-connected and assigns a percentage rating that is intended to represent the average earning reduction a veteran with that condition would experience in a civilian occupation. The veteran is then notified of that decision. For reopened claims, the assigned diagnostic codes affect the veteran's overall combined percentage of disability. The ROs, regional offices personnel applying the appropriate diagnostic code percentages to determine the combined level of disability. Once that claim is completed, it is declared an end product and the result of that claim is cleared and a work credit is given to the regional office. So a completed claim and corresponding cleared end product is then subject to review by STAR reviewers. In the 110th Congress, I introduced H.R. 5892, which sought to improve VBA's quality control measures. This bill was incorporated into an omnibus veterans' package, which was signed into law as Public Law 110-389. The Veterans Disability Benefits Claims Modernization Act of 2008, H.R. 5892, was part of that. So our hearing should also provide a chance to gauge how well these quality control measures are working. I thank you for being here, in advance for your testimony, and now I would like to recognize Ranking Member Lamborn for his opening statement. [The prepared statement of Chairman Hall appears on p. 30.] OPENING STATEMENT OF HON. DOUG LAMBORN Mr. Lamborn. Well, thank you, Mr. Chairman. And thank you for waiting, although I would have been happy for you to go ahead and start. But I know as a courtesy, you in a spirit of bipartisanship, you wanted to wait. So I thank you for that. And I do apologize. There was a misunderstanding between my staff and I. I thought that we were going to start after this first series of votes because I specifically asked that. And once they found the mistake, they notified me immediately, but I was two buildings away. So I got here as quickly as I could. But I know everyone's time here is extremely valuable and so I apologize. It is totally the fault of me and my office. Mr. Chairman, I want to, like you, welcome everyone to this hearing on the Department of Veterans Affairs' STAR Program. Throughout my tenure on this Committee, my fellow Members and I have called for stronger accountability within the VA claims system. For too long, the primary focus has been on production and this has led to an error rate that is unacceptable. I believe that the VA's greatest challenge, the claims backlog, is largely attributable to hasty decisions made without proper regard for accuracy. The ramifications of this approach can be seen throughout the entire system. Therefore, VA employee performance awards cannot be based entirely on production. There must also be a valid measure of quality. Under the STAR Program, a statistically valid sample of rating decisions from various regional offices is reviewed for accuracy. While this method may be useful from a macro perspective, it is not sufficient for ensuring individual accountability. VA must be able to identify employees in need of individualized remedial training. Without this essential component of the quality assurance process, VA will have perpetual problems in its claims system. In the 110th Congress, this Committee passed a provision that was included in the Veterans Benefits Improvement Act of 2008 that required VA to conduct a study on the effectiveness of the current employee work credit system. I believe the upcoming report, along with the testimony that we will hear today, will provide valuable feedback for the Department to improve its Quality Assurance and Accountability Program. I look forward to hearing from our witnesses today, and I thank you all for your participation. And, Mr. Chairman, I thank you also and I yield back. [The prepared statement of Congressman Lamborn appears on p. 31.] Mr. Hall. Thank you, Mr. Lamborn. We are moving at lightning speed now. We would like to ask our first panel to join us at the witness table. We have Belinda J. Finn, Assistant Inspector General for Audits and Evaluations, Office of Inspector General, U.S. Department of Veterans Affairs, and Daniel Bertoni, Director of Education, Workforce, and Income Security with the Government Accountability Office. Welcome to you both, and your full written statements are entered in the record. So we will recognize you for 5 minutes each for however much of that you would like to give to us directly. Ms. Finn, welcome, and you are now recognized for 5 minutes. STATEMENTS OF BELINDA J. FINN, ASSISTANT INSPECTOR GENERAL FOR AUDITS AND EVALUATIONS, OFFICE OF INSPECTOR GENERAL, U.S. DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY LARRY REINKEMEYER, DIRECTOR, KANSAS CITY AUDIT OPERATIONS DIVISION, OFFICE OF INSPECTOR GENERAL, U.S. DEPARTMENT OF VETERANS AFFAIRS; AND DANIEL BERTONI, DIRECTOR, EDUCATION, WORKFORCE, AND INCOME SECURITY, U.S. GOVERNMENT ACCOUNTABILITY OFFICE STATEMENT OF BELINDA J. FINN Ms. Finn. Thank you, Chairman Hall. I am fighting some allergies that sometimes cause coughing fits, so I am trying to avoid that. Mr. Hall. You are excused. Ms. Finn. Thank you for having us here today. I am pleased to be here to discuss our review of VBA's Quality Assurance Program and how VBA can improve the programs to better serve our veterans. I am joined today by Larry Reinkemeyer, who is the Director of the Kansas City Audit Operations Division. The OIG is committed to proactively reviewing the Department's key internal controls to identify weaknesses before they escalate into significant problems. Over the past 2 years, my office has audited three components of VBA's Quality Assurance Program, rating accuracy, rating consistency, and VBA's Site Visit Program. Although our written statement covers our work in all three areas, my comments today are only addressing our audit of the Systematic Technical Accuracy Review or STAR Program. In March 2009, we issued the audit of VBA's compensation rating accuracy and consistency reviews. We concluded that VBA's STAR process did not effectively identify and report errors in compensation claim rating decisions. We projected that about 77 percent of claims were accurate as opposed to VBA's reported accuracy rate of 87 percent. This equated to approximately 88,000 additional claims where veterans' monthly benefits may be incorrect. We identified five areas where VBA needed to improve the STAR Program. VBA agreed with all of our recommendations and just recently reported that all corrective actions were complete. First, the STAR reviewers did not identify some errors because they either did not thoroughly review available evidence or they inappropriately misclassified benefit entitlement errors as comments that did not count against the accuracy rate. Second, STAR management required regional offices to report quarterly on actions taken to correct errors but did not follow-up to ensure that these offices actually took the corrective actions on comments. Third, VBA excluded brokered claims from STAR reviews. The officials told us that the STAR Program's primary focus was to assess and report rating accuracy for each of the individual regional offices. Since two regional offices are involved in brokered claims, these officials believed it would be difficult to assign responsibility for rating accuracy on a brokered claim. Fourth, the STAR reviewers did not ensure regional offices submitted all of the selected compensation claim ratings for review. In fact, the offices did not submit about seven percent of the requested ratings for the 12-month period ending February 2008. We reviewed a sample of these unsubmitted claims and identified a benefit entitlement error rate of approximately 22 percent. Last, the STAR reviewers were not required to complete formal training on an annual basis. In contrast, the regional office staff that prepare and complete ratings are required to complete 80 hours of training per year to stay current on laws, policies, and processes that affect rating claims. The STAR management relies on the regional offices to take corrective actions on the issues identified by the STAR team. Since April 2009, the OIG Benefits Inspection Division has issued eight reports where we looked at regional office procedures to ensure the accurate and timely correction of errors identified by STAR. Our analysis of 148 errors found that regional office staff had not corrected 27 percent of these and in some cases had erroneously reported to STAR that the errors had been corrected. In closing, VBA is under tremendous pressure to process claims and reduce the growing backlog. Without an effective and reliable Quality Assurance Program, VBA leadership cannot adequately monitor performance to make necessary program improvements and ensure veterans receive accurate and consistent ratings. Mr. Chairman, thank you again for the opportunity to be here today. Mr. Reinkemeyer and I would be pleased to answer any questions you may have. [The prepared statement of Ms. Finn appears on p. 32.] Mr. Hall. Thank you, Ms. Finn. You had 4 seconds remaining. Good job. Ms. Finn. Thank you. Mr. Hall. Mr. Bertoni, welcome. You are now recognized for 5 minutes. STATEMENT OF DANIEL BERTONI Mr. Bertoni. Mr. Chairman, Members of the Subcommittee, good afternoon. I am pleased to be here to discuss the Department of Veterans Affairs' efforts to improve the quality of disability decisions. For years, we have noted that VA's claims processing challenges not only include making quicker decisions in reducing its claims backlog but also improving accuracy and consistency. My statement today focuses on steps VA has taken in response to recommendations from us and others to enhance its quality assurance tools, namely the Systematic Technical Accuracy Review or STAR Program, as well as other programs designed to address decisional consistency. Since STAR was first implemented, we have made numerous recommendations for improvement. For example, very early on, we noted that STAR reviewers lacked organizational independence because they also had claims processing duties and reported directly to regional managers whose claims they may review. Per our recommendation, VA moved to require organizationally independent STAR reviewers who are precluded from making claims decisions. In subsequent work, we found that STAR sampling was insufficient to ensure the accuracy of disability pension decisions. VA addressed our findings by consolidating pension processing into three locations and establishing a separate STAR review for pension claims. More recently we reported that VA is not using STAR to separately assess the accuracy of the benefits delivery at discharge and quick-start claims, alternative processes for fast tracking VA disability compensation claims for active-duty servicemembers. To date, the Agency has opted not to evaluate the extent to which staff are accurately developing and rating these claims, although such information could better inform training and focus program monitoring efforts. VA's Office of Inspector General has also recommended changes to the STAR Program which VA has begun to address, including establishing minimum annual training requirements for reviewers, mandating an additional supervisory review of STAR reports, sampling brokered claims for accuracy, and implementing more stringent procedures for conducting STAR reviews. Finally, the Agency has begun to take steps to address deficiencies that both we and VA's Inspector General have identified with its consistency review programs, which assess the extent to which regional offices and individual raters make consistent decisions on the same claim. We recommended that VA conduct systematic studies of impairments identified as having potentially inconsistent decisions. And in fiscal year 2008, VA did initiate this effort. However, last year, the Inspector General reported that VA had not followed through on its plans to conduct additional reviews, which was attributed in part to insufficient STAR staffing resources. The Agency has since developed a consistency review strategy and is in the process of conducting fiscal year 2010 reviews. However, these efforts have only recently begun and it is too early to assess their impact. And despite various recommendations for improvement and actions to address them, VA has struggled over the years to improve the accuracy rate for disability compensation decisions which was 84 percent in fiscal year 2009 and well short of VA's stated goal of 90 percent. VA has attributed its inability to meet accuracy goals in part to the large numbers of newly-hired personnel conducting claims development work and their general lack of training and experience. We have also noted that human capital challenges associated with providing training to help new staff become more proficient will likely continue into the near future and could impact quality. Thus, it is important that VA continue to improve and maintain a robust quality assurance framework that not only supports staff in their understanding of the very complex business of making timely, accurate, and consistent disability decisions but also ensures that all veterans receive the benefits they are legally entitled to. Over time, VA's newly-implemented quality assurance initiatives have the potential to improve decisional accuracy and consistency if compliance with enhanced protocols, procedures, and standards is sustained. However, it is imperative that VA remain proactive in its quality assurance efforts going forward, especially as aging veterans and more veterans from current conflicts add to VA's already substantial claims workloads. And, Mr. Chairman, this does conclude my statement. I am happy to answer any questions you may have. Thank you. [The prepared statement of Mr. Bertoni appears on p. 36.] Mr. Hall. Thank you, Mr. Bertoni. I am just going to yield to Mr. Lamborn to make a comment about his questions. Mr. Lamborn. Mr. Chairman, I am going to waive my questions for the sake of time and just follow-up in writing to the degree that we need further explanations. So thank you. Mr. Hall. Thank you, Mr. Lamborn. And I will try to squeeze a few questions in here before we have to go across the street to vote. Ms. Finn, your OIG report indicates that there are about 203,000 claims where a veteran is receiving incorrect monthly benefits. Does this number reflect only veterans who are being underpaid? I ask because it is important to know whether there is a systematic bias toward underpaying rather than overpaying veterans. What does your data suggest. Ms. Finn. No. We found errors are reported both for overpayments and underpayments. Mr. Hall. Can you give us a ratio or is that roughly equal. Mr. Reinkemeyer. No. I think most of the errors are underpayments where the claims examiner did not identify all the issues. A veteran may have filed his claim and had eight or nine issues and a couple of them were omitted in the rating decision. So I would say most, but we have no numbers on that. And the 203,000 is the projected number based on our sample. Mr. Hall. Okay. Well, that would be a good thing if you can cull that from your data. That would be good for us to know if it is tilted toward underpayment and how much. Mr. Reinkemeyer. Sure. [The VA Inspector General George Opfer, subsequently followed up in a letter dated April 29, 2010, which appears on p. 55.] Mr. Hall. Ms. Finn, your testimony also indicates that VA reported that it has completed actions to implement the recommendations of the OIG report of March 2009. Are there plans at OIG to follow-up on this? Ms. Finn. We do not have immediate plans for follow-up right now. We do over time, though, follow-up on selected reports to actually assess the effectiveness of corrective actions. We always follow-up with management to ensure that they can report to us what they have done. Mr. Hall. Thank you. Your study observed that STAR reviewers do not identify some errors because they either do not thoroughly review available evidence, they fail to identify the absence of necessary medical information, and sometimes they misclassify errors in a way that resulted in errors not being counted against the accuracy rate. What do you believe can cure these deficiencies? Is it a training issue alone? Ms. Finn. I believe our recommendation in the report related to increased supervisory reviews on those issues. Certainly to ensure that comments that should be corrected should be counted as errors were not counted. Mr. Hall. Thank you. And you noted that in your testimony that STAR management required that regional offices report quarterly on actions taken to correct benefit entitlement errors, but they did not require or follow-up to ensure regional offices were actually taking corrective actions on the comments made by the STAR reviewers. Do you have any indication that VA has remedied this concern and what assurances the corrective action at VA as purported has actually been implemented in a satisfactory manner? Ms. Finn. The only assurance we have is VBA's response to our recommendation. And as I alluded to in my oral and written testimony, we look at this issue when we go on our regional office inspections and we do not always find that STAR errors have been actually corrected. Mr. Hall. And lastly for you, Ms. Finn, your review indicates that brokered claims are experiencing a 69 percent accuracy rate. What impact do you believe brokered claims will have on the overall VBA claims accuracy rate in the short term and long term or can you tell yet? Ms. Finn. No, I cannot tell you. We are conducting a national audit looking at the brokering or redistribution of claims program. And we are assessing the impact on the timeliness of claims processing and also how this could possibly impact accuracy or how the brokering program is being implemented in respect to accuracy. So I do not at this point have any real solid projections. Our sampling that we did a year ago was based on a sample of the brokered claims to determine our error rate at that time. Mr. Hall. I would ask you, Ms. Finn, and then Mr. Bertoni also, the following question. In the list of the 2009 STAR accuracy ratings for all regional offices, that we were given, the most accurate RO was Des Moines with an accuracy rating of 92.34 percent, ranging down to Baltimore with a 69.34 percent accuracy rate. The Washington Regional Office is not rated but we understand its quality level may be lower than Baltimore. Have you already or do you have plans to try to identify what it is that the Des Moines office is doing that others maybe should be emulating or what it is that Baltimore is doing that others should avoid or is it not that simple? Are there other factor? Ms. Finn. I do not know that it is that simple. We have not yet visited the Des Moines office on our benefit inspection. We have, however, gone to Baltimore. And we do a selected review of claims. We are looking at specifically claims for PTSD, TBI. We have looked at diabetes and also we look at brokered claims. So our review of claims is not consistent necessarily with the STAR methodology and, therefore, the numbers are not directly comparable. We did find when we looked at those selected claims in Baltimore that they had about 38 percent inaccuracy on those types of selected claims. We also found a number of management issues in Baltimore in that they had had leadership vacancies and they also had had staff removed from the regular regional office operations to work on a disability pilot. Mr. Hall. Right. Ms. Finn. So we felt that impacted their ability to do quality work. Mr. Hall. So the virtual RO pilot project and DES pilot may have been hurting, at least temporarily, the overall accuracy rating? Ms. Finn. We felt it was an issue that needed to be addressed. We could not directly quantify the impact. Mr. Hall. Okay. Well, thank you very much. Mr. Bertoni, would you care to address that same question. Mr. Bertoni. Yes. In terms of the variation in quality, I think the OIG testimony bears out some interesting issues with the accuracy rate itself. I think one question might be variation might be due perhaps to quality of the review across the regions. I mean, we see lack of training, folks who may not know completely the rules of the game who may be reviewing these claims. And in some regions, they may be doing a more thorough job. In other regions, they may not. The other part is that there is some blame to lay on the regions in terms of quality. I think you cannot underrate regional management. There are good managers out there who really embrace quality and it is ingrained in the staff. They are doing innovative things in terms of training. And we have a report we are about to issue to this Committee in regard to training and some actually best practice type things that are going on out there. And also the ability to hire staff and keep staff. And I think it is more difficult in certain areas than others, especially some of the larger urban areas, and supervisors. So I think it is a range of things. It is a combination of things that sort of lead to this variance. But I would also, again, hearken back to the quality of the reviews. Is it true that these two regions are entirely different? It may be reviewers are making mistakes and it might lead to some of this disparity. Mr. Hall. Mr. Bertoni, in our last hearing examining the benefit delivery at discharge (BDD) and quick-start programs, you noted that VA lacks sufficient and specific performance measures for assessing the accuracy of decisions on BDD claims. And you recommended that VA consider options for separately estimating the accuracy of such decisions. However, VA has asserted that the cost of sampling pre- discharge claims as part of STAR would outweigh the benefits. If VA subjected BDD and quick-start claims to STAR review, could we make these programs better, even better than they apparently are? If so, what staffing, training, or other resources do you think would be needed? Mr. Bertoni. I think the answer is yes. There were 250,000 initial claims that year and BDD represented a little over 50,000. So that is about 20 percent of the new claims coming into the pipeline. So if you can make a difference there, you can make a difference. And the goal is to make BDD claims an even larger part of the initial claims. The idea that it is not cost effective, I do not know if I have seen enough to agree to that. VA did do some analysis for us, but it was only on the cost side, full-time equivalents, space, modifications to STAR. They never pulled a sample of BDD claims to say what did we get from this additional review. So what is the return on investment on the other side is not clear. And I continue to believe that VA continues to process more claims outside the traditional process. They are reengineering processes. They are reengineering, you know, how they do work. And I think they are using in many cases sort of STAR as more of a blunt instrument rather than using it to really drill down into these specialized new business processes to see how they are doing. And I think at the end of the day, they need to do more of that to find out where they are getting the return on investment and where they need to walk away from things. Mr. Hall. You asserted in your testimony that the accuracy rate was 86 percent in fiscal year 2008 and 84 percent for fiscal year 2009, well short of VBA's goal of 90 percent. So at this point, what additional recommendations do you have for improving the STAR Program. Mr. Bertoni. I think there is a mixture here again. There are program design issues that they have to address, but there is also management and oversight. And I think the OIG really hit on many of those. I was extremely surprised to hear that you have such an important program and had no training requirements. I was very surprised. The fact that they have not done that yet is something that they really need to move on. A supervisory review, you know, having another look at these cases before they are blessed or deemed to be correct is very important. It is part of their quality assurance process. They need to build in some management controls and certainly training. We have a lot of new staff and integrating those staff and putting them in a position to be able to accurately assess claims is going to take good management, good supervisory review to sort of teach people as to, you know, what needs to be done. So I think that is very important. Mr. Hall. Thank you, Mr. Bertoni. You also note that due to increased Congressional support, VBA has increased its compensation and pension staffing. Specifically you point out that VBA more than doubled the size of the quality assurance staff, allowing it to increase the scope of quality assurance reviews. Do you believe this additional staff has made a difference or are you of the opinion that staffing alone is insufficient to address deficiencies? I think you started to answer this question, but---- Mr. Bertoni. Staffing never hurts. And if you staff up a program and you give folks the tools that they need to do the job they need to do, it could make a difference. And that means again training them up and putting them in a position to make a difference. So the numbers will help. But, again, it is going to take time. There is a learning curve. But over time, I think it will be effective, but you also have to build in the management controls to make sure that the reviews do have the authority and the oversight teeth that they should have. Mr. Hall. Thank you. Some VSOs point to the high rate of remand ordered by the Board of Veterans' Appeals (BVA) or the U.S. Court of Appeals for Veterans Claims (CAVC) as indications that the accuracy rate is a lot lower than reported by VBA. What is your assessment of how the errors found by the Board and the Court impact VBA's reported claims processing accuracy rate? And maybe, Ms. Finn, if you could comment on that as well. Ms. Finn. Mr. Chairman, we have not done any specific work on the appeals process, so I would prefer not to comment. I would note, however, that remands involve other issues other than just whether the rating decision was accurate. There could be procedural reasons for a remand. But other than that, I will leave that one to Mr. Bertoni. Mr. Bertoni. We had worked in that area not very recently, but I would say again there are a range of reasons why a case would be remanded back. Sometimes it is a matter of it has been so long that the impairments change, new impairments are introduced, other evidence has not been considered. So there are a lot of reasons. But I think overall in hearing what I have heard today with the OIG report, what we know and work that we have done in the past in terms of missing claims and how that could affect error rates, I would say the error rate--I do not have a whole lot of faith in the error rate or, I am sorry, the accuracy rate of 84 percent. It is probably some other lower figure, but I do not know what that is. Mr. Hall. And, lastly, Mr. Bertoni, before the changes in the STAR Program were mandated by Public Law 110-389, only ten cases per month for each RO were being reviewed by the STAR system. Today VA staff advises us that twice as many cases are being reviewed, 240 cases per RO per year. How many total cases were decided by the regional offices last year and is a sample of 20 cases per month per RO sufficient or should that sample be larger, particularly for ROs to process a larger number of claims? Mr. Bertoni. Unfortunately, I do not have the number of total cases. But any time you can ramp up the numbers is going to make the projectability of what you do more rigorous. Our concern again with the current system is STAR tends to be used more as a blunt instrument to account for accuracy of all claims regardless of varied business processes. More and more, we see these claims being processed through these alternative or re-engineered processes in order to expedite things. And the Agency could be using STAR more effectively to assess how they are doing with these alternative processes. They are not. And BDD again is a good example of that. Every re-engineered process should include some kind of quality assurance whether it is continuous or a snapshot or a limited investment. Without doing that, you waste a lot of time. You do not know what you are getting for your money and you might end up where you do not want to be at the end of the day. I think failure to build any regular or limited investment quality assurance into any of these alternative processes is a failure of management. And we have noted along the way these varied again re-engineered or alternative means by which cases are being decided where the quality assurance, the STAR review was not sufficient in our view to get a good sense of how accurate and consistent many claims are. Mr. Hall. Well, thank you, Mr. Bertoni and Ms. Finn and Mr. Reinkemeyer. I am grateful for your testimony. And we may have further questions for you that we will send you in writing. We now have a series of votes called on the floor, approximately 30 minutes we are told, so we will ask our next panels to be patient, please, as you are used to, I am sure, by now. And the first panel is excused. Thank you again. The Subcommittee is recessed for votes. [Recess.] Mr. Hall. Welcome again, and thank you for your patience. The Subcommittee on Disability Assistance and Memorial Affairs will now resume its hearing on the STAR Review, Making the Grade. And our second panel, thank you for joining us again. Ronald B. Abrams, Joint Executive Director, National Veterans Legal Services Program (NVLSP); John L. Wilson, Assistant National Legislative Director, Disabled American Veterans (DAV); Raymond C. Kelley, National Legislative Director of AMVETS; and Ian C. de Planque, Assistant Director, Veterans Affairs and Rehabilitation Commission of the American Legion. Gentlemen, thank you so much for being here today. And your full statement, written statement, of course is entered into the record. So we will give you each 5 minutes to expound upon it, and starting with Mr. Abrams. You are now recognized. STATEMENTS OF RONALD B. ABRAMS, JOINT EXECUTIVE DIRECTOR, NATIONAL VETERANS LEGAL SERVICES PROGRAM; JOHN L. WILSON, ASSISTANT NATIONAL LEGISLATIVE DIRECTOR, DISABLED AMERICAN VETERANS; RAYMOND C. KELLEY, NATIONAL LEGISLATIVE DIRECTOR, AMERICAN VETERANS (AMVETS); AND IAN C. DE PLANQUE, ASSISTANT DIRECTOR, VETERANS AFFAIRS AND REHABILITATION COMMISSION, AMERICAN LEGION STATEMENT OF RONALD B. ABRAMS Mr. Abrams. Thank you, Mr. Chairman. I have some good news. We have just met with hundreds of service officers for Legion and Purple Heart. And I am happy to tell you that when they talked to me they said STAR does catch errors, they do order the VA to fix them, and there is some training based on the errors caught by STAR. However, based on the same comments of these service officers, and our own personal experience, we find that the training based on the errors caught by STAR is not generally doing the job. The errors still continue and the emphasis is still on production in the regional offices. NVLSP believes that the quality of the regional office adjudications is much worse than what is reported by the VA. And all you have to do if you want to look at it in an independent check is look at the statistics produced by the Board and the Court of Appeals for Veterans Claims. And I can tell you as far as consistency goes, I defy almost anyone to look at 20 evaluations of mental conditions and make rhyme or reason out of them. The rating schedule is not applied on a consistent basis. I am not going to over the BVA statistics. You all have them. I will say that the results of what we do with the American Legion, and we have done over forty quality checks in regional offices, we find that the quality is much worse. In many instances claims are denied in a premature manner or benefits are not paid at the proper rate because the regional office was more concerned about claiming work credit and reducing the backlog than taking the time to develop and analyze the claim properly. STAR shows about a 16 percent error rate. We are finding over a 30 percent error rate, in general. Here are some of the errors that we have found that you should know about. Assignment of erroneous low ratings for service-connected mental conditions. Erroneous denial of claims for service-connection for mental conditions. Failure to consider 38 USC 1154(b), the Combat Veteran Statute. Erroneous denials of claims for individual unemployability. Failure to consider presumptive service-connection. And inadequate requests for medical opinions. Sometimes doctors are not asked the right questions and that leads to a litany of problems. Based on that we ask that the VA work measurement system be altered so that quality, production, and timeliness are concepts that really drive the system. That is the most important thing. You need to change that to make any effective change. We would also like a quality control with teeth and an independent quality check. I will leave it at that except to say that what the VA is doing now is obviously not working. We need to have things changed. Thank you. [The prepared statement of Mr. Abrams appears on p. 42.] Mr. Hall. Thank you, Mr. Abrams. Mr. Wilson, you are now recognized. STATEMENT OF JOHN L. WILSON Mr. Wilson. Thank you, sir. Mr. Chairman and Members of the Subcommittee, I am pleased to appear before you on behalf of Disabled American Veterans to address VBA's Systematic Technical Accuracy Review program. Successfully reforming the veterans benefits claims process will require training and accountability, two elements central to producing quality results for veterans. VBA continues to struggle with the quantity of claims, with quality, training, and accountability taking a backseat. VBA's primary quality assurance program is the STAR program. The STAR program can identify three types of errors: benefit entitlement, decision documentation and notification, and administrative. The STAR program was evaluated by the VA OIG in the March 2009 report, which determined the program did not provide a complete assessment of rating accuracy. The OIG found STAR processors did not effectively identify and report all errors in compensation claim rating decisions. While VBA stated STAR reviewers achieved an 87 percent technical accuracy rate, the OIG projected an accuracy rate of only 78 percent based on its review of STAR reviewed cases. The OIG determined that this equates to approximately 203,000 claims in that 1 year alone with veterans monthly benefits may be therefore incorrect. The OIG determined the STAR reviewers did not identify some of the missed errors because they either did not thoroughly review available medical and non-medical evidence, did not identify the absence of necessary medical information, or inappropriately misclassified benefit entitlement errors in the comments section. These findings point to the need for greater management oversight and an effective formal training program for the STAR reviewers. We had heard reports that while STAR reviewers could benefit from formal training current workload requirements do not allow for the necessary training time. This common theme for VBA underlies even STAR reviews; quantity over quality. Even in the area that is supposed to ensure quality of at least a technical nature. The need for a quality control program as an adjunct to the STAR quality assurance program can also be seen when considered through a review of the Board of Veterans' Appeals summary of remands. Nineteen thousand one hundred cases, or 34 percent of appeals reaching the BVA in fiscal year 2009, were due to remands for notice not being provided to claimants, failed requests for service medical records and personnel records, or ignored travel Board requests. These elementary errors were either undetected or ignored. A 34 percent error rate on such basic elements in the claims process is simply unacceptable. With no incentive to prevent such errors, and a constant focus on production, quality will continue to decline. DAV agrees with the VA OIG recommendations to improve the STAR program. In addition, we recommend VBA establish a quality control program that looks at claims in process in order to determine not just whether a proper decision was made but how it was arrived at in order to identify ways to improve the system. Combining results from such quality control reviews with STAR's quality assurance results, and the data from remands from the Board of Veterans' Appeals, and the Court of Appeals for Veterans Claims, could yield valuable information on trends and causes of errors. If this data could be incorporated into a robust IT system, proper analysis of such data would provide management and employees insights into processes and decisions. With a modern IT system, VBA would be able to do quality control in real time, not just after the fact. This in turn would lead to quicker and more accurate decisions on benefits claims and more importantly to the delivery of all benefits earned by the veteran, particularly disabled veterans, in a timely manner. That concludes my testimony. I would be happy to answer any questions. [The prepared statement of Mr. Wilson appears on p. 45.] Mr. Hall. Thank you, Mr. Wilson. Mr. Kelley. You are recognized. STATEMENT OF RAYMOND C. KELLEY Mr. Kelley. Thank you for giving AMVETS the opportunity to present our views on the STAR program. AMVETS agrees with VA's Office of Inspector General's March, 2009 report that identified eight issues that will improve the process of reviewing claims for errors, and AMVETS is please to see VBA is taking action to correct these issues. AMVETS is concerned, however, with what is done with the information that is gleaned from STAR. For the STAR program to truly be effective AMVETS believes three things must be done. First, STAR must be enhanced so trends and errors can be easily identified by regional offices. With this information, VBA must hold ROs accountable for failures in accuracy and insist that the ROs develop improvement strategies and include training for these accuracy issues. Second, VBA must change its culture of timeliness and strive for a culture of accuracy. Whether or not STAR is completely accurate in its review is important but not nearly as important as what is done to ensure the same mistakes are not made again. Third, OIG must conduct periodic reviews of the STAR program to ensure that its accuracy ratings are within a 3 percent error margin. Even though AMVETS believes the STAR program is effective, we believe it could be expanded to ensure that specific programs such as BDD and specific conditions can be tracked for anomalies that occur so improvement strategies and specific training can be implemented. After the Veterans Claims Assistance Act (VCAA) was introduced in the first quarter of fiscal year 2002, nearly half of all errors were VCAA related. VBA had the ROs retrain their claims processors and in the last 8 months of the fiscal year these types of errors were reduced by one-third. This type of accountability needs to be the rule and not the exception. As you know, the STAR program identifies errors in claims decisions each month through random sampling of each regional office. The results of the reviews are sent to the ROs. Errors that are found are to be corrected by the RO who made the decision. The corrections are made, but all too often the ROs do not implement strategies to ensure that claims processors do not continually repeat the same mistakes. AMVETS believes the reason these strategies are not developed is that the culture within the regional offices is one of timeliness and not one of accuracy. STAR has consistently found that nearly 20 percent of claims are in error over the past decade, but VBA has done little to ensure that mistakes that are made in the regional offices are understood and that strategies for improvements are put in place. On the other hand, VBA does require ROs to develop corrective action plans if they do not reach strategic goals for production, inventory, and timeliness. This paradigm must be flipped. The March, 2009 OIG report clearly defines the gaps in the STAR program that have caused the 10 percent disparity in compensation and pension rating accuracy. AMVETS believes VBA is taking action to close these gaps, however we believe it is important to have this accuracy fall within a 3 percent margin of error. Therefore, AMVETS requests that OIG conduct a follow up to the 2009 report to ensure VBA's gap solutions are productive and that OIG continue to conduct periodic reviews of STAR to be sure the program reaches and maintains that 3 percent margin of error. Mr. Chairman, this concludes my remarks and I will be happy to answer any questions that you have. [The prepared statement of Mr. Kelley appears on p. 48.] Mr. Hall. Thank you, Mr. Kelley. Mr. de Planque. STATEMENT OF IAN C. DE PLANQUE Mr. De Planque. Thank you Mr. Chairman and Members of the Committee. On behalf of the American Legion I would like to say that the STAR program is a potentially effective tool that if used in a more effective manner could really help VA deal with the accuracy issues. It is refreshing to see that in the OIG report from March 2009, that the issues that OIG identified VA concurred with and has recently reported that they are making the changes to correct those. The American Legion would recommend three points which could help VA enhance the efficiency of STAR and the ability for it to be effective. The first point is to create an aggregate record of all of the errors that are reported through STAR so that it can be analyzed. The second point would be to use the collected data to develop a targeted training program. And the third point would be to have regular, outside, impartial oversight of the process. Going back to elaborate on the first part. The errors are being reported back to the ROs. However, to the best of our knowledge there is no one consolidated effort to aggregate these mistakes, common errors, and deficiencies that are being noted. If you combine this with common errors and deficiencies noted from the Board of Veterans' Appeals, the Appeals Management Center (AMC), and the Court of Appeals for Veterans Claims, you can develop an overall picture both on a regional office level, perhaps even on a team level, but more importantly on a national level of what the common errors within VA are. And you can use that to determine where to devote the resources to improve accuracy within VA. That leads directly into point number two, targeted training. We have reports and when we have spoken to VA employees on the American Legion quality review visits to regional offices, it is noted and VA notes that when STAR identifies problems they are reported back to the teams and the coaches for correction involved. However, if you can notice a national trend, if you can notice that consistently VA is experiencing problems with, say, rating mental health disorders, or improperly asking for exams, then this can be set into a targeted training program so that VA is getting the most use possible out of their 80 to 85 hours of mandated training, that it is going to correct the areas that you need corrected. If you take a math test and you realize that you are having problems with binomial equations, then you need to go back and do some extra work on binomial equations so that the next time you take that math test you get it right. This is exactly the same sort of thing that VA could use this aggregate data to accomplish. And the third point that I want to make is about outside oversight, third party oversight. In the recent report this morning that was published in the Federal Register on the OIG investigation of Togus, Maine, one of the things that was pointed out, 25 percent of the STAR errors were not corrected in accordance with VBA policy. And two examples that are listed here, STAR instructed the regional office to inform a widow that her child could be entitled to Dependency and Indemnity Compensation (DIC) benefits. There is no evidence in the claims folder showing staff informed the widow of this potential entitlement. Furthermore, the RO staff erroneously informed STAR that they corrected the error. Secondly, STAR instructed the RO to send a notification letter for a burial claim to the proper claimant. While the staff informed STAR that they corrected the error they did not send the revised error to the proper claimant. So clearly, and this is not new that we have seen in the reports of the OIG of the various specific regional offices, even if STAR is capturing the errors they are not necessarily being followed up on, which is why you need third party oversight. You need somebody to go in and double check that they are crossing the I's, dotting the T's, and correcting these errors. VA has potentially a very effective tool here and we want them to be able to use it effectively. And with the three points that we believe will be effective to create an aggregate of that information that can be searched for trends, to use those trends to target the training, and to add outside oversight, we believe that this can be an effective tool for VA. I thank you very much, Mr. Chairman and the Committee, and we would be happy to answer any questions. [The prepared statement of Mr. de Planque appears on p. 48.] Mr. Hall. Thank you, is it Mr. de Planque or de Planque, just---- Mr. De Planque. It is de Planque, technically. Mr. Hall. De Planque, okay, thank you. Just like to try to get the correct pronunciation. You mentioned that training should be more specialized. Who do you think should oversee this training? Should it be standardized at either the RO or national levels, or individualized for each local VA office? Mr. De Planque. What I am proposing here, what the American Legion is proposing here, is to have some national training that is targeted to national areas of deficiency. However, within the individual ROs you also, there are different training staff within the individual ROs. And so when they notice something that is within their particular office obviously they have some initiative to direct the training on their own. However, what we are talking about here is, when you are identifying a national problem that a consistent number, or even a regional problem, that ROs in a certain area are deficient in performance in a certain aspect, such as applying the mental health disability schedule, or properly sending out VCAA notice, then it is important to have the national training specifically craft training that will address the issues that are noted. And that is why it is important to actually have a method for capturing that that takes into account not only STAR but also, as I said, there needs to be a method for capturing the errors that are being seen at the Appeals Management Center, that are being seen at the Board of Veterans' Appeals, and that are being seen at the courts. And if you combine those four elements, STAR, the AMC, the BVA, and the CAVC, then you actually have a big picture, the picture from, you know, 3,000 feet of what is going on. And you can create the targeted training that is going to be more effective. Mr. Hall. Thank you. And just one more question. Sir, you suggested incentives for bonuses in the management of the ROs are driven by quantity rather than quality. How would you change the incentive structure to help encourage the accuracy and quality of claims? Mr. De Planque. There is technically one point that addresses accuracy. However, it does not substantially impact the bonus structure. The vast majority of it is driven by timeliness or quantity of claims. And what the American Legion has maintained for a long time is that the work credit system needs to be addressed. We have proposed, although we are open to other proposals, the idea that work credit not be counted until a claim is finally adjudicated so that you can determine that it was correctly done every step of the way. That may be a more complicated system. However, some system that takes into account the accuracy of the claim in counting the credit for that work rather than simply that the claim was passed along to another desk needs to be implemented. As the system stands right now where you are counting people on three and a half claims a day rather than three and a half correctly done claims a day, that is where you are running into the problem. If you start counting them on accurately and effectively completed claims, then that is when VA is going to be in a situation where the incentive is to get the claims done right and not to get a large number of claims done. Mr. Hall. Thank you. Mr. Kelley, in your testimony AMVETS asserts that the STAR program should be expanded to ensure that specific programs such as BDD and specific conditions can be tracked for anomalies that occur so that improvement strategies and specific training can be implemented. VA has claimed that the benefits of such changes do not surpass the cost of these measures. Do you concur with that position, and if not why? Mr. Kelley. I do not agree with the, that it is not worth the cost. These are veterans who are waiting for a benefit, an accurate benefit. And I do not think there should be a cost analysis against what veterans are waiting and deserve to have done accurately. Mr. Hall. You point out that not only is training for VBA raters needed but sometimes retraining is needed to reteach key lessons based on updated procedures. In the 10 years since STAR's establishment, the VA has yet to surpass a 90 percent rating accuracy. With that in mind, do you think that the STAR review team needs to be retrained to make sure that they are correctly employing the STAR quality control system? Mr. Kelley. I have not really put any thought into that. It is a good concept. I think just having enough people in place within the STAR, I think they had 24 billets and they were using eight to process these, you are putting a lot of pressure on individuals again to do a lot of work in a little bit of time. I think making sure that they have the right resources and that they are trained properly, and retrained. If you start seeing deficiencies it is absolutely time to go back and retrain. Mr. Hall. Thank you, Mr. Kelley. I just wanted to comment that we had three veterans who were from, members of AMVETS, with their service dogs in our office last week. And it was a remarkable experience. My dogs were very interested when I got home. But it is a great program and I want to thank you for the organization's work with service dogs, and trying to make them more available to all veterans who need them. Mr. Abrams, you suggested VBA should increase its claim processing accuracy by employing a system of awards and disincentives. What would the system look like in your mind? Mr. ABRAMS. Well, if you adjudicate claims from the point of view of the veteran as opposed to VA, and the disincentive is if you do not do it right and the appellate period extends, the managers in the regional office cannot claim work credit, that is a disincentive. If you give the managers the incentive to do it right because you are going to reward them with work credit, they are going to encourage their workers to do it right. VA workers will work to what their managers want them to do, as do postal workers, and patent office workers. And you have read about the problems with those groups. Set the system up so that what the managers want is what we want, quality adjudications. Then we could also reward people who do good work and plenty of it. But instead of just plenty of it work, that is not enough. That is a good way to make people do the right thing, and get benefits promptly to our veterans. Mr. Hall. Which was what this Committee intended in the legislation we passed in 2008. Mr. Abrams. We certainly support that. We were happy that we did it, and we anxiously await the response. Mr. Hall. As do we. You suggest in your testimony that VBA staff, or some VBA staff, take an adversarial attitude toward claims filed by veterans seeking benefits for mental conditions such as PTSD and TBI. What examples do you have of such bias that you can come up with off the top of your head? Mr. Abrams. If you want, we have plenty of cases where not only the regional office but the Board of Veterans' Appeals denied claims even though the veteran met the requirements in the diagnostic code to get the higher evaluation. In fact we had one, I just trained on it yesterday, where we screened it. It was a BVA denial, and the VA General Counsel did not even let us brief the case. They said let us award the benefit. They completely conceded. There is a reason. There is something going on where the VA does not want to evaluate properly. And I am not saying every case. We are talking about a certain percentage of cases that need to be corrected. But we are finding a high level there. And we have other instances, but I would have to go back through files and go over them, and if they are public I can share them. Mr. Hall. I would appreciate that and I am sure the Committee would. Mr. Wilson, in your testimony you indicate that the primary problem with the current accountability system is that VBA employees are not usually held responsible for their errors. In what specific way would you propose that VBA hold its employees accountable? Are Mr. Abrams' suggestions similar to what you would suggest? Mr. Wilson. We view the STAR program as a quality assurance program. It does what it does rather well generally speaking, although it certainly has an error rate that is nonetheless unacceptable. But it does measure the process itself. It tells you what the results of that process will provide. It does that. What it does not provide, however, is quality control. You do not have a quality control program where you can determine if rating decisions are correct based on the merits of the case. There is no effective program in place beyond each review to determine whether this process is giving the correct rating decisions. What we need is a program that will provide quality inspections and deliverable quality peer reviews of the of the work done at the end of the day. This work, currently by coaches, is done only on an occasional basis, unfortunately. What is needed is a program that would aggregate this quality data, roll it up and provide VBA a national level, trend analysis providing opportunities for recognition and training, focused on various regional offices or individual employees. That data could also be tied into the kinds of compensation packages and appraisals that would be provided to not only the senior supervisors, but to all individuals in the VBA. Mr. Hall. Your testimony alludes to the benefits of creating a new, more robust IT system that would incorporate many different measurements of accuracy. We heard from our IT witness previously that at least temporarily in Baltimore there may be a lengthening of the time or an increase in inaccuracy, a decrease in accuracy, due to the fact that they have siphoned people off to work on the virtual RO who otherwise would be working on the normal, old-fashioned way of processing claims. So I am just curious what your thoughts are about this move that is being attempted into the virtual world and increased use of IT? Are there less costly alternatives? Or should the VA pursue this and keep going down the road that they are currently on. Mr. Wilson. It is an important initiative and I would like to respond for the record to the question. [Mr. Wilson subsequently provided the following information:] VA must continue moving forward on the path towards a fully electronic IT system--the Veterans Benefits Management System (VBMS)-- which will become the backbone of the claims processing system. Although there may be temporary losses in production, and potentially accuracy as well, while new methods and technology are developed and implemented, VA must remain focused on long term reform. Over the next few years, VA should seek to minimize such transitory losses of production and quality through increased resources and personnel. If VA were simply focused on reducing the backlog quickly they could probably achieve some short term reductions through non- technological means. However that would neither be sustainable nor conducive to creating a claims processing system that ensures veterans receive accurate and timely benefits. We also continue to have concerns about whether VA's focus on production could come at the expense of quality and accuracy, and how that is influencing the development of the VBMS. Given the enormous pressure on VA to reduce the backlog, we believe that an independent, outside expert review of the VBMS development program would provide Congress with greater certainty that VA is on the right path, as well as provide early warnings if changes to the VBMS program are required. Mr. Hall. Okay. I have more questions for all of you that we are going to submit in writing because of the possibility of votes being called, and we still have another panel to hear testimony from. We will send you some questions in writing in addition to the ones that have been asked already. But one last one maybe each of you could respond to. According to VA's assessment of accuracy we range from a high in Des Moines of 92.34 percent accuracy to a low in Baltimore of 69.33 percent, and Washington lower than that, although we do not have a number. I am sorry to say that New York, my home State, is close to the bottom at 76 percent, according to VA's estimate which the OIG suggests may be actually optimistic by as much as ten percentage points. But I, maybe starting with Mr. Abrams you could tell what you suggest that the VA do to take, to learn from those who are getting the best results? Mr. Abrams, do you want to---- Mr. Abrams. We have been to Des Moines to do a quality check, and outside of the fact that we could not get back to DC from there, which is a long story, and an ice storm somewhere involved, too, we did not find such a high quality of work. It was better than some offices, but not at 92 percent. In my experience, I do not really trust VA statistics except to note that they are generally worse than what are reported. I say that because I have seen that over 36 years. However, some offices in rural areas attract a higher level of worker. And we do not talk about that a lot, but working for the VA in New York with the kicker to get a little more money does not give you the same quality of life as working in Des Moines. So one of the problems is attracting a higher level of worker in the regional office. Also, a smaller office tends to be on top of things a little more. The worst office we found was St. Petersburg, the largest one. It was a disaster, in my view. Mr. Hall. Right. And they are about two-thirds of the way down the list here, but not all the way at the bottom. Mr. Abrams. Well, I am only looking at what we actually found. And since I did a lot of those, you know, cases I can talk to it personally. Mr. Hall. Sure. Mr. Wilson. Mr. Wilson. This is a response for the record on that as well, sir. [Mr. Wilson subsequently provided the following information:] Rather than focusing on the disparities in performance at various Regional Offices, we would emphasize that quality and accuracy must be prioritized in every location, as well as at level of leadership and management. While VA's STAR program provides a standard assessment of technical accuracy of claims work, it fails to provide true quality control at every stage of the process. Furthermore, the IG's review of the STAR program found that it failed to properly measure even technical accuracy, with about an additional 10 percent error rate going undetected. The only solution is for VA to foster a management culture that measures and rewards quality of results, not just quantity, and provides sufficient training of VA's management and workforce in order to achieve this outcome. Employees at all levels of the organization must be encouraged to provide constructive feedback at each step in the claims process to ensure that optimum performance is achieved. Creating and maintaining such an atmosphere will benefit employees, management and veterans. In the long run, the only way that VA will ever reduce and eliminate the backlog of overdue claims is by building a quality system that `gets it right the first time.' Mr. Hall. Thank you. Mr. Kelley. Mr. Kelley. I think if you look at the ROs there is a personality that comes out in each one of these ROs. And in that there is a culture. And we need to figure out how can we get everybody to look at things the same way. Localized training on certain issues that have been identified as a true problem for that one area need to be identified, but there needs to be national, standardized training throughout to ensure that everybody is thinking from the same book and everybody is moving forward with the same thought in mind. Mr. Hall. Thank you. Mr. de Planque. Mr. De Planque. Well the simple answer that you would want to do in a case like that is figure out a way to clone Iowa, if that is your top performing thing, and replicate it everywhere else. But as Mr. Abrams pointed out, you do not necessarily have the same base, the same resources, the same workload at every regional office. And so there are going to be fluctuations. We agree completely on what you have to address in the pool of workers that you can attract with the salaries that are there for VA. Obviously, any employment is great in a difficult economy. But people want to be competitive, and they want to be in a position to do that. And in some areas where the cost of living is much higher it is less attractive and so that is difficult. And we do not talk about that a lot, but it is something that is there, and it is there to consider. The nationally standardizing training and having good, solid means of national analysis to see, ``Hey, how do we replicate the Iowa results in other offices?'' Is there a way to condense that, put it into a training package and get that to everyplace else? That is what you are dealing with. But also in each regional office there is different staff, there is different management. They are all trying to do the same task, but you have a different climate in each one. The accuracy rates that we have seen have been fairly consistently, when the American Legion does quality reviews, have been fairly consistently in the 25 percent to 33 percent range of error, which is, and that is an across the board sort of average. Which is below obviously what the STAR averages are finding, and if you look across the board, then something like 92 percent may be optimistic, although you should still consider that that is better than what a number of others are doing. So in addition to finding out where you are seeing the deficiencies are, also look for where you are seeing the strengths are and use that in your training programs to, again, like we said, create targeted training. Create training that is targeting what you know is both positive and negative about your organization and to correct those errors. Play to your strengths, and beef up your weaknesses. Mr. Hall. Thank you, Mr. de Planque, and Mr. Kelley, Mr. Wilson, Mr. Abrams, for your written and oral testimony, and for your patience while we wait for votes to happen today. It is always good to hear from you and to see you. So I thank you for your service to our veterans and to our country. And the second panel may now be excused. And we will ask our third panel to join us. Bradley G. Mayes, the Director of Compensation and Pension Service for the Veterans Benefits Administration, U.S. Department of Veterans Affairs; accompanied by Edna MacDonald, Acting Deputy Director of Compensation and Pension Service of VBA; and Terence Meehan, Director of Employee Development and Training of the VBA. Thank you all for being here. Thank you for your patience. As you know, your full written statement is already made a part of the hearing record so feel free to abridge it or add to it, or whatever you would like to do in 5 minutes. Mr. Mayes. STATEMENT OF BRADLEY G. MAYES, DIRECTOR, COMPENSATION AND PENSION SERVICE, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY EDNA MACDONALD, ACTING DEPUTY DIRECTOR, COMPENSATION AND PENSION SERVICE FOR POLICY AND PROCEDURES, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; AND TERENCE MEEHAN, DIRECTOR OF EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS Mr. Mayes. Mr. Chairman, thank you for providing me this opportunity to discuss the Veterans Benefits Administration's quality assurance program and the positive effect it has on the processing of veterans disability claims. And joining me today, as you said, are Edna MacDonald to my right, Acting Deputy Director of Compensation and Pension Service for Policy and Procedures; and to my left, Terence Meehan, Director of VBA Employee Development and Training. Mr. Chairman, I have prepared remarks. But let me just say before I get into that, that I agree completely with virtually every panelist that has been up here, that quality must be at the core of our philosophy for serving veterans. There is no question about that. The Subcommittee has indicated a special interest in our Systematic Technical Accuracy Review program. However, I want to emphasize that STAR is only one of four tiers of our multifaceted national quality assurance program. The STAR component focuses on claims processing accuracy, while the other three components address regional office oversight, rating consistency, and special focus reviews. Along with the STAR program, these components collaborate to ensure high quality and consistent decisions for veterans. STAR is the quality assurance component that focuses on claims processing accuracy. STAR reviews are focused on outcomes for veterans rather than specific processes by which the outcomes are reached. STAR reviews evaluate the quality of the rating decision product that VBA provides for veterans. It is from the veteran's perspective that there is an expectation that we understand the claim, evaluate it accurately and fairly, and provide proper compensation under the law. The purpose of STAR reviews is to ensure that rating decision outcomes meet these expectations. The STAR program incorporates review of three types of work, claims that usually require a rating decision, authorization work that does not usually require a rating decision, and fiduciary work. A determination of benefit entitlement accuracy for rating and authorization work utilizes a structured checklist to ensure all issues were addressed, claims assistance was provided, and the decision was correct. Accuracy results are calculated on the results of the benefit entitlement reviews. STAR findings provide statistically valid accuracy results at both the regional office and the national level. VBA continues to focus on expanding and improving the STAR program. In 2008, the STAR staff was consolidated to Nashville, which provided space for expansion and allowed aggressive recruitment for more staff. Since then, STAR has completed an extensive expansion effort, more than doubling the staff and increasing the sample size to obtain a statistically valid sample at the regional office level. In 2010, quality review of fiduciary cases was also transferred to Nashville. During fiscal year 2009, a little over 20,000 cases were reviewed under the program. The targeted number of cases for review in 2010 is 37,932. The STAR sample was expanded in 2009 to include a review of brokered work completed by VBA's resource centers and Tiger Team. Sampling was increased for the pension management centers to allow measurement of pension entitlement decisions. Ongoing reviews of the disability evaluation system cases, and Appeals Management Center cases became part of the monthly compensation quality sample in fiscal year 2009. The VBA quality assurance program has undergone significant change over the past several years, and has become more comprehensive by expanding the type and breadth of cases reviewed. But we acknowledge this is a journey that we are on, and we look forward to working with our stakeholders to continuously improve our quality assurance program. Mr. Chairman, thank you for the opportunity to provide you an update on our accomplishments at this hearing. And with that, I would be glad to answer any questions that you may have. [The prepared statement of Mr. Mayes appears on p. 51.] Mr. Hall. Thank you, Mr. Mayes. I appreciate the challenge that you face, and the need to evaluate things even as numbers are growing, numbers of staff are growing, numbers of claims being brought in by the new diseases that are service-connected automatically, not to mention the returning Operation Iraqi Freedom/Operation Enduring Freedom (OIF/OEF) soldiers and veterans, and the aging of our older veterans population. But I am curious just, you know, for instance what action you have, or have taken, or are considering taking regarding for instance Baltimore or Washington given the disparity between, you know, their accuracy rate under your evaluation and the more successful. I mean, I would just guess that maybe Nashville is rated near the top because STAR is based there? Is that a correct assumption? And, but what, is there something underway already to try to bring Baltimore and Washington up. Mr. Mayes. Well first, I will tackle that last one. I do not think it hurts that we have our quality assurance operation in Nashville, although it is not actually in the regional office. But in Nashville you had a director who I understand is going to be retiring soon, Brian Corley, who came up through the system understanding the C&P program and the adjudication process, and was very dedicated, I know personally, in training and focusing on developing his workforce. He knew the value of developing that workforce because that in the end would reap a long-term reward. While you may have to pay the price early on to train these employees, down the road they would become more effective. He preached that, and he enforced that. Now, I think one of the earlier panelists mentioned that this has to be a philosophy, and I echoed that. I am not suggesting that in Baltimore they do not have that philosophy or they do. What I am saying, though, is what the service has to do, is help shine the light on areas where there are deficiencies. STAR helps do that. With STAR, you have that array of the quality by regional offices. But we also do site surveys. We have gone into Baltimore on a site visit, and we have highlighted some areas that they need to focus their attention on. We work closely with our operations element through our Office of Field Operations. We share this information with them. And then the expectation is that the management team will execute on these recommendations. That is how this process works. We have provided that information, and I believe that they have set about executing a plan for addressing these deficiencies that we have identified. It has resulted, I believe, in a higher quality rating. Mr. Hall. Does the VA keep records of how performances changed or improved with new leadership at the ROs? For instance, in New York, I know, as it serves my constituents in the Hudson Valley and other New York regional veterans, recently had a new director appointed. And I am curious if, well currently in this ranking has the fifth lowest accuracy rating. Is that changing? Have you had time yet to ascertain whether the quality, the accuracy rate has started to improve since the new director has taken over there. Mr. Mayes. Mr. Chairman, I want to be clear. I am beginning to get out of my lane a little bit. The way we are organized, with me being the head of the policy component in VBA, our program goes in and takes a look and identifies these gaps, and then we work closely with that operation's component to then come back and rectify those deficiencies. My understanding is the Director just reported there. So I think it might be too early for us to tell whether some of the countermeasures that she has put in place are beginning to reap some benefit. Mr. Hall. Do you have accuracy ratings for the previous year, 2008? Mr. Mayes. We do. It is a 12-month cumulative accuracy score, so it is looking back at the 12-month cumulative number. We have those numbers going back historically, yes. Mr. Hall. Thank you, Mr. Mayes. Could you provide the Committee, please, with any changes in the directors, the different ROs within the last 2 years? And any correlation that you see in terms of the accuracy before and after the new management took their office? That is something, just homework for you. [The VA subsequently provided the information in the answer to Question 5 of the Post-Hearing Questions and Responses for the Record, which appears on p. 58.] Mr. Mayes. All right, we can provide that. Mr. Hall. I think it is worth looking at. Would you address the question that the previous panel was talking about, in terms of incentives and disincentives, to try to encourage the higher accuracy performers and maybe disincentivize those who are not producing the same rate of accuracy? Mr. Mayes. Well one thing I would say is that accuracy is in the performance plan for every individual that is involved in the claims process, from the director right down to the claims examiner who is developing the claim, to the RVSR who is rating the claim. Every one of those decision makers, and the managers in that process, every one of those employees has a critical component related to accuracy. So the assertion that we somehow do not value accuracy is not borne out by how we measure individuals' performance. It is production as well, but there is equal weight in the performance plan for all of those employees for accuracy. Mr. Hall. I appreciate that. And I note that it is a complicated thing that is not explained by any one factor. For instance, you do have areas where it may be easier to attract, due to the cost of living and the standard of living one can achieve on a given salary. Just as with air traffic controllers, or with doctors in the private medical field and, you know, other areas. There is obviously in Des Moines, as was pointed out before, a chance to do better on a given salary than in New York or Washington. But there has been conversation in the veterans community, in the public press, and here in Congress and in this Committee about the evaluations, year end evaluations and bonuses that are given to people at different, in different positions, at VBA. And, you know, it seems to me that that is something that I would look at if I were going to try to get some teeth into this quickly. We need to try to motivate people to try to get up to where Des Moines is. In fact, we need to get more Des Moines here. So are you looking at or have you already made any changes in the way bonuses are set? Mr. Mayes. Well Mr. Chairman, I am making the assumption that you are talking about the senior leadership level. I am not involved in the process by which the bonuses are determined. Mr. Hall. But I believe it goes down below that, also. Are not bonuses given out to career VBA employees? Mr. Mayes. Yes. We can certainly give performance awards to employees, up and down the line, across the spectrum of employees. I was saying, quality, even for the journey-level employee that is processing this work, is a core component of their performance plan. I know when I was in Cleveland as the Director, or previously in Milwaukee, that if somebody was failing to achieve their performance plan target for quality, then that counted against them in the mix with respect to who was going to get performance awards, because we valued quality. Mr. Hall. So if we looked at offices that are not performing as well we would see lower year end bonuses or performance awards? Mr. Mayes. Mr. Chairman, because I have not been involved in that process. That is one I have to take back for the record. [The VA subsequently provided the information in the answer to Question 6 of the Post-Hearing Questions and Responses for the Record, which appears on p. 58.] Mr. Hall. Okay. We will try to find out, and if you could find out for us that would be good. Is it safe to conclude since Atlanta, Chicago, New York, L.A., and Baltimore are all near the bottom of the list that accuracy levels are driven down by volume? Mr. Mayes. I think that Mr. Abrams made some points about smaller stations. It is easier to control your workload. I think that is a valid observation. But there is evidence that larger stations can successfully manage that workload. It can be more challenging because you have a lot more work coming through there; you have a lot more employees that you have to manage. Even though the span of control at the first line supervisor level may be the same, there certainly are some complexities when you have a large operation. Edna, do you want to add to that? Ms. MacDonald. I believe that some of the other panels already talked about the challenges in hiring and retaining a good workforce. As you point out, most of those are significantly urban cities. We do recognize that is a challenge for us to compete and to retain a workforce. Not that we cannot do it, but it is sometimes easier in your smaller stations like Muskogee, which is a very large office for us, but we are able to retain a very strong workforce. Mr. Mayes. That is precisely why we built out our quality assurance operation in Nashville because we were having a challenge in recruiting and retaining employees at that grade level here in the Washington, DC, area. I know the OIG report mentioned our site survey staff and the difficulty that we have had in getting employees to come in and stay and do that work. We have made the decision we are going to move that part of our operation to Nashville as well because we are achieving more success there. Mr. Hall. And when is that going to happen? Mr. Mayes. We have an announcement on the street right now for employees for part of that operation in Nashville. So as soon as we can through the hiring process, we will begin that transition, just like we did for the reviewers. We moved that from Washington to Nashville. Mr. Hall. Could you please provide us with, if you have it, the incentives or inducements that the Department is offering to hire people in the cities that you are having a harder time retaining or hiring staff in? Just so we have some idea of what the range is. As I said, there are other branches of government that are doing the same thing for the same reasons. And it would be helpful for us to know that. [The VA subsequently provided the following information:] VBA offers relocation incentive bonuses on a case-by-case basis for management positions in geographic areas where recruitment and retention of a skilled and qualified workforce are challenging. These challenges historically occur in areas where:Employers outside of the Federal Government pay higher salaries for similar positions; Local candidates do not possess the unique competencies required for VBA positions; or, The geographic location, position, or duties are undesirable to qualified candidates. All relocation incentive bonus requests are pre-approved by the Associate Deputy Under Secretary for Field Operations and the Under Secretary for Benefits prior to announcing the position. Relocation incentive amounts typically range from $5,000 to $15,000 depending on the position, location, and past recruitment efforts. OIG reports that the STAR quality control system was hindered by VBA's exclusion of brokered claims from STAR reviews. Specifically, based on OIG's 2009 review it found a 69 percent accuracy rate. While VBA initially, as I understand it, resisted the idea of reviewing brokered claims there is an indication that recently you have embraced this recommendation and begun to review those claims. What impact do you believe a review of brokered claims will have on the overall accuracy rate? Mr. Mayes. Well, we are reporting it now. We started with cases worked in April. Right now the national accuracy rate for brokered work is 77 percent. For the Tiger Team it is 78 percent. When you factor that in to the national numbers, I think it stays the same at 83 percent. Is that correct, Edna? Right now given the sample sizes, it is not impacting the overall nationwide accuracy rate. But I think the fact that we are looking at this work now is important, because it means that folks that are promulgating decisions out of our resource centers know that there is the possibility that you are going to have your case selected for review at that national level. Mr. Hall. How do you explain the difference between STAR comments on a claim and issues identified as errors? What can you do, or what are you doing, to ensure that all issues are correctly identified as errors or comments. Mr. Mayes. Well, when we select our sample, we are sampling claims that have been completed over a 1-month period previous to the month that we are doing the reviews. The sample is driven by the end product control that is associated with the work that was done. The way we set our system up, we are doing the review for the claim that was at issue associated with that control. But sometimes during the course of a review we might find where we missed something, but it is not associated with the claim that is subject to the review. So we will annotate that as a comment, rather than call an error on the claim that is subject to the review in this particular review. So what the OIG commented on was that for error calls where we are saying, ``You made a benefit entitlement error on that particular claim,'' we were requiring a second level review. But in those reviews where there was no error associated with that claim, but we noted a comment, we were not requiring a second level review. We concurred with the OIG said that we need to have the second level review not just on those cases where we call a benefit entitlement error but also in those situations where we found something in the record that warranted a comment. We have implemented that. So that was something that we agreed with the Inspector General on. Mr. Hall. Thank you, Mr. Mayes. Thank you, Ms. MacDonald and Mr. Meehan. I am grateful to you for the work you are doing and for coming here again, yet again, to tell us about it. I will also ask you to put yourselves in the shoes of those veterans in Washington and Baltimore, New York, and in areas served by the other RO regions which according to the OIG, have quality levels maybe as much as 10 percent below what your numbers show. And I realize you are dealing with a very complicated situation. But each veteran is also dealing with a very complicated situation that they may not have the resources to deal with. So I know you understand the seriousness of this, and the Committee does also. You and all the witnesses will have, and Members of the Committee will have, 5 days to extend or revise remarks. And thank you to all of our panels. The hearing is now adjourned. [Whereupon, at 4:55 p.m., the Subcommittee was adjourned.] A P P E N D I X ---------- Prepared Statement Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs Good Afternoon. Would everyone please rise for the Pledge of Allegiance? Flags are located at the front and back of the room. Ladies and gentlemen, I welcome you all to the halls of Congress in what has been a profoundly historic and important week for the nation and for our veterans. Over the past seven days the Full Committee convened a successful Claims Summit which brought together dozens of the nation's top veterans' stakeholders including VA officials, GAO representatives, Veterans Service Organizations (VSOs), veterans' advocates, and academia and even industry leaders in this area. The big news from the Summit is that VA admitted what many of us have known for a while, the system for processing compensation and pension (C&P) claims is broken beyond repair, and must be replaced. We welcome this acknowledgement and look forward to working with VA in developing this new system. Of course, on a rare Sunday session, Congress passed, and yesterday, the President signed part one of the sweeping package to reform the nation's health care system. I for one look forward to the second part of health reform reaching the President's desk. Finally, this week, we have passed a number of bills in the full House that will significantly help our veterans. These include the End Veteran Homelessness Act of 2010 to provide $200 million in support of Sec. Shinseki's goal of ending homelessness for America's warriors. We also passed the Helping Heroes Keep Their Homes Act of 2009, sponsored by our colleague Tom Perriello, which would protect home mortgages of veterans and servicemembers. We also passed the disability compensation COLA bill, that would allow a Cost of living increase by the end of the year, also sponsored by Mr. Perriello. The National Guard Employment Protection Act of 2010 which aims to help preserve jobs of soldiers ordered to full time deployments with the National Guard is set to be voted on today. And I remain committed to helping Chairman Filner push legislation to help our veterans who were harmed by Agent Orange exposure. In this afternoon's hearing entitled: Examination of VA Regional Office Disability Quality Review Methods: Is VBA's Systematic Technical Review System (STAR) Making the Grade?, we will examine the primary quality review tool employed by the Department of Veteran Affairs (VA) to assess the accuracy of disability compensation and pension claims processing. The STAR system can help VBA monitor the quality and accuracy of ratings at its regional offices (ROs). Through this quality control tool, VBA can focus attention on poorly performing ROs and help the agency direct additional staff and training to problem offices. At the end of the day the goal is for VA to get the claim right the first time so that veterans are not improperly denied the benefits they deserve or faced with lengthy appeals cycles. The STAR system was implemented by VBA in October 1998 to improve the measurement of the accuracy of claims processing. Since FY 2007, VBA has set for itself a performance goal of completing compensation claim ratings without error, 90% of the time. Its long term strategic accuracy goal is 98%. Unfortunately, the VA is far from achieving this goal. Until the STAR system provides an accurate accounting of the error rate at the VBA, it is difficult to envision a path for meeting this goal. VA's Office of Inspector General (OIG) and the Government Accountability Office (GAO) produced studies which revealed several issues that impede the efficiency and consistency of the STAR system. Specifically, OIG suggests that any claims ratings accuracy numbers reported by VA should be discounted by as much as 10% to yield an accurate performance measurement. I'm also personally troubled by GAO's finding that VBA claim processing accuracy rate is particularly low in cases filed by veterans seeking benefits based upon Post Traumatic Stress Disorder (PTSD) and/or Traumatic Brain Injury (TBI). Today, we intend to analyze these studies through hearing testimony from representatives of VA's OIG and GAO. VBA has made some improvements to its claims rating system since the implementation of STAR system. We look forward to hearing from them about these improvements and how we can help them with any areas of concern. To fully understand the STAR quality assurance program, it's important to review the C&P rating system itself. Through its disability compensation program, VBA pays monthly benefits to veterans for injuries or diseases incurred or aggravated while on active military duty. To access a claim, the application for disability benefits must be ``developed,'' a process that involves obtaining all necessary evidence of the veteran's service and disability to support the claim. After development, claims go to a Rating Veterans Service Representative (RVSR) for a decision. RVSRs determine if a veteran's disability is service connected and assigns a percentage rating (ranging from 0 to 100 percent) that is intended to represent the average earning reduction a veteran with that condition would experience in civilian occupations. The veteran is then notified of the decision. For reopened claims, the veteran's previously assigned diagnostic codes with evaluations also affect the veteran's overall combined percentage of disability. Regional offices use previously assigned diagnostic codes, along with their percentages, in combination with current assigned diagnostic code percentages to determine the combined level of disability. Once a claim is completed, the result of that claim or ``end product'' is cleared, and work credit is given to the regional office. A completed claim and corresponding cleared end product is then subject to review by STAR reviewers based on a statistical sample of all completed rating end products. In the 110th Congress, I introduced H.R. 5892, which outlined a series of steps to improve the quality of VA's claims processing system. These recommendations formed the core of directives which were codified by P.L. 110-389, the Veterans Disability Benefits Claims Modernization Act of 2008. Today's hearing also provide us a chance to gauge how these quality control measures have been implemented by VA. With that, I look forward to the informed testimony of our witnesses and insightful comments and questions from my colleagues on the Subcommittee. I now recognize Ranking Member Lamborn for his Opening Statement. Prepared Statement of Hon. Doug Lamborn, Ranking Republican Member, Subcommittee on Disability Assistance and Memorial Affairs Thank you Mr. Chairman, And welcome everyone, to this hearing on the Department of Veterans Affairs STAR program. STAR is an acronym for Systematic Technical Accuracy Review, which is VA's program for quality assurance. Throughout my tenure on this Committee, my fellow members and I have called for stronger accountability within the VA claims system. For too long, the primary focus has been on production and this has led to an error rate that is unacceptable. I believe that the VA's greatest challenge, the claims backlog, is largely attributable to hasty decisions made without proper regard for accuracy. The ramifications of this approach can be seen throughout the entire system. Therefore, VA employee performance awards cannot be based entirely on production, there must also be a valid measure of quality. Under the STAR program, a statistically valid sample of rating decisions from various regional offices is reviewed for accuracy. While this method may be useful from a macro perspective, it is not sufficient for ensuring individual accountability. VA must be able to identify employees in need of individualized remedial training. Without this essential component of the quality assurance process, VA will have perpetual problems in its claims system. I also have other concerns based on the written statements of today's participants. There seems to be some contradiction as to whether brokered claims are excluded from STAR review, or whether STAR requires follow-up to ensure that VA Regional Offices provide corrective action or remedial training. Perhaps these were former deficiencies that have since been corrected. I am encouraged by GAO's report that VA has made a number of improvements, but I also agree that challenges remain. In the 110th Congress this committee passed a provision that was included in the Veterans' Benefits Improvement Act of 2008 that required VA to conduct a study on the effectiveness of the current employee work-credit system. I believe the upcoming report, along with the testimony that we will hear today, will provide valuable feedback for the Department to improve its quality assurance and accountability program. I look forward to hearing from our witnesses today, and I thank you all for your participation Thank you, I yield back. Prepared Statement of Belinda J. Finn, Assistant Inspector General for Audits and Evaluations, Office of Inspector General, U.S. Department of Veterans Affairs INTRODUCTION Mr. Chairman and Members of the Subcommittee, thank you for the opportunity to discuss the Office of Inspector General's (OIG) work regarding the Veterans Benefits Administration's Systematic Technical Accuracy Review (STAR) program. I am accompanied by Mr. Larry Reinkemeyer, Director of the OIG's Kansas City Audit Operations Division. The OIG is committed to proactively reviewing the effectiveness of key management controls to assure the accomplishment of mission critical service responsibilities to veterans, such as the delivery of accurate and timely disability benefits. Further, we strive to focus our efforts on identifying control weaknesses before they escalate into significant problems. Over the past 2 years, we issued audit reports covering aspects of VBA's quality assurance process--the STAR, Rating Consistency Review, and Site Visit programs. In March 2009, we issued the Audit of Veterans Benefits Administration Compensation Rating Accuracy and Consistency Reviews (Report No. 08-02073-96). In May 2009, we issued the Audit of Veterans Benefits Administration Compensation and Pension Site Visit Program (Report No. 08-02436-126). Also in fiscal year (FY) 2009, we established a Benefits Inspection Division to provide recurring oversight of regional offices by focusing on disability compensation claims processing and performance of Veterans Service Center (VSC) operations. STAR PROGRAM Improving the quality of rating decisions, which includes accuracy and consistency of disability compensation claim rating decisions, is among VBA's highest priorities. The STAR program is a key mechanism for evaluating regional office performance in processing accurate benefit claims for veterans and beneficiaries. The STAR program provides a comprehensive review and analysis of compensation rating processing associated with specific claims or issues. VBA's FY 2008 and FY 2009 goal for compensation claim ratings was a 90 percent accuracy rate. STAR reviewers select a number of claims to review from VA regional offices nationwide and use a checklist designed to facilitate a consistent, structured review and classify errors into three categories: benefit entitlement errors, decision documentation/ notification errors, and administrative errors. Results During the 12-month period ending February 2008, STAR reviewers found that VBA staff accurately rated about 87 percent of the claims (approximately 767,000 claims of 882,000 claims reviewed). We reviewed a random sample of STAR reviewed claims and found additional errors affecting veterans' benefits that STAR reviewers did not identify. As a result, we projected VBA's accuracy rate for claims reviewed was only 78 percent. The STAR program does not provide a complete assessment of compensation claim rating accuracy. We found that VBA officials excluded brokered claims from STAR reviews. Brokered claims are those assigned to one regional office but sent to another office to be rated. When we combined the results of our review of brokered claims with our review of STAR reviewed claims, we projected that about 77 percent (approximately 679,000 claims) of compensation claims were accurate and VBA's reported error rate was understated by approximately 10 percentage points. This equates to approximately 88,000 additional, or about 203,000 total, claims where veterans' monthly benefits may be incorrect. This difference occurred because STAR reviewers did not identify all errors, and VBA officials excluded a significant number of compensation claim ratings from review. Additionally, VBA officials had not implemented an effective formal training program for the STAR reviewers. We identified five areas in our review of the STAR program where VBA needed to take action to improve its quality assurance and oversight efforts. STAR reviewers did not identify some errors because they either did not thoroughly review available medical and non-medical evidence or identify the absence of necessary medical information. STAR reviewers also misclassified some errors in a way that resulted in the error not being counted against the regional office's accuracy rate. In these cases, they recorded a ``comment'' instead of an ``error'' although the errors clearly affected veterans' benefits entitlements and should have been counted as errors. STAR management required regional offices to report quarterly on actions taken to correct benefit entitlement errors but they did not require or follow up to ensure regional offices took corrective actions on comments made by STAR reviewers. Ultimately, they relied on the regional office to take corrective action on all issues identified whether the STAR reviewer identified an error or a comment. From our sample, we identified 33 compensation claim ratings where STAR reviewers made comments instead of reporting issues as errors. At least six of the comments related to issues that could affect the veterans' benefits. We found that regional office staff had not corrected any of the six comments potentially affecting the veteran's benefits. VBA officials excluded brokered claims from STAR reviews. VBA officials told us STAR reviewers do not review brokered claims because the STAR program's primary focus is on assessing and reporting rating accuracy for each of the regional offices. Since two or more regional offices are involved in brokered work, VBA officials stated it would be difficult to assign responsibility for the rating accuracy to one specific regional office. Thus we found that STAR management was replacing brokered claims selected for review with non-brokered claims. We reviewed a sample of brokered claims that were not evaluated and found a 69 percent accuracy rate. The STAR reviewers did not ensure regional offices submitted all of the selected compensation claim ratings to the STAR program for review. Regional offices did not submit about 600 (7 percent) of the approximately 9,000 requested claim ratings for the 12-month period ending February 2008. We reviewed 54 of the 600 pending requests and identified 12 (22 percent) benefit entitlement errors. STAR management relies on regional office staff to submit the requested claims and only follows up with the regional offices that do not submit any requested claims for a given month. A STAR manager stated they did not have sufficient resources to follow up on individual claims that regional office staff do not submit. Therefore, regional office staff can cherry pick claims because STAR reviewers do not reconcile the claim requests. This control weakness provided opportunities for regional office staff to withhold claims if they suspect the claims to have errors. STAR reviewers are not required to complete formal training on an annual basis. The reviewers met infrequently to discuss issues, and had no set formal training schedule or requirements. Regional office staffs that prepare and complete ratings and awards for compensation claims are required to achieve 80 hours of training per year to stay competent on laws, policies, and processes on rating-related issues. However, the STAR program manager stated their program workload requirements currently do not allow for the amount of training time necessary, yet agreed the program staff could benefit from formal training. We recommended that VBA: Ensure STAR reviewers evaluate all documentation related to the claim selected for review. Establish a requirement that all STAR reviewer comments receive a second review to make sure the comment was not a benefit entitlement error. Establish procedures to review brokered claims as part of the STAR program. Enforce procedures requiring regional offices to submit all requested claims to the STAR program office for their review or submit written justification to the STAR program's office requesting to exclude the claim from the review. Establish minimum annual training requirements for STAR reviewers that are comparable to regional office rating staff training requirements. VBA agreed with all five recommendations and reported that it had completed actions to implement the recommendations in our March 2009 report. RATING CONSISTENCY PROGRAM To address variances in compensation rating decisions within individual VA regional offices and across the Nation, VBA developed a rating consistency review plan that included metrics to monitor rating consistency and a method to identify variances in compensation claim ratings. VBA's plan would identify unusual patterns of variance in claims, and then review selected variances to assess the level of decision consistency within and between regional offices. However, as of March 2009, VBA had not fully implemented its rating consistency review plan. Results In FY 2008, VBA officials identified 61 diagnostic codes where offices appeared to be making inconsistent decisions in the evaluation of a granted claim or whether offices granted or denied the claim. VBA officials planned to conduct 22 of the 61 reviews in FY 2008 consisting of 20 grant/denial rate and 2 evaluation reviews. However, they only initiated two grant/denial rate reviews and did not complete either review until December 2008. Additionally, VBA did not initiate either of the evaluation reviews designed to reduce variances in compensation claim ratings. In March 2010, VBA informed us that insufficient staffing prevented them from completing any consistency reviews in FY 2009. However, they have now hired the necessary employees. The first FY 2010 review was completed in January 2010; a second review started in February 2010. We identified three areas that impaired VBA's ability to fully implement its rating consistency review plan. VBA did not have an accurate universe of claims to review. From April through May 2008, VBA officials encountered a delay in completing planned consistency reviews because the universe of claims included completed claims with diagnostic codes outside the scope for the requested period. VBA officials notified the Office of Performance, Accountability, and Integrity of the data integrity issue and worked with Compensation and Pension (C&P) Service staff to identify an appropriate universe by June 2008. Data captured by STAR reviewers contained veterans' personally identifiable information (PII). In July 2008, VBA officials stopped all consistency reviews until they could take action to secure PII in STAR's electronic records used to capture and review data and analyze the results. VBA officials took the necessary actions to correct this condition and secured the database in December 2008. VBA officials did not assign a sufficient number of staff to accomplish consistency reviews. The STAR program office was authorized 26 reviewers for rating accuracy and consistency reviews, and had 18 reviewers on board. However, only 8 of the 18 reviewers conducted consistency reviews. The eight reviewers tasked with completing reviews of consistency were not sufficient to complete the assigned work. A STAR manager estimated that approximately 12 reviewers were needed to complete the 22 planned reviews. However, in addition to the consistency reviews, STAR management also assigned these same eight reviewers to conduct at least seven special focus reviews involving thousands of compensation claim ratings. We concluded that the assigned staffing was insufficient to complete this planned work. As part of future rating consistency review plans, inter-rater reliability reviews (IRRR) should be included, along with the annual rating consistency reviews. In July 2008, C&P Service staff conducted an IRRR and found that 76 (31 percent) of the 246 participants incorrectly rated a relatively simple back strain claim. In August 2008, C&P Service staff conducted another IRRR and found that 30 (13 percent) of the 247 participants incorrectly rated a Post Traumatic Stress Disorder compensation claim. According to VBA officials, the most common errors were incorrect evaluations due to misinterpretation of the appropriate facts and criteria. C&P Service managers used the results of the IRRRs to plan focused training efforts, and they plan to conduct follow-up IRRRs to evaluate the effectiveness of the training. The IRRRs allow VBA officials to target a single rating issue to ensure the consistent application of policies and procedures nationally. We recommended VBA develop an annual rating consistency review schedule and complete all planned reviews as scheduled, dedicate sufficient staff to conduct consistency reviews in order to complete planned workload and reviews, and include inter-rater reliability reviews as a permanent component of their consistency review program. VBA agreed with these recommendations and reported that it had completed actions to implement the recommendations. Site Visit Program The C&P Service's Site Visit program was established to ensure centralized oversight and provide technical assistance to VBA's 57 regional offices. Site Visit teams monitor compliance with policies and procedures and identify best practices to assist in achieving high performance. This includes determining if regional offices are correcting errors identified by STAR teams. Results The Site Visit program lacks an adequate infrastructure and management strategy to meet its mission and goals. We identified three areas that VBA needed to address to leverage the benefits of their Site Visit program. The Site Visit team experienced significant turnover during FY 2006 and FY 2008 and has never been fully staffed to the allotted eight full-time equivalent positions. Program officials stated that they have been unable to maintain adequate staffing of the Site Visit program because of difficulties in recruiting from field offices qualified candidates who are willing to relocate to the Washington, DC, area. In addition, we found that C&P Service cannot ensure that onsite evaluations are performed in compliance with generally applicable governmental standards for independence or that sufficient independence exists between the Site Visit program's employees and VSCs reviewed. C&P Service did not review all 57 VSCs in any 3-year period, and 7 (12 percent) of 57 VSCs were only visited once from FY 2001 to FY 2008. Because the planned 3-year cycle of review coverage has not been met, potentially low-performing VSCs who could most benefit from a Site Visit program evaluation may not be visited frequently enough. In addition, C&P Service does not have formal policies and procedures to ensure site survey protocols are modified to reflect emerging C&P issues and systemic deficiencies identified during site visits. C&P Service has not established procedures and guidelines to identify and disseminate best practices. Also, C&P Service has not developed reports that adequately develop the causes of errors identified, and a follow-up process to ensure that action items are resolved. In addition, C&P Service does not adequately identify and report system-wide trends to senior VBA managers, thus missing out on opportunities to proactively address issues and concerns found during individual site visits nationwide. We recommended C&P Service: Develop a staffing plan to ensure that sufficient resources are made available to complete VSC reviews on a 3- year cycle. Comply with generally applicable government standards for independence when performing site visits. Develop a procedure to continuously monitor and update protocols to address systemic issues identified during Site Visit reviews, management concerns and priorities, and changes in program operations. Develop a process for the identification of best practices and resolution of inconsistencies in the application of policies and procedures. Develop and implement policies, procedures, and performance measures to strengthen follow-up on corrective action plans developed by regional offices on issues identified during onsite evaluations. Ensure Site Visit reports issued for VSC operations to more fully identify the root cause of issues affecting VSC performance. VBA agreed with all six recommendations and reported that it had completed actions to implement the recommendations. VBA agreed with these recommendations and reported that it had completed actions to implement the recommendations. OIG REGIONAL OFFICE INSPECTIONS RESULTS STAR management relies on the regional office managers to take corrective action on all issues identified by the STAR team. Since April 2009, the OIG's Benefits Inspection Division has issued eight reports that include a review of regional office procedures to ensure the accurate and timely correction of errors identified by the VBA's STAR program. We found that regional offices did not have formal procedures established to ensure employees took corrective actions on the identified errors and as a result, five of the eight regional offices had not corrected all of the errors identified by the STAR team. Our analysis of 145 errors identified by STAR found that regional office staff did not correct 40 (28 percent) of the errors. Further, regional office staff erroneously reported to STAR that 21 (53 percent) of those 40 errors were corrected, although no corrections were made. We will continue to review and report on regional offices' performance in correcting errors identified during STAR reviews in future OIG benefit inspections and to report on other issues affecting accuracy and timeliness of claims processing. We will also confirm during these inspections whether the actions taken by VBA to implement our recommendations to improve the C&P Site Visit and Rating Consistency programs were effective. Currently, we plan on conducting up to 18 inspections in FY 2010, which means that each regional office will be inspected on a 3-year basis. Once we have sufficient data, we plan to issue roll-up reports that identify trends affecting accuracy and timeliness and include recommendations for improvement across the VBA system. CONCLUSION VBA is under tremendous pressure to process claims and reduce the growing backlog. Without an effective and reliable quality assurance program, VBA leadership cannot adequately monitor performance to make necessary program improvements and ensure veterans receive accurate and consistent ratings. Mr. Chairman, thank you for the opportunity to discuss these important issues. We would be pleased to answer any questions that you or other members of the Subcommittee may have. Prepared Statement of Daniel Bertoni, Director, Education, Workforce, and Income Security, U.S. Government Accountability Office VETERANS' DISABILITY BENEFITS: VA Has Improved Its Programs for Measuring Accuracy and Consistency, but Challenges Remain GAO Highlights Why GAO Did This Study For years, in addition to experiencing challenges in making disability claims decisions more quickly and reducing its claims backlog, the Department of Veterans Affairs (VA) has faced challenges in improving the accuracy and consistency of its decisions. GAO was asked to discuss issues surrounding VA's Systematic Technical Accuracy Review (STAR) program, a disability compensation and pension quality assurance program, and possible ways, if any, this program could be improved. This statement focuses on actions VA has taken; including those in response to past GAO recommendations, to (1) address identified weaknesses with STAR and (2) improve efforts to monitor the consistency of claims decisions. This statement is based on GAO's prior work, which examined several aspects of STAR, as well as VA's consistency review activities, and on updated information GAO obtained from VA on quality assurance issues that GAO and VA's Office of Inspector General (OIG) have identified. GAO also reviewed VA's OIG March 2009 report on STAR. GAO is not making any new recommendations. What GAO Found Over the past several years, GAO has identified several deficiencies with the Veterans Benefit Administration's (VBA) STAR program, and although VBA has taken actions to address these issues, it continues to face challenges in improving claims accuracy. For example, GAO found that STAR reviewers lacked organizational independence, a basic internal control principle. In response to our finding, VA began utilizing organizationally independent reviewers that do not make claims decisions. GAO also found that sample sizes for pension claims were insufficient to provide assurance about decision accuracy. In response to GAO's recommendation, in fiscal year 2009, VA began increasing the number of pension claims decisions it reviews annually at each of its offices that process pension decisions. VA has also taken a number of other steps to address weaknesses that VA's OIG found in the STAR program, including (1) establishing minimum annual training requirements for reviewers and (2) requiring additional supervisory review of STAR reviewers' work. Although it has made or has started making these improvements, VBA remains challenged to improve its decision accuracy for disability compensation decisions, and it has not met its stated accuracy goal of 90 percent. VBA's performance has remained about the same over the past several fiscal years. In addition, VA has taken steps to address deficiencies that GAO and the VA's OIG have identified with consistency reviews--assessments of the extent to which individual raters make consistent decisions on the same claims. For example, in prior work, GAO reported that VA did not conduct systematic studies of impairments that it had identified as having potentially inconsistent decisions. In response to GAO's recommendation, in fiscal year 2008, VBA's quality assurance staff began conducting studies to monitor the extent to which veterans with similar disabilities receive consistent ratings across regional offices and individual raters. However, last year, VA's OIG reported that VA had not followed through on its plans to conduct such reviews. In response to this and other OIG findings and recommendations, VA took a number of actions, including developing an annual consistency review schedule and hiring additional quality assurance staff. However, VBA has only recently begun these programs to improve consistency, and it is too early to assess the effectiveness of their actions. __________ Mr. Chairman and Members of the Subcommittee: I am pleased to have the opportunity to comment on the Department of Veterans Affairs' (VA) efforts to improve the accuracy and consistency of its disability compensation and pension benefit decisions. As we and other organizations have reported over the last decade, VA's claims processing challenges are not limited to making decisions more quickly and reducing its claims backlog; but also includes improving the accuracy and consistency of its decisions. The number of veterans awaiting decisions could grow as servicemembers returning from ongoing conflicts and aging veterans submit claims. According to VA, about 35 percent of veterans from ongoing hostilities file claims. It is important not only that decisions be timely, but also accurate. Accurate initial claims decisions can help ensure that VA is paying cash disability benefits to those entitled to such benefits and also help prevent lengthy appeals. Meanwhile, consistent decisions help ensure that comparable medical conditions of veterans are rated the same, regardless of which VA regional benefits office processes the claim. You asked us to discuss issues surrounding VA's disability compensation and pension quality assurance programs; particularly, the Systematic Technical Accuracy Review (STAR) program. My statement focuses on STAR, which deals with accuracy, and two other VA quality assurance activities that focus on consistency.\1\ More specifically, my remarks will focus on actions VA has taken to (1) address deficiencies identified with STAR and (2) improve efforts to monitor the consistency of claim decisions. This statement is based on our prior work, which examined several aspects of STAR, as well as VA's consistency review programs, and on updated information we obtained from VA on quality assurance vulnerabilities that we and VA's Office of Inspector General (OIG) have identified. We also reviewed VA OIG's March 2009 report on STAR and consistency reviews.\2\ Our work was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. --------------------------------------------------------------------------- \1\ These are (1) reviews of consistency of claims decisions across VA's Veterans Benefits Administration, which is responsible for administering VA's disability compensation and pension programs, by type of disabling condition; and (2) inter-rater reliability reviews, which examine the consistency of raters when evaluating the same condition based on a comparable body of evidence. \2\ Office of Inspector General, Department of Veterans Affairs, Audit of Veterans Benefits Administration Compensation Accuracy and Consistency Reviews (Washington, D.C.: Mar. 12, 2009). --------------------------------------------------------------------------- Background Through its disability compensation program, VA pays monthly benefits to veterans with service-connected disabilities.\3\ Under its disability pension program, VA pays monthly benefits to low-income veterans who have disabilities not related to their military service or are age 65 or older. VA also pays compensation to the survivors of certain veterans who had service-connected disabilities and of servicemembers who died while on active duty. --------------------------------------------------------------------------- \3\ The amount of disability compensation depends largely on the severity of the disability, which VA measures in 10 percent increments on a scale of 0 percent to 100 percent. In 2010, basic monthly payments for veterans range from $123 for 10 percent disability to $2,673 for 100 percent disability. --------------------------------------------------------------------------- Veterans and their survivors claim benefits at one of the Veterans Benefits Administration's (VBA) 57 regional offices. Once the claim is received, a service representative assists the veteran in gathering the relevant evidence to evaluate the claim. Such evidence includes the veteran's military service records, medical examinations, and treatment records from VA medical facilities and private medical service providers. Also, if necessary for reaching a decision on a claim, the regional office arranges for the veteran to receive a medical examination. Once all necessary evidence has been collected, a rating specialist evaluates the claim and determines whether the claimant is eligible for benefits. If so, the rating specialist assigns a percentage rating. Veterans with multiple disabilities receive a single composite rating. Since 2001, VBA has created 15 resource centers that are staffed exclusively to process claims or appeals from backlogged regional offices. Most of these centers focus either on making rating decisions, or on developing the information needed to evaluate claims. In addition to the traditional claims process, any member of the armed forces who has seen active duty--including those in the National Guard or Reserves--is eligible to apply for VA disability benefits prior to leaving military service through VA's Benefits Delivery at Discharge (BDD) program or the related Quick Start program.\4\ In 2006, VA completed its consolidation of BDD rating activity into its Salt Lake City, Utah, and Winston-Salem, North Carolina, regional offices, to increase the consistency of BDD claims decisions. Also, under the Department of Defense (DoD)--VA disability evaluation system pilot program, servicemembers undergoing disability evaluations, if found medically unfit for duty, receive VA disability ratings. This rating covers both the unfitting conditions identified by the military service and conditions identified by the servicemember during the process. The rating is used by both DoD and VA to determine entitlement for disability benefits.\5\ --------------------------------------------------------------------------- \4\ In order to be eligible for the BDD program, servicemembers must meet several requirements, which include filing a VA claim 60 to 180 days prior to an honorable discharge and completing a medical examination. Under BDD, the examination also serves as Department of Defense's separation physical examination. Quick Start is for those servicemembers--primarily members of the National Guard and Reserve-- who cannot meet the BDD time frame. \5\ For our review of the DoD--VA disability evaluation system pilot program, see GAO, Military Disability System: Increased Supports for Servicemembers and Better Pilot Planning Could Improve the Disability Evaluation Process, GAO-08-1137 (Washington, D.C.: Sept. 24, 2008). --------------------------------------------------------------------------- Enacted in October 2008, the Veterans' Benefits Improvement Act of 2008 required VA to contract for an independent, 3-year review of VBA's quality assurance program.\6\ This review is to include, among other items, assessments of the accuracy of disability ratings and their consistency across VA regional offices. VA contracted with the Institute for Defense Analyses (IDA) to conduct this study. According to VA, IDA will provide preliminary findings in the Summer of 2010, and VA is scheduled to report to the Congress in October 2011. --------------------------------------------------------------------------- \6\ Pub. L. No. 110-389, Sec. 224; 38 U.S.C. Sec. 7731(c). --------------------------------------------------------------------------- STAR Program Under the STAR program, which was implemented in fiscal year 1999, VBA selects a random sample of completed claims decisions each month from each of its regional offices to review for accuracy. STAR reviewers assess decision accuracy using a standard checklist. For decisions affecting benefit entitlement, this review includes an assessment of whether (1) all issues in the claim were addressed; (2) assistance was provided to the claimant, as required by the Veterans Claims Assistance Act of 2000; and (3) the benefit entitlement decision was correct. If a claim has any error, VBA counts the entire claim as incorrect for accuracy rate computation purposes. The STAR reviewer then returns the case file and the results of the review to the regional office that made the decision. If an error was found, the regional office is required to either correct it or request reconsideration of the error determination. VBA uses the national accuracy rate from STAR reviews of compensation entitlement decisions as one of its key claims processing performance measures. VA also uses STAR data to estimate improper compensation and pension benefit payments. Consistency Review Activities One VA consistency review activity involves conducting studies of regional offices' decisions on specific conditions such as post- traumatic stress disorder where VBA found differences, such as in benefit grant rates, across regional offices through comparative statistical analysis. VBA uses the results of these reviews to identify root causes of inconsistencies and to target training. Under another VA consistency review activity, called inter-rater reliability reviews, VBA provides rating specialists a sample case file to assess how well raters from various regional offices agree on an eligibility determination when reviewing the same body of evidence. These reviews allow VBA officials to target a single rating issue and take remedial action to ensure the consistent application of policies and procedures nationally. VA Has Implemented Procedures to Address Deficiencies Identified with the STAR Program, but Continues to Face Challenges in Improving Accuracy Over the past decade, VBA has taken several actions to improve its STAR program and to address deficiencies identified by both GAO and VA's OIG. For example, in March 1999, we found that STAR review staff lacked sufficient organizational independence because they were also responsible for making claims decisions and reported to regional office managers responsible for claims processing.\7\ In response to our findings, VBA took steps to address this by utilizing reviewers who do not process claims and who do not report to managers responsible for claims processing. More recently, in February 2008, we found that STAR was not sampling enough initial pension claims to ensure the accuracy of pension claims decisions.\8\ Because initial pension claims constituted only about 11 percent of the combined compensation and pension caseload subject to accuracy review, few were likely to be included in the STAR review sample. We recommended that VBA take steps to improve its quality assurance review of initial claims, which could include reviewing a larger sample of pension claims. According to VBA, it has addressed this issue by consolidating pension claims processing in its three Pension Management Centers \9\ and establishing a separate STAR sample for pension claims. During fiscal year 2009, VBA began reviewing more pension claim decisions and reported that, for fiscal year 2009, its pension entitlement accuracy was 95 percent, exceeding its goal. --------------------------------------------------------------------------- \7\ GAO, Veterans' Benefits Claims: Further Improvements Needed in Claims-Processing Accuracy, GAO/HEHS-99-35 (Washington, D.C.: Mar. 1, 1999). \8\ GAO, Veterans' Benefits: Improved Management Would Enhance VA's Pension Program, GAO-08-112 (Washington, D.C.: Feb. 14, 2008). \9\ The Pension Management Centers are located in St. Paul, Minnesota; Philadelphia, Pennsylvania; and Milwaukee, Wisconsin. --------------------------------------------------------------------------- In a September 2008 report, we noted that VA lacked sufficient and specific performance measures for assessing the accuracy of decisions on BDD claims and recommended that VA consider options for separately estimating the accuracy of such claims decisions.\10\ VA conducted an analysis of the costs of sampling pre-discharge claims as part of STAR and concluded that the costs would outweigh possible, unquantifiable benefits. VA also noted that the two sites that rate BDD claims surpassed the national average in accuracy for claims overall.\11\ While generally responsive to our recommendation, VA's analysis did not specifically review the accuracy of BDD claims relative to traditional claims. Moreover, because BDD claims do not comprise all claims reviewed at the two rating sites, we continue to believe VA's analysis was not sufficient to estimate the relative accuracy of BDD claims at these sites. While we agree that the benefits of reviewing accuracy are difficult to measure, if VA had better information on the accuracy of BDD claims, VA could use such information to inform training and focus its monitoring efforts. In contrast, VA currently performs STAR reviews that target rating decisions made by its Baltimore and Seattle offices under the DoD--VA disability evaluation system pilot program. Such a targeted review could also be conducted for BDD claims. --------------------------------------------------------------------------- \10\ GAO, Veterans' Disability Benefits: Better Accountability and Access Would Improve the Benefits Delivery at Discharge Program, GAO- 08-901 (Washington, D.C.: Sept. 9, 2008). \11\ BDD claims are rated at the regional offices in Winston-Salem, North Carolina, and Salt Lake City, Utah. --------------------------------------------------------------------------- In its March 2009 report, VA's OIG also identified several deficiencies in the STAR program and recommended corrective actions. The OIG found that (1) regional offices did not always submit all requested sample cases for review, (2) reviewers did not evaluate all documentation in sample files, and (3) reviewers were not properly recording some errors. The OIG also found that VBA was not conducting STAR reviews of redistributed cases (for example, claims assigned to resource centers for rating). The OIG reviewed a sample of redistributed claims and found that 69 percent had accurate entitlement decisions, well below VBA's reported rate of 87 percent for the 12- month period ending in February 2008. Further, the OIG found that VBA did not have minimum training requirements for STAR reviewers. As of March 2010, VBA had taken actions to respond to all of the OIG's recommendations related to STAR, including (1) implementing procedures to follow up on cases not submitted by regional offices; (2) adding a mechanism to the STAR database to remind reviewers of key decision points; (3) requiring a second-level review of STAR reviewers' work; and (4) establishing a requirement that STAR reviewers receive 80 hours of training per year. In addition, during fiscal year 2009, based in part on the OIG's recommendation, VBA also began monitoring the accuracy of claims decided by rating resource centers as it does for regional offices. As we noted in our January 2010 report, VBA has significantly expanded its practice of redistributing regional offices' disability claims workloads in recent years,\12\ and gathering timeliness and accuracy data on redistributed claims could help VBA assess the effectiveness of workload redistribution. --------------------------------------------------------------------------- \12\ VBA refers to the practice of redistributing claims as ``brokering.'' --------------------------------------------------------------------------- In addition, as the Congress has provided more resources to VBA to increase compensation and pension staffing, VBA has devoted more resources to quality review. In fiscal year 2008, VBA more than doubled the size of the quality assurance staff, allowing it to increase the scope of quality assurance reviews. VA states that in the 12-month period ending in May 2009, STAR staff reviewed over 14,000 compensation and pension benefit entitlement decisions. Although VBA has taken steps to address deficiencies in the STAR program, the accuracy of its benefit entitlement decisions has not improved. The accuracy rate was 86 percent in fiscal year 2008 and 84 percent in fiscal year 2009, well short of VBA's fiscal year 2009 goal of 90 percent.\13\ VA attributed this performance to the relatively large number of newly hired personnel conducting claims development work and a general lack of training and experience. Human capital challenges associated with providing the needed training and acquiring the experience these new claims processors need to become proficient at their jobs will likely continue in the near future. According to VBA officials, it can take 3 to 5 years for rating specialists to become proficient. --------------------------------------------------------------------------- \13\ This rating-related accuracy measure includes original and reopened claims for disability compensation and dependency and indemnity (survivor) compensation benefits. Reopened claims include cases where a veteran seeks a higher rating for a disability or seeks compensation for an additional condition. VA Has Taken Actions to Strengthen Efforts to Monitor Consistency of --------------------------------------------------------------------------- Claims Decisions VA has taken actions to address deficiencies identified with its consistency review programs, but it is still too early to determine whether these actions will be effective. In prior work, we reported that VBA did not systematically assess the consistency of decision- making for any specific impairments included in veterans' disability claims. We noted that if rating data identified indications of decision inconsistency, VA should systematically study and determine the extent and causes of such inconsistencies and identify ways to reduce unacceptable levels of variations among regional offices. Based on our recommendation, VBA's quality assurance staff began conducting studies to monitor the extent to which veterans with similar disabilities receive consistent ratings across regional offices and individual raters.\14\ VBA began these studies in fiscal year 2008. VBA identified 61 types of impairments for consistency review and conducted at least two inter-rater reliability reviews, which found significant error rates. --------------------------------------------------------------------------- \14\ GAO, Veterans' Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved, GAO-02-806 (Washington, D.C.: Aug. 16, 2002). --------------------------------------------------------------------------- In its March 2009 report, the OIG noted that, while VBA had developed an adequate rating consistency review plan, including metrics to monitor rating consistency and a method to identify variances in compensation claim ratings, it had not performed these reviews as scheduled. In fact, VBA had initiated only 2 of 22 planned consistency reviews in fiscal year 2008. The OIG reported that VBA had not conducted these reviews because STAR staffing resources were not sufficient to perform all of their assigned responsibilities and noted that VBA's quality review office had not staffed all of its authorized positions. In addition, the OIG found that inter-rater reliability reviews were not included in VBA's quality assurance plan. The OIG recommended that VBA (1) develop an annual rating consistency review schedule and complete all planned reviews as scheduled; (2) dedicate sufficient staff to conduct consistency reviews in order to complete planned workload and reviews; and (3) include inter-rater reliability reviews as a permanent component of its consistency review program. VBA reported that it has developed an annual consistency review schedule and is in the process of conducting scheduled fiscal year 2010 reviews. As of January 2010, VBA also added six staff members to perform quality assurance reviews. Further, VBA incorporated inter- rater reliability reviews into its fiscal year 2009 quality assurance plan. Because VBA has only recently implemented these initiatives, it is too early to determine their impact on the consistency of claims decisions. Conclusion Over the years, VA has been challenged in its efforts to ensure that veterans get the correct decisions on disability claims the first time they apply for them, regardless of where the claims are decided. Making accurate, consistent, and timely disability decisions is not easy, but it is important. Our veterans deserve timely service and accurate decisions regardless of where their claims for disability benefits are processed. To fulfill its commitment to quality service, it is imperative that VA continue to be vigilant in its quality assurance efforts, as this challenge will likely become even more difficult as aging veterans and veterans returning from ongoing conflicts add to VA's workload. Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions you or Members of the Subcommittee may have at this time. GAO Contact and Staff Acknowledgments For further information about this testimony, please contact Daniel Bertoni at (202) 512-7215 or [email protected]. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. In addition to the contact named above, key contributors to this statement include Shelia Drake, Jessica Orr, Martin Scire, and Greg Whitney. Related GAO Products Veterans' Disability Benefits: Further Evaluation of Ongoing Initiatives Could Help Identify Effective Approaches for Improving Claims Processing. GAO-10-213. Washington, D.C.: January 29, 2010. Veterans' Disability Benefits: Preliminary Findings on Claims Processing Trends and Improvement Efforts. GAO-09-910T. Washington, D.C.: July 29, 2009. Military Disability System: Increased Supports for Servicemembers and Better Pilot Planning Could Improve the Disability Evaluation Process. GAO-08-1137. Washington, D.C.: September 24, 2008. Veterans' Disability Benefits: Better Accountability and Access Would Improve the Benefits Delivery at Discharge Program. GAO-08-901. Washington, D.C.: September 9, 2008. Veterans' Benefits: Improved Management Would Enhance VA's Pension Program. GAO-08-112. Washington, D.C.: February 14, 2008. Veterans' Benefits: Further Changes in VBA's Field Office Structure Could Help Improve Disability Claims Processing. GAO-06-149. Washington, D.C.: December 9, 2005. Veterans Benefits: VA Needs Plan for Assessing Consistency of Decisions. GAO-05-99. Washington, D.C.: November 19, 2004. VA Disability Benefits: Routine Monitoring of Disability Decisions Could Improve Consistency. GAO-06-120T. Washington, D.C.: October 20, 2005. Veterans' Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, D.C.: September 30, 2003. Veterans' Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. Washington, D.C.: August 16, 2002. Veterans' Benefits: Quality Assurance for Disability Claims Processing. GAO-01-930R. Washington, D.C.: August 23, 2001. Veterans' Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999. Prepared Statement of Ronald B. Abrams, Joint Executive Director, National Veterans Legal Services Program Mr. Chairman and Members of the Committee: I am pleased to have the opportunity to submit this testimony on behalf of the National Veterans Legal Services Program (NVLSP). NVLSP is a nonprofit veterans service organization founded in 1980 that has been assisting veterans and their advocates for thirty years. We publish numerous advocacy materials, recruit and train volunteer attorneys, train service officers from such veterans service organizations as The American Legion and Military Order of the Purple Heart in veterans benefits law, and conduct quality reviews of the VA regional offices on behalf of The American Legion. NVLSP also represents veterans and their families on claims for veterans benefits before VA, the U.S. Court of Appeals for Veterans Claims (CAVC), and other federal courts. Since its founding, NVLSP has represented over 2,000 claimants before the Board of Veterans' Appeals and the Court of Appeals for Veterans Claims (CAVC). NVLSP is one of the four veterans service organizations that comprise the Veterans Consortium Pro Bono Program, which recruits and trains volunteer lawyers to represent veterans who have appealed a Board of Veterans' Appeals decision to the CAVC without a representative. In addition to its activities with the Pro Bono Program, NVLSP has trained thousands of veterans service officers and lawyers in veterans benefits law, and has written educational publications that thousands of veterans advocates regularly use as practice tools to assist them in their representation of VA claimants. It is clear that the quality of VA adjudications is not satisfactory and is a major contributor to the size of the backlog. Because many claims are improperly denied, because many VA adjudicators are inadequately trained, because many VA regional offices are improperly managed, because many VA regional offices are inadequately staffed, and because VA Central Office management has not acted to fix these problems in any meaningful way, many veterans and other claimants for VA benefits have to file unnecessary appeals, wait several years for a BVA remand, and wait for the VA to obtain evidence that should have been requested during the original adjudication of the claim. These appeals clog the system and create unneeded work for the VA. Of course, it would have been better for the VARO to do the work correctly the first time. NVLSP believes that the quality of VARO adjudications is much worse than what is reported by the VA. Valid indicators of the poor quality of adjudications performed by the VAROs are the remand and reversal statistics produced by decisions issued by the Board of Veterans' Appeals (BVA or Board) and decisions issued by the U.S. Court of Appeals for Veterans Claims (CAVC). BVA statistics provided by the Board for FY 2009 reveal that Board decided over 48,000 appeals. The Board granted additional benefits in 24 percent of the appeals and remanded 37.3 percent of these appeals back to the VAROs. Therefore, 61.3 percent of the VARO decisions that were appealed and decided by the BVA were either reversed or remanded. These statistics are about 4 percent worse than the statistics I reported to you in 2008. The news gets worse. The BVA, in its rush to make final decisions and to avoid remands quite often prematurely denies claims that should have been remanded. The CAVC in a large number of appeals finds fault with how the VA processed the claim and remands the case back to the VA to correct claims processing errors. The CAVC reports that of the 3,270 merits decisions it issued in fiscal year 2009, it affirmed the BVA decision in less than 20 percent of the cases. In over 80 percent of all the BVA decisions appealed to the CAVC and decided on the merits, the CAVC either reversed or remanded the BVA decision Even Chief Justice Roberts was startled when he learned that in litigating with veterans before the CAVC, the government more often than not takes a position that is substantially unjustified. These statistics are shocking and revealing. The VA is required to conduct ex-parte, non-adversarial claims adjudications in a veteran friendly environment. (38 U.S.C. 5107(b), 38 C.F.R. 3.102) and provide veteran claimant's with a non-adversarial adjudication system. The results of the Legion/NVLSP quality reviews continue to be discomforting. The American Legion/NVLSP team usually spends a week in a VARO reviewing the quality of recently adjudicated claims where The American Legion represented the veteran. The results of these quality reviews reveal that in many instances claims are improperly denied or benefits are not paid at the proper rate because the RO was more concerned about claiming work credit and reducing the VARO backlog than taking the time to develop and analyze the claim properly. The Legion team generally finds a much higher error rate than the 12 percent generally reported by STAR. The good news is that most of the VA service center: managers, coaches, decision review officers, and raters that we have interviewed on these many quality reviews are sincere when they mention the need for quality adjudications. We have, however, met many VA regional office officials who are free to admit that their main focus is on production, not quality. INDEPENDENT QUALITY REVIEWS During the past several years the National Veterans Legal Services Program (NVLSP) has performed, on behalf of The American Legion and several state departments of veterans' affairs, quality reviews of decisions issued by several VA Regional Offices (ROs). Our conclusion, based on these reviews and on information received at our service officer trainings, is that although the VA is to be commended for initiatives to stop blatant work measurement (end-product) cheating and to emphasize quality, the most needed change--full and fair adjudication of veterans' claims--has not become a reality. Essentially, while NVLSP commends VBA for its quality initiatives, we are forced to conclude that these initiatives combined with the STAR program have not achieved the desired result. Premature Adjudications Resulting in Adverse Decisions The most important and pervasive problem facing veterans seeking VA disability benefits is the eagerness of some ROs to adjudicate claims before all necessary evidence has been obtained. For example, some ROs prematurely deny claims based on inadequate VA examinations. In some cases, even where the VA examiner clearly fails to respond to a specific question asked by the RO, the examination report is not returned as inadequate. Instead, the claim is adjudicated and denied on the basis of the inadequate report. In other instances, claims are denied before all service medical records are received. Other claims are sometimes denied before the veteran has a fair opportunity to submit independent medical evidence. These all-too-frequent cases of premature denial result from an over-emphasis on timeliness and a lack of accountability. We certainly believe that claims for VA disability benefits should be accurately adjudicated in a timely manner. However, because of a management emphasis on timeliness, or a perceived emphasis on timeliness, some VA adjudicators appear to believe that they are pressured to make premature final decisions. In most instances, we have discovered that a decision made prematurely is likely to take the form of a denial of benefits rather than an award of benefits. Let us make something very clear: The timeliness of VA adjudication is but one factor in the overall assessment of the VA disability claims adjudication system. We realize that the overall timeliness statistics provided by the VBA show that VBA has not met its goal to reduce the time it takes to adjudicate claims for disability benefits. Even though the VA has not met its goal in this respect, we urge that you not overemphasize timeliness to the detriment of quality. It does veterans little good to have their claims promptly, but inaccurately, denied. The errors found by STAR and the subsequent trainings based on STAR findings have not significantly improved this situation. One may wonder why VA adjudicators would want to prematurely deny claims. The answer lies in the VA work measurement system. When a claim for VA benefits is prematurely and inaccurately denied, many veterans submit new evidence to reopen their claim. The VA considers the new evidence a second claim and the employee earns double work credit. Adjudication officers, now called service center managers, have informed us off-the-record that they feel pressured to prematurely adjudicate claims because they expect other ROs will do the same, and they want to show that their productivity and timeliness is as good as other ROs. The VA work measurement system should encourage a timely and accurate adjudication, not just a timely adjudication. Section 2 of H.R. 3047 would change when VA regional offices (VAROs) can claim work credit, and was a good bill that would have helped to accomplish this goal. NVLSP looks forward to reviewing the overdue VA report that was mandated by PL 110-389. Adversarial Attitude Our quality review has identified a systemic attitude problem in some ROs, which may take one of several forms. One example is that despite the general tendency to deny prematurely, some ROs ``develop to deny.'' That is, these ROs consistently seek to develop negative evidence in cases where all the evidence of record before the RO, without further development, would reasonably support the grant of benefits. Another attitude problem is that some ROs have biases against certain types of VA claims for benefits. For example, veterans seeking service connection for mental conditions, entitlement to individual unemployability benefits, or entitlement to compensation based upon secondary service connection, in some instances, have to jump over a higher bar than those who file other types of claims. In addition, some ROs either refuse to consider or are unaware of beneficial statutes in Title 38, United States Code. For example our quality reviews have found that 38 U.S.C. Sec. 1154(b), which provides in most cases that the statement of a combat veteran about an injury that occurred during combat will be accepted as true even though there is no official record of the injury, is sometimes conspicuously disregarded. Communication Problems In many cases, the VA's communication with its veteran-claimants causes real problems. For example, VA notifications often fail to provide an adequate explanation of the reasons and bases for the adverse VA determination. Other communication problems noted by NVLSP are: Inadequate development letters (development letters are sent by the VA to the veteran and his or her representative, asking for further information or evidence) that do not comply with VA's guidance that letters should clearly tell the claimant what evidence is needed and what exactly has to be done to establish entitlement to the benefit sought (see M21-1, Part III, para. 1.04a.); and Telephone communication with the veteran that is not monitored or sanctioned by the veteran's representative (the VA does not even inform the representative that it is about to contact the representative's client). Widespread Errors The following is a list of a systemic pattern of errors that we have noticed during our quality review checks. These errors are: Assignment of erroneously low disability ratings for service-connected mental conditions; Erroneous denial of claims for service connection for mental conditions; Failure to consider 38 U.S.C. Sec. 1154(b); Erroneous denial of claims of individual unemployability; Inadequate requests for medical opinions (for example, the standard of proof in the VA claims process is rarely explained to VA doctors, and in many instances conclusions regarding critical facts are not communicated to doctors who are asked to provide medical opinions); and Non-responsive VA examination reports (for example, some VA examiners do not comply with the AMIE protocol, and other examiners fail to respond to specific questions), coupled with the acceptance of these inadequate examination reports by ROs. In general, there is a lack of coordinated local (RO) quality control and a subsequent failure to act on recognized patterns of errors. NVLSP Recommendations Based on the foregoing observations, NVLSP makes the following suggestions: VA's work measurement system should be altered so that quality as well as timeliness are twin concepts that together drive the system. To provide VA quality control with ``teeth'' and prevent end-product and work measurement abuses, an aggressive independent quality control should be performed. VBA should conduct regular meetings with its stakeholders to inform them of any actions VBA has taken to correct systemic adjudication problems. The stakeholders should be informed about the patterns of errors identified nationally, the ROs where there are significant problems, VBA's plans to correct these problems, changes in management, progress reports on previous initiatives, and an invitation for the stakeholders to participate and coordinate in the correction of problems. VA should institute a system of awards and disincentives for managers and adjudicators. VA managers and adjudicators who perform accurate and timely work should be rewarded. Managers who do not perform adequately should be appropriately chastised. VA employees who do a good job should be paid a reasonable salary, receive bonuses and be promoted. VA management should more clearly communicate with its employees what it wants from them. If management focuses on quality as well as efficient work, veterans will be better off. NVLSP acknowledges that the adjudication of claims for VA benefits is very complicated. However, we believe the stakeholders want to help correct adjudication problems. We would be happy to meet regularly with the VA to talk about the problems we have identified and suggested solutions. We would like to commend VBA managers for initiatives in reducing outright end-product and work measurement dishonesty and efforts to emphasize quality. While these efforts are commendable, it is time to see results. Our experience has taught us that VA managers are reasonable people who want to do the right thing. These managers care about veterans and know that the claims adjudication system is not working properly. To help these managers we ask you to encourage the VA to make at least the most necessary changes, alter VA's work measurement system, institute an aggressive quality control program, and support its efforts to coordinate with its stakeholders. We appreciate the opportunity to provide the subcommittee with this testimony. Thank you. Prepared Statement of John L. Wilson, Assistant National Legislative Director, Disabled American Veterans Mr. Chairman and Members of the Committee: The Disabled American Veterans (DAV) has consistently stated that the keys to successfully reforming the veterans' benefits claims process are training and accountability, two elements central to producing quality results for veterans. However, today the Veterans Benefits Administration (VBA) remains production driven from the Monday morning workload reports to personnel awards. The DAV strongly believes that quality should be rewarded at least on parity with production. However, in order for this to occur, VBA must first implement and then inculcate a comprehensive quality control program to complement its quality assurance program. VBA's primary quality assurance program is the Systematic Technical Accuracy Review (STAR) program. The STAR program can identify three types of errors--benefit entitlement, decision documentation/ notification, and administrative. STAR looks at whether a proper VCAA pre-decision ``notice'' was provided and whether the rating decision was merited based on the available evidence. Under the STAR program, VA reviews a sampling of decisions from regional offices and bases its national accuracy measures on the percentage with errors that affect entitlement, benefit amount, and effective date. This is a technical review to ensure a variety of transactions are properly carried out. Inconsistency in technical accuracy may signal several types of processing problems, including: uneven or insufficient understanding of governing criteria, operating rules that are too vague and allow an overly broad interpretation which, in turn, leads to significant variance, or outright arbitrariness in decision-making. Obviously, VA must detect inconsistencies before the cause or causes can be determined and remedied. The STAR program was implemented in the late 1990s. It was intended to be a national quality assurance program that would assist VBA in identifying processing vulnerabilities and error trends. It was designed to draw statistically significant case samples from all regional offices, using stratified random sampling algorithms. Using this tool, VBA could review a statistically valid case sample each year, and calculate its national error rates. Using the STAR program was also intended to identify major national error trends, so the Compensation and Pension (C&P) program could initiate corrective measures. Such corrective measures could include training, improved procedural guidance, or automated system improvements. The STAR program was not designed to provide evaluative data at the working unit or individual level. There are two major reasons for this. First, the sample sizes used in STAR are small. While totals may be significant at the national level, breaking out the data to Regional Offices may not provide numbers significant enough to ascertain a trend with an individual employee. Second, the STAR program essentially assesses the outcome, not the process of getting there. So, a claim that took two years to process because of piece-meal development would not have an error called if the resulting decision was correct and all pre- and post-notification requirements were met. Such processing delays would fall under the purview of quality control in-process reviews and not quality assurance programs such as STAR. Quality control findings from local in-process reviews would assist a station in assessing overall performance towards achieving the goal of timely and accurate decisions on veterans' claims. VBA recognized the importance of such quality control when the STAR program was created to replace the ineffective Statistical Quality Control program. At that time, VBA requested funding to implement a local C&P quality control program. That program--called Systematic Individual Performance Assessment (SIPA)--was announced in 2000 as a new initiative to monitor individual performance. Under this program, the VA would review an annual sample of 100 decisions for each adjudicator to identify individual deficiencies, ensure maintenance of skills, promote accuracy and consistency of claims adjudication, and restore credibility to the system. The reviewers would have performed related administrative functions, such as providing feedback on reviews, maintaining reports, and playing a role in employee development and ongoing training. Unfortunately, the VA abandoned this initiative during 2002, and proficiency is now apparently subjectively assessed by supervisors based on their day-to-day perceptions of employee performance. The SIPA program may have been abandoned due to inadequate resources. Without any quality assurance review on the individual level, the VA is unlikely to impose effective accountability down to the individual adjudicator level, where it must go if optimum quality is expected. VBA elected instead to install a much reduced quality control procedure, where coaches' review several cases per month for Veterans Service Representatives (VSR) and Rating Veterans Service Representatives (RVSR), as their quality control mechanism. This hybrid process has not provided adequate individual accountability, or sufficiently robust data to identify local process improvements. Coaches typically do not have the time to manage day-to-day operations and pull case files for ad hoc reviews of employees. With significant attention on the C&P claims backlog, it is understandable that VBA wants to maximize its personnel and resources to direct claim decision-making. However, technical accuracy is arguably the most important component of the C&P claims process. Pushing cases faster will not help if significant numbers of cases are done wrong. VBA needs to elevate quality to the highest priority. This means they should dedicate adequate resources to both quality assurance and quality control programs. The VSRs, RVSRs and local management teams need to understand that high quality work will be recognized and rewarded. Further, they need to understand that there will be clear thresholds for individual quality, repeated errors will be identified and associated with their processor, and that there will be appropriate consequences for insufficient accuracy rates. The STAR program was evaluated by the VA Office of Inspector General as part of its review of compensation rating accuracy in March 2009, in the report titled ``Audit of Veterans Benefits Administration Compensation Rating Accuracy and Consistency Reviews.'' The OIG determined that VBA's STAR program does not provide a complete assessment of rating accuracy. During the 12-month period ending in February 2008, VBA's STAR process did not effectively identify and report all errors in compensation claim rating decisions. VBA identified a national compensation claim rating accuracy of 87 percent. Of the approximately 882,000 compensation claims measured by STAR reviewers, VBA estimated that about 87 percent were technically accurate. The OIG, on the other hand, reviewed a random sampling of cases that had also been reviewed by STAR reviewers and found additional errors. They projected an accuracy rate of only 78 percent. They also audited brokered cases. Of that sampling, they found an accuracy rate of 69 percent. Combining the audit of brokered claims with those STAR reviewed claims, results in a projected accuracy rate of about 77 percent of claims. The OIG determined that this equates to approximately 203,000 claims in that one year alone where veterans' monthly benefits may be incorrect. The OIG found that STAR reviewers did not identify some of the missed errors because they either did not thoroughly review available medical and non-medical evidence, did not identify the absence of necessary medical information, or inappropriately misclassified benefit entitlement errors as comments. These findings are, on their surface, a result of STAR reviewers not finding all the errors in the cases they reviewed. They also point to the need for greater management oversight and an effective formal training program for the STAR reviewers. STAR reviewers could benefit from formal training; however STAR managers have said that current workload requirements do not allow for the amount of training time necessary. This is a common theme for VBA that underlies even STAR reviews--quantity over quality--even in the area that is supposed to ensure quality of at least a technical nature. The need for a quality control program as an adjunct to the STAR program can also be seen when considered through a review of the Board of Appeals for Veterans Claims' Summary of Remands. The summary represents a statistically large and reliable sample of certain measurable trends. The examples must be viewed in the context of the VA (1) deciding over 880,000 cases per year; (2) receiving over 133,000 Notice of Disagreements; and (3) over 49,000 appeals to the Board. The examples below are from fiscal year (FY) 2009: 1. Remands resulted in 801 cases because no ``notice'' under section 5103 was ever provided to the claimant. In addition, there were 4,048 remanded for inadequate or incorrect notice, some of which may result from the current generic notice letters sent VBA. The DAV continues to call for changes to these letters to include more specific information, which could help lower the incidence of this error. 2. VA failed to request for Service Medical Records in 1,739 cases and failed to request for personnel records in 1,511 cases. These numbers are disturbing because initially requesting a veteran's service records is the foundation to every compensation claim. 3. The Board remanded 7,814 cases for failure to request VA medical records. The disturbing factor here is that a VA employee can usually obtain VA medical records without ever leaving the confines of one's computer screen. 4. Another 3,187 cases were remanded because the claimant had requested a travel board hearing or video-conference hearing. Again, there is a disturbing factor here. A checklist is utilized prior to sending an appeal to the Board that contains a section that specifically asked whether the claimant has asked for such a hearing. The examples above totaled 19,100 cases or 34 percent of appeals reaching the Board, all of which cleared the local rating board and the local appeals board with errors that are elementary in nature. Yet they were either not detected or they were ignored. Many more cases were returned for more complex errors. Regardless, a 34 percent error rate on such basic elements in the claims process involving VBA's most senior rating specialists is simply unacceptable. The problem with the current accountability system is that VBA employees who commit such errors are usually not held responsible. With no incentive to prevent such errors and a constant focus on production, quality will continue to decline. DAV agrees with the VA OIG that VBA could improve the STAR program by establishing: a mechanism to ensure STAR reviewers evaluate all documentation related to the claim selected for review; a requirement that all STAR reviewer comments receive a second review to make sure the reviewer appropriately recorded the comment instead of a benefit entitlement error; procedures to review brokered claims as part of the STAR program; and minimum annual training requirements for each STAR reviewer that are comparable to regional office rating staff training requirements. In addition, DAV recommends that VBA establish a quality control program that looks at claims in-process in order to determine not just whether a proper decision was made, but how it was arrived at in order to identify ways to improve the system. Combining results from such quality control reviews with STAR's quality assurance results and the data from remands from the Board of Veterans' Appeals and the Court of Appeals for Veterans Claims could yield valuable information on trends and cause of errors. If the data from all such reviews could be incorporated into a robust IT system, proper analysis of such data would provide management and employees important insights into processes and decisions. This in turn would lead to quicker and more accurate decisions on benefits claims, and most importantly, to the delivery of all earned benefits to veterans, particularly disabled veterans, in a timely manner That concludes my testimony. I would be pleased to answer any questions the Committee may have. Prepared Statement of Raymond C. Kelley, National Legislative Director, American Veterans (AMVETS) Chairman Hall, Ranking Member Lamborn, and members of the subcommittee, thank you for the opportunity to appear before you today to provide AMVETS' views regarding VBA's Systematic Technical Accuracy Review (STAR) Program. The Systematic Technical Accuracy Review (STAR) Program is VBA's compensation and pension accuracy measurement system. AMVETS agrees with the VA's Office of Inspector General's March 12, 2009 report that identified eight issues that will improve the process of reviewing claims for errors. And AMVETS is pleased to see VBA is taking action to correct these issues. AMVETS is concerned though, with what is done with the information that is gleaned from STAR. For the STAR program to truly be effective, AMVETS believes three things must be done. First, STAR must be enhanced so trends in errors can be easily identified by the regional offices. With this information, VBA must hold ROs accountable for failures in accuracy and insist that ROs develop improvement strategies that include training for all accuracy issues. Second, VBA must change its culture of timeliness and strive for a culture of accuracy. Wither or not STAR is completely accurate in its reviews is important but not nearly as important as what is done to ensure the same mistakes are not repeated. Third, OIG must conduct periodic review of the STAR program to ensure that STAR's accuracy ratings are within a 3 percent margin of error. Even though AMVETS believes the STAR program is effective, we believe it should be expanded to ensure that specific programs, such as the Benefits Delivery at Discharge (BDD) program and specific conditions can be tracked for anomalies that occur, so improvement strategies and specific training can be implemented. When the Veterans Claims Assistance Act (VCAA) was introduced in the first quarter of FY 2002, nearly half of all errors were VCAA related. VBA had the ROs re- train their claims processors and in the last 8 months of the FY, these types of errors were reduced by one-third. This type of accountability needs to be the rule and not the exception. As you know, the STAR program identifies errors in claims decisions each month through random sampling of each Regional Office (RO). The results of the reviews are sent to the ROs. Errors that are found are to be corrected by the RO that made the decision. The corrections are made, but all too often, the ROs don't implement strategies to ensure that claims processors don't continually repeat the same mistakes. AMVETS believes the reason these strategies are not developed is that the culture within the regional offices is one of timeliness and not one of accuracy. STAR has consistently found that 20 to 25 percent of claims are in error over the past decade, but VBA has done little to ensure that mistakes that are made in the regional offices are understood and that strategies for improvements are put in place. On the other hand, VBA does require ROs to develop corrective action plans if they do not reach strategic goals for production, inventory and timeliness. This paradigm must be flipped. The March, 2009 OIG Report clearly defined the gaps in the STAR program that have caused the 10 percent disparity in compensation and pension rating accuracy. AMVETS believes VBA is taking action to close these gaps. However, AMVETS believes it is important to have this accuracy fall within a 3 percent margin of error. Therefore, AMVETS requests that OIG conduct a follow-up to the 2009 report to ensure VBA's gap solutions are productive, and that OIG continue to conduct periodic reviews of STAR to ensure the program reaches and maintains a 3 percent margin of error. Mr. Chairman, thank you again for providing AMVETS the opportunity to present our views on the STAR program. This concludes my testimony and I will be happy to answer any questions you may have. Prepared Statement of Ian C. de Planque, Assistant Director, Veterans Affairs and Rehabilitation Commission, American Legion Mr. Chairman and Members of the Subcommittee: I appreciate this opportunity to express the views of the nearly 3 million members of The American Legion on the Department of Veterans Affairs' (VA's) Systemic Technical Accuracy Review (STAR). VA maintains, as stated by Acting Under Secretary for Benefits Michael Walcoff in a January 3, 2010 appearance on 60 Minutes, that: ``We stress over and over again to our employees that quality is our number one indicator, that's absolutely a requirement for successful performance.'' STAR has the potential to be an effective tool to achieve this end; however, VA is currently falling short in their effective implementation of this tool. This failure can, and should be corrected. The American Legion recommends VA consider taking focused actions to reverse this current situation. On March 12, 2009, VA's Office of the Inspector General (VAOIG) issued Report No. 08-02073-96, Audit of Veterans Benefits Administration Compensation Rating Accuracy and Consistency Reviews. This report detailed numerous flaws in the current implementation of the STAR system. Although VA acknowledged some of those criticisms, The American Legion is unaware of any significant corrective actions by VA leadership. The VAOIG audit was conducted to evaluate the Veterans Benefits Administration's (VBA's) quality assurance program, of which STAR is a component. Some of the more troubling findings included: VBA's STAR process did not effectively identify and report errors in compensation claim rating decisions. VBA identified a national compensation claim rating accuracy of 87 percent for the 12-month period ending February 2008. We projected that VBA officials understated the error rate by about 10 percent, which equates to approximately 88,000 additional claims where veterans' monthly benefits may be incorrect. In total, we projected about 77 percent (approximately 679,000) of the almost 882,000 claims completed were accurate for the 12-month period ending February 2008. The 87 percent rate is not comparable to the accuracy rate VBA reports in the Performance and Accountability Report because that rate includes pension claims. Further, this accuracy rate only included errors that affected a veteran's benefits. STAR identifies, but does not report, other errors related to the rating decision's documentation, notification, and administration. That 88,000 claims are potentially incorrect is certainly troubling and clearly unacceptable to The American Legion. This possible trend will undoubtedly add to the current claims backlog challenge being aggressively addressed by VA Secretary Eric Shinseki. Further problematic is the inaccuracy of VA's system of reporting its own errors and isolating problem areas and/or responsible individuals in the claims rating process. Without external observation, such as this audit by VAOIG, such errors may never have come to light and an inaccurate picture of the overall flaws in the disability system may never have been recorded. More importantly, corrective actions cannot be taken in a timely manner. Furthermore, the VAOIG audit reports: VBA officials planned to conduct 22 reviews in FY 2008 consisting of 20 grant/denial rate and 2 evaluation reviews. However, they only initiated two grant/denial rate reviews and these were not completed until December 2008. Furthermore, VBA officials did not initiate either of the two planned evaluation reviews to analyze and improve the consistency of disability compensation ratings and to reduce the variances between states. Even where problem areas or potential problem areas are identified, VA is not conducting followup or analysis on their projected plans. Effective as STAR may be, if it is not implemented as intended, it cannot help correct problems. Aggressive congressional oversight would seem essential to assuring timely application of these procedures. VAOIG concluded in this instance: Without an effective and reliable quality assurance program, VBA leadership cannot adequately monitor performance to make necessary program improvements and ensure veterans receive accurate ratings. Further, without implementing an effective rating consistency program, VBA officials cannot successfully assess or prioritize the improvements needed for claim rating consistency. If this was the only problem discovered, it would be enough to call into question the effectiveness of the system as currently configured. However, with further investigation it becomes clear that the current use of STAR does not accurately assess the claims benefits operational picture. The report states: In addition, VBA officials excluded brokered claims from STAR reviews. We reviewed a sample of brokered claims and found an accuracy rate of 69 percent. The brokering of claims is increasingly becoming an integral part of the way VBA conducts operations today. Brokering is a system utilized by VA to shift claims from Regional Offices with a larger workload to Regional Offices with less substantial workloads to increase the ability to process a greater number of claims overall. Brokering claims also raises other serious issues that merit further investigation. Accountability is one of the major concerns. The American Legion believes that without STAR analysis of the brokered claims there is a lack of accountability for these claims, which is deeply troubling. How effective is the actual STAR analysis? This quote from the report raises additional unsettling issues for The American Legion: STAR reviewers did not identify some of the missed errors because they either did not thoroughly review available medical and non-medical evidence (or identify the absence of necessary medical information), or they inappropriately misclassified benefit entitlement errors as comments. The American Legion asks how can a system intended to assess the quality and accuracy of actions by VBA be effective, if evaluators do not have access to the full file information required to make the decision. Even with these errors and flaws in the current system, such a potentially powerful tool for self correction should not be abandoned completely. Perhaps a better solution is to look for ways, both internal and external, in which the present system could be adapted to effectively improve the use of the existing components. In order to rectify existing problems within STAR, The American Legion suggests VA could make improvements by implementing a three-step process for change. 1. Compile results nationwide of rating errors identified by STAR evaluations. Currently there appears to be no systemic method to track errors. This data could be critical to identify patterns, whether in an individual sense, on a Regional Office (RO) level, or nationally across the scope of VA operations. If this information is currently gathered, it does not appear to be used for analysis to detect trends which could indicate systemic problems within VA. This data, coupled with data gathered on errors at lower levels from the Board of Veterans Appeals (BVA) and the Appeals Management Center (AMC), would be an excellent tool to assess VA's quality overall by supplying details that could indicate problem areas. It is not enough to know what VA's accuracy rate across the system is. VA must also know where are their greatest areas of concern in order to determine areas that could be addressed and provide the most efficient and effective use of resources. 2. Utilize data and patterns gathered from STAR to plan and implement a program of training. Adequate and effective training is a key concern noted often in the adjudication of claims. This data could show specific topics in which employees need additional training to improve accuracy of ratings. Such information could help VA leadership craft an effective training schedule to maximize the training resources. Future training would not be generalized, but rather targeted to fix specifically identified problems. This focused approach would assure that training resources would be used to the greatest possible impact. 3. Augment STAR for accuracy with outside oversight to ensure the effectiveness of the program. One obvious complaint about the current implementation of STAR is a lack of adequate followup. The American Legion believes third-party oversight offers the opportunity to provide impartial and critical followup to assure compliance. The American Legion strongly advocates the use of external oversight for validation. The American Legion recommends VA should closely monitor and record common errors from previous BVA and AMC remands and grants. Such status monitoring and documentation could help VA identify errors consistently made in a Regional Office or by specific individuals that are eventually recognized and corrected later in the process. The American Legion believes this would help isolate trends needing immediate corrective action. Unless there is a mechanism for identifying, compiling, reporting and correcting those errors, the office(s) or individual(s) making repeated mistakes continue. This concept also applies to the systemic review nationwide of claims by STAR. The American Legion believes if the error reporting of all three entities is combined, it would constitute an even more effective pool of data to enhance VA's analysis of internal problems. As Acting Under Secretary Walcoff stated, that quality of processing and not quantity of processing is the primary concern of VA, then an overarching and thorough assessment of every aspect of claims rating in which quality falls short is essential to rectifying the system. The American Legion believes there is plenty of data available to be harvested that would reveal exactly where VA is making its most common mistakes. There is no question that there are issues where VA struggles more than others. The American Legion recommends VA should effectively use this data to focus available resources where VA could improve accuracy. VBA should analyze where the process is weakest and use the analysis to develop meaningful training to improve performance. The most obvious solution to improve accuracy is meaningful and timely training of employees. Utilizing the wealth of information on weak points that can be generated by compiling data on common errors, VA can create a targeted training plan focused on weakness most in need of improvement. If focused and meaningful, VA's continuing education and training can maximize effectiveness of performance. When The American Legion conducts its quality assessment visits to Regional Offices around the country, one of the most common complaints from VA personnel, who actually process the claims, is the lack of useful training. An all too often made complaint from VA personnel is that the training is repetitive, unclear and not in areas that they struggle with most. Employees identify segments of the rating schedule that they find: most confusing, raise questions about compliance with notification due to the Veterans' Claims Assistance Act (VCAA), or questions on how to better communicate with examining physicians to achieve exams that provide enough information to adequately rate a claim. STAR review, in conjunction with data gathered from the BVA, AMC and perhaps employee review can find where VA employees are struggling most and get them the training they need to get the job done right the first time. The American Legion is confident that VA employees want to get the claims right, but struggle under the dual burdens of inadequate training in a complex system and time pressures that place a greater value on production rather than on quality. VA recently increased the required training time annually from 80 hours to 85 hours. More of this training will be standardized. By reforming training with regard to common errors identified through internal review processes such as STAR, VA can build on its commitment to more effective and meaningful training and provide its employees all the skill sets and knowledge they need to do their jobs right--the first time. The fact that VA has been inconsistent following up on corrections identified by STAR highlights the need for outside oversight. Even when problems have been identified, VA has fallen short at the goal of rectifying those problems. VA states a target of 90 percent accuracy, yet VAOIG has pointed out that VA's internal numbers are artificially inflated by inaccurate self-reporting and still fall short of those numbers. Incentives for bonuses in the management of Regional Offices are all numbers-based and driven by workload measurement (quantity not quality). Consequently, there appears to be little to no incentive within VBA to accurately report numbers that would work counter to achieving these bonus goals. The American Legion recommends that external audits and congressional oversight would greatly help redirect emphasis on quality over quantity. Under the provisions of Public Law 110-389, the Veterans' Benefits Improvement Act of 2008, the VA Secretary is to contract with an independent entity to conduct, over a three-year period, an assessment of the VA quality assurance program. In addition, the VA Secretary is to report the entity's findings and conclusions to Congress. The American Legion looks forward to this report, but would encourage VA to aggressively strive for marked improvements in accuracy before, during and after this mandated assessment. The American Legion is confident that VA is capable of internal improvement. There is sufficient information now to begin making STAR work the way it should for VA employees and veterans. The American Legion stands ready to answer any questions of this Subcommittee and thank you again for this opportunity to provide testimony. Prepared Statement of Bradley G. Mayes, Director, Compensation and Pension Service, Veterans Benefits Administration, U.S. Department of Veterans Affairs Mr. Chairman and Members of the Subcommittee, thank you for providing me with this opportunity to discuss the Veterans Benefits Administration's (VBA) Quality Assurance Program and the positive effect it has on the processing of Veterans' disability claims. Joining me today are Edna MacDonald, Acting Deputy Director of Compensation & Pension Service for Policy and Procedures, and Terence Meehan, Director of VBA Employee Development and Training. The Subcommittee has indicated a special interest in our Systemic Technical Accuracy Review (STAR) Program. However, I want to emphasize that STAR is only one of four tiers of our multi-faceted national Quality Assurance Program. The STAR component focuses on claims processing accuracy, while the other three components address regional office oversight, rating consistency, and special focus reviews. Along with the STAR Program, these components collaborate to ensure high quality and consistent decisions for Veterans. Before discussing the STAR program, I will briefly discuss the three other facets of our Quality Assurance Program, which endeavors to ensure that compensation and pension benefits are provided in a timely, accurate, and consistent manner. Quality Assurance Program Oversight The oversight component involves compliance oversight visits to regional offices by members of Central Office site survey teams. Each regional office is visited on a three-year rotating basis. All operations of the visited regional office are reviewed and evaluated, with recommendations provided for improving claims processing efficiency, accuracy, and timeliness. Additionally, the site visit team assesses whether the regional office is in compliance with VBA policy and procedures and consistent with national standards. A Web site has recently been created to share the ``best practices'' identified during site visits. Consistency The consistency component is based on mining of source data from the VBA corporate database that houses information from all regional office rating decisions. Given the possibility for variation in disability decisions, it is incumbent on VA to ensure program integrity by having a credible system for identifying indications of inconsistency among its regional offices and then remedying any inconsistencies found to be unreasonable. The two key pieces of information obtained are the grant rate and the evaluation distribution. Data analysis of recently completed rating decisions identifies the most frequently rated diagnostic codes, and plots both the grant/denial rate and evaluation assigned across all regional offices. This information focuses on rating decisions for specific disabilities and provides a method to evaluate consistency and accuracy on a regional office level. A focused review of files from the regional offices that are above or below the national average is conducted with the goal of identifying causes for the statistical anomaly. Once root causes are identified, the regional offices are notified of any recommendations, which may include training to correct problems or improper procedures identified during the review. Special Focus Reviews Special focus reviews address topics of special interest to VBA or other stakeholders where accuracy and consistency are an issue and are conducted as needed in support of VA's mission and needs. They address a specified purpose or type of claim and may involve a nationwide review or a review of work assigned to a specific regional office. The ad hoc reviews can be one-time or recurring in nature. For example, consolidation of the processing of radiation claims began on October 16, 2006. In 2008 the STAR staff conducted a special focused review of radiation claims completed between October 2006 and October 2007. The findings from this review provided a means for assessing the consolidation effort. The review found the overall accuracy and processing timeliness of radiation claims improved following consolidation. STAR STAR is the quality assurance component that focuses on claims processing accuracy. STAR reviews are focused on outcomes for Veterans rather than specific processes by which the outcomes are reached. STAR reviews evaluate the quality of the rating decision product that VBA provides for Veterans. From the Veteran's perspective, there is an expectation that we understand the claim, evaluate it accurately and fairly, and provide proper compensation under the law. The purpose of STAR reviews is to ensure that rating decision outcomes meet these expectations. The STAR system includes review of three types of work: claims that usually require a rating decision; authorization work that does not generally require a rating decision; and fiduciary work. The focus on rating-related decisions and authorization actions is a benefit entitlement review using a structured checklist to ensure all issues were addressed, claims assistance was provided, and the decision was correct (to include a correct effective date). Accuracy results are calculated on the results of the benefit entitlement review. STAR findings provide statistically valid accuracy results at both the regional office and national level. In addition, quality reviews of the accuracy of VBA examination requests and VHA examination reports are conducted in collaboration with the Compensation and Pension Examination Program (CPEP) office. VBA continues to focus on expanding and improving the STAR Program. In 2008 the STAR staff was consolidated to Nashville, which provided space for expansion and allowed aggressive recruitment for more STAR staff Since then, STAR has completed an extensive expansion effort, more than doubling the staff and increasing the sample size to obtain a statistically valid sample at the regional office level. In 2010, quality review of fiduciary cases was transferred to Nashville. During fiscal year (FY) 2009, 24,747 cases were reviewed for rating and authorization accuracy, and 3,671 cases were reviewed for fiduciary accuracy. The targeted number of cases for STAR review in FY 2011 is 37,932. The STAR sample was expanded in 2009 to include a review of brokered work completed by VBA's 12 Resource Centers and one Tiger Team (a Cleveland-based team focus on processing a unique subset of claims), and sampling was increased for the Pension Management Centers to allow measurement of pension entitlement decisions. Ongoing reviews of the Disability Evaluation System cases and Appeals Management Center cases became part of the monthly compensation quality sample in FY 2009. STAR error trends are identified and used as training topics. Training continues to be a priority and is conducted using a variety of methods, including a monthly national Quality Call, where Compensation & Pension Service's training, policy, and procedures staffs collaborate with the STAR staff to address national error trends identified in STAR assessments. To assure accuracy of STAR findings, a second-level peer review of all comments was implemented in FY 2009. Regional offices are provided explanations on all error calls, and they are required to take corrective action. On a quarterly basis, regional offices are required to certify to VBA headquarters the corrective action taken for all errors identified by STAR. The reported actions are validated during the oversight visits conducted by the site survey teams. Section 224 of the Veterans' Benefits Improvement Act of 2008 (PL 110-389) required VA to contract with a third-party entity to conduct an assessment of the quality assurance program, evaluate a sample of employees' work, measure the performance of VA regional offices and accuracy of rating, assess employee and manager performance, and produce data to help identify trends. VBA contracted with the Institute for Defense Analyses (IDA) to conduct this assessment. IDA furnished preliminary findings concerning its evaluation of the national accuracy of work, the regional office accuracy of work, the accuracy of disability ratings, and the consistency of disability ratings. Its preliminary findings include an assessment that the current STAR accuracy review program is adequately designed to estimate, within a 5 percent margin of error, the percentage of claims completed that contain errors both nationally and by regional office. IDA is continuing to determine options for identifying the accuracy and consistency of disability rating decisions. VBA anticipates that further improvements in the STAR Program, as well as other components of the Quality Assurance Program, will result from recommendations from the IDA study. We are looking forward to the recommendations from IDA and working collaboratively with them to further improve our Quality Assurance Program. Quality Assurance and Training Training Improvements VBA is committed to using the error trends and accuracy findings to improve overall quality. VBA uses nationwide error patterns identified by STAR reviews, as well as information from other components of the Quality Assurance Program, to adjust and develop the employee training curricula. All employees, regardless of training level, must receive 80 hours of instruction annually. Instructional methods may include Training Performance Support System (TPSS) modules, lectures, or practical application exercises. For intermediate and journey-level employees, the 85 hours must include 40 Core Technical Training Requirement (CTTR) hours. These involve standardized training curricula of essential topics and information. Employees must complete an additional 20 hours of training from a list of standardized topics provided by VBA. The final 20 hours may be used by regional offices to train on local issues and areas of concern. This approach ensures that new and experienced employees are grounded in standardized claims processing fundamentals. Data from STAR reviews, consistency reviews, special focus reviews, and regional office site visits, are used to develop training for our new hires, as well as our intermediate and journey-level employees. Claims processing personnel are timely informed of errors and inconsistency trends and provided with constructive feedback, including instructions on how to avoid such errors in the future. The error trends identified in STAR reviews provide us the information we need to assess the effectiveness of our training programs and make necessary adjustments. This promotes our goal of providing accurate, fair, and consistent claims processing. Office of Employee Development and Training To ensure that our training is up to date and incorporates the lessons learned from the Quality Assurance Program, VBA has a robust training evaluation program conducted by the Technical Training and Evaluation section of the Office of Employee Development and Training. With the assistance of professionally qualified outside evaluators, this office has undertaken two formal evaluations of the centralized Challenge program for newly hired claims personnel. The first formal evaluation, conducted during 2007-2008, included visits to 16 regional offices and surveys with 1,405 respondent trainees. This led to the following three recommendations, which were adopted in reformulating the Challenge curricula: Re-sequencing the content to eliminate redundancy and make better use of resources, Providing standardized hands-on claim processing during the centralized training portion, and Creating more formal requirements for delivery and compliance. The second evaluation of Challenge training began at the end of 2009. Preliminary findings show that the changes made to Challenge significantly improved its effectiveness, as demonstrated by new trainees now being able to correctly process simple cases immediately after completing the centralized portion of the training. TPSS training was separately evaluated during 2005-2006 and 2006-2007, and then subsumed into the Challenge evaluations because of its high usage during the initial training phase. In total, the Office of Employee Development and Training visited 34 regional offices during the programmatic evaluations of Challenge and collected more than 3,200 surveys. This is in addition to the hundreds of individual participant Challenge surveys done in the course of the three phases of Challenge classes. Performance support training tools allied to TPSS modules continue to show high and increasing usage, reflecting their utility to the field. For example, the Medical Electronic Performance Support System provides computerized visual images of the various human body systems. It was developed with STAR review input to assist with identifying the appropriate rating codes associated with different body systems and to facilitate medical examination requests. This tool had 633,585 unique user sessions in FY 2009, a 32 percent increase from FY 2008. Another training tool, the Veterans Service Representative Assistant, was designed to assist with claims development and had 34,696 unique sessions in FY 2009, a 62 percent increase from FY 2008. The increased use of these performance support tools may be attributable to a growing population of new claims processors who have learned about them during Challenge training, where their use is integrated into practical applications. VBA continues to improve its assessment of training program quality and effectiveness by collecting more timely feedback from participants. Improvements in the last two years, for instance, include adding surveys during the home-station segments of Challenge training and promulgating an official requirement to complete surveys for both Challenge and CTTR training. Conclusion The VBA Quality Assurance program has undergone significant change over the past several years and has become more comprehensive by expanding the type and breadth of cases reviewed. This is a journey that we are on, and we look forward to learning from the IDA review and working with our stakeholders to continuously improve our Quality Assurance Program. Thank you for the opportunity to provide you an update on our accomplishments. MATERIAL SUBMITTED FOR THE RECORD U.S. Department of Veterans Affairs Office of Inspector General Washington, DC. April 29, 2010 The Honorable John Hall Chairman, Subcommittee on Disability Assistance and Memorial Affairs Committee on Veterans' Affairs United States House of Representatives Washington, DC 20515 Dear Mr. Chairman: At the March 24, 2010, hearing before the Subcommittee on Examination of VA Regional Office Disability Claims Quality Review Methods--Is VBA's Systematic Technical Accuracy Review (STAR) Making the Grade?, you requested additional information on the results of file reviews associated with our March 2009 report, Audit of Veterans Benefits Administration Compensation Rating Accuracy and Consistency Reviews. Specifically, you asked for the ratio of underpayments and overpayments in files that contained errors. We did not collect data from the VA regional offices during our audit to determine the ratio. The Veterans Benefits Administration may be able to determine the ratio from other data sources. Thank you for your interest in the Department of Veterans Affairs. Sincerely, /S/Joanne M. Moffett for GEORGE J. OPFER Inspector General Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. April 5, 2010 Daniel Bertoni Director, Education, Workforce, and Security U.S. Government Accountability Office 441 G Street, NW Washington, DC 20548 Dear Mr. Bertoni: Thank you for testifying at the House Committee on Veterans' Affairs' Subcommittee on Disability Assistance and Memorial Affairs' oversight hearing on the ``Examination of VA Regional Office Disability Claims Quality Review Methods--Is VBA's Systematic Technical Accuracy Review (STAR) Making the Grade?,'' held on March 24, 2010. I would greatly appreciate if you would provide answers to the enclosed follow- up hearing questions by Thursday, May 6, 2010. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Due to the delay in receiving mail, please provide your responses to Ms. Megan Williams by fax at (202) 225-2034. If you have any questions, please call (202) 225-3608. Sincerely, John J. Hall Chairman __________ United States Government Accountability Office Washington, DC. May 4, 2010 The Honorable John J. Hall Chairman Subcommittee on Disability Assistance and Memorial Affairs Committee on Veterans' Affairs House of Representatives Subject: VA Has Improved Its Programs for Measuring Accuracy and Consistency, but Challenges Remain This letter responds to your April 5, 2010, request that we answer questions related to our testimony on March 24, 2010.\1\ During that hearing, we discussed the progress the Department of Veterans Affairs (VA) has made in improving its ability to measure the accuracy and consistency of its decisions on disability benefit claims; and remaining challenges to improving accuracy and consistency. Your questions, along with our responses, follow. --------------------------------------------------------------------------- \1\ Veterans' Disability Benefits: VA Has Improved Its Programs for Measuring Accuracy and Consistency, but Challenges Remain, GAO-10-530T (Washington, D.C.: March 24, 2010). 1. In your testimony, you observed that while the STAR system was used to review the accuracy of some Veterans Benefits Administration (VBA) claims programs, Benefits Delivery at Discharge (BDD) and Quick Start pre-discharge claims were not assessed for quality by the STAR program. Are there other programs that are not being assessed by the STAR system, which should be, in your opinion? If there are such programs, what are the benefits of using STAR to examine the quality of these VBA programs? Could STAR reviews for all VBA ratings programs help improve the agency's ability to more timely deliver --------------------------------------------------------------------------- benefits to deserving veterans. In our September 2008 report on BDD,\2\ we noted that VBA lacked measures that would allow it to assess whether decisions on BDD claims were more or less accurate than decisions under its traditional claims process, or whether BDD locations faced challenges in processing claims. As noted in my statement, improved accuracy would improve VBA's ability to provide veterans and their survivors the benefits to which they are entitled. Also, improved accuracy helps to ensure that decisions are correct the first time, which can prevent delays in VA's lengthy appeals process. In addition, VBA's consistency reviews have the potential to improve VBA's ability to provide comparable benefits to comparable veterans across its 57 regional offices. We have not identified any other specific types of claims where we have recommended targeted reviews under the Systematic Technical Accuracy Review (STAR) program. --------------------------------------------------------------------------- \2\ GAO, Veterans' Disability Benefits: Better Accountability and Access Would Improve the Benefits Delivery at Discharge Program, GAO- 08-901 (Washington, D.C.: Sept. 9, 2008). 2. Detail all the VBA programs that are currently being assessed by the STAR system, and provide the annual accuracy --------------------------------------------------------------------------- rates for those programs for 2009. VA provided information on several types of claims where it was conducting STAR reviews, with fiscal year 2009 accuracy rates. Table 1: Accuracy rates of VBA programs assessed by STAR, Fiscal Year 2009 ------------------------------------------------------------------------ Accuracy rate (percent) ------------------------------------------------------------------------ Compensation entitlement (key 84 performance measure) ------------------------------------------------------------------------ Pension entitlement 97 ------------------------------------------------------------------------ Redistributed (brokered) cases 77 \a\ ------------------------------------------------------------------------ Disability Evaluation System pilot cases 88 \b\ ------------------------------------------------------------------------ Appeals Management Center \c\ 83 ------------------------------------------------------------------------ \a\ Benefit entitlement decisions made by VBA's 9 rating resource centers. VBA also reported a 78 percent accuracy rate for ratings by the Tiger Team at the Cleveland Regional Office. According to VBA, it began STAR reviews of these cases in July 2009. \b\ According to VBA, it began STAR reviews of Disability Evaluation System pilot ratings by the Baltimore and Seattle regional offices in June 2009. \c\ According to VBA, it began reviewing Appeals Management Center decisions in July 2009. Source: VBA. 3. You both observed that STAR review staff lacked the manpower and training to fully utilize the quality control mechanisms offered by the STAR program. With these issues in mind, how many Full Time Equivalent (FTE) employees would be ideal, and what training for those employees would you recommend? In its March 2009 report, VA's Office of Inspector General noted that VBA lacked sufficient staff to conduct planned consistency reviews. Since then, VBA has hired additional quality assurance staff. We have not done the work needed to estimate the number of Full Time Equivalent (FTE) staff the quality assurance programs need to conduct its planned quality assurance reviews. In responding to these questions, we relied on information we collected in preparing our March 24, 2010, written statement. For further information, please contact Daniel Bertoni at (202) 512-7215 or [email protected]. Daniel Bertoni Director, Education, Workforce, and Income Security Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. April 5, 2010 Bradley Mayes Director, Compensation and Pension Service Veterans Benefits Administration U.S. Department of Veterans Affairs 810 Vermont Avenue, NW Washington, DC 20240 Dear Mr. Mayes: Thank you for testifying at the House Committee on Veterans' Affairs' Subcommittee on Disability Assistance and Memorial Affairs' oversight hearing on the ``Examination of VA Regional Office Disability Claims Quality Review Methods--Is VBA's Systematic Technical Accuracy Review (STAR) Making the Grade?,'' held on March 24, 2010. I would greatly appreciate if you would provide answers to the enclosed follow- up hearing questions by Thursday, May 6, 2010. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Due to the delay in receiving mail, please provide your responses to Ms. Megan Williams by fax at (202) 225-2034. If you have any questions, please call (202) 225-3608. Sincerely, John J. Hall Chairman __________ Questions for the Record from the HVAC DAMA Subcommittee Hearing on Examination of VA Regional Office Disability Claims Quality Review Methods Is VBA's Systematic Technical Accuracy Review (STAR) Making the Grade? March 24, 2010 Question 1. Does VA know why the Des Moines Regional Office (RO) is the highest performing RO with an accuracy rate reported by STAR of 92 percent? Response: Regional office performance varies as a result of a number of factors including workforce experience, local economic and employment environment, and staff turnover. Regional offices that consistently perform well are in areas where VA is an employer of choice. In these locations, VA is able to recruit and retain high- performing employees. The Des Moines RO has a stable and well-trained workforce, allowing the RO to achieve a high accuracy rate. Question 2. Does the VA know why the Des Moines RO is performing at such a higher rate than the Baltimore RO (69 percent)? Response: ROs that have difficulty in meeting performance targets are predominantly in high-cost metropolitan areas with high employee turnover. The Baltimore RO is located in a densely populated area and faces significant competition in recruiting and retaining a highly skilled workforce. Question 3. Please detail the plans that VA has in place to take lessons learned from what is being done right in the Des Moines RO and apply them to lower performing ROs like Baltimore. Response: VA aggressively monitors regional office performance and develops specific action plans to improve problem areas. Performance is evaluated against both national and individual targets that are established at the beginning of each fiscal year. Oversight is provided through site visits conducted by both the Compensation and Pension Service and the Area Directors. Lessons learned and specific examples of ``best practices'' from these visits are provided to assist ROs increase their performance. Question 4: Please provide the Subcommittee with data about the breakdown of claims processing errors reported by STAR and the VA OIG report. What percentages of these errors were Veterans being under compensated, and what percentage were Veterans being overcompensated? Response: STAR performed 21,747 national quality reviews in FY 2009 and cited 2,676 benefit entitlement errors. Of the benefit entitlement errors, 18 percent were potential Veteran/Beneficiary overpayments and 24 percent were potential Veteran/Beneficiary underpayments. Question 5. Please provide the Subcommittee with data for outgoing and new RO directors for the past three years, along with RO error data for the same time. Does the VA find any correlation between new leadership and a change in the accuracy of an RO? Response: The RO director is a critical component in the performance of a regional office. However, VBA does not believe that the STAR accuracy rate, be it positive or negative, can be attributed to just one factor or one individual. Data requested is attached (Attachment 1). Question 6. Please provide the list of all the names of the VA Regional Office Directors, the date on which they were hired, their current salaries, last bonus, and the 2009 accuracy rate per STAR review. Response: See spreadsheet. [Attachment 2, which lists the current salaries and bonuses of the VA Regional Directors is being retained in the Committee files. ] Question 7: Please provide the name and location of the STAR team director. Response: The STAR team director is Edna MacDonald, Assistant Director for Quality Assurance. She is located in VA Central Office, Washington DC. Question 8: Please provide the number of STAR reviewers. Response: There are 38 STAR team reviewers, which is an increase of 66 percent from FY 2009. Attachment 1 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Time in STAR Rating STAR Rating STAR Rating Station Area Director Effective Date of Position Previous Director Accuracy Accuracy Accuracy Appointment (Yrs) Rate (FY09) Rate (FY08) Rate (FY07) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Philadelphia Eastern Lastowka, Thomas 7/1/1990 19.8 Cary, Robert 79.9% 87.4% 91.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Columbia Southern Hawkins, Carl 10/1/1998 11.6 Sloan, Stedman 88.1% 87.6% 88.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Boise Western Vance, James 1/30/2000 10.2 Barker, Barry 82.4% 88.2% 88.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Waco Central Lowe, Carl 3/12/2000 10.1 McRae, Jerry 84.5% 81.7% 90.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Baltimore Eastern Wolohojian, George 12/15/2002 7.4 Quinton, Newell 67.9% 82.7% 88.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Phoenix Western Flint, Sandy 6/1/2003 6.9 Nappi, Patrick 86.1% 86.8% 91.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ St. Louis Central Unterwagner, Dave 9/21/2003 6.6 Graves, Kim 84.0% 84.0% 93.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ New Orleans Central Christian, Rowland 2/8/2004 6.2 Jackson, Barry 81.9% 88.4% 93.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ San Diego Western Fetzer, Lily 8/8/2004 5.7 Dusenbery, Michael 83.5% 82.2% 86.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Buffalo Eastern Terrell, Donna 10/31/2004 5.5 McCoy, Jack 82.9% 87.5% 90.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Montgomery Southern Randle, Ric 6/26/2005 4.8 Watson, Montgomery 81.6% 84.4% 84.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Muskogee Central Jarvis, Sam 10/2/2005 4.6 Burks, Larry 89.2% 86.9% 94.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Manila Western Skelly, Jon 1/8/2006 4.3 Barker, Barry 89.7% 89.7% 95.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Louisville Southern Thompson, Keith 4/16/2006 4.0 Wardle, Jimmy 87.6% 86.5% 89.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Manchester Eastern Woolford, Charles 8/20/2006 3.7 Cully, Maribeth 76.0% 81.0% 85.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ White River Junction Eastern Woolford, Charles 8/20/2006 3.7 Cully, Maribeth 79.1% 83.3% 81.6% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Los Angeles Western Witty, Kerrie 9/3/2006 3.6 Liff, Stewart 75.5% 81.5% 81.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Denver Western Jacobs, Janice 11/26/2006 3.4 Alger, Jeffrey 86.0% 89.7% 90.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Detroit Eastern Walker, Linda 12/24/2006 3.3 Thompson, Keith 78.1% 76.1% 90.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Roanoke Southern Nicholas, William 2/4/2007 3.2 Smith, John 85.5% 89.5% 90.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Cleveland Eastern Cange, Joyce 7/8/2007 2.8 Mayes, Brad 88.6% 84.9% 95.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Oakland Western Flint, Lynn 7/8/2007 2.8 Smith, Catherine 82.3% 89.6% 91.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Hartford Eastern Leonard, Dave 7/22/2007 2.8 Walker, Linda 83.1% 87.4% 86.1% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Little Rock Central Rawls, Cheryl 8/5/2007 2.7 Nicholas, William 87.9% 86.9% 86.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Providence Eastern Navaratnasingam, Pritz 9/2/2007 2.6 Witty, Kerrie 86.0% 90.3% 91.6% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Huntington Southern Commens, Zita 3/30/2008 2.1 Rawls, Cheryl 82.9% 86.0% 94.1% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Atlanta Southern Bocchicchio, Al 4/27/2008 2.0 Burks, Larry 79.0% 77.5% 84.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Winston-Salem Southern Umlauf, Dan 6/22/2008 1.8 Montgomery, John 86.0% 90.6% 89.6% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Salt Lake City Western Bilosz, Mark 7/20/2008 1.8 Wadsworth, Douglas 80.4% 88.2% 88.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Lincoln Central Miller, Loren 8/3/2008 1.7 Smith, Peter 92.9% 94.9% 93.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Wichita Central Waller, Antione 10/12/2008 1.5 Umlauf, Dan 79.4% 84.7% 86.7% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Chicago Central Honeycutt, Duane 11/9/2008 1.4 Olson, Michael 81.6% 86.5% 88.7% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Indianapolis Eastern Stephens, Michael 1/4/2009 1.3 Kuewa, Dennis 84.6% 86.0% 88.1% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ St. Paul Central Fillman, Carol 1/18/2009 1.3 Alger, Jeffrey 86.1% 91.7% 90.9% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Togus Eastern Karczewski, Scott 2/25/2009 1.2 Bilosz, Mark 90.4% 92.9% 90.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Des Moines Central Davis, Dave 3/15/2009 1.1 Braley, Richard 92.7% 91.5% 93.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Honolulu Western Betts, Tracy 3/29/2009 1.1 Reed, Greg 75.9% 83.7% 78.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Jackson Southern Moore, Craig 7/19/2009 0.8 Adair, Joseph 81.8% 82.3% 87.1% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ San Juan Southern Murphy, Thomas 8/2/2009 0.7 Moreno, Sonia 72.1% 75.7% 82.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Pittsburgh Eastern McCoy, Beth 10/11/2009 0.5 Cully, Maribeth 80.6% 88.7% 92.4% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Albuquerque Western Singleton, Grant 10/25/2009 0.5 Moore, Craig 79.8% 90.9% 93.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ New York Eastern Malley, Sue 10/25/2009 0.5 Amberg-Blyskal, Patricia 75.9% 76.0% 83.0% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Seattle Western Prieb, Patrick 10/25/2009 0.5 Fillman, Carol 81.6% 83.5% 88.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Reno Western Russell, Ed 11/8/2009 0.5 Iddings, Donald 90.7% 89.2% 89.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Portland Western Marshall, Chris 11/22/2009 0.4 McClellan, 86.6% 85.8% 87.0% Christopher ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sioux Falls Central Brubaker, James 1/4/2010 0.3 Smith, John 86.9% 94.5% 90.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Washington Southern Reyes-Maggio, Judy 10/31/2004 5.5 Wilson, Keith N/A 59.7% 75.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Appeals Mgmt Cntr OFO Burke, Ronald 2/1/2009 1.2 Russo, Arnold N/A N/A N/A ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ St. Louis RMC Central Hawthorne, Corey 3/28/2010 0.1 Prieb, Patrick N/A N/A N/A ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Anchorage Western Remotely managed by Salt 86.5% 84.9% 86.3% Lake City Director (Bilosz, Mark) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Boston Eastern VACANT Braley, Richard 77.5% 81.3% 81.1% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Cheyenne Western Remotely managed by the N/A N/A N/A Denver Director (Jacobs, Janice) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Fargo Central Remotely managed by 91.6% 91.3% 86.7% Sioux Falls Director (Brubaker, James) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Fort Harrison Western Remotely managed by Salt 91.4% 91.0% 92.6% Lake City Director (Bilosz, Mark) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Houston Central VACANT Henderson, Ursula 82.4% 82.5% 84.3% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Milwaukee Central VACANT Olson, Michael 92.7% 93.0% 92.6% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Nashville Southern VACANT Corley, Brian 88.2% 92.6% 93.2% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Newark Eastern VACANT McCourt, John 81.0% 83.7% 82.8% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ St. Petersburg Southern VACANT Barker, Barry 84.7% 87.0% 91.7% ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Wilmington Eastern Remotely managed by the 82.4% 83.6% 83.6% Philadelphia Director (Lastowka, Thomas) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------