[House Hearing, 110 Congress] [From the U.S. Government Publishing Office] EXAMINING THE EFFECTIVENESS OF THE VETERANS BENEFITS ADMINISTRATION'S TRAINING, PERFORMANCE MANAGEMENT AND ACCOUNTABILITY ======================================================================= HEARING before the SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS of the COMMITTEE ON VETERANS' AFFAIRS U.S. HOUSE OF REPRESENTATIVES ONE HUNDRED TENTH CONGRESS SECOND SESSION __________ SEPTEMBER 18, 2008 __________ Serial No. 110-105 __________ Printed for the use of the Committee on Veterans' Affairs U.S. GOVERNMENT PRINTING OFFICE 44-932 WASHINGTON : 2009 ---------------------------------------------------------------------- For sale by the Superintendent of Documents, U.S. Government Printing Office Internet: bookstore.gpo.gov Phone: toll free(866) 512-1800; DC area (202) 512-1800 Fax: (202) 512-2104 Mail: Stop IDCC, Washington, DC 20402-0001 COMMITTEE ON VETERANS' AFFAIRS BOB FILNER, California, Chairman CORRINE BROWN, Florida STEVE BUYER, Indiana, Ranking VIC SNYDER, Arkansas CLIFF STEARNS, Florida MICHAEL H. MICHAUD, Maine JERRY MORAN, Kansas STEPHANIE HERSETH SANDLIN, South HENRY E. BROWN, Jr., South Dakota Carolina HARRY E. MITCHELL, Arizona JEFF MILLER, Florida JOHN J. HALL, New York JOHN BOOZMAN, Arkansas PHIL HARE, Illinois GINNY BROWN-WAITE, Florida SHELLEY BERKLEY, Nevada MICHAEL R. TURNER, Ohio JOHN T. SALAZAR, Colorado BRIAN P. BILBRAY, California CIRO D. RODRIGUEZ, Texas DOUG LAMBORN, Colorado JOE DONNELLY, Indiana GUS M. BILIRAKIS, Florida JERRY McNERNEY, California VERN BUCHANAN, Florida ZACHARY T. SPACE, Ohio STEVE SCALISE, Louisiana TIMOTHY J. WALZ, Minnesota DONALD J. CAZAYOUX, Jr., Louisiana Malcom A. Shorter, Staff Director ______ SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS JOHN J. HALL, New York, Chairman CIRO D. RODRIGUEZ, Texas DOUG LAMBORN, Colorado, Ranking PHIL HARE, Illinois MICHAEL R. TURNER, Ohio SHELLEY BERKLEY, Nevada GUS M. BILIRAKIS, Florida Pursuant to clause 2(e)(4) of Rule XI of the Rules of the House, public hearing records of the Committee on Veterans' Affairs are also published in electronic form. The printed hearing record remains the official version. Because electronic submissions are used to prepare both printed and electronic versions of the hearing record, the process of converting between various electronic formats may introduce unintentional errors or omissions. Such occurrences are inherent in the current publication process and should diminish as the process is further refined. C O N T E N T S __________ September 18, 2008 Page Examining the Effectiveness of the Veterans Benefits Administration's Training, Performance Management and Accountability................................................. 1 OPENING STATEMENTS Chairman John J. Hall............................................ 1 Prepared statement of Chairman Hall.......................... 31 Hon. Michael R. Turner........................................... 3 Hon. Doug Lamborn, Ranking Republican Member, prepared statement of............................................................. 32 WITNESSES U.S. Government Accountability Office, Daniel Bertoni, Director, Education, Workforce, and Income Security Issues............... 4 Prepared statement of Mr. Bertoni............................ 32 U.S. Department of Veterans Affairs, Michael Walcoff, Deputy Under Secretary for Benefits, Veterans Benefits Administration. 20 Prepared statement of Mr. Walcoff............................ 68 ______ American Federation of Government Employees, AFL-CIO, Michael Ratajczak, Decision Review Officer, Cleveland Veterans Affairs Regional Office, Veterans Benefits Administration, U.S. Department of Veterans......................................... 9 Prepared statement of Mr. Ratajczak.......................... 39 Bartzis, Nicholas T., Cleveland, OH.............................. 15 Prepared statement of Mr. Bartzis............................ 65 Disabled American Veterans, Kerry Baker, Assistant National Legislative Director........................................... 11 Prepared statement of Mr. Baker.............................. 42 National Veterans Legal Services Program, Ronald B. Abrams, Joint Executive Director............................................. 13 Prepared statement of Mr. Abrams............................. 47 Human Resources Research Organization, Patricia A. Keenan, Ph.D., Program Manager................................................ 14 Prepared statement of Ms. Keenan............................. 61 MATERIAL SUBMITTED FOR THE RECORD Reports: Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors, Report No. GAO-08-561, dated May 2008............................................. 71 Post-Hearing Questions and Responses for the Record: Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Michael Ratajczak, Decision Review Officer, Cleveland Veterans Affairs Regional Office, letter dated September 25, 2008, and Mr. Ratajczak's responses.......... 108 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Kerry Baker, Assistant National Legislative Director, Disabled American Veterans, letter dated September 22, 2008, and Mr. Baker's responses.............. 113 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Ronald B. Abrams, Joint Executive Director, National Veterans Legal Services Program, letter dated September 25, 2008, and Mr. Abrams responses............... 114 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Dr. Patricia Keenan, Program Manager, Human Resources Research Organization, letter dated September 22, 2008, and Dr. Keenan's response............................ 117 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Mr. Michael Walcoff, Deputy Under Secretary for Benefits, Veterans Benefits Administration, U.S. Department of Veterans Affairs, letter dated September 22, 2008, and VA responses............................................... 119 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Bradley Mayes, Director, Compensation and Pension Service, Veterans Benefits Administration, U.S. Department of Veterans Affairs, letter dated September 22, 2008, and VA responses..................................... 121 Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, to Dorothy Mackay, Director, Employee Development and Training, Veterans Benefits Administration, U.S. Department of Veterans Affairs, letter dated September 25, 2008, and VA responses..................................... 122 EXAMINING THE EFFECTIVENESS OF THE VETERANS BENEFITS ADMINISTRATION'S TRAINING, PERFORMANCE MANAGEMENT AND ACCOUNTABILITY ---------- THURSDAY, SEPTEMBER 18, 2008 U.S. House of Representatives, Committee on Veterans' Affairs, Subcommittee on Disability Assistance and Memorial Affairs, Washington, DC. The Subcommittee met, pursuant to notice, at 10:06 a.m., in Room 340, Cannon House Office Building, Hon. John Hall [Chairman of the Subcommittee] presiding. Present: Representatives Hall, Lamborn, and Turner. OPENING STATEMENT OF CHAIRMAN HALL Mr. Hall. Good morning. The House Committee on Veterans' Affairs Disability Assistance and Memorial Affairs Subcommittee hearing on examining the effectiveness of the Veterans Benefits Administration's (VBA) training, performance management, and accountability will now come to order. I would ask everyone to rise for the Pledge of Allegiance. Flags are located at both ends, actually this end. [Pledge of Allegiance.] Mr. Hall. Thank you. And welcome again. Our Nation's veterans understand the necessity of proper and adequate training. Their lives have depended on it. The military trains for its operations and everyone knows every detail of their job prior to their mission. These same veterans should be able to expect the same level of competence when they seek assistance from the Veterans Benefits Administration. Unfortunately, that is not always the case as we have heard at other meetings and hearings throughout the year that this Subcommittee has held regarding the U.S. Department of Veterans Affairs (VA) disability claims processing system. The VA has standardized its training curriculum and requires that all claims processors must complete 80 hours of annual training. This is a lot of hours because, in fact, some healthcare providers do not need to meet that level of continuing education to maintain their clinical license or credentialing. The VBA training topics are identified by the Central Office (CO) or by the individual's Regional Office (RO). New employees go through an orientation process and there are additional online learning tools available through the VBA's training and performance support system. Yet, with all this effort, VA training seems to fall short of its intended goals. Less than 50 percent of the Ratings Veterans Service Representatives or RVSRs passed the certification exam even though it was an open book test. Frankly, I have seen the training manual and it could be measured in pounds, not pages. So I do not know how useful the book is, especially given the workload that the people being trained are already under. But that is sort of the crux of the matter. As outlined in previous hearings, there are significant inconsistencies in ratings between the VA's 57 Regional Offices and a high rate of remanded cases. I am pleased that the American Federation of Government Employees (AFGE) is here to shed light on the issue. You are a critical link to those on the front lines working to improve outcomes for our disabled veterans. The U.S. Government Accountability Office (GAO) released a report in May 2008 entitled, ``Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors.'' That is a long title, but sums it up. [The GAO Report No. GAO-08-561, which was attached to Mr. Bertoni's prepared statement, appears on p. 71.] The report documented areas in which the VA needs to improve its training and hold accountable those it does train. According to the GAO, staff is not held accountable for completing the required training since the VBA does not track completion, so there are no consequences for not taking the training. Additionally, the VBA does not evaluate its training, so it does not know if it is successfully designed and implemented in educational program. Feedback is not consistently collected from RO employees on the training that they do receive and many have reported difficulty in accessing training because of their stringent productivity demands. I look forward to hearing more from the GAO about this report, but these are not surprising conclusions to the Veterans Service Organizations who have complained for years about the inadequacies of the VBA training program. So I am grateful that they have joined us here today as well. Training is not an issue that should be taken lightly. We all know the importance of good training, but effective implementation that ensures consistency and accountability can be elusive and that is what I hope we can address today. I have taken steps to ensure improved training outcomes, we have on this Committee when we passed and the full House passed H.R. 5892, the ``Veterans Disability Benefits Claims Modernization Act.'' These policy enhancements will hopefully lead to compensation claims processing improvements and more accurate claims adjudication results for our veterans and their families. Moreover, I am not sure that the VBA's current training regimen complements its current claims processing improvement model or CPI. In fact, I am positive that the current coupling detracts from increased accountability efforts. I am pleased to report that with the help of many in this room, H.R. 5892 passed unanimously on July 31, 2008, by the full House. On August 1, 2008, Senator Clinton introduced companion legislation, S. 3419, in the Senate. So, Congress hopefully is on its way to rectifying the inadequacies in the VBA training system that have already been identified. Today's oversight hearing will allow us to look deeper into this issue and gauge where VA is in terms of its training protocol and see what other improvements can still be made. I look forward to hearing from our witnesses. I hope to learn more about best practices and strategies for measuring performance, building better training protocols, and accountability standards. [The prepared statement of Chairman Hall appears on p. 31.] And Mr. Lamborn, our Ranking Member, is not present yet, but, Mr. Turner, would you like to make an opening statement? If so, you are recognized. OPENING STATEMENT OF HON. MICHAEL R. TURNER Mr. Turner. Sure. Thank you, Chairman Hall. Thank you for this opportunity for collective discussion on the effectiveness of the Veterans Benefits Administration's training, VBA's performance management and accounting requirements. Over the course of the past several months, this Subcommittee has examined nearly every facet of the VA benefits claims process system in order to identify how we might help the Department overcome the claims backlog crisis. While the recent expansion of its workforce will certainly have a positive impact, VA must ensure that newly hired claims workers receive training that is commensurate with their responsibilities. It is critical that the training it provides meets the needs of the Department and its employees. It is equally important that the results of the training are evaluated. Without feedback, VA may never know whether or not the training is accomplishing its goals. Any viable training program should be able to identify deficiencies and demonstrate the intended and actual outcome of the curriculum. VA training must be connected to its vision and mission and VA managers need to be assured that if employees are pulled off the floor for training that it will result in long-term benefits. With the growing number of pending claims, there is a certain level of trepidation that there is too much work to do already and we will just get further behind if we have to conduct training. There must be clear support from the top down in order to conduct adequate training and acquire the expected outcomes. Certainly training new employees of everything they need to know in order to make sound rating decisions is a daunting task. The VA rating schedule itself is complex and it is merely a portion of the array of knowledge a competent adjudicator must possess to perform his or her job. Today's hearing is an opportunity to not only learn more about the training and assessment program VA provides its employees but also to reiterate to the Department that it should be forthright about any additional resources deemed necessary to fulfill this critical requirement. Mr. Chairman, I thank you for hosting this hearing on VA's training program and I yield back. Mr. Hall. Thank you, Mr. Turner. I would like to remind all panelists that your complete written statements have been made a part of the hearing record so that you can limit your remarks to 5 minutes so we can have sufficient time for follow-up questions once everyone has had the opportunity to testify. Our first panel, the entire panel, is Mr. Daniel Bertoni, Director of Education, Workforce, and Income Security Issues at the U.S. Government Accountability Office. Welcome, Mr. Bertoni, and you are now recognized for 5 minutes. STATEMENT OF DANIEL BERTONI, DIRECTOR, EDUCATION, WORKFORCE, AND INCOME SECURITY ISSUES, U.S. GOVERNMENT ACCOUNTABILITY OFFICE Mr. Bertoni. Mr. Chairman, Members of the Subcommittee, good morning. I am pleased to discuss training, performance management for Department of Veterans Affairs' disability claims processors. Last year, VA paid about $38 billion in benefits to nearly four million veterans and their families. The disability claims process has chronically suffered from long waits for decisions, large backlogs, and problems with accuracy and consistency. We have also noted that VA's program is in urgent need of transformation, especially in regard to how it assesses work capacity and provides interventions and support services to veterans. To address its management challenges, VA has hired thousands of additional staff. However, increased staff alone will not guarantee more timely, accurate, and consistent decisions. Among other things, adequate training and performance management will be key to developing new staff and ensuring that more experienced staff maintain needed skills. My remarks today draw from our prior work for this Subcommittee and focus on two areas, VA's training program for disability claims staff and its performance management system. Summary. Consistent with accepted training practice, VA has taken steps to align its training with the agency's overall mission and goals. For example, in 2004, VA established the Training and Learning Board to ensure that training support of VA's strategic and business plans. Various analyses. VA has also identified the skills and abilities needed by staff and taken steps to determine the appropriate level of investment in their training and prioritize funding. Finally, we found that VA's training program for new staff defined pertinent terms and concepts and provided many realistic examples of claims work. However, we did identify areas for improvement. While the VA collects feedback to assess initial training for new staff, not all training is evaluated to determine how relevant or effective it is. None of the Regional Offices we visited consistently collected feedback on training they provided either for new or experienced staff. Thus, VA's Central Office lacks key information on its entire training activities. Both new and experienced staff reported concerns with their training. Some staff noted that VA's computer-based learning tool, Training and Performance Support System (TPSS), was often out of date and too theoretical. The more experienced staff reported that they struggle to meet VA's annual 80-hour training requirement due to workload pressures or the lack of training relevant to their experience level. It is unclear what criteria was used to justify the 80-hour requirement, but identifying the right amount of training is crucial. Overly burdensome requirements can take staff away from essential tasks while too little training can contribute to errors. Putting aside the appropriateness of the current requirement, VA has no policies to hold staff accountable for meeting it and may be missing an opportunity to convey the importance of training as a means to meet individual goals as well as broader agency performance goals. Regarding performance management, VA's system generally conforms to accepted practices in that individual performance measures such as quality and productivity are also aligned with organizational performance measures. Staff are required to receive regular feedback on performance and employees and other key stakeholders were actively involved in developing the current system. However, we are concerned that under the current rating formula, VA's system may not make meaningful distinctions in staff performance. Although VA has a five category rating system in the field offices we visited, 90 percent of all staff ended up in only two, fully successful and outstanding as noted in the chart we provided. We have reported that four or five category systems are most useful for making performance distinctions. However, if staff ratings are clustered in only one or two categories as with VA, this may be problematic. Broad, overlapping performance categories may deprive managers of information needed to reward top performers, address performance issues, deprive staff of much needed feedback. VA has acknowledged potential issues with the rating formula and is considering some adjustments. However, the agency has never examined the distribution of all staff across the five performance categories. Absent this analysis, VA will lack a clear picture of what adjustments are actually needed. Conclusion. While VA recognizes the importance of developing and maintaining high performing staff, it must devote more attention to training and performance management. Additional study on such issues as the effectiveness of regional training, the appropriateness of the current training requirement, and the usefulness of computer-based learning tools is needed. Additional means for holding staff and managers accountable for completing training should be explored as well as options for enhancing the current rating system. We acknowledge VA's efforts to hire more staff. However, in the longer term, more fundamental changes are needed. VA and other Federal disability programs must adopt a more modernized approach to determining eligibility for benefits as well as the timing and portfolio of services that are provided. If progress is made in this area, effective training and performance management will be of crucial importance and will impact the degree to which service to veterans is ultimately improved. Mr. Chairman, this concludes my statement. I am happy to answer any questions you may have. Thank you. [The prepared statement of Mr. Bertoni appears on p. 32.] Mr. Hall. Thank you, Mr. Bertoni, for that most incisive statement and for your written statement as well, which goes into considerably more detail, of course. In your opinion, what is more effective, the classroom training or online tools? Mr. Bertoni. I do not know what individual mode of training is most effective. I think any training program should have a mix of classroom training, on the job, and online, computer- based tools. Agencies should be looking at and evaluating each of those, call it the three-legged stool, to ensure that they complement each other, they are consistent in terms of the information they convey, and they are meeting the needs of staff. Mr. Hall. What did you think of the VA's learning management system and training and performance support system? Mr. Bertoni. Generally we used an in-house criteria to assess the four components of training system and generally there are four. Our criteria is based on various external experts, internal experts at GAO, private-sector individuals, nonprofits, and academia. And we came up with a criteria that we use quite often to evaluate training and performance management. Generally we break this apart into four categories, planning, design, implementation, and ultimately evaluation. And as our report notes, I think VBA and VA are doing an adequate job in many respects. They have aligned their training with broader agency goals. They have aligned their performance management system and with their measuring folks on with their broader agency goals and mission. So they are doing, I think, many things well relative to accepted practices, but still are falling short in terms of in the area of feedback. Feedback is essentially your evaluation loop. That is how you find and determine whether your training is relevant, whether it is effective, and whether it is being delivered at the appropriate time. To the extent you are not getting that feedback, I think you are missing an opportunity. They do a great job or a better job early on for the initial orientation training, but I think things fall off as staff return to their home units and begin taking training. And it is harder for us to get a handle on uniformity, consistency, and whether it is as effective as it could be. Mr. Hall. Thank you, Mr. Bertoni. I understand that proper training is vital to efficiency and accuracy, but is training the bottom line problem at VBA? Would you correlate the problems inherent in the backlog as an issue with training? Mr. Bertoni. It is hard to isolate all the factors that contribute to the backlog. There are laws, regulation that have contributed. There are management inefficiencies that have contributed. But training certainly is a tool to get at or to address many of the problems that agencies face. We would not do it if we did not think it was worthy. And clearly when you have good training, you can see results. Whether you can, you know, measure clear cause and effect, that is probably not something that you can always do. But generally a good, solid designed training system is effective. Beyond training, I do really believe that part and parcel with that is a good quality assurance process. Until you know where your soft spots are, where your problems are through your quality assurance reviews, it is very hard to design training that is going to really target and get at the problematic areas in your process. We have gone many years with timeliness, accuracy, and consistency issues, and I have to believe based on some work that we have done looking at the STAR system and other quality assurance processes that having more robust quality assurance could help VBA, VA identify some of the real root causes for inconsistency and inaccuracies. And I think they really need to focus on that. And it appears from the statement I read from VA this morning they are heading in that direction. Mr. Hall. In reviewing VBA's performance management system, how different was it from other Federal agencies? Three-part question here. Are there other agencies with a better performance management system that VA could adopt and would it make sense to give performance credits for training as well as for work completion so that there is not pressure to stay on the workload and not train? Mr. Bertoni. I do not know if it is a matter of giving credits for workload completion. I believe their rating should reflect the fact that they are doing a good job or a great job in terms of processing the workloads. Our concern with their performance management system is that it may not be rewarding. It may, in fact, be potentially demoralizing for some staff. I will give you an example. We'll use three people. You take the first person who is clearly a high flyer who rates exceptional across all the critical and noncritical dimensions. That person, it is a no-brainer. That person would be listed in the outstanding category. We will take another person who is just barely at the fully successful level and, you know, just barely eking out a fully successful rating, marginally falling below that, but at the end of the performance year ends up with a fully successful rating. Take a third person who is clearly also a high performer. That person gets exceptional ratings on all dimensions, critical and noncritical. With the exception of one, that person gets a fully successful. That individual will be dumped into the bucket with that fully successful person. Clearly that person falls somewhere between exceptional and outstanding, but because of that one fully successful in a critical dimension are dropping down into that lower bucket and all the incentives, the pay incentives, the other monetary incentives are not available. So if you are trying to create a world-class organization and incentivize people for hard work, this example really disturbs us. We see in the four regions that we visited 90 percent of all folks in two categories and nobody in that second excellent category. It gives me pause. Mr. Hall. Thank you, Mr. Bertoni. My time is expired. I will now recognize and welcome our Ranking Member, the gentleman from Colorado, Mr. Lamborn. Mr. Lamborn. Yeah. And thank you, Mr. Chairman. Mr. Bertoni, you apparently have some familiarity with other departments other than the VA because of looking at training programs in other parts of the Federal Government. What are some successful components that you have seen in some of these other areas that you would suggest the VA should imitate? Mr. Bertoni. Again, I will hearken back to what we identified in our report. I think you really need to foster a rigorous system for feedback and evaluation. If you are focusing on new employees at the initial orientation level and getting good information there, fine. We acknowledge that. But when these folks transfer back or go out into their home units and are taking this, you know, training at the regional level, there is no feedback loop that is feeding back into VBA's headquarters so they can use this information to identify areas where they are falling short and ultimately to devise training to get at some of the problems that we have talked about in the area of accuracy and consistency. World-class organizations have completed that wheel, planning, design, implemented, and your evaluation loop. To the extent that all those elements are firing, you have better systems. And there are some organizations that do better than others. Mr. Lamborn. Thank you. Mr. Chairman, I yield back. Mr. Hall. Thank you, Mr. Lamborn. And one more question, Mr. Bertoni. In 2001, the GAO issued a report ``Veterans Benefits Training for Claims Processors Needs Evaluation.'' At that time, the GAO found that the TPSS was being implemented differently in each Regional Office. I know that the VBA has mandated some of the training, but employees have raised concerns about the training and claim limited time for it because of workload demands. So what differences do you see in the VA's training process from that time, from 2001 to the present? Mr. Bertoni. We really did not delve into the TPSS system. We had the prior report. We followed up our recommendations. I know that VA has taken steps to monitor and approve that system. I think there is an ongoing effort and I think almost a report to be issued shortly also. But we really did not delve into the system exclusively. We were only able to sort of give a snapshot of what was going on in the four Regional Offices and there was clearly some noise that the TPSS system was not all it could be for some folks in terms of the timeliness of the information in there, the relevance, especially in regard to the collection of medical evidence. I believe that came up frequently. But this was not a look at TPSS. It was sort of a broad- based review of all the elements and tools that are there for staff to use. Mr. Hall. Okay. Well, that is all the questions I have. Mr. Lamborn, if you are done, I would thank you, Mr. Bertoni, again for your testimony. And your panel is now excused. Mr. Bertoni. Thank you. Mr. Hall. And thank you so much. And joining us now on our second panel, we would like to welcome Mr. Michael Ratajczak--is that correct---- Mr. Ratajczak. Ratajczak. Thank you. Mr. HALL [continuing]. Decision Review Officer at the Cleveland Veterans Affairs Regional Office on behalf of the American Federation of Government Employees; Mr. Kerry Baker, Assistant National Legislative Director of the Disabled American Veterans (DAV); Ronald Abrams, Joint Executive Director of the National Veterans Legal Services Program (NVLSP); Dr. Patricia Keenan, Manager of Employee Performance Enhancement and Growth Program at the Human Resources Research Organization (HumRRO); and Mr. Nick Bartzis, a veteran from Cleveland, Ohio. Welcome, all, and your statements, written statements are in the record so that we will allow you to deviate and ask you to stay within the 5 minutes so we have time for questions for everybody. Tell me one more time. Ratajczak. Mr. Ratajczak. Ratajczak. Mr. Hall. Ratajczak. Mr. Ratajczak, you are now recognized for 5 minutes. STATEMENTS OF MICHAEL RATAJCZAK, DECISION REVIEW OFFICER, CLEVELAND VETERANS AFFAIRS REGIONAL OFFICE, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS, ON BEHALF OF AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES, AFL-CIO; KERRY BAKER, ASSISTANT NATIONAL LEGISLATIVE DIRECTOR, DISABLED AMERICAN VETERANS; RONALD B. ABRAMS, JOINT EXECUTIVE DIRECTOR, NATIONAL VETERANS LEGAL SERVICES PROGRAM; PATRICIA A. KEENAN, PH.D., PROGRAM MANAGER, HUMAN RESOURCES RESEARCH ORGANIZATION (HUMRRO); AND NICHOLAS T. BARTZIS, CLEVELAND, OH (VETERAN) STATEMENT OF MICHAEL RATAJCZAK Mr. Ratajczak. Thank you. I think I have as much practical experience with regard to training issues at the VBA as anyone in the room. I served as centralized challenge instructor. I worked on the certification design Committees. I have identified trends in errors by interacting with Board of Veterans' Appeals (BVA) personnel. I have taken some remedial action to fix errors resulting from what ultimately is improper and effective training. As a Decision Review Officer every day, I kind of am serving as a forensic pathologist where I look at a case where a mistake was made and trying to assign a reason for that mistake. When I have done that more often than not, I will effectively as possible take the case to the person who may have made the mistake and say this is what you did, let us try and avoid that in the future. More often than not as an explanation for their actions, the folks who made an error will attribute the error to the fact that, well, I just did not know that. I was not trained. I do not even know where to find that information. To the extent that errors are made, it is obviously a reflection of improper training on behalf of the VBA. And what I would like to submit is that the folks who are making these errors, the front-line claims adjudicators, veterans representatives, specialists, decision officers, they have an interest in effective training. They do not want to make mistakes because as we all do, they have service to veterans at heart ultimately. But beyond that, if they are making mistakes, adverse employment action can be taken against them. They can be put on a production improvement plan. So they have a very real interest in effective training and participating in effective training. To the extent that that opportunity for effective training is not presented to them, I think we can do better. I was struck by the testimony in previous hearings before this Subcommittee and others regarding complexity, increasing complexity of the process, judicial review, explanations for why there is a backlog in the VBA. And all that is true. VBA no longer operates in splendid isolation for judicial review, so the court comes back and looks at VBA's processes and says, well, this is what you need to do, this is what you are not doing correctly. Those kind of decisions should be viewed more as an opportunity than as an impediment because we can take those decisions, take the reasoning in them, and identify things they need to be trained on so we avoid the mistakes in the future. Recognizing that there is a component to the backlog which is attributable to complexity in legal review, VBA, I do not think, has made the same leap to the fact that those same elements have a detrimental impact on an individual employee's ability to do their job. Obviously if a given claim is more complex and there are additional legal requirements placed on the Veterans Service Representatives (VSRs) and Rating Specialist, it is going to take them longer to do their job and maybe what we could do is design some effective training to make them more efficient in doing their job in the face of increasing complexity and increasing legal requirements. I am going to suggest briefly a couple of methods of doing that based on my experiences. I just want to note that I did read the GAO report. I think overall it is accurate, but I want to caution anybody who will listen to the extent that it may imply that individual claims processors are responsible for improper training, that cannot be the case. They do not have the ability to go to their management and say, hey, I need more training. You need to do that so I can do my job better. Frankly, that is someone else's job to do that training and give them the opportunity for it. Several actions can be taken to include training. Number one, centralize curriculum from Central Office implemented by instructors that are responsible to Central Office. That would provide a uniform voice to all Regional Offices. This is CO's policy. To the extent that adjudicators' experience will be informed by that uniform training, you would expect to get some uniformity and some more consistency in the decisions. Also, I think Veterans Service Representatives, Rating Specialists in particular, should be given work credit for doing deferred actions where they have direct additional development in a claim to the person who is responsible for developing the evidence. That would provide feedback to people developing the evidence and also an incentive to the Rating Specialists. Thank you. [The prepared statement of Mr. Ratajczak appears on p. 39.] Mr. Hall. Thank you, Mr. Ratajczak. Mr. Baker, now you are recognized for your statement. STATEMENT OF KERRY BAKER Mr. Baker. Mr. Chairman and Members of the Subcommittee, on behalf of the DAV, I am pleased to offer my testimony to address VA's training, accountability, and performance measurement system. VA has a standard training curriculum for new claims processors and an 80-hour annual training requirement for all claims processors. DAV has long maintained that the VA should invest more in training and hold employees accountable for higher standards of accuracy. VA's problems caused by a lack of accountability do not begin in the development and the rating process and in the training program of which we can find little, if any, measurable accountability. For example, some employees inform DAV that many candidates begin centralized training before they complete or even start phase one training. Candidates are then not held responsible via formal testing on subjects before advancing to phase two training. As in phase one, VA refuses to test phase two trainees. Without such testing, VA cannot gauge the success of its learning objectives. During phase three training, employees work on real world cases. That notwithstanding, no accountability, no testing, and no oversight other than that provided locally. That oversight is not measured nationally. The result of such an unsupervised and unaccountable training system is that no distinction exists between unsatisfactory performance and outstanding performance. Lack of accountability during training eliminates employee motivation to excel. The need for improvements in VA's training program became evident when it began the skills certification test for VSRs. The first two attempts at the test produced a 25- and a 29- percent pass rate. The third produced a 42-percent pass rate but only after employees completed a 20-hour prep course to pass the test. Mandatory, comprehensive testing designed cumulatively one subject area to the next for which VA then holds trainees accountable should be the number one priority of any plan to improve VA's training program. Further, VA should not allow trainees to advance to subsequent stages of training until they have successfully completed such testing. DAV has also long stated that in addition to training, accountability is the key to quality. Therefore, timeliness. We believe VA's quality assurance or STAR Program is severely inadequate. In 2006, the STAR Program reviewed just over 11,000 compensation and pension (C&P) claims for improper payments. There was only .72 percent of the total number of cases available for review. However, a more reliable measure of VA's accuracy and its lack of accountability is shown through the Board of Veterans' Appeals' annual report. Fiscal year 2007, the Board decided approximately 40,000 appeals. The Board reversed 21 percent of those appeals and remanded over 35 percent. Of those remands, over 7,000 of those cases or nearly 20 percent were remanded because of errors in the most basic of procedures, procedures that should be elementary to every single decision-maker. The problem is compounded when one considers that those errors cleared the local Board, local Rating Board and the local Appeals Board which contain VA's most senior Rating Specialists. These facts clearly show little incentive for many employees to concern themselves with the quality of their work. Congress should require the Secretary to report on how the Department could establish a quality assurance and accountability program that will detect, track, and hold responsible those employees who commit errors. We believe that effective accountability can be engineered in a manner that would hold each VA employee responsible for his or her work as a claim moves through the system while simultaneously holding all employees responsible. If such errors are discovered, employees responsible for such errors should forfeit a portion of their work credit. This idea is discussed more in my written testimony, but I would be happy to explain further if the Committee has questions. I believe this type of accountability system will draw on the strengths of VA's performance measurement system, thereby allowing easier and less disruptive implementation of stronger and more effective accountability. Mr. Chairman, that concludes my statement, and I will be happy to try to answer any questions you might have. [The prepared statement of Mr. Baker appears on p. 42.] Mr. Hall. Thank you, Mr. Baker. Now, Mr. Abrams, you are recognized for 5 minutes. STATEMENT OF RONALD B. ABRAMS Mr. Abrams. Good morning, Mr. Chairman and Members of the Committee. I am pleased to have the opportunity to submit this testimony on behalf of NVLSP. As some of you know, we have trained thousands of service officers and lawyers in veterans' benefits law. We have written books, textbooks, and guides on veterans' benefits and we help people represent veterans seeking VA benefits. We also conduct quality reviews of the VA Regional Offices on behalf of the American Legion. We believe after reviewing all of this that the effectiveness of VBA training should be measured by the quality of the work product produced by VA adjudicators. Therefore, the quality and timeliness of VA adjudication should reflect the effectiveness or lack thereof of VA training. Because the quality of VA adjudications is in our opinion inadequate, NVLSP must conclude that VBA training is not effective or adequate. In the experience of NVLSP over 10 years of quality checks, over 40 VA Regional Offices combined with extensive representation before the Veterans Court, we find that most of the worst VA errors are the result of premature Regional Office decisions. We find that many VA managers emphasize quantity over quality, making many aspects of training not relevant. That is because VA workers want to please the people they work for. Training is de-emphasized because production is paramount. VA employees consistently let us know that they are told that time spent training reduces the time available to produce decisions. Also training VA workers regarding the procedures designed to protect the rights of claimants, developing the claim fully, reviewing the entire file instead of top sheeting, which is just looking at the first couple of pages--all takes time. The essence of fairness in this particular process is giving the VA worker the time to do the job. In fact, in September 2008, courageous VA Regional Office employees filed a grievance exposing this overemphasis on production. That grievance is attached to my testimony. The grievance asserts that the Regional Office created and encouraged a culture in which quantity is emphasized to the detriment of quality, that the RO failed to properly implement training to assure that those reviewing claims are sufficiently trained and that the RO failed to properly implement a fair and impartial performance appraisal system that assures that quantitative measures of performance are not emphasized to the detriment of measures of quality. These VA workers allege they were told to produce cases. It did do not care about the quality of their work or it did not care that much. And we think that the quality of the VA work is much worse than what is reported by the VA. As Kerry testified, the BVA statistics are scary. If over 50 percent of what goes up to the Board of Veterans' Appeals has to be returned because the VA in general failed to follow proper process, we have a problem. That is the best quality check possible. I want to thank you for permitting NVLSP to testify on this important issue. I will be happy to take any questions. [The prepared statement of Mr. Abrams appears on p. 47.] Mr. Hall. Thank you, Mr. Abrams. We will have some questions for you. Dr. Keenan, you are now recognized. STATEMENT OF PATRICIA A. KEENAN, PH.D. Dr. Keenan. Good morning, Mr. Chairman and Members of the Subcommittee. I am Patricia Keenan, a Program Manager at the Human Resources Research Organization or less formerly and more shortly is HumRRO. HumRRO is a nonprofit 501(c)(3) research and development organization established in 1951 that works with Government agencies and other organizations to improve their effectiveness through improved human capital development and management. Our comment today is about the compensation and pension services training program as well as on methods to reduce training variance. I am the project leader for HumRRO's work with VBA skills certification programs. HumRRO has worked closely with C&P service on skill cert programs for Veterans Service Representatives or VSRs since 2001. We have seen passing rates on the VSR test rise steadily since the beginning indicating to us that C&P's training initiatives are having a positive effect. As part of our work, we conducted numerous focus groups and interviews at Regional Offices. As a result, we identified several factors that decrease rating consistency. First, the RVSR job is very cognitively demanding. It requires knowledge of medical conditions, regulations, and the rating schedule, attention to detail, decisiveness, and strong analytical ability. These requirements exacerbate the other challenges RVSRs face. The second factor we identified is the problem of incomplete information in the rating schedule. While the schedule contains over 700 diagnostic codes, by comparison, the international classification of diseases by the medical profession contains thousands of codes. Many areas of the rating schedule leave room for interpretation and RVSRs develop individual rules for matching the medical evidence to the schedule. This allows them to reduce the cognitive load and work more quickly, but it also is a source of rating variance. The rating schedule does not have diagnostic codes for some conditions such as Parkinson's disease or carpal tunnel syndrome. When a claim includes an unlisted condition, the RVSR rates it by analogy to a closely related disease or injury. By their nature, these analogous codes lack criteria for rating, so raters have to research different conditions to make the evaluation that may not be very straightforward. The third challenge to the RVSR is related to workload. The sheer volume of cases awaiting adjudication sometimes results in a lack of attention to detail. One way this shows itself is in cases that are not ready to rate and require additional development. This means the RVSR has spent time reviewing a case, but the decision must wait for new information and the case must be reviewed a second time. The pressure also shows itself in rating decisions that rely too heavily on the use of templates rather than clearly establishing the connection between the medical evidence presented and the regulations in a way that is easily understandable. It is very important that veterans feel confident that their cases have been fairly evaluated. VBA's recent wave of hiring was the best long-term response to the influx of claims. However, one result has been that newly hired RVSRs do not understand the development process well. Although the workload of the RVSR is not going to lighten in the near future, there are some actions that could help improve rating consistency. First, to address one of the workload issues, newly hired RVSRs should work predetermination for several weeks to learn the system. The obvious drawback to this is that it would take longer for new RVSRs to begin rating cases. But we believe having this additional knowledge would pay off in the longer term. Second, job aids that include more specific medical information and rating schedule would ease the cognitive complexity of the job and reduce the individual interpretation of the ratings. However, job aids cannot help with the complex problem of writing good rating decisions. One long-term solution to this problem would be to develop a selection tool that assesses an applicant's ability to synthesize information and present it in a well-structured, easily comprehensible document. Finally, VBA should ensure that all RVSRs continue to receive standardized training. It has been HumRRO's pleasure to work with the C&P service for the past 7 years. We are honored to be even a small part of the valuable work the Veterans Benefits Administration does for America's veterans. We have watched both the skill certification program and C&P service over this time. The effort devoted to training has been steadily improving pass rates on the test. Beyond training, we have identified several areas that could further increase rating consistency. Addressing the problems of cognitive complexities, ambiguity in the rating schedule, and workload is something VBA should continue. Thank you for this opportunity. [The prepared statement of Ms. Keenan appears on p. 61.] Mr. Hall. Thank you, Dr. Keenan. Mr. Bartzis, you are now recognized for 5 minutes. STATEMENT OF NICHOLAS T. BARTZIS Mr. Bartzis. Thank you, sir. Thank you for the invitation to speak here today as a veteran and I am here today as a private citizen. However, I have a few statements beyond my written statement to go through. Most of them are in the form of---- I raise my right hand approximately 20 inches. I am going to show approximately a half an inch with my left hand. If we determine we have two cases, the cases make the mistake, make the same mistake on the same issue, how long is it going to take to go through all of the medical evidence on the case on the right as opposed to the case on the left? The answer to that question to me or the question itself is how much time is it going to take? But what I am really talking about is time. If I have to go through 20 inches of medical evidence or if I have to go through \1/4\ inch or \1/2\ inch of medical evidence, I am going to spend a significantly different amount of time to find the error. It has been stated earlier that part of the problem is the development issue, but the actual issue is how much time does it take and who is going to do it. That is a question I believe the Committee should ask. The second issue is accountability. Who is going to do it and when and where is a failure, if any? The rest of the statements are in my written statement, and I will defer to that. Thank you very much. [The prepared statement of Mr. Bartzis appears on p. 65.] Mr. Hall. Thank you, Mr. Bartzis. I will recognize myself now for a round of questions. Mr. Ratajczak, what happens when RO staff cannot complete the 80 hours of training? Mr. Ratajczak. To the best of my knowledge, nothing happens. I do not think there is any detrimental effect on the individual adjudicator if they have not concluded their training requirements. I suppose that there would be some method in place for them to do some remedial training. But to the extent that they are not held accountable for ensuring the training to begin with, nothing happens. As far as a practical matter, what happens is they do not get enough training and obviously we get poor decisions and increasing appeals. Mr. Hall. Dr. Keenan, you spoke of new hires having an initial period of, if I understood you correctly, of training before they start working on cases. What amount of time do you recommend and is that something that you have seen succeed in other similar situations in other agencies? Dr. Keenan. For them to work predetermination before they start rating cases? Mr. Hall. Right. To just have a concentrated training period before they start rating cases. Dr. Keenan. Well, right now they do have a 3-month training period before they start rating, but they do not have enough time to learn all the intricacies of predetermination which is all the evidence gathering that is required for the RVSR then to be able to make ratings, solid ratings. So if they could spend an extra 6 or 8 weeks doing pre-d, just learning the whole claims process. What we heard in our visits to the ROs was that the people who are promoted internally to RVSRs, it is a road running much more than people who are hired from the outside. Mr. Hall. Yes. Dr. Keenan. And that makes perfect sense. It is a very complicated process. There is a lot to learn in PR and when you are doing a rating decision, you have to understand, you have to be able to look at the evidence and know that you have enough there to make your decision. If the VSR has not developed all of that information and you do not know this and you spend time going through your case and then have to defer it, as Michael mentioned before, so if they could spend more time once they have gone through their initial training and then go through pre-d, spend some time, they will still be working cases in pre-d, so they are helping in that sense getting things ready to rate, they will have a much better sense of the whole process. Mr. Hall. So rather than 3 months, you suggest 4 or 5 months, in effect? An additional 6 to 8 weeks, you said? Dr. Keenan. Yes. Mr. Hall. Why do you think Dr. Keenan, so many VBA employees had a problem in passing the open book test that you developed? Dr. Keenan. When VBA first told us they wanted an open book test, we were totally dumfounded. We do a lot of certification work and it is always closed book. You come in on a Saturday. You take the test. But once we realized the complexity of the job and the fact that people really cannot do their job just with their memory, it is just too complicated, there are too many rules, too much money, too many regulations, they have to have training aids, job aids, you know, regulations available to them so that they can make these decisions. The tests are fairly applied, so it is not just a matter of recall which screen you use for that, although those are important to know because it makes you work more efficiently. The---- Mr. Hall. So you think they need more time to take the test? Dr. Keenan [continuing]. Well, we actually just had a meeting about that yesterday. We are going to add---- Mr. Hall. Because it is a learning experience? The test itself? Dr. Keenan [continuing]. We want them to learn while they are testing. Mr. Hall. Right. Dr. Keenan. We hope that they will come in knowing that. And we had originally set the time limit. It is a two part test, half in the morning and half in the afternoon, 2 hours and 45 minutes in the morning and 2 and 45 in the afternoon. We are going to expand that to a full 3 hours both times. Mr. Hall. Thank you. I am sorry to have to interrupt you, but I only have a little bit of time left in this round. Mr. Baker, in your testimony, you mentioned that the VBA does not test participants after each phase of training. Do you think it would be better to test at these intervals instead of the certification exam or in addition to the certification exam? Mr. Baker. Well, if I may clarify. It sounded like I was pretty rough on VA employees there and I want to make sure that you know I was not aiming at VA employees. It was the system that I would like to see improved. But to answer your question, I think it should be both, absolutely testing during the training phases, cumulative testing, so that you are responsible for what you learned in the previous phase and the current phase and so on and so on. I agree with the open book because my opinion this is more of a legal system than anything else and as complex it is, people need to know where to find it. And I would go as far as to have those resources cited in the answers. But if there was some incentive to do that, I think you would end up with a lot better training right up front. Mr. Hall. With Mr. Lamborn's patience and agreement, I am going to just go on and ask a couple more questions even though the red light is on. Mr. Abrams, what steps would you recommend that VA follow to improve their training program and, in particular, should VBA training be contracted out to lawyers and doctors who specialize in disability compensation cases? Mr. Abrams. The first thing that we would recommend is that we make upper management accountable by evaluating the BVA reversal and remand rate. We are not confident in the STAR Program. We do know many of the VA managers and they are good people. They are smart. And if their promotions and raises were contingent in part on the quality of the work performed at their Regional Office or for Central Office, the national figures, I have confidence that they would implement a real good training program because it would affect them. Right now it does not affect anybody when things are not good at the Board or even at the court where I know that we screen many, many cases. And the ones we take, we win over 90 percent of the time because there are many process errors. Can you ask your second question again? Mr. Hall. The second question, and this relates to various aspects of the VA's work, because of the backlog, the workload and the complexity and funding or staffing difficulties over the years, we keep running up against privatization proposals. But in particular, this is training I am talking about. Mr. Abrams. Right. Mr. Hall. Should VBA training be contracted out to lawyers and doctors who specialize in disability compensation cases? Mr. Abrams. Well, I will tell you, on one of our quality trips, I was asked by the Regional Office manager to train the Rating Boards in an area where we all agree they were making error after error. And after I was finished training, they came up to me and said we would like to do what you say, but we do not have time. When I got back to DC, I realized that we have an ethical issue here. If we are going to sue the VA in Federal Court as a matter of practice, if we are going to represent veterans, I am not certain that most knowledgeable lawyers who represent veterans before the VA, the Board, and the courts should be involved in training the VA too. It seems to put the advocate on both sides of the table. I think that if the VA wants to contract, I think that the people who do that may have to give up their other duties. There is a real tension there to me. Mr. Hall. Okay. Thank you so much, Mr. Abrams. Mr. Bartzis, you testified in your written statement that nationwide for the period of January 1, 2007, through September 5, 2008, the training and performance support system basic rating completion report lists 2,115 RVSR employees. Only 124 have completed the TPSS portion of the training. This is a completion rate of 17 percent. Could you elaborate on this point, please? Mr. Bartzis. First, I would like to state that there is a calculation error. The error is, it is not 17 percent completion rate. It is a 5.8 percent completion rate. Second, it is common knowledge of myself and other people, VA employees that I speak with that TPSS does not get completed. What we have is a complex system and we have multiple layers of people who get employed at different times. One layer of employee hired at a certain date may only complete portions of the basic completion report. Another section might complete half of it. Another section might complete a very small portion. In order to elaborate, what this shows me is a very low percentage of VA employees have completed the scheduled training. Whether it is 5.8 percent or 17 percent or 50 percent, what we have is a significantly low portion of a completion of training during a specified time period. Mr. Hall. Thank you, Mr. Bartzis. I will now recognize Mr. Lamborn. Mr. Lamborn. Thank you, Mr. Chairman. And I have a couple of questions for Mr. Ratajczak. There seemed to be three main bodies of knowledge that the VA employees who are doing this need to be aware of, the VA rating schedule, Title 38, and precedent court decisions. Are you finding in your review of erroneous decisions that one or more of those areas are where there needs to be extra training? Mr. Ratajczak. I would say it is the application of court decisions and continuing legal requirements to the actual development of cases insofar as that interacts with the veterans. It is not hard, well, I should not say it is not hard, but it is easier to rate or adjudicate a claim, make the decision with regard to the entitlement service connection, evaluation, and effective date if all the evidence is before you. When the evidence is not before you, when the case is not properly developed, it becomes difficult if not impossible to do that correctly. Given the time constrains that folks have to make these decisions, oftentimes they are not given any incentive to actually make the case right before they make the decision. So I think the real crux of the problem is that the folks who are developing the evidence need to know what they are developing and why they are developing it and how to do that. And that, I think, is informed by the case law and the regulations. Mr. Lamborn. Okay. Thank you. And, secondly, in the area of centralized training curriculum, which you were referring to from the Central Office, who do you think should develop this? Should it be experienced field employees or is there also a role for outside experts? Mr. Ratajczak. Well, the one thing that the GAO report points out is that Central Office staff does a very good job with regard to developing a curriculum for initial hires. And to that extent, I think that the Central Office staff as it exists now maybe with some additional personnel would be able to design a curriculum that is relevant to new issues as they develop and design a curriculum that sets forth the best practice with regard to how these issues should be dealt with by folks out in the field, some language that they can use in their rating decisions so that they are legally sufficient. I think that might be the better way to go simply because those folks that are already in VBA are already familiar with the process. If we were to go out and try to contract out, get someone else to do it, I think that there would be a very large lag time, number one. And, number two, ultimately the training has to be responsive and coordinated with what the policy of the VBA is. Contracted lawyers, et cetera, who are not interpreting that policy and those practices, we may get a good disconnect. Mr. Lamborn. Okay. Thank you for your answers. And thank you all for your testimony. Mr. Hall. Thank you, Mr. Lamborn. We have more questions, but what we are going to do is submit them to you in writing. And you have provided lots of answers in your testimony to begin with. So, Mr. Ratajczak, Mr. Baker, Mr. Abrams, Dr. Keenan, Mr. Bartzis, thank you for your work on behalf of our Nation's servicemembers and veterans. And you are now excused. Thank you very much. And joining us in our third panel is Michael Walcoff, Deputy Under Secretary for Benefits of the Veterans Benefits Administration at the Department of Veterans Affairs. Mr. Walcoff is accompanied by Dorothy Mackay, Director of Employee Development and Training of the Veterans Benefits Administration, and Bradley Mayes, Director of Compensation and Pension Service of the VBA at the Department of Veterans Affairs. Of course, you know the routine. Your written statement is made a part of the record, so feel free to deviate from it. Mr. Walcoff, you are now recognized for 5 minutes. STATEMENT OF MICHAEL WALCOFF, DEPUTY UNDER SECRETARY FOR BENEFITS, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY DOROTHY MACKAY, DIRECTOR, EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; AND BRADLEY MAYES, DIRECTOR, COMPENSATION AND PENSION SERVICE, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS Mr. Walcoff. Thank you, Mr. Chairman. Actually, I did have an oral statement that was derived from my written statement, but instead I think because the written statement is part of the record, I would like to use this time just to address some of the issues that were raised by the previous panels. Hopefully that will be okay. I want to start with some of the testimony presented by my old friend, Mr. Abrams. And in his testimony, he talked about what he saw as an emphasis within VBA on productivity at the expense of quality. And he said that he has seen that and heard that from employees who work within VBA. What I would like to offer concerning that is the objective criteria that we actually use to evaluate our employees and our managers, and I will give you three instances to refute what Mr. Abrams said. First of all, in our Directors' performance standards, we have in their standards equal elements for both productivity and quality. They are weighted exactly the same. In our employees' performance standards, again we have two elements, two of the elements are productivity and quality, again weighted exactly the same. And in our incentive award criteria, we go a step further. In that criteria, in order for a station to qualify for what we call level two money, they have to meet a variety of factors and normally we will list, say, five or six factors and say you have to meet four of them. But in addition to the four of the six factors, they have to meet all of their quality factors. That is one that never can be waived. Always have to meet quality. And I think that gives an indication of the fact that we do put a lot of value on quality, certainly as compared to productivity. Another statement Mr. Abrams made had to do with work sent back from the Board. And he said that 50 percent, over 50 percent of work that is sent to the Board comes back to the VA as incorrect. I will tell you that is basically a totally false statement. The remands, as he was referring to, the actual avoidable remand rate, meaning cases that go back to the ROs because of mistakes made by those Regional Offices is 18 percent. Now, I am not saying 18 percent is something that is, you know, as good as we would like it to be, but it is a far cry from over 50 percent. And I wanted to clarify that on the record. Mr. Chairman, in your opening statement and then in questions with Dr. Keenan, you talked about certification and said that you had been told that over 50 percent of the Rating Specialists, RVSRs, who took the certification test failed it. And I wanted to clarify that. The actual pass rate on the one RVSR certification test that we have given is 85 percent. What you were referring to is the pass rate on the VSR test which was at 49 percent. And what I would say to you is that you have to understand that the VSR certification test measures the employee's ability to move into the senior technical position in authorization, which is what a VSR does. The person who gets that position has to be at a point where they can authorize the work of other employees. We do not believe that every employee necessarily will get to the point where they can authorize the work of other employees. So, frankly, 50 percent as a pass rate did not surprise us and I do not believe there is anything wrong with that. The 85 percent on Rating Specialists is different because that is a test that measures basic knowledge of rating skills. So I would expect the pass rate to be higher in that and, of course, it was. I want to address something that Dr. Keenan said where she talked about the desire that Rating Specialists should have an extra 3 weeks of training in predetermination. Normally in ideal circumstances, we promote people to the Rating Specialist from the VSR position where they have been doing that development work throughout their career. So it is really not necessary to do additional training. Because of the unusual circumstance that we have been going through over the last year and a half where we were hiring so many people, bringing most of them in as VSRs, we were put in a position where we really could not promote the number of Rating Specialists that we would need to do the work that is being developed by these new VSRs all from the VSR position because then we would be left with no experienced VSRs and we wound up having to hire some Rating Specialists off the street. And they are the people that she is talking about that should spend some time in predetermination. That is a very small percentage of the total Rating Specialists that we hire, but I think her point is well taken and it is something that we will definitely look at. There are three points that I wanted to make just addressing some of the previous remarks. And at this point, I will turn it over and say I will be very willing to---- Mr. Hall. Even though the light is red, you are the guy that we are here to hear from, so feel free to continue. Mr. Walcoff [continuing]. Okay. Then let me address one other thing that you had mentioned and that was that employees should get credit for training. And, in effect, they do get credit for training in that when we measure their productivity during the day, they get excluded time for any time that they are in training. So that does not count against them. If anything, it makes it so that the ratio of hours available for work and the work they do is equal so that they do not get penalized for being in training. And I think that is an important point. But at this point, I think I would like to end my statement and be available to answer any questions you have. [The prepared statement of Mr. Walcoff appears on p. 68.] Mr. Hall. Thank you, Mr. Walcoff. Mr. Lamborn is double booked for most of this day, so we are going to let him go first, the Ranking Member, Mr. Lamborn. Mr. Lamborn. Thank you, Mr. Chairman. At this point, I will just defer to you because I do not have any questions. You have done a good job in your written testimony and I will be looking that over further. And I will submit further questions to you if any arise. Thank you. Mr. Walcoff. Thank you, sir. Mr. Hall. Thank you, Mr. Lamborn. Mr. Walcoff, we talked earlier with one of the panelists about getting 80 hours of training and what happens to an employee who does not complete it. Could you answer that same question? What does VA do about employees who do not complete their training? Are there incentives to complete it and are there any penalties to not completing it? Mr. Walcoff. Let me answer that by saying that I agree with the statement that was made by Mr. Ratajczak on the second panel that I do not believe that it should be the responsibility of the employee to complete the training. I think it is management's responsibility to make sure that all their employees complete the training because, frankly, an employee is not going to go up to the manager and say, oh, I am leaving my desk now to go sit in on training and I will be back in an hour. The manager has to be the one to orchestrate that. So the person that should be held responsible for employees meeting their training are the managers. And I will tell you that we have an element in the Directors' performance standards and it is in most of the coaches' standards that talks about, and it is in the Human Resource responsibilities, I believe, it is named a critical element, and it talks about the fact that the manager is responsible for doing a bunch of different things involving Human Resources (HR), one of which is making sure that training is done. But I will tell you that in reviewing that and, frankly, in reviewing it in preparation for this hearing, I spoke with the Associate Deputy Under Secretary for Field Operations and discussed with her the need to probably make this a little more specific and make it so that we have in there very specifically that our managers will be responsible for making sure that all their employees complete the number of hours for that position that is required because the requirement we currently have, while I think it is specific enough, I think that a third party might say, well, it is too generous. So we are going to make it even more specific because we do believe that somebody should be responsible for that and it should be the managers. Mr. Hall. Well, that makes sense and I am happy to hear it. In your testimony, you mentioned that last year, you trained 2,150 new VSRs and RVSRs. What percentage of your rating workforce does that represent approximately? Mr. Walcoff. Well, the 2,100 is Rating Specialists and VSRs. Mr. Hall. Right. Mr. Walcoff. I mean, I can tell you that we have about 1,800 Rating Specialists and we have somewhere around 4,500 VSRs. But, of course, some of those are people that have been there for many, many years. And the numbers you are referring to are just the people that went through the challenge training during this year. So it is kind of tough. It is a hundred percent of the new employees that we hired is what I would say. You know, I know I am not answering your specific question, but challenge is really geared to new employees and every employee must go to challenge training. Ms. Mackay. We can indicate that fiscal year 2008, 727 of the 2,000 or 2,200 people that we have trained in centralized training were RVSRs. So that gives you an indication of what percent they represent of the workforce. Mr. Hall. Mr. Walcoff, you noted that VBA did training with an independent consultant to evaluate the challenge training. What were the results of that? You contracted with an independent consultant. What were the results of that study? Mr. Walcoff. I am going to ask Ms. Mackay to answer that. Ms. Mackay. Yes. I do want to make the point that we have actually done evaluation on a number of different programs. In fact, we have done two different evaluations of the TPSS program. As GAO noted, one of those was in 2007 and one was in 2008. We also reviewed the Challenge program. Some of the recommendations that have come out of that and the point that was made was that it does provide value to the organization, that the program itself is providing value to the organization. There were some recommendations about resequencing some of the training modules. Right now there are many that are done on the post end of Challenge training, if you will, and we want to move some of them to the beginning so that there is less duplication. The TPSS modules are done on the tail end of Challenge training at home station training. Those are very explicit with tests at the end of them. In the Challenge centralized training, there is some repetition of some of the learning that is taking place, so we want to move some of those modules forward. Those were some of the results that came out of the Challenge evaluation. It is a good program. The employees like it. We got excellent results. Of the three studies that we did, two on TPSS and one on Challenge. We have data from 37 ROs. We did 470 interviews and we interviewed through survey, survey response 2,700 employees. So we really got a very representative sample of the population out there including, you know, subject matter experts and bargaining unit employees in the field. Mr. Hall. Has the VA also conducted a time and motion study? Mr. Walcoff. I am going to ask Brad to answer that. Mr. Mayes. Yes, we have. We conducted one in fiscal year 2007, I think, was the last time a motion study was conducted. Mr. Hall. And any results you would care to share with us? Mr. Mayes. Well, what we do with information that we get from the time and motion study is we determine if the weights are appropriate for the end products that we use for performance management. In other words, how many hours does it take to do an original compensation claim, how many hours does it take to do a reopened compensation claim or a Dependency and Indemnity Compensation (DIC) claim. And then those end products as employees work those types of cases, they get work credit, the station gets credit. Ultimately that is used for resource allocation as well. So there were some adjustments made to the work credits based on that study. But off the top of my head, I do not remember exactly what the---- Mr. Hall. Would you please send the Subcommittee the study? It is something we would like to have a look at and save you from having to tell us or remember all the details of it. Mr. Mayes, how long can an employee be on a performance improvement plan? Mr. Mayes [continuing]. Well, the---- Mr. Walcoff. Yeah. If you do not mind, I will answer that. I am a former HR guy, so I have knowledge of that. The performance improvement plan (PIP) as laid out in our negotiated agreement is 90 days. And they have 90 days during which they meet regularly with their supervisor. They get counseling sessions in terms of how they are doing. We ask them what help do you need in order to meet your performance standards. Is there training you need. Is there any particular information that you need. And then they are evaluated at the end of the 90 days and then a decision is made on whether they should be kept in their position or removed from their position. Mr. Hall. So if the employee improves, but then does not achieve the fully successful rating on their evaluation, do you do another improvement plan or just move them to a different position or move them? Mr. Walcoff. There are situations where an employee will be not meeting their standards. They go on a PIP. They meet their standards. They go off the PIP. They do not meet their standards and then you put them on another PIP. The case law actually in that situation says that if that happens a number of times, and it is undefined what the number is, that you may actually be looking at a conduct problem rather than a performance problem. It is something that occasionally we run into. Most of the time when employees are on a PIP, hopefully we are able to give them the assistance and the tutoring that they need so that they can become satisfactory in their jobs and never be on a PIP again. Mr. Hall. Is the ASPEN system, which tracks employee performance, integrated with the STAR Quality Review Program and how is that information cross-referenced? Mr. Walcoff. The ASPEN system is merely just a shell that we use for tracking. You know, we have to do quality control, performance evaluations of individuals because that is part of their performance standards. We needed some vehicle to track that. In other words, if we are looking at Dorothy Mackay as an employee, how many cases did she do this month, how many were correct, and be able to track that over a year. That is all that Aspen is. The quality assurance program that is STAR is a totally separate program. They are not connected in any way other than one measures quality of an individual whereas the quality assurance program, the STAR Program measures quality for the organization. Mr. Hall. The primary goal of training obviously is to improve the accuracy of the rating decisions. However, there are still accuracy issues at many ROs. Should there be additional training in those offices and are those employees still eligible for bonuses and promotions even if the office's accuracy rate is below the national standard? Mr. Walcoff. Let me answer the second part of the question in terms of the bonuses. As I had mentioned earlier, the level two awards, which are the primary source of bonuses for employees, is dependent on the station performance and quality. So if a station is not making quality, that station will not get level two money that they could give to those employees. There is something called level one which is a smaller amount of money that every station gets for their own use. And if you happen to have an individual in a C&P, in a service center who is an outstanding employee, whose individual work is outstanding, even though the work of that division is not outstanding, you have at least that level one money where you can reward that individual. So---- Mr. Hall. Management discretion? Mr. Walcoff [continuing]. Right. Exactly. Mr. Hall. I want to ask Ms. Mackay how many actual days of training do you provide at the Academy? Do trainees have to sign in and out every day and how do you track their participation? Ms. Mackay. I assume we are talking about the Challenge students related to C&P training because we do do training at the Academy as well. It is 3 weeks of training. And I noted in DAV's testimony that they thought that the travel time really cut into the 3 weeks of training. These are bargaining unit employees, so we do have to pay attention to the fact that, you know, we do not want them traveling on the weekend. But we only cut one and a half days off of that 3 weeks. So it is 1 day on the front end and then a half day on the back end. So it is, you know, virtually 3 weeks long of training. And they are there. There is an accountability. There are instructors there along with a course manager. So any coming and going of students during that 3 weeks of centralized training is duly noted. It is only for emergency purposes. We have had some folks who might have gotten sick or whatever. They are monitored. And not only are they monitored in terms of their health and getting them back in the classroom, but there is a communication with the Regional Office that they have been removed from the classroom for whatever reason. So that notification is made to the RO as well. So we keep a good track of the students when they are at the Academy for that 3 weeks of training. Mr. Hall. What is the pass/fail rate at the Baltimore Academy? Ms. Mackay. In terms of? Mr. Hall. The employees you train. Ms. Mackay. Well, if they go through the modules and the training, the 3 weeks of training, there is not any graduation per se of them. There is enforcement every single day. Again, whenever you have a classroom full of students with varying levels of ability, it is the job of the instructors to really assess that classroom and see whether they need to pay particular attention and give special effort and interest to particular students, that they get it before they leave the classroom, that particular lesson. So in that sense, they do or should when they leave the Academy have learned the modules and the lessons that they need in order to go back and do the post training at their station. Mr. Hall. If we could just talk about the training manual, the five volume training manual for a second. It seems like it may not be the most effective tool. Have you consulted with the private-sector organizations that specialize in developing training protocols or Federal and private-sector agencies? Ms. Mackay. I think I am going to pass that to Mr. Mayes. Mr. Mayes. I am not familiar with the five volume training manual. I think you might be talking about what we call our Manual of Adjudication Procedures. We call it 11MR right now. It should just be M21-1. We had rewritten the manual a couple of years ago and completed it. It is a number of volumes. The way I look at the manual is that we start with statute and regulations and sometimes, as you know, some of these statutes and regulations are rather difficult to apply or understand or interpret at the Regional Office level. So the manual then takes those procedures or those regulations and interprets those in a more, I would call it, a more straightforward fashion so that our VSRs and our RVSRs in the field have a ready reference to apply the laws and regulations properly. And it is complex. When we did the task analysis for our VSRs a number of years ago, actually, Admiral Cooper noted this, and they were working with the Claims Commission, there is something on the order of 11,000 tasks. It is complex work and it does take a lot of rules to make sure that we comply with the law. Mr. Hall. We have been known to write some complicated and challenging laws. Mr. Mayes. I was not referring to any of your legislation. Mr. Hall. We held a roundtable with survivors back in the spring. At that time, we heard tales of folks who have encountered VBA employees who did not seem to know much about survivors' benefits such as DIC. Do you do specialized training around those issues? Mr. Mayes. Well, training on the Dependency and Indemnity Compensation benefit is part of the training curriculum for our new VSRs. So, yes, that is included in that curriculum. And that curriculum, by the way, requires 26 weeks. So the formal curriculum, 26 weeks. It is two to 3 weeks of prerequisite training. Then we send our VSRs off to Challenge where they have another 3 weeks of formal training. We utilize the TPSS modules, the training, performance, support system modules in this training and then they go back to the Regional Office. And it is a combination, as the gentleman from GAO said, it is a combination of computer-based training and then moving away from that and practically applying it. And DIC is part of that process, that curriculum. Mr. Hall. Thank you. Now, Mr. Walcoff, I want to ask you a couple more questions here. VHA has had success with teleconferencing capabilities. I am curious if VBA has sought out the same opportunities to train with teleconferencing. Mr. Walcoff. I am looking at Dorothy to see whether we actually use teleconferencing for training. Ms. Mackay. Yes. Mr. Mayes. I can tell you most recently, we identified in the field some challenges applying the DeLuca rules. In other words, a court case a number of years ago that said that when there is limitation resulting--when pain results in limitation of function or motion, that we can adjust the evaluation commensurately. And so what we did recently was we had a national conference call that was targeted at our Rating Veteran Service Representatives. We identified this through our quality assurance program. So when we find through our quality assurance program that we have a targeted area that needs focus, then frequently we will have a teleconference with RVSRs or managers out in the field. And we are doing that right now as part of our followup from the findings on STAR. We are doing that monthly with the Regional Office. So there is an example where we would use conference calls to initiate discussion on topics that are causing some problems in the field. Mr. Hall. You know, that is obviously a developing and more frequently used tool that you may find yourselves wanting to use more. I want to also ask what training is required of Board of Veterans' Appeals employees. Mr. Walcoff. Well, you know, that is something that we are not really involved in. I mean, BVA has their own training programs for their employees and I do not think any of us are in a position to be able to answer that. Mr. Hall. Then we will not ask you that question then. So the Subcommittee staff recently conducted a site visit to Okinawa, Guam, and Hawaii to review the claims processing issues there. And a point that was brought up in Japan was the role of the U.S. State Department in compiling claims information for the Islands of Micronesia, Palau, and the Marshall Islands. The Department of the Interior handles veterans' claims. How does VA inform other agencies and train them in the claims application process and educate them on other benefits information? Mr. Walcoff. That is going to be yours again. Mr. Mayes. Well, we do work with representatives from the State Department who are involved in communicating information about veterans' benefits. We recently had a training seminar. Members of the Compensation and Pension Service staff conducted that training. So we are involved in that. Primarily what we have State Department personnel doing is again giving out information and also they are involved in the coordination of compensation and pension exams for veterans who reside overseas. So we give them information and provide training for that as well. Mr. Hall. Okay. Lastly, I wanted to ask a question about dose reconstruction. The Veterans' Advisory Board on Dose Reconstruction met this month and discussed the advantages of having the specialization and ionizing radiation claims in one RO so the training could be targeted. If this is true for atomic veterans' claims, would specialization and targeted training benefit veterans with other conditions such as Post Traumatic Stress Disorder (PTSD) or Traumatic Brain Injury (TBI)? Mr. Walcoff. I think that the idea of specializing in that sense is something that we have had a lot of discussion about. And at this point, the problem is that when somebody, for instance, files a claim for PTSD, invariably there are going to be other issues that they are also claiming. And it is very difficult to send just the PTSD part of a file, you know, to another office and say you work on that and we will leave everything else at the home office because everybody needs access to the entire file. What we do believe is as we move, I hope steadily, toward getting to the point where we go paperless and we have electronic files, one of the real advantages of that will be that if we want a particular office to, say, do the PTSD part of the claim, they will be able to access the file the same time as the home office is accessing the file because it is electronic. And that really would make it so that we would be able to do that type of specialization on a more frequent basis. The radiation claims are so complex and there are so few of them that we felt that it made sense to do it even with the fact that we are in paper. But ultimately I believe that when we go paperless, it will open all kinds of doors for us exactly as you are talking. Mr. Hall. You are a moving target, as you know, and in our oversight capability or legislative capability, we are trying to encourage or fund or to help you move in directions that you are already moving in many circumstances. So you just brought up the paperless thing. I want to ask you how that is going. I know even though our bill has passed the House, we are waiting for Senate action, which hopefully we will see. But at the same time, I am sure you are somewhere down the road. Any reports on that? Mr. Walcoff. There are a lot of things that are happening. The biggest thing right now is we are awaiting for the awarding of a contract for our lead systems integrator, which is going to be a very key piece as we move forward because the integrator is going to be the person that or the organization that works with us in coordinating all the different parts of this, you know, project that we call paperless, you know, including the part that deals with a veteran's ability to apply online for benefits, including the ability for a veteran to go into our system and be able to determine the status of his claim without having to pick up the phone and call us, including the part about making all the actual paper electronic. There are so many pieces to this and the integrator is going to be a key player. That contract is supposed to be awarded before the end of the fiscal year. So that is a major, major step forward with us. So we are very, very optimistic in terms of the future. The Secretary, I will tell you, Mr. Chairman, the Secretary has been relentless in terms of telling us that we will be paperless. So we are absolutely getting a lot of encouragement. Mr. Hall. In terms of your continuing efforts to do the things that we are trying to tell you to do or ask you to do, is there any report or progress? Have you been working on updating the rating schedule at the same time? Mr. Walcoff. Brad, you want---- Mr. Mayes. Yes. I do not know if you saw the press release. I think it came out yesterday or the day before. But we are creating a Federal Advisory Committee that will be tasked with taking a look at the schedule, looking at the studies that have been done recently. And we have the Marsh-West study. We have the Disability Benefits or Veterans Disability Benefits Commission, the Dole- Shalala Commission. There are a lot of folks out here that have done some work and looked at it. And, of course, I think you know that we have contracted with a firm to review the aspects of the Dole-Shalala proposed legislation that the President sent over to Congress. So this Advisory Committee will be looking at all of that and then making recommendations to the Secretary on, I would say, how do we get moving and prioritize and what is next. And, finally, I want to mention to you that we had proposed a new rule for traumatic brain injury. We think that is going to go final very quickly. It is going to change the rating criteria for TBI. We just updated the skin portion that was proposed. We think we are close to going final on that. And we are working on adding presumptive for Amyotrophic Lateral Sclerosis (ALS). Mr. Hall. And this is really the last question now. Where do you stand with the quality of life study? Mr. Mayes. Well, that is the study that I just mentioned. We are in the process of going through it. Our technical comments have been requested. So as we move through that and provide that feedback to the contractor. Once that is complete, then it will be published and available to the Committee. Mr. Hall. Great. We look forward to receiving that. Thank you all for your statements, for your answers, for the work you do for our veterans. And I would like to thank everybody from all the previous panels for your insight and opinions. And this hearing now stands adjourned. [Whereupon, 11:45 a.m., the Subcommittee was adjourned.] A P P E N D I X ---------- Prepared Statement of Hon. John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs Our Nation's veterans understand the necessity of proper and adequate training--their lives have depended on it. The military trains for its operations and everyone knows every detail of their job prior to the mission. These same veterans should be able to expect the same level of competence when they seek assistance from the Veterans Benefits Administration. Unfortunately, that is not the case, as we have heard at other meetings and hearings throughout the year that this Subcommittee has held regarding the VA disability claims processing system. VA has standardized its training curriculum and requires that all claims processors must complete 80 hours of annual training. This is a lot of hours because in fact some healthcare providers don't need to meet that level of continuing education to maintain their clinical license or credentialing. The VBA training topics are identified by Central Office or by the individual's Regional Office (RO). New employees go through an orientation process and there are additional online learning tools available through the VBA's Training and Performance Support System. Yet, with all of this effort, VA training seems to be falling short of its intended goals. Less than 50 percent of the Ratings Veterans Service Representatives (RVSRs) passed the certification exam, even though it was an open-book test. But, frankly, I've seen the training manual and it can be measured in pounds not pages, so I don't know how useful their book is and that can be the crux of the matter right there. More importantly, as outlined in previous hearings, there are significant inconsistencies in ratings between VA's 57 ROs and a high rate of remanded cases. I am pleased that the AFGE is here to shed light on this issue. You are a critical link to those on the frontlines working to improve outcomes for our disabled veterans. The Government Accountability Office (GAO) released a report in May 2008 entitled, ``Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors'' that documented the areas in which VA needs to improve its training and hold accountable those it does train. According to the GAO, staff is not held accountable for completing the required training, since VBA does not track completion, so there are no consequences for not taking the training. Additionally, VBA does not evaluate its training, so it does not know if it has successfully designed and implemented an educational program. Feedback is not consistently collected from RO employees on the training that they do receive and many have reported difficulty in accessing training because of their stringent productivity demands. I look forward to hearing more from the GAO about this report. But, these are not surprising conclusions to the Veterans Service Organizations, which have complained for years about the inadequacies of the VBA training program. So, I am grateful that they have joined us here today as well. Training is not an issue that should be taken lightly. We all know the importance of good training, but effective implementation that ensures consistency and accountability can be elusive and that is what I hope we can address today. I have taken steps to ensure improved training outcomes when I introduced the ``Veterans Disability Benefits Claims Modernization Act'', H.R. 5892. These policy enhancements will lead to compensation claims processing improvements and more accurate claims adjudication results for our veterans and their families. Moreover, I am not sure that VBA's current training regimen complements its current Claims Processing Improvement model, or CPI. In fact, I am positive that the current coupling detracts from increased accountability efforts. I am pleased to report, with the help of many in this room, H.R. 5892 passed the House on July 31, 2008, by a 429 to 0 vote. On August 1, 2008, Senator Clinton introduced companion legislation to my bill, S. 3419. So, Congress is well on its way to rectifying the inadequacies in the VBA that have already been identified to us. Today's oversight hearing will allow us to drill deeper into this issue and gauge where VA is at in its training protocol and see where other improvements can still be made. I look forward to hearing from our witnesses today and hope to learn more about best practices and strategies for measuring performance and building better training protocols and accountability standards.Prepared Statement of Hon. Doug Lamborn, Ranking Republican Member, Subcommittee on Disability Assistance and Memorial Affairs Thank you Chairman Hall for yielding. I am pleased to have this opportunity for a collective discussion on the effectiveness of the Veterans Benefits Administration's (VBA's) training, performance management, and accountability requirements. Over the course of the past several months, this Subcommittee has examined nearly every facet of the VA benefits claims processing system in an effort to identify how we might help the Department overcome the claims backlog crisis. While the recent expansion of its workforce will certainly have a positive impact, VA must ensure that newly hired claims workers receive training that is commensurate with their responsibilities. It is critical that the training it provides meets the needs of the Department and its employees. It is equally important that the results of the training are evaluated. Without feedback, VA may never know whether or not the training is accomplishing its goal. Any viable training program should be able to identify deficiencies and demonstrate the intended and actual outcome of its curriculum. VA training must be connected to its vision and mission, and VA managers need to be assured that if employees are pulled off the floor for training that it will result in long-term benefits. I'm sure that with a growing number of pending claims, there is a certain level of trepidation that, ``There's too much work to do already and we'll just get further behind if we have to conduct training.'' There must be clear support, from the top down, in order to conduct adequate training and acquire the expected outcomes. Certainly, training new employees on everything they need to know in order to make sound rating decisions is a daunting task. The VA rating schedule itself is complex and it's merely a portion of the array of knowledge a competent adjudicator must possess to perform his or her job. I view today's hearing as an opportunity to not only learn more about the training and assessment program VA provides its employees, but also to reiterate to the Department that it should be forthright about any additional resources deemed necessary to fulfill this critical requirement. I look forward to our witnesses' testimony and a productive discussion on VA's training program. I yield back. Prepared Statement of Daniel Bertoni, Director, Education, Workforce, and Income Security Issues, U.S. Government Accountability Office Improvements Needed in VA's Training and Performance Management Systems GAO Highlights Why GAO Did This Study The Department of Veterans Affairs' (VA) disability claims process has long been a subject of concern because of long waits for decisions and large backlogs of claims pending decisions. To address these issues, VA has hired almost 3,000 new claims processors since January 2007. However, adequate training and performance management are essential to developing highly competent disability claims processors and ensuring that experienced staff maintain the skills needed to issue timely, accurate, and consistent decisions. The Subcommittee on Disability Assistance and Memorial Affairs, House Veterans' Affairs Committee asked GAO to present its views on (1) VA's training for its claims processors, and (2) VA's performance management of this staff. This statement is based on a May 2008 report on VA's training and performance management (GAO-08-561) and has been updated as appropriate. What GAO Recommends In its May report, GAO recommended that VA collect feedback from staff on the training provided in the regional offices and use this feedback to improve training; hold staff accountable for meeting their training requirement; and assess, and if necessary adjust its process for placing staff in overall performance categories. VA concurred with these recommendations, but has not yet reported any significant progress in implementing them. What GAO Found Training for VA disability claims processors complies with some accepted training practices, but VA does not adequately evaluate its training and may have opportunities to improve training design and implementation. VA has a highly structured, three-phase training program for new staff and an SO-hour annual training requirement for all staff. GAO found that VA has taken steps to plan this training strategically and that its training program for new staff appears well- designed and conforms to adult learning principles. However, while VA collects some feedback on training for new staff, it does not collect feedback on all the training conducted at its regional offices. Moreover, both new and experienced staff reported problems with their training. Some new staff told us a computer-based learning tool is too theoretical and often out of date. More experienced staff said they struggled to meet the annual SO- hour training requirement because of workload pressures or could not always find courses relevant given their experience level. Finally, the agency does not hold claims processors accountable for meeting the annual training requirement. VA's performance management system for claims processing staff generally conforms to accepted practices. For example, individual performance measures, such as quality and productivity, are aligned with the agency's organizational performance measures, and VA provides staff with regular performance feedback. However, the system may not clearly differentiate among staff performance levels. In each of the regional offices we visited, at least 90 percent of claims processors were placed in just two of five overall performance categories. Broad, overlapping performance categories may deprive managers of the information they need to reward top performers and address performance issues, as well as deprive staff of the feedback they need to improve. Fiscal Year 2007 Appraisals for Four Offices Were Concentrated in Two Categories [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: VBA regional office. Mr. Chairman and Members of the Subcommittee: I am pleased to have the opportunity to comment on training and performance management for Department of Veterans Affairs' (VA) disability claims processors. In fiscal year 2007, VA paid about $37\1/ 2\ billion in benefits to more than 3.6 million veterans and their families. The disability claims process has long been a subject of concern for VA, the Congress, and veterans' service organizations due to long waits for decisions, large backlogs of pending claims, and problems with the accuracy and consistency of decisions. Moreover, we have noted that VA's current disability process is in urgent need of re-examination and transformation, especially in regard to how it assesses the work capacity of individuals with disabilities in today's world and in its ability to provide timely and appropriate benefits. With an increase in claims resulting from injuries sustained in Iraq and Afghanistan and from an aging veteran population, these issues will likely persist. To address them, VA added almost 3,000 new claims processors from January 2007 to July 2008 and has plans to add even more staff by the end of September 2008. Earlier this year, I testified before this Subcommittee that enlarging VA's disability workforce is likely to produce certain human capital challenges for the agency.\1\ More staff alone will not guarantee effective disability claims processing. Among other things, adequate training and performance management are essential to developing highly competent new disability claims processors and ensuring that experienced staff maintain the skills needed to issue timely, accurate, and consistent disability decisions. --------------------------------------------------------------------------- \1\ GAO, Veterans' Disability Benefits: Claims Processing Challenges Persist, While VA Continues to Take Steps to Address Them, GAO-08-473T (Washington, DC: Feb. 14, 2008). --------------------------------------------------------------------------- My remarks today primarily draw from our May 2008 report for the Committee on Veterans' Affairs and focus on (1) VA's training for its disability claims processing staff and (2) its performance management system for claims processors. We conducted our work in accordance with generally accepted Government auditing standards.\2\ For this testimony, we updated information from our report, as appropriate, to reflect the current status of VA training and performance management systems. --------------------------------------------------------------------------- \2\ GAO, Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors, GAO-08-561 (Washington, DC: May 27, 2008). --------------------------------------------------------------------------- In summary, although we found that training for VA disability claims processors complies with some accepted training practices, it is not adequately evaluated, and some aspects of training design and implementation could be improved. We found that VA has taken steps to strategically plan its training, including the establishment of a training board to evaluate the agency's training needs. Also, VA's training program for new staff appears well-designed and conforms with adult learning principles. However, while VA collects feedback on many of the training methods and tools for new staff, not all the training VA conducts is evaluated to determine how relevant or effective it is. Moreover, both new and experienced staff reported problems with their training. Some new staff members reported that a computer-based learning tool was not useful. Also, VA requires 80 hours of training annually for all claims processors, but some experienced claims processors struggled to meet this requirement because of workload pressures, and some could not always find relevant courses. It is not clear what criteria VA uses to justify the number of required training hours. Furthermore, individual claims processors are not held accountable for meeting the annual training requirement, although according to VA, the agency has implemented a new learning management system allowing it to monitor staffs completion of the training requirement. VA's performance management system for claims processing staff generally conforms to accepted performance management practices. For example, individual performance measures, such as quality and productivity, are aligned with the agency's organizational performance measures, and VA provides claims processing staff with regular feedback on their performance. However, the system may not clearly differentiate among performance levels. Broad, overlapping performance categories may deprive managers of the information they need to reward top performers and address performance issues, as well as deprive staff of the feedback they need to improve. Background The Veterans Benefits Administration (VEA) within VA administers the disability compensation and pension programs, whereby VA claims processing staff assess veterans' applications for disability compensation and pension benefits. Aside from benefits for veterans, VEA claims processing staff make eligibility determinations for deceased veterans' spouses, children, and parents. In short, they are responsible for ensuring that the decisions that lead to paying disability compensation and pension benefits are timely, accurate, and consistent. The VA disability claims process involves multiple steps and usually involves more than one claims processor. When a veteran submits a claim to one of VEA's 57 regional offices, staff in that office are responsible for obtaining evidence to evaluate the claim, such as medical and military service records; determining whether the claimant is eligible for benefits; and assigning a disability rating specifying the severity of each of the veteran's impairments. These ratings determine the amount of benefits eligible veterans will receive. VA has faced questions about the timeliness, accuracy, and consistency of its disability decisions. GAO designated Federal disability programs, including VA and other programs, as a high-risk area in 2003. In particular, our prior work found VA relied on outmoded criteria for determining program eligibility that did not fully reflect advances in medicine and technology or changes in the labor market. As a result, VA's disability program may not recognize an individual's full potential to work. In addition, VA has seen processing times for their disability claims increase over the past several years, and inconsistencies in disability decisions across locations have raised questions about fairness and integrity. Some have suggested that VA needs to address its training and guidance related to claims processing in order to improve consistency and that it should conduct periodic evaluations of decisions to ensure the accuracy of ratings across disability categories and regions. VA has reported that some of the inconsistency in its decisions is due to complex claims, such as those involving post-traumatic stress disorder, but it has also acknowledged that the accuracy and consistency of claims decisions needs further improvement. Training Complies With Some Accepted Practices, But VBA Does Not Adequately Evaluate Training and May Be Falling Short in Training Design and Implementation To prepare newly hired staff to perform the tasks associated with processing disability claims, VEA has developed a highly structured, three-phase program designed to deliver standardized training. The first phase is designed to lay the foundation for future training by introducing new staff to topics such as medical terminology and the computer applications used to process and track claims. The second provides an overview of the technical aspects of claims processing, including records management, how to review medical records, and how to interpret a medical exam. The third includes a combination of classroom, on-the-job, and computer-based trainings. The second and third phases in this program are designed to both introduce new material and reinforce material from the previous phase. To help ensure that claims processing staff continually maintain their knowledge after their initial training and keep up with changing policies and procedures, VEA's Compensation and Pension Service requires all claims processing staff to complete a minimum of 80 hours of technical training annually. This training requirement can be met through a mix of classroom instruction, electronic-based training from sources such as the Training and Performance Support System (TPSS), or guest lecturers. VBA's regional offices have some flexibility over what courses they provide to their staff to help them meet the training requirement. These courses can cover such topics as establishing veteran status, asbestos claims development, and eye-vision issues. We found that VBA has taken some steps to strategically plan its training for claims processors in accordance with generally accepted training practices identified in our prior work.\3\ For example, VBA has taken steps to align training with the agency's mission and goals. In 2004, VBA established an Employee Training and Learning Board (board) to, among other things, ensure that the agency's training decisions support its strategic and business plans, goals, and objectives. Also, VBA has identified the skills and competencies needed by its claims processing staff by developing a decision tree and task analysis of the claims process. In addition, VBA has taken steps to determine the appropriate level of investment in training and to prioritize funding. The board's responsibilities include developing an annual training budget and recommending training initiatives to the Under Secretary of Benefits. Further, we found that VBA's training program for new claims processing staff appears well-designed, in that it conforms to adult learning principles by carefully defining all pertinent terms and concepts and providing abundant and realistic examples of claims work. --------------------------------------------------------------------------- \3\ GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, DC: March 2004). --------------------------------------------------------------------------- However, while VBA has developed a system to collect feedback from new claims processing staff on their training, the agency does not consistently collect feedback on all of the training it provides. For example, none of the regional offices we visited consistently collected feedback on the training they conduct. Without feedback on regional office training, VBA may not be aware of how effective all of its training tools are. Moreover, both new and experienced claims processing staff we interviewed reported some issues with their training. A number of staff told us the TPSS was difficult to use, often out-of-date, and too theoretical. Some claims processing staff with more experience reported that they struggled to meet the annual training requirement because of workload pressures or that training topics were not always relevant for staff with their level of experience. VBA officials reported that they have reviewed the 80-hour training requirement to determine if it is appropriate, but they could not identify the criteria or any analysis that were used to make this determination. Identifying the right amount of training is crucial. An overly burdensome training requirement may needlessly take staff away from essential claims processing duties, while too little training could contribute to processing and quality errors. In addition to lacking a clear process for assessing the appropriateness of the SO-hour training requirement, VBA also has no policy outlining consequences for individual staff who do not complete the requirement. Because it does not hold staff accountable, VBA is missing an opportunity to clearly convey to staff the importance of managing their time to meet training requirements, as well as production and accuracy goals. In fiscal year 2008, VBA implemented a new learning management system that allows it to track the training hours completed by individual staff. Although VBA now has the capacity to monitor staffs completion of the training requirement, the agency has not indicated any specific consequences for staff who fail to meet the requirement. VA's Performance Management System Generally Conforms With Accepted Practices, But May Not Clearly Differentiate Among Staff's Performance Levels VA's performance management system for claims processors is consistent with a number of accepted practices for effective performance management systems in the public sector.\4\ For example, the elements used to evaluate individual claims processors-such as quality, productivity, and workload management-appear to be generally aligned with VBA's organizational performance measures. Aligning individual and organizational performance measures helps staff see the connection between their daily work activities and their organization's goals and the importance of their roles and responsibilities in helping to achieve these goals. VA also requires supervisors to provide claims processors with regular feedback on their performance, and it has actively involved its employees and other stakeholders in developing its performance management system. --------------------------------------------------------------------------- \4\ GAO, Results Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success, GAO-03-488 (Washington, DC: Mar. 14, 2003). --------------------------------------------------------------------------- However, VA's system may not be consistent with a key accepted practice-clear differentiation among staff performance levels. We have previously reported that, in order to provide meaningful distinctions in performance for experienced staff, agencies should preferably use rating systems with four or five performance categories.\5\ If staff members' ratings are concentrated in just one or two of multiple categories, the system may not be making meaningful distinctions in performance. Systems that do not make meaningful distinctions in performance fail to give (1) employees the constructive feedback they need to improve and (2) managers the information they need to reward top performers and address performance issues. --------------------------------------------------------------------------- \5\ GAO, Human Capital: Preliminary Observations on the Administration's Draft Proposed ``Working for America Act,'' GAO-06- 142T (Washington, DC: Oct. 5, 2005). --------------------------------------------------------------------------- VA's performance appraisal system has the potential to clearly differentiate among staff performance levels. Each fiscal year, regional offices give their staff a rating on each individual performance element: exceptional, fully successful, or less than fully successful. For example, a staff member might be rated exceptional on quality, fully successful on productivity, and so forth. Some elements are considered critical elements, and some are considered noncritical. Staff members are then assigned to one of five overall performance categories, ranging from unsatisfactory to outstanding, based on a formula that converts a staff member's combination of ratings on the individual performance elements into an overall performance category (see fig. 1). Figure 1--VA Overall Performance Appraisal Formula [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: GAO analysis of VBA information. However, there is evidence to suggest that the performance management system for claims processing staff may not clearly or accurately differentiate among staffs performance. Central office officials and managers in two of the four regional offices we visited said that, under the formula for assigning overall performance categories, it is more difficult to place staff in certain overall performance categories than in others-even if staffs performance truly does fall within that category. These managers said it is especially difficult for staff to be placed in the excellent category. In fact, at least 90 percent of all claims processors in the regional offices we visited ended up in only two of the five performance categories in fiscal year 2007: fully successful and outstanding (see fig. 2). Figure 2--Fiscal Year 2007 Overall Performance Ratings for Claims Processors in Four Regional Offices Were Concentrated in the Outstanding and Fully Successful Categories [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: VBA regional offices. Some managers told us that there are staff whose performance is better than fully successful but not quite outstanding, but that under VA's formula, it is difficult for these staff to be placed in the excellent category. To be placed in the excellent category, a staff member must be rated exceptional in all the critical elements and fully successful in at least one noncritical element. However, managers told us that virtually all staff who are exceptional in the critical elements are also exceptional in the noncritical elements, and they are appropriately placed in the outstanding category. On the other hand, if a staff member is rated fully successful on just one critical element, even if all other elements are rated as exceptional, the staff member's overall performance category falls from outstanding to fully successful. Neither VBA nor VA central office officials have examined the distribution of claims processing staff across the five overall performance categories. However, VA has acknowledged that there may be an issue with its formula, and the agency is considering changes to its performance management system designed to allow for greater differentiation in performance. Absent additional examination of the distribution of claims processors among overall performance categories, VA lacks a clear picture of whether its system is working as intended and whether any adjustments are needed. In conclusion, VA appears to have recognized the importance of developing and maintaining high performing claims processors. It needs to devote more attention, however, to ensuring that its training and performance management systems are better aligned to equip both new and experienced staff to handle a burgeoning workload. Specifically, in our May 2008 report, we recommended that VA should collect feedback from staff on training provided in the regional offices in order to assess issues such as the appropriateness of the 80-hour annual training requirement and the usefulness of TPSS. We also recommended that the agency should use information from its new learning management system to hold staff members accountable for meeting the training requirement. In addition, we recommended that VA should assess whether its performance management system is making meaningful distinctions in performance. In its comments on our May 2008 report, VA concurred with our recommendations, but it has not yet reported making any significant progress in implementing them. While hiring, training, and evaluating the performance of staff is essential, commensurate attention should be focused on reviewing and aligning disability benefits and service outcomes to today's world. In prior work, we have noted that VA and other Federal disability programs must adopt a more modern understanding of how technology and labor market changes determine an individual's eligibility for benefits, as well as the timing and portfolio of support services they are provided. To the extent progress is made in this area, effective training and performance management systems will be of crucial importance. Moreover, the way VA's larger workforce is distributed and aligned nationwide can also significantly impact the degree to which it succeeds in meeting the agency's responsibilities to veterans in the future. In short, VA should seize this opportunity to think more strategically about where to best deploy its new staff and how to develop and maintain their skills. GAO Contact and Staff Acknowledgments Mr. Chairman, this concludes my remarks. I would be happy to answer any questions that you or other Members of the Subcommittee may have. For further information, please contact Daniel Bertoni at (202) 512-7215 or [email protected]. Also contributing to this statement were Clarita Mrena, Lorin Obler, David Forgosh, and Susan Bernstein. Related GAO Products Veterans' Benefits: Improved Management Would Enhance VA's Pension Program. GAO-08-112. Washington, DC: February 14, 2008. Veterans' Disability Benefits: Claims Processing Challenges Persist, while VA Continues to Take Steps to Address Them. GAO-08-473T. Washington, DC: February 14, 2008. Disabled Veterans' Employment: Additional Planning, Monitoring, and Data Collection Efforts Would Improve Assistance. GAO-07-1020. Washington, DC: September 12, 2007. Veterans' Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, DC: September 30, 2003. Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government. GAO-03-893G. Washington, DC: July 2003. Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. GAO-03-488. Washington DC: March 14, 2003. Major Management Challenges and Program Risks: Department of Veterans Affairs. GAO-03-110. Washington, DC: January 1, 2003. Veterans' Benefits: Claims Processing Timeliness Performance Measures Could Be Improved. GAO-03-282. Washington, DC: December 19, 2002. Veterans' Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. Washington, DC: August 16, 2002. Veterans' Benefits: Training for Claims Processors Needs Evaluation. GAO-01-601. Washington, DC: May 31, 2001. Veterans Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, DC: March 1, 1999. Prepared Statement of Statement of Michael Ratajczak, Decision Review Officer, Cleveland Veterans Affairs Regional Office, Veterans Benefits Administration, U.S. Department of Veterans, on Behalf of American Federation of Government Employees, AFL-CIO Dear Chairman and Members of the Subcommittee: Thank you for the opportunity to testify today on behalf of the American Federation of Government Employees, AFL-CIO (AFGE), the exclusive representative of employees in the Veterans Benefits Administration (VBA). I currently serve VBA as a Decision Review Officer (DRO) at the Cleveland VA Regional Office (RO). I joined VBA's workforce in September of 2001. I worked with the Tiger Team Remand Unit that resolved some of the oldest pending Board of Veterans Appeals remands in the country and served as an operational model for VBA's Appeals Management Center. I also served as a Specialized Rating Veterans Service Representative (RVSR) in the Tiger Team and the Cleveland Resource Unit of the Appeals Management Center. My duties with VBA have also included temporary assignments to a Remand Quality Review Project which was intended, at least in part, to identify common errors occurring at the RO level which necessitate remand of cases to the RO by the Board of Veterans Appeals. I have also served as an instructor in Centralized Challenge Courses for newly hired RVSRs. I currently serve as AFGE's representative on the VBA Design Committees for Basic and Journey-level RVSR Certification Testing. Prior to my employment with the VA, I was an attorney in private practice. In reviewing past Congressional testimony by VBA, I was struck by the consistent reference to increased complexity of claims and continuing judicial refinement of the duty to assist veterans under the Veterans Claims Assistance Act of 2000 (VCAA) as factors contributing to the claims backlog. Yet, VBA's testimony never mentioned the obvious impact of increasing claims complexity and duty to assist mandates on employees' ability to develop and adjudicate claims, or review case development and adjudications upon notice of disagreement from claimants within the time constraints imposed by productivity requirements set by management. Training and Performance Management are Closely Linked All who work in VBA claims processing agree that increased claims complexity and additional duties imposed by law are perhaps the most important factors slowing performance and reducing productivity. Therefore, in addition to adequate hiring, effective training must be an essential component of any VBA effort to increase the timeliness, accuracy and consistency of claims processing. Ultimately, the veterans seeking benefits are shortchanged by VBA's failure to make any accommodation in performance standards to reflect growing claims complexity and new legal requirements. VBA's insistence on holding claims processors to production standards that do not allow adequate time to develop, consider, and resolve complex claims in accordance with the duties imposed by law is a disservice to veterans and undermines the intent of the Veterans Claims Assistance Act. GAO found in its May 2008 report that VBA's training program for claims processors does not consistently track completion of training or ensure that training requirements are uniformly adhered to at the RO level. To the extent that GAO recommends that VBA be held accountable for fully implementing and tracking the training it mandates, it is laudable. However, I am very concerned that the report could be misread to imply that individual claims processors are ultimately responsible for ensuring that they complete their mandatory training requirements. Ultimately, it is VBA management's responsibility to provide the necessary time and resources for employees to complete well designed, comprehensive, up-to-date training programs. No individual claims processor can demand of local management that he or she receive training when a determination is made that employee resources are better devoted to other concerns, such as fulfilling a monthly or fiscal year production goal. Management's failure to devote time to the initial and continuing education of claims processors in favor of fulfilling a short-term production goal is similar to VBA eating its seed corn, since it deprives claims processors of the means to become more efficient, accurate, and fulfill their ever changing and increasingly complex duties. Similarly, management's failure to ensure adequate training time impairs the ability of VBA's certification testing program for VSRs and RVSRs to be an objective measure of training effectiveness. This places an unfair burden on individual claims processors who are thus less likely to achieve a passing score on certification testing, and in turn, advance their careers. In order for training to be meaningful, management must afford the participant time to read and analyze the material, and internalize its meaning through cognitive effort and practical application. The complexity of the claims process administered by VBA does not admit to simple resolution by reference to checklists, decision trees or presentation of information without elucidation. This complexity is well illustrated by a videotape that was recently shown to VBA claims processors on the application of GAF (global assessment of functioning) scores in the context of assigning disability ratings for service connected posttraumatic stress disorder. This 2 hour training video focuses on one very discrete issue in rating a very specific type of claim that is fraught with difficulties. It serves as a reminder that even well-prepared training material presented by highly competent and learned professionals can be of limited value in the absence of an opportunity for meaningful interactive learning. Training materials must be combined with the opportunity for trainees to receive timely feedback from an individual who understands the subject and can provide relevant, consistent, and immediate guidance to trainees struggling to internalize the meaning of the material in the context of the duties attendant to their positions. Similarly, while materials such as Fast Letters, Training Letters, and Decision Assessment Documents may be useful in drawing a claims processor's attention to a particular nuance or change in the claims process, they do not necessarily serve as useful training tools. The usefulness of such materials is further diminished if claims processors are not given adequate time to digest the materials VBA provides to notify them of changes in interpretation or implementation of policies, regulations, statutes, or case law. GAO's favorable findings about the effectiveness of centralized new hire training confirm the benefits of a curriculum that is developed, designed and implemented through Central Office. Therefore, I urge this Subcommittee to support greater centralization of mandatory continuing training, specifically a nationally uniform curriculum taught by a cadre of instructors at every RO who have completed the Instructor Development Curriculum, and who are available to implement training materials designed by Central Office. The presence of a team of qualified instructors at every RO, charged with implementing a relevant curriculum developed by Central Office staff and accountable to Central Office, would help close the gap between VBA Central Office's expectations for training and conflicting or incompatible goals of local management. The breakdown in VBA's training process seems to occur at two critical junctures: first, at the RO where time devoted toward training may be viewed as an unwarranted impediment to achieving immediate production goals; and second, between Central Office and ROs because Central Office does not adequately identify trends in errors that are amenable to training or provide enough specific curricula for continuing employee education. VBA should apply lessons learned from its Remand Quality Review Project to acquire data pertaining to common development errors by reviewing a statistically relevant sample of deferred rating decisions from a wide range of ROs. VBA should then use that data to tailor training to common development errors, including improper deferrals of rating decisions, and develop effective strategies to help employees avoid those time consuming errors in the first instance. This type of feedback loop already exists in the healthcare arena to prevent medical errors, including the Veterans Health Administration that has long played a leadership role in medical error reduction. Performance Standards Must Reflect Claims Complexity and Legal Requirements VBA recognizes that increasingly complex claims and continuous refinement of the legal requirements attendant to claims processing has a detrimental effect on the size of VBA's pending workload inventory. However, VBA has either failed or refused to recognize that those same factors have a detrimental effect on the productivity of individual claims processors insofar as additional time is needed to develop, adjudicate, or review claims. By not adjusting individual productivity standards to reflect the increasing complexity and difficulty of the claims process, VBA may once again be failing to provide the service its claimants deserve. The needs of claimants intersect with the requirements imposed on claims adjudicators precisely in the implementation of performance standards. If claims processors are required to choose between developing, rating, or reviewing a case in accordance with all legal requirements and fulfilling their production requirements, a temptation is presented to make a decision in favor of their own immediate interests. As an example, RVSRs are generally given no production credit for identifying development deficiencies in a given case, or directing action to correct those deficiencies via deferred rating actions. Often, such deferred actions are time- consuming and complex. Consequently, RVSRs are often met with a choice between meeting their productivity requirements and ensuring that decisions are rendered in accordance with all applicable duty to assist requirements and are, in essence, given no meaningful credit for ensuring that claims are adequately developed. One of the many unfortunate side effects of this problem is that the VSRs charged with developing claims oftentimes receive no meaningful feedback from RVSRs concerning development deficiencies. Of course, the ultimate effect of this system is felt by claimants, and reflected in an increasing appeal rate. Such a system of measuring ``productivity'' is disrespectful to claims processors and claimants alike. Productivity requirements that do not take into account the increased time necessary to develop, adjudicate, or review increasingly complex claims in an increasingly stringent legal environment ultimately lead to bad service for claimants, since productivity is often rewarded over quality. As AFGE has urged this Subcommittee in the past, I cannot overstate the importance of requiring thorough time-motion studies so that VBA performance standards for claims processors may be informed and adjusted by reference to valid and scientific data. Unless and until VBA has valid empirical evidence concerning what claims adjudicators can reasonably be expected to accomplish in a given period, any mandate concerning what those employees must do in the same period will be suspect. Moreover, insofar as such suspect standards are used to project the need for additional human resources or trends concerning future claims inventory levels, any projections upon which they are based will also be suspect. The proposal in H.R. 5892 for a study of the VBA work credit system will provide valuable data toward this goal. Nor should VBA continue to set performance standards for claims processors by fiat with reference to the ``experience'' of managers who are years removed from any meaningful contact with the day-to-day exigencies of claims processing, and expect to meaningfully address workload trends or human capital requirements. These managers should be required to devote a portion of their workday (e.g. 50 percent) to developing or adjudicating claims in order to keep a fresh perspective of what it takes to conform to an individual position description. Such a requirement would also have the benefit of reducing the pending claims inventory. Therefore, I also support the requirement in H.R. 5892 that managers pass the same certification tests as the employees they supervise. The absence of such a requirement would permit non- certified supervisors to be charged with critiquing the work of certified employees, thereby seriously undermining the credibility of the certification process. Increased tracking and implementation of continuing training at VBA is laudable, but only insofar as it will ultimately serve claimants. An educated and well-trained workforce should be one of VBA's highest goals. However, VBA's claims processing workforce must also be afforded adequate time to perform their duties in accordance with their training, and that consideration must be reflected in individual performance standards. Methods to Increase Accountability and Reduce Rating Variances If continuing training of VBA claims processors is made consistent by reference to a curriculum created by VBA centralized training staff and implemented by a corps of instructors accountable to VBA's Central Office, rating variances could reasonably be expected to decrease as claims processors conform their individual activities to their uniform training. Conclusion I view the GAO report not as an indictment on claims processors' skills, abilities, or willingness to learn as much as a description of VBA's failure to provide relevant, useful continuing training or adequately track the efficacy of that training. Ultimately, training deficiencies at VBA are not the product of nor should they be the responsibility of individual claims processors. The claims processing workforce at VBA is among of the finest and most dedicated workforces in Government. Unfortunately, insofar as VBA inadequately provides continuing training to claims processors and then attempts to hold them responsible for deficiencies in quality and productivity, claims processors may be disproportionately affected by inadequate training. To illustrate that fact it might be instructive for this Subcommittee to inquire of VBA whether they can recall a single instance of any VBA manager who has ever been disciplined, demoted, or formally reprimanded for failing to adequately train an employee. In contrast, there are numerous instances of long term claims processors with good work histories being disciplined, demoted, formally reprimanded, or even discharged for failing to meet their productivity or quality requirements. The crux of the problem is a failure by VBA to provide the continuing training necessary for our Members to be productive, efficient and accurate in fulfillment of their duties. The onus of inadequate training should not be disproportionately borne by employees charged with the day to day processing of claims or the constituency they are honored to serve. Thank you. Prepared Statement of Kerry Baker, Assistant National Legislative Director, Disabled American Veterans Mr. Chairman and Members of the Subcommittee: On behalf of the 1.3 million members of the Disabled American Veterans (DAV), I am honored to appear before you today to discuss the effectiveness of the Veterans Benefits Administration's (VBA's) training, performance management, and accountability requirements. In accordance with our congressional charter, the DAV's mission is to ``advance the interests, and work for the betterment, of all wounded, injured, and disabled American veterans.'' TRAINING VBA has a standard training curriculum for new claims processors and an 80-hour annual training requirement for all claims processors. The training program in VBA is basically a three-stage system. First, VBA policy requires new staff to complete some orientation training, which is provided in their home offices. Second, they are required to attend a 2- to 3-week centralized training course that provides a basic introduction to job responsibilities. Third, new staff are required to spend several more months in training at their home offices, which includes on-the-job training and/or instructor-led training that follow a required curriculum via use of an online learning tool called the Training and Performance Support System (TPSS). VBA policy states that all claims processors are required to complete a minimum of 80 hours of training annually. VA Regional Offices (ROs) have some discretion over what training they provide to meet this requirement. The VA's three-phased program for new claims processors is designed to deliver standardized training, regardless of training location or individual instructors. Topics included in the training program contain a lesson plan with review exercises, student handouts, and copies of slides used during the instructor's presentation. The VBA also has an annual 80-hour training requirement for new and experienced rating veterans' service representatives (RVSRs) and veteran service representatives (VSRs). The first phase of training for new RVSRs is prerequisite training. It begins at new RVRSs' home regional offices when they begin working. The prerequisite training is designed to lay the foundation for future training by introducing new employees to topics such as the software applications used to process and track claims, medical terminology, the system for maintaining and filing a case folder, and the process for requesting medical records. VBA specifies the topics that must be covered during prerequisite training: however, ROs can choose the format for the training and the timeframe. New VSRs and RVSRs typically spend two to 3 weeks completing prerequisite training in their home office before they begin the second program phase. Nonetheless, VA personnel informed the DAV that many new employees are only allowed approximately 7 days to complete the training. The second phase of training is known as centralized training, wherein new VSRs and RVSRs spend approximately 3 weeks in classroom training. The 3-week timeframe is misleading because the first and last portions of week one and three are utilized for travel. Therefore, the actual training time is closer to 2 weeks. Participants from multiple ROs are typically brought together in centralized training sessions, which is usually at the Veterans Benefits Academy in Baltimore, Maryland. Centralized training provides an overview of the technical aspects of the VSR and RVSR positions. These training classes usually have at least three instructors, but the actual number can vary depending on the size of the group. VBA's goal is to maintain a minimum ratio of instructors to students. To practice processing different types of claims, VSRs work on either real or hypothetical claims specifically designed for training. Centralized training for new RVSRs focuses on topics such as: systems of the human body; how to review medical records; and, how to interpret medical exams. To provide instructors for centralized training, VBA relies on RO staff who have received training on how to be an instructor. Centralized training instructors may be VSRs, RVSRs, supervisors, or other staff identified by RO managers as having the capability to be effective instructors. The VBA has increased the number of training sessions because of the influx of new staff. In fiscal year 2007, VBA increased the frequency of centralized training and its student capacity at the Veterans Benefits Academy. During fiscal year 2007, VBA held 67 centralized training sessions for 1,458 new VSRs and RVSRs. Centralized training sessions were conducted at 26 different ROs during fiscal year 2007, in addition to the Veterans Benefits Academy. By comparison, during fiscal year 2006, VBA held 27 centralized training sessions for 678 new claims processors. Nonetheless, VBA has not run its benefits academy near to full capacity in 2008, the reasons for which are unclear. When new VSRs and RVSRs return to their home office after centralized training, they are required to begin their third phase of training, which is supposed to include on-the-job, classroom, and computer-based training modules that are part of VBA's Training and Performance Support Systems (TPSS), all conducted by and at the RO. New VSRs and RVSRs typically take about 6 to 12 months after they return from centralized training to complete all the training requirements for new staff. In addition to the foregoing three-phase training program, VBA also requires 80 hours of annual training for all VSRs and RVSRs. The training is divided into two parts. At least 60 hours must come from a list of core technical training topics identified by Compensation and Pension (C&P) Service. VBA specifies more core topics than are necessary to meet the 60-hour requirement, so regional offices can choose those topics most relevant to their needs. They can also choose the training method used to address each topic, such as classroom or TPSS training. The RO managers decide the specificities of the remaining 20 hours. Analysis and Recommendations The DAV has consistently maintained that VA should invest more in training adjudicators and decision-makers, and that it should hold them accountable for higher standards of accuracy. Nonetheless, such training has not been a high priority in VBA. We have further consistently stated that proper training leads to better quality, and that quality is the key to timeliness. Timeliness follows from quality because omissions in record development, failure to afford due process, and erroneous decisions require duplicative work, which add to the load of an already overburdened system. The VBA will only achieve such quality when it devotes adequate resources to perform comprehensive and ongoing training, devotes sufficient time to each case, and imposes and enforces quality standards through effective quality assurance methods and accountability mechanisms. One of the most essential resources is experienced and knowledgeable personnel devoted to training. More management devotion to training and quality requires a break from the status quo of production goals above all else. In a 2005 report from VA's Office of Inspector General, VBA employees were quoted as stating: ``Although management wants to meet quality goals, they are much more concerned with quantity. An RVSR is much more likely to be disciplined for failure to meet production standards than for failing to meet quality standards,'' and that ``there is a lot of pressure to make your production standard. In fact, your performance standard centers around production and a lot of awards are based on it. Those who don't produce could miss out on individual bonuses, etc.'' \1\ Little if anything has changed since the Inspector General has issued this report. --------------------------------------------------------------------------- \1\ Department of Veterans Affairs Office of Inspector General, Rep. No. 05-00765-137, Review of State Variances in VA Disability Compensation Payments 61 (May 19, 2005). --------------------------------------------------------------------------- A review of VBA's training programs mentioned above reveals that its problems caused by a lack of accountability do not begin in the claims development and rating process--they begin in the training program. Essentially, we can find little, if any, measurable accountability in VBA's training program. For example, despite VBA's program requirements for its new hires to complete phase-one training before advancing to the phase-two centralized training, some VA employees anonymously informed the DAV that many candidates begin centralized training without having had the opportunity to participate in and/or complete phase-one training. Additionally, candidates are not held responsible via formal testing on subjects taught during phase-one training. While oversight may or may not exist for this portion of training, the DAV could find none. Without resorting to a critique of the substance of VBA's subject matter taught during phase-two training, or any other phase for that matter, we limit our analysis again to accountability. As in phase one, VBA refuses to test participants of phase-two training. The obvious goal is to ensure employees attend the required course--ensuring that employees achieve VBA's learning objectives appears to have no priority whatsoever. By now, a new employee has approximately 1 month of training and is supposedly prepared for phase-three training. Keep in mind that during phase three, new employees will work on real-world cases whose outcomes affect the lives and livelihoods of disabled veterans and their families. Real cases notwithstanding, again there is no accountability, no testing, and no oversight outside that of which is provided locally; again, that oversight is not measured nationally. The result of such an unsupervised and unaccountable training system is that no distinction exists between unsatisfactory performance and outstanding performance. This lack of accountability during training further reduces, or even eliminates, employee motivation to excel. This institutional mindset is further epitomized in VBA's day- to-day performance, where employees throughout VBA are reminded daily that optimum work output is far more important quality performance and accurate work. The effect of VBA's lack of accountability in its training program was demonstrated when it began offering skills certification tests to support certain promotions. Beginning in late 2002, VSR job announcements began identifying VSRs at the GS-11 level, contingent upon successful completion of a certification test. The test consisted of 100 multiple-choice questions, which were open-book. The VA allowed participants to use online references and any other reference material, including individually prepared notes in order to pass the test. The first validation test was performed in August 2003. There were 298 participants in the first test. Of these, 75 passed for a pass rate of 25 percent. VBA conducted a second test in April 2004. Out of 650 participants, 188 passed for a pass rate of 29 percent. Because of the low pass rates on the first two tests, a 20-hour VSR ``readiness'' training curriculum was developed to prepare VSRs for the test. A third test was administered on May 3, 2006, to 934 VSRs nationwide. Still, the pass rate was only 42 percent. Keep in mind that these tests were not for training, they were to determine promotions from GS-10 to GS- 11. The VBA recently began similar testing with RVSRs. The DAV was unable to obtain those tests result. VA employees nonetheless informed us that VBA's test results for RVSRs were no better than test results for VSRs. These results reveal a certain irony, in that VBA will offer a skills certification test for promotion purposes, but does not require comprehensive testing throughout its training curriculum. The following ``accountability'' portion of this testimony further illustrates the product of inadequate training. Mandatory and comprehensive testing designed cumulatively from one subject area to the next, for which VBA then holds trainees accountable, should be the number one priority of any plan to improve VBA's training program. Further, VBA should not allow trainees to advance to subsequent stages of training until they have successfully completed such testing. To be fair, the DAV understands that VBA is not solely at fault on this subject. The VA employees union has objected to the type of testing mentioned herein. We do not expect such objections to cease. In fact, we feel the only way to moot these objections is for Congress to mandate such testing through statutory change. Section 105 of H.R. 5892 mandates some testing for claims processors and VBA managers, which is an improvement over current practices; however, it does not mandate the type of testing during the training process as explain herein. Measurable improvement in the quality of and accountability for training will not occur until such mandates exist. ACCOUNTABILITY The DAV has consistently stated that, in addition to training, accountability is the key to quality, and therefore to timeliness as well. As it currently stands, almost everything in VBA is production driven. In addition to basing personnel awards on production, the DAV strongly believes that quality should be awarded at least on parity with production. However, in order for this to occur, VBA must implement stronger accountability measures for quality assurance. VA's quality assurance tool for compensation and pension claims is the Systematic Technical Accuracy Review (STAR) program. Under the STAR program, VA reviews a sampling of decisions from regional offices and bases its national accuracy measures on the percentage with errors that effect entitlement, benefit amount, and effective date. Inconsistency signals outright arbitrariness in decision-making, uneven or insufficient understanding of governing criteria, or rules for decisions that are too vague or overly broad and allows them to be applied according to the prevailing mindset of a particular group of decision-makers. Obviously, VA must detect inconsistencies before the cause or causes can be determined and remedied. Simply put, there is a gap in quality assurance for purposes of individual accountability in quality decision-making. In the STAR program, a sample is drawn each month from a regional office workload divided between rating, authorization, and fiduciary end-products. For example, a monthly sample of ``rating''-related cases generally requires a STAR review of 10 rating-related end products.\2\ Reviewing 10 rating-related cases per month for an average size regional office, an office that would easily employee more than three times that number of raters, is undeniable evidence of a total void in individual accountability. If an average size regional office produced only 1,000 decisions per month, which we feel is quite conservative, the STAR program would only review 1 percent of the total cases decided by that regional office. Those figures leave no room for trend analysis, much less personal accountability. --------------------------------------------------------------------------- \2\ See M21-4, Ch. 3, Sec. 3.02. --------------------------------------------------------------------------- To put this in better perspective, according to VA's 2007 performance and accountability report, the STAR program reviewed 11,056 compensation and pension (C&P) cases in 2006 for improper payments. While this number appears significant, the total number of C&P cases available for review was 1,540,211. Therefore, the percentage of cases reviewed was approximately seven tenths of 1 percent, or 0.72 percent. Another method of measuring the VA's need for more accountability is an analysis of the Board's Summary of Remands, while keeping in mind that its summary represents a statistically large and reliable sample of certain measurable trends. The examples must be viewed in the context of the VA (1) deciding 700,000 to 800,000 cases per year; (2) receives over 100,000 NODs; and (3) submits 40,000 appeals to the Board. The examples below are from October 2006 to October 2007. Remands resulted in 998 cases because no ``notice'' under section 5103 was ever provided to the claimant. The remand rate was much higher for inadequate or incorrect notice; however, considering the confusing (and evolving) nature of the law concerning ``notice,'' we can only fault the VA when it fails to provide any notice. VA failed to make initial requests for SMRs in 667 cases and failed to make initial requests for personnel records in 578 cases. The number was higher for additional followup records requests following the first request. This number is disturbing because initially requesting a veteran's service records are the foundation to every compensation claim. It is claims development 101. The Board remanded 2,594 cases for initial requests for VA medical records and 3,393 cases for additional requests for VA medical records. The disturbing factor here is that a VA employee can usually obtain VA medical records without ever leaving the confines of one's computer screen. Another 2,461 cases were remanded because the claimant had requested a travel board hearing or video-conference hearing. Again, there is a disturbing factor here. A checklist is utilized prior to sending an appeal to the Board that contains a section that specifically asked whether the claimant has asked for such a hearing. The examples above totaled 7,298 cases or nearly 20 percent of appeals reaching the Board, all of which cleared the local rating board and the local appeals board with errors that are elementary in nature. Yet they were either not detected or they were ignored. Many more cases were returned for more complex errors. But for nearly a 20-percent error rate on such basic elements in the claims process passing through VBA's most senior of rating specialist is simply unacceptable. The problem with the VA's current system of accountability is that it does not matter if VBA employees ignored these errors because those that commit such errors are usually not held responsible. They therefore have no incentive to concern themselves with the quality of their work. Above all else, these figures showing that the VA's quality assurance and accountability systems require significant enhancement. Recommendation Congress should require the Secretary to report on how the Department could establish a quality assurance and accountability program that will detect, track, and hold responsible those VA employees who commit egregious errors. Such report should be generated in consultation with veterans' service organizations most experienced in the VBA claims process. The DAV believes that effective accountability can be engineered in a manner that holds each VBA employee responsible for his/her work as a claim moves through the system while at the same time holds all employees responsible simultaneously. As errors are discovered (definition of such errors to be determined, but specific to employee responsibility), employees responsible for such errors must be held accountable by forfeiture of work credit percentage. For example, if a Decision Review Officer (DRO) reverses a decision from a frontline rating specialist because of error, as opposed to difference of opinion or receipt of new evidence, then the frontline employee should be subject to forfeiture of a portion of work credit that is normally used to track production standards applicable to performance bonuses. In turn, if a case proceeds to the BVA and is reversed or remanded on similar grounds, then both the frontline rater as well as the DRO should forfeit work credit, and so one. The same should apply to Veterans' Law Judges at BVA when the Court of Appeals for Veterans Claims finds error in a BVA decision. Such a cumulative accountability system would effectively eliminate potential abuse of the system through the proverbial good-old-boy club. One employee would be far less likely to cover for errors or look the other way from errors committed by a fellow employee if they knew their performance standards were equally at risk. This type of system would ensure personal accountability at every stage in the claims process without seriously disrupting or dismantling VBA's current performance measurement system. PERFORMANCE MEASUREMENT VA's benefits delivery system has become particularly multifaceted, especially when considering the various types of claims a beneficiary may file, the various stages of development and decision-making within each claim, and the potential changes that can occur at any particular stage of the claim. Currently, VA utilizes over 50 pending end-product codes\3\ for a multitude of actions. The number of end-product codes may be further expanded by using ``modifiers'' that designate specific ``issues'' for types of claims within a certain broader category. --------------------------------------------------------------------------- \3\ M21-4, App. A, Glossary of Terms and Definitions. Manpower Control and Utilization in Adjudication Divisions (Pending End Product: ``A claim or issue on which final action has not been completed. The classification code identified refers to the end product work unit to be recorded when final disposition action has been taken.''). --------------------------------------------------------------------------- The VA's end product codes are used in conjunction with its productivity and work measurement system. The productivity system is the basic system of work measurement used by C&P Service, but it is also used for report and tracking. Additionally, VA's end-product codes are also utilized in the STAR program. The program is also a tool used for quantitative measurement, a tool utilized in preparing budget forecasts, and in distributing available staffing. Quantitative and productivity measurement are also tools used in comparing and tracking employment of resources. Both productivity measurement and work measurement are tools available to management for this purpose. Quantitative measurement also allows Central Office and Area Offices to compare stations and to track both local and national trends. Productivity measurement and work measurement are complementary measurement systems that each depend, in part, on VA's end product code system. The end-product code system is further used in determining work credit provided to VA's employees and is therefore vital in measuring employee production goals and awarding performance awards. Changes should not be mandated that would cause VA to lose the ability to manage and track its day-to-day functions. The DAV finds no measurable flaws in VBA's overall measurement systems. In fact, our foregoing recommendations concerning improvements in VBA's accountability would draw on the strengths in its measurement systems, thereby allowing easier and less disruptive implementation of stronger and more effective accountability. *** FORCED SO BODONI DASH IS NOT AT TOP OF PAGE *** deg. We hope the Subcommittee will review these recommendations and give them consideration for inclusion in your legislative plans. Mr. Chairman, thank you for inviting the DAV to testify before you today. Prepared Statement of Ronald B. Abrams, Joint Executive Director, National Veterans Legal Services Program Mr. Chairman and Members of the Committee: I am pleased to have the opportunity to submit this testimony on behalf on behalf of the National Veterans Legal Services Program (NVLSP). NVLSP is a nonprofit veterans service organization founded in 1980 that has been assisting veterans and their advocates for 28 years. NVLSP has trained thousands of service officers and lawyers in veterans benefits law, and has written educational publications that thousands of veterans advocates regularly use as practice tools to assist them in their representation of VA claimants. NVLSP also conducts quality reviews of the VA regional offices on behalf of The American Legion. NVLSP also represents veterans and their families on claims for veterans benefits before VA, the U.S. Court of Appeals for Veterans Claims (CAVC), and other Federal courts. Since its founding, NVLSP has represented over 1,000 claimants before the Board of Veterans' Appeals and the Court of Appeals for Veterans Claims (CAVC). NVLSP is one of the four veterans service organizations that comprise the Veterans Consortium Pro Bono Program, which recruits and trains volunteer lawyers to represent veterans who have appealed a Board of Veterans' Appeals decision to the CAVC without a representative. We believe that the effectiveness of VBA training should be measured by the quality of the work product produced by VA adjudicators. Therefore, the quality and timeliness of VA adjudications should reflect the effectiveness or lack thereof, of VA training. Because the quality of VA adjudications is inadequate NVLSP must conclude that VBA training is not effective or adequate. In the experience of NVLSP (over 10 years of quality reviews, in conjunction with The American Legion of approximately 40 different VAROs combined with extensive NVLSP representation before the CAVC), most of the most egregious VA errors are a result of premature adjudications. Many VA managers emphasize quantity over quality. VA managers and VA adjudicators have let us know that because production is paramount, training is deemphasized because the time spent training reduces the time available to produce decisions, and the training of VA adjudicators regarding the procedures designed to protect the right of claimants seeking VA benefits also reduces production. In September 2008, courageous VA regional office employees filed a grievance exposing this overemphasis on production. This grievance is attached to my testimony. The grievance asserts that: the regional office created and encouraged a culture in which quantity is emphasized to the detriment of quality; the VARO failed to properly implement training initiatives to assure that those reviewing claims are sufficiently trained in the relevant disciplines so as to reduce errors, improve the quality of claims processing, and successfully complete newly implemented certification requirements; and the VARO failed to properly implement a fair and impartial performance appraisal system that assures that quantitative measures of performance are not emphasized to the detriment of measures of the quality of that performance. NVLSP believes that the quality of VARO adjudications is much worse than what is reported by the VA. The remand and reversal statistics produced by decisions issued by the Board of Veterans' Appeals (BVA or Board) can be considered an independent review of the quality of adjudications performed by the VAROs. BVA statistics provided by the Veterans Appeals Control and Locator System (VACOLS) for FY 07 reveal that Board decided over 40,000 appeals. The Board allowed 21.12 percent (that is, granted the claim that the VA regional office had denied) and remanded 35.36 percent of these appeals back to the VARO because the VARO had wrongly failed to obtain all of the evidence it should have attempted to obtain. Therefore, in 56.48 percent of the appeals decided by the BVA, the BVA either reversed or remanded the VARO decision. This 56.48 percent statistic could be considered an error rate. Even if we take into account the fact that new evidence can be added at the Board and deduct 20 percent from the 56.48 percent, an error rate as high as 36 percent is not acceptable and does not verify the low error rate claimed by VA in its VA Star Reports (close to a 90 percent ``accuracy rate''). The reversal/remand rate thus far for FY 2008 is 59.4 percent. The news gets worse. The BVA, in its rush to make final decisions and to avoid remands also quite often prematurely denies claims that should have been remanded. Of course, the error was originally committed by the VARO, not the BVA. In September 2007, my fellow Joint Executive Director, Bart Stichman, testified that ``[f]or more than a decade, the Court's [Court of Appeals for Veterans Claims (Court or CAVC)] annual report card of the BVA's performance has been remarkably consistent. The 12 annual report cards issued over the last 12 years yields the following startling fact: of the 16,550 Board decisions that the Court individually assessed over that period (that is, from FY 1995 to FY 2006), the Court set aside a whopping 77.7 percent of them (that is, 12,866 individual Board decisions). In each of these 12,866 cases, the Court set aside the Board decision and either remanded the claim to the Board for further proceedings or ordered the Board to award the benefits it had previously denied. In the overwhelming majority of these 12,866 cases, the Court took this action because it concluded that the Board decision contained one or more specific legal errors that prejudiced the rights of the VA claimant to a proper decision.'' How should a veteran seeking VA disability benefits feel? The Board of Veterans' Appeals reverses and remands over 50 percent of all VARO adjudications and the CAVC sets aside over 77 percent of the BVA decisions that it reviews. These numbers do not inspire confidence in the quality of VA adjudications. It is clear that the adjudication culture at the VAROs needs to be changed. Many VA managers act like they are producing widgets rather than adjudicating claims filed by real people. Their goal should not be just prompt adjudication; the goal should be a timely, accurate and fair adjudication. Even VA employees are frustrated and upset by their lack of training and the overemphasis on production. The following is a quote from the grievance attached to my testimony. ``As set forth in the Grievance, management has: (1) established an environment in which ``there is an extreme pressure to produce minimally acceptable work at any cost''; (2) ``developed an employee culture where striving to achieve unreasonable production criteria is paramount and . . . actually doing difficult necessary work on cases is strongly discouraged on penalty of removal''; and (3) [management] commonly expresses sentiments such as, ``just decide the case and let the veteran appeal.'' If the assertions in this grievance are true then we call upon VA Central Office management to hold regional office management accountable. Thank you for permitting NVLSP to testify on such an important issue. __________ Kirkland and Ellis LLP Chicago, IL August 29, 2008 Via OVERNIGHT MAIL Joyce A. Cange Director, Cleveland Regional Office Department of Veterans Affairs, Veterans Benefits Administration A.J. Celebrezze Federal Building 1240 East 9th Street Cleveland, Ohio 44199 Re: April 2008 Grievance Of Local 2823 Dear Ms. Cange: I write to advise that Kirkland & Ellis LLP represents Local 2823 of the American Federation of Government Employees (``Local 2823'') in connection with the grievance that Local 2823 filed and amended in April 2008 (the ``Grievance'' or ``GRV''), and I further write to respond to an April 25, 2008 memorandum (the ``Memorandum'' or ``MEM'') from your office that denied the Grievance for various reasons.\1\ --------------------------------------------------------------------------- \1\ Local 2823 does not address herein all the matters raised by its Grievance or assertions made by your office's Memorandum in response, as Local 2823 does not believe that such unaddressed matters need to be resolved at this juncture to have a productive discussion of the principal disputes between the parties. However, Local 2823 does not waive and expressly reserves all rights and claims it has, including those that are not expressly addressed herein but are otherwise fairly encompassed by the Grievance. --------------------------------------------------------------------------- 1. The Grievance Local 2823's Grievance demands resolution of several disputes that Local 2823 believes are negatively affecting the processing of veterans' benefits claims at the Cleveland VA Regional Office (the ``Cleveland VARO''). Specifically, the Grievance states that management of the Cleveland VARO has, among other things: 1. created and encouraged a culture in which the quantity of benefits claims rated is emphasized to the detriment of the quality of the rating of those claims (see GRV at 3-4 para.para. (6)(b)-(d); hereafter, the ``Culture Claim''); \2\ --------------------------------------------------------------------------- \2\ This may be one of the reasons why Ohio ranks second to last nationally in the average compensation awarded to its disabled veterans according to the VBA's Annual Benefits Report for FY2006. See, e.g., The Plain Dealer at A7 (Apr. 12, 2008) (summarizing disparate disability compensation averages across states, with only Indiana having a lower average than Ohio, but having fewer than half of the number of disability recipients as Ohio); accord VBA Annual Benefits Report--Fiscal Year 2006 at 103Sec. 1A0953 (state-by-state figures). --------------------------------------------------------------------------- 2. failed to properly, equitably and promptly implement training initiatives to assure that those reviewing claims are sufficiently trained in the relevant disciplines so as to reduce errors, improve the quality of claims processing, and successfully complete newly implemented certification requirements (see GRV at 2-3 para.para. 5(d)- (e); hereafter the ``Training Claim''); \3\ and --------------------------------------------------------------------------- \3\ See, e.g., 1997 Master Agreement between the Dep't of Veterans Affairs and the Am. Fed'n of Gov't Employees [hereafter, ``MA''] at 142 (Art. 34) Sec. 1(A) (``The Department and the Union agree that the training and development of employees is of critical importance in carrying out the mission of the Department.''); id. Sec. 9 (``Procedures which ensure fair and equitable training opportunities are appropriate subjects for local bargaining.''). See also 1/20/00 VBA-AFGE Mem. of Understanding [hereafter ``1/20/00 MOU''] para. 2 (``VBA commits to a standard of excellence in the quality and quantity of training for all employees.''); id. para. 10 (``Local unions will be given the opportunity to bargain over appropriate issues not otherwise in conflict with this or other national level agreements, prior to local implementation.''); 5/19/06 Mem. of Understanding; Interim Cert. MOU for FY2006 [hereafter ``5/19/06 MOU''] at preamble (``The terms of the original MOU on Certification dated 2000 are still in effect.''); id. para. 15 (``The parties may negotiate locally on this subject provided it does not conflict, interfere with, or impair the implementation of this MOU and the Master Agreement.''). --------------------------------------------------------------------------- 3. failed to properly implement a fair and impartial performance appraisal system that assures that quantitative measures of performance are not emphasized to the detriment of measures of the quality of that performance (see GRV at 4 para. 6(a); hereafter the ``Performance Appraisal Claim'').\4\ --------------------------------------------------------------------------- \4\ See, e.g., Am. Art. 26 Sec. 1(D) (``The parties share an interest in improving the performance of the Department's workforce''); id. Sec. 1(H) (``The performance appraisal process as set forth in this Article is intended to be innovative and evolutionary in nature. Its effectiveness is critical to the Department achieving its mission.''). While recent disputes between management and Local 2823 (and certain of its officers) unfortunately have created an adversarial atmosphere and tone, it is important to underscore and reaffirm that Local 2823's fundamental reason for instituting this Grievance is to improve the quality of the processing of benefits claims by assuring that those responsible for reviewing such claims are (i) encouraged and expected to perform high quality reviews (even where doing so may reduce the quantity of claims that can be reviewed), (ii) sufficiently trained to perform these high quality reviews, and (iii) appraised based both on the quality as well as the quantity of their ratings--all of which, Local 2823 believes, is consistent with and required by VA policy, applicable agreements of the parties, and applicable law. Local 2823 believes that the best way to achieve these objectives is by working collaboratively and cooperatively with management to achieve them.\5\ Accordingly, Local 2823 and the undersigned request that within the next 2 weeks you provide them with a date and time when we can sit down with you and your management team in the next 60 days to see if there is a way that the parties can work together to accomplish the aforementioned objectives and resolve their differences by means of such ADR.\6\ --------------------------------------------------------------------------- \5\ See, e.g., MA at 4 (Preamble) Sec. 2 (``The Department and the Union agree that a constructive and cooperative working relationship between labor and management is essential to achieving the Department's mission and to ensuring a quality work environment for all employees. The parties recognize that this relationship must be built on a solid foundation of trust, mutual respect, and a shared responsibility for organizational success. [para.] Therefore, the parties agree to work together in partnership and through this Master Agreement to identify problems and craft solutions, enhance productivity, and deliver the best quality of service to the Nation's veterans.'') (emphasis added). See also id. at 8 (Art. 3) Sec. 2 (noting important ``Partnership'' principles that are to characterize parties' relationships, including ``Shared responsibility,'' ``Sharing of information,'' ``Reaching joint agreements and making joint recommendations,'' ``Use of alternative dispute resolution, interest-based, problem-solving techniques, and facilitation,'' and ``Integration of interests''--all addressed to the common goal of establishing and improving ``effective Partnerships which are designed to ensure a quality work environment for employees, more efficient administration of VA programs, and improved service to veterans'') (emphasis added); id. Sec. 3 (``The scope of partnership will include issues raised by either party regarding: A. Matters involving personnel policies, practices, and working conditions.''); id. at 15 (Art. 7) Sec. 1 (``Service to the veteran is the cornerstone of the relationship between the Department and employees.''); id. at 177 (Art. 146) Sec. 1 (``The parties recognize that a new relationship between the Union and the Department as full partners is essential for reforming the Department into an organization that works more efficiently and effectively and better serves customer needs, employees, Union representatives and managers.''). \6\ See, e.g., Master Agreement at 13 (Art. 6) Sec. 1 (``Union and Management at all levels should be committed to use of ADR problem- solving methods as a priority to resolve disputed matters.''); id. Sec. 4(A) (``ADR is an appropriate subject matter for local negotiations.''); id. at 14 (Art. 6) Sec. 4(D) (``ADR methods may be used prior to or during a grievance/arbitration or statutory appeal.''); id. at 165 (Art. 42) Sec. 6 (``The use of ADR is encouraged. The parties agree that every effort will be made to settle grievances at the lowest possible level.''); VA Dir. 5023 Sec. 2(a) (``It is the policy of VA to recognize and deal with lawful labor organizations on matters of concern to the employees they represent, and to place primary reliance on informal settlement of any differences or disputes at the earliest stage possible by discussion between VA management and representatives of labor organizations.''). --------------------------------------------------------------------------- If, however, management is unwilling to work collaboratively to achieve these objectives, then Local 2823 has a responsibility to its members (and to veterans) to prosecute the Grievance vigorously by commencing arbitration proceedings, which Local 2823 will not hesitate to do. Moreover, should the matter proceed to arbitration, for the reasons set forth below, we are confident that the arbitrator will find in Local 2823's favor and that your legal and other objections to the Grievance that are recited in the Memorandum will not be sustained. 2. The Culture Claim As set forth in the Grievance, management has: (1) established an environment in which ``there is an extreme pressure to produce minimally acceptable work at any cost''; (2) ``developed an employee culture where striving to achieve unreasonable production criteria is paramount and . . . actually doing difficult necessary work on cases is strongly discouraged on penalty of removal''; and (3) commonly expresses sentiments such as, ``just decide the case and let the veteran appeal.'' (See GRV at 4 para. 6(a); id. para.para. 6(a), (b) & (d).) A. The Culture Claim I Sufficiently Particular The Memorandum your office issued takes exception to the Grievance, inter alia, on the grounds that it is insufficiently particular in that it does not identify: (1) ``any specific acts that are being grieved''; (2) ``any specific times, dates and places for any acts being grieved''; and (3) ``any rationale supporting or even explaining how VBA's alleged conduct violated any negotiated agreements or law.'' (Mem. at 1 para. 3.) The Memorandum, however, notably fails to identify any agreement or law requiring that Local 2823's step-three grievance (see GRV at 1 para. 1) be stated with greater particularity--because there is none. While the Master Agreement recites that a step-two grievance ``must state, in detail, the basis for the grievance and the corrective action desired'' (MA at 166 (Art. 42) Sec. 7 (emphasis added)), no such requirement is prescribed for a step-three grievance like the one at issue here. Regardless, even were the Grievance deemed a step-two grievance (or deemed bound by a similar ``detailed basis'' requirement), the Grievance does ``state, in detail, the basis for the grievance''--e.g., the Culture Claim, the Training Claim, and the Performance Appraisal Claim. Moreover, the Master Agreement does not state that ``all facts'' supporting a grievance must be recited in the grievance in order to satisfy the ``detailed basis'' requirement of Article 42 Sec. 7; it does not require that all ``acts'' encompassed by the grievance be separately listed; it does not require specification of times, dates, and places for all acts encompassed by a grievance; and it does not require an explanation of each of the ways in which the conduct at issue violated each applicable agreement and/or law. Nor would it be reasonable to impose such a particularized pleading requirement on Local 2823--before any documents have been turned over by management relating to the claims pled as the basis for the Grievance. B. The Culture Claim Is Based on Violations of Applicable VA Policies, Agreements, and/or Law Actions of management giving rise to the Culture Claim violate VA policy as well as applicable agreements and law. First, VA Directive 5023 properly states and acknowledges that ``[t]he public interest demands the highest standards of employee performance and the continued development and implementation of modern and progressive work practices to facilitate and improve employee performance and the efficient accomplishment of the operation of the Government.'' See VA Dir. 5023 Sec. 2(b). Further, VA Directive 5023 provides that ``VA management shall carry out its duties in a manner consistent with the terms and spirit of human resources policies, principles and procedures that encourage the highest standard of employee performance and the most efficient accomplishment of VA operations.'' Id. Sec. 2(c) (emphasis added). Thus, the VBA violates its own policy by promoting a working culture that discourages the highest standard of employee performance so as to more expeditiously process claims. Moreover, the VBA further violates its policy respecting the efficient accomplishment of VA tasks by promoting a culture that likely results in more claims decisions that are vulnerable to reversal on appeal and re-processing on remand--a less efficient manner of proceeding. Second, as set forth in Article 7 Sec. 1 of the Master Agreement, ``[s]ervice to the veteran is the cornerstone of the relationship between the Department and employees,'' and the ``parties recognize the importance of a strong commitment to a comprehensive Total Quality Improvement Program (TQI) in the Department.'' (Emphasis added). Thus, discouraging quality reviews of benefits claims in the name of processing a greater quantity of benefits claims violates the service- to-the-veteran imperative that is to inform the work of both the Department and its employees and ignores the importance of quality considerations to the reviews undertaken--both of which violate the letter and spirit of Article 7 Sec. 1 of the Master Agreement. (See also Am. Art. 26 Sec. 1(A) (``The Department will strive for continuous improvement in performance to fulfill the Department's commitment to providing quality customer service.'') (emphasis added); Agreement Between Dep't of Veterans' Affairs Cleveland Regional Office and AFGE Local 2823 [hereafter ``Local Agreement'' or ``LA''] Sec. 1(A) (``The mission of the Department of Veterans Affairs and this Regional Office is to service the veteran and beneficiaries with timely and quality service.'') (emphasis added).) Further, pursuant to Article 43 Sec. 3(B) of the Master Agreement, management has an obligation to bargain locally with Local 2823 as to, inter alia, the ``methods and means of performing work.'' Thus, by refusing to bargain over the Culture Claim, which is squarely addressed to the Cleveland VARO's methods and means of performing work, the Cleveland VARO is abrogating this provision of the Master Agreement as well. Third, an atmosphere that promotes quantity at the expense of quality is fundamentally contrary to the Department's statutory obligations to improve the quality of the claims rating process as well as increase the quantity of claims processed. See, e.g., 38 U.S.C. Sec. 7734(1)(E) (requiring an annual report to Congress that includes, inter alia, ``actions taken to improve the quality of services provided and the results obtained'') (emphasis added); id. Sec. 7734(2) (requiring an annual report to Congress that includes, inter alia, ``information with respect to the accuracy of decisions, including trends in that information.''). C. Local 2823's Culture Claim Has Evidentiary Support There is ample evidentiary support for Local 2823's Culture Claim, both nationally and locally. First, recently completed national studies and surveys have confirmed the perception among many claims ratings personnel nationwide that the Department has fostered an atmosphere in which quantitative objectives are pursued to the detriment of qualitative objectives. For example, in a recent report issued by the Veterans' Disability Commission in October 2007, they noted the following: In respect of the criticism concerning balancing quality and quantity in employee performance, CNAC \7\ discovered there exists a perception that VA emphasized quantity over quality. In a national survey, 80 percent of raters said having enough time to process a claim was one of their top three challenges. They were also asked to rate the availability of time to decide a claim, 54 percent of raters said availability of time was fair or poor. It can be argued that this creates incentives for RVSRs to make decisions that are not always fully backed by evidence, which leads to more appeals, and remands, and increases backlogs in the system. . . . --------------------------------------------------------------------------- \7\ ``CNAC'' is a shorthand reference to the CNA Corporation, which conducted scientifically valid and reliable surveys. See 10/07 VDBC Report at 19 (``The Commission also examined the results of studies undertaken on its behalf by the CNA Corporation (CNAC). . . . Additionally, CNAC surveyed VA raters, service officers from veterans service organizations, and disabled veterans and survivors. These surveys were scientifically valid and reliable.''). --------------------------------------------------------------------------- * * * CNAC believes that the VA's training difficulties are made exponentially worse because staff feel a need for more training and that training seems to be sacrificed to meet work quotas. This emphasis has encouraged a high staff turnover at VA. The quality of claims is lessened since inexperienced individuals are taking over for experienced raters. Veterans Disability Benefits Comm'n, Honoring The Call To Duty: Veterans' Disability Benefits In The 21st Century [hereafter, ``10/07 VDBC Report''] 342 (Oct. 2007) (emphasis added).\8\ --------------------------------------------------------------------------- \8\ Local 2823 further notes that it believes the failure of the Cleveland VARO to share the results of the aforementioned study with Local 2823 was a violation of Master Agreement Article 46 Sec. 8(C), which on relevant part provides that ``[i]f a third party conducts a survey and the results are distributed to the Department, the results will be shared with the Union.'' See also Dep't of Veterans Affairs (VA) FY2007 Performance and Accountability Report [hereafter, ``11/07 VA Report''] 78 (Nov. 15, 2007) (``The Veterans' Disability Benefits Commission began work in May 2005 and issued its report in October 2007. VA will study the Commission's recommendations and begin taking appropriate actions in 2008.''). Moreover, in recent testimony to Congress, the Deputy Under Secretary for Benefits, Michael Walcoff, acknowledged that current quantitative measures of productivity, which do not take into account the complexity of benefits claims and numbers of issues raised by a --------------------------------------------------------------------------- claimant, are in need of revision: To further enhance our ability to monitor performance, the study team recommends the creation of a performance measurement system focused on tracking the number of medical disabilities or issues claimed. IBM believes that this issue-based performance measurement system, in conjunction with the existing claim-based performance measurement system, will result in a more accurate and detailed measure of productivity and workload. Under the current claim-based performance measurement system, a regional office is given the same credit for completing a claim with one issue as a claim with forty issues. The study team believes that measuring work output by both number of claims and number of issues at an organizational level is a more accurate assessment of a regional office's productivity. In addition, an issued-based performance measurement at an individual level will provide more specificity in the activities of staff and result in increased accountability overall. [para.] VBA agrees with the idea of adding an issue-based performance measurement system to our current reporting structure. February 18, 2008 Statement of Michael Walcoff Before The House Committee On Veterans' Affairs Subcommittee On Disability Assistance And Memorial Affairs, Examining The VA Claims Processing System (emphasis added). Second, in addition to such national evidence supporting Local 2823's Culture Claim, Local 2823 will be prepared at the arbitration hearing to present evidence of specific instances of actions taken by Cleveland VARO management contributing to the establishment and promotion of a culture in which the quantity of ratings decisions is valued to the detriment of quality. For example, VSR production standards at the Cleveland VARO are set at 10 points per day, 25 percent higher than the national standard of 8 points per day. Also, rating specialists at the Cleveland VARO only receive production credit for cases they have decided, and not for reviewing cases that are sent back to VSRs for further development. This means that RVSRs are incentivized to decide cases without correcting errors in the file instead of ``losing time'' while they rework cases that are procedurally or developmentally flawed. D. Local 2823's Clarified and Revised Information Requests Addressed to the Culture Claim are Proper In addition, as noted in the Grievance, Local 2823 requests information from the Cleveland VARO that is normally maintained, reasonably available, and necessary for Local 2823 to fulfill its representational functions and responsibilities with respect to the prosecution of the Culture Claim, and, to that end clarifies and revises its requests for information, hereby requesting production of the following information created on or after January 20, 2000: Any communications between and among Cleveland VARO management and supervisors respecting efforts to achieve quantitative production requirements; Any communications between and among Cleveland VARO supervisors and employees respecting efforts to achieve quantitative production requirements; Documents sufficient to identify the numbers of employees who have received awards, bonuses, and/or promotions for achieving or exceeding quantitative production requirements; Documents sufficient to identify the numbers of supervisors who have received awards, bonuses, and/or promotions for achieving or exceeding quantitative production requirements; Documents sufficient to identify the numbers of employees (if any) who have received awards, bonuses, and/or promotions for achieving or exceeding qualitative performance standards; Documents sufficient to identify the numbers of supervisors (if any) who have received awards, bonuses, and/or promotions for achieving or exceeding qualitative performance standards; Documents sufficient to identify the number of employees who have received demotions, non-satisfactory ratings, or other disciplinary actions or negative performance ratings for their failure to achieve or exceed quantitative production requirements; Documents sufficient to identify the number of supervisors who have received demotions, non-satisfactory ratings, or other disciplinary actions or negative performance ratings for their failure to achieve or exceed quantitative production requirements; Documents sufficient to identify the number of employees (if any) who have received demotions, non-satisfactory ratings, or other disciplinary actions or negative performance ratings for their failure to achieve or exceed qualitative performance standards; Documents sufficient to identify the number of supervisors (if any) who have received demotions, non-satisfactory ratings, or other disciplinary actions or negative performance ratings for their failure to achieve or exceed qualitative performance standards; Any communications from employees in the Cleveland VARO to supervisors or other Cleveland VARO management complaining about quantitative production requirements and their effect on the quality of the claims processing; Any communications from supervisors in the Cleveland VARO complaining about quantitative production requirements and their effect on the quality of the claims processing; Any statistical information tracking the quantitative performance of the Cleveland VARO in terms of processing clams; Any statistical information tracking the quality of the Cleveland VARO in terms of processing claims; and Any statistical information respecting the number and rate of Cleveland VARO claims determinations that are reversed or remanded on appeal. Moreover, it is plain that such information respecting the treatment of employees and supervisors with respect to the achievement of quantitative production requirements as compared to the treatment of employees and supervisors with respect to the achievement of qualitative performance standards is both relevant and necessary to understand and assess the working culture created by management favoring quantitative attainments over qualitative achievements. (See MA at 177 (Art. 46) Sec. 5 (``The Department agrees to provide the Union, upon request, with information that is normally maintained, reasonably available, and necessary for the Union to effectively fulfill its representational functions and responsibilities. This information will be provided to the Union within a reasonable time and at no cost to the Union.'').). See also AFGE Local 1345 v. Fed'l Labor Relations Auth., 793 F.2d 1360 (DC Cir. 1986) (Union entitled under 5 U.S.C. Sec. 7114 to obtain information regarding two employees who had been dismissed from jobs within union's bargaining unit upon request for information from employer, as union's status as bargaining representative required it to have access to information to assess its responsibility, including information regarding dismissal of unit employees).\9\ Thus, Local 2823 expects to use the aforementioned information to evidence a consistent and longstanding emphasis on and encouragement of the attainment of quantitative production, with little or no concomitant emphasis or regard for the attainment of qualitative performance standards. --------------------------------------------------------------------------- \9\ While the Memorandum asserts that such information requests need be answered only insofar as the requirements of 5 U.S.C. Sec. 7114 are met, contending that Article 46 Sec. 5 of the Master Agreement merely restates 5 U.S.C. Sec. 7114, section 5 does not so state, nor is its text as limited as the provisions of 5 U.S.C. Sec. 7114 that are addressed solely to data ``which is reasonably available and necessary for full and proper discussion, understanding, and negotiation of subjects within the scope of collective bargaining.'' Cf. NAGE Local R14-143, 55 FLRA 317 (1999) (Chair Segal, Concurring & Dissenting In Part) (dissenting from majority finding that Union conceded statutory and contractual rights were the same, noting that ``such concession appears particularly unlikely in view of the fact that Article 11 makes no reference to, and does not otherwise restate, the Statute'') (noting that where provision merely reiterates statute, then authority need only take care to assure that contractual interpretation is not inconsistent with statute). We further note that even if the Cleveland VARO had legitimate grounds to withhold some of the information requested, that does not mean it can withhold all information requested. See generally AFGE Local 2263 v. Fed'l Labor Relations Auth., 454 F.3d 1101 (10th Cir. 2006). --------------------------------------------------------------------------- 3. The Training Claim As set forth in the Grievance, management has: (1) failed to honor its commitment ``to a standard of excellence in the quality and quantity of training for all employees'' (1/20/00 MOU para. 2), ``as evidenced by the low pass rates of employees on previous tests and great disparities between . . . veterans served by different regional offices''; (2) failed ``to provide training that significantly furthers the employee's knowledge, skills and abilities to serve veterans''; (3) provided ``ongoing training that . . . bears little resemblance to the training described in the January 20, 2000 MOU''; (3) failed to rotate employees in a manner that would enable them ``to gain and maintain proficiency in all aspects of their job''; (4) disadvantaged employees seeking promotion by failing to adequately train them to process claims, thereby preventing them from successfully competing for and obtaining promotions, bonuses, awards, advances, and other merit-based compensation and/or benefits. (GRV at 2-3 para.para. 5(d), (1), (2), (3), 5(e).) A. The Training Claim is Sufficiently Particular For the same reasons that Local 2823's Culture Claim has been stated with sufficient particularity in the Grievance, its Training Claim has been stated in a sufficiently particular manner. Again, your office's Memorandum does not identify any authority requiring a more particularized statement of the facts underlying the Training Claim at this juncture, and we are aware of none. B. The Training Claim is Based on Violations of Applicable VA Policies and/or Agreements Like the Culture Claim, Local 2823's Training Claim is based on violations of VA policies, applicable agreements, and applicable law. First, as noted by Deputy Under Secretary for Benefits, Ronald R. Aument, in a statement to Congress last October, training is important, because ``[c]ritical to improving claims accuracy and consistency is ensuring that our employees receive the essential guidance, materials, and tools to meet the ever-changing and increasingly complex demands of their decision-making responsibilities.'' October 16, 2007 Statement of Ronald R. Aument, Deputy Under Secretary For Benefits, Before The House Veterans' Affairs Subcommittee On Oversight And Investigations [hereafter, 10/16/07 Aument Testimony]. As summarized by Under Secretary Aument, the training regimen that the VA is supposed to follow is intended to be centralized, standardized, and comprehensive: [1] New hires receive comprehensive training and a consistent foundation in claims processing principles through a national centralized training program called ``Challenge.'' [2] After the initial centralized training, employees follow a national standardized training curriculum (full lesson plans, handouts, student guides, instructor guides, and slides for classroom instruction) available to all regional offices. Standardized computer-based tools have been developed for training decision- makers (71 courses completed and an additional 5 in development). Training letters and satellite broadcasts on the proper approach to rating complex issues are provided to the field stations. [3] In addition, a mandatory cycle of training for all Veterans Service Center employees has been developed consisting of an 80-hour annual curriculum. 10/16/07 Aument Testimony. And the VA's most recent annual report again reiterates that ``[t]raining remains a priority. . . . '' 11/07 VA Report at 199. But notwithstanding these published policies and statements, and as set forth below, Local 2823 has reason to believe that the training provided by the Cleveland VARO: (1) does not provide new hires with ``comprehensive training and a consistent foundation in claims processing principles''; (2) does not assure that incumbent employees follow a ``national standardized training curriculum'' that is made equally and fully available to all within the Cleveland VARO; and (3) does not assure that all employees receive the full cycle of training and complete an 80-hour curriculum each year. Second, the applicable agreements similarly make clear that training is critical and the Department is responsible for providing it on a fair and equitable basis. Specifically, Master Agreement Article 34 Sec. 1(A) provides as follows: The Department and the Union agree that the training and development of employees is of critical importance in carrying out the mission of the Department. In recognition of this, the Department will provide training and career development opportunities to employees of the bargaining unit. The Department is responsible for ensuring that all employees receive the training necessary for the performance of the employees' assigned duties. (MA at 142 (Art. 34) Sec. 1(A) (emphasis added).) Moreover, the Master Agreement requires fair and equitable administration of training among employees. (See, e.g., MA at 143 (Art. 34) Sec. 3(C) (``When resources for training are limited, approval for training funds will be based on fair criteria that are equitably applied.'') (emphasis added).) Further, the Master Agreement requires the Department to inform employees, at least annually, about training opportunities, policies, and nomination procedures. (See id. at 143 (Art. 34) Sec. 6(A) (``The Department shall inform employees, at least annually, about Department training opportunities, policies, and nomination procedures. Upon request, the Department will advise individual employees of training opportunities that meet identified educational or career objectives.''). Finally, and perhaps most importantly, the Master Agreement makes clear that such training matters are appropriate subjects for local bargaining, stating: ``[p]rocedures which ensure fair and equitable training opportunities are appropriate subjects for local bargaining.'' (MA at 144 (Art. 34) Sec. 9 (emphasis added).\10\) --------------------------------------------------------------------------- \10\ See also MA at 172 (Art. 44) Sec. 4(A) (``On all policies and directives or other changes for which the Department meets its bargaining obligation at the national level, appropriate local bargaining shall take place at individual facilities and may include substantive bargaining that does not conflict with negotiated national policy and agreements.''); id. at 172-73 (Art. 44) Sec. 4(B) (``Proposed changes in personnel policies, practices, or working conditions affecting the interests of one local Union shall require notice to the President of that local. Proposed changes in personnel policies, practices, or working conditions affecting the interests of two or more local Unions within a facility shall require notice to a party designated by the NVAC President with a copy to the affected local Unions. Proposed changes in personnel policies, practices, or working conditions affecting the interests of more than one facility shall require notice to a party designated by the national VA Council President.''); id. at 169 (Art. 43) Sec. 1 (``Recognizing that the Master Agreement cannot cover all aspects or provide definitive language for local adaptability on each subject addressed, it is understood that Local Supplements may include substantive bargaining on all subjects covered in the Master Agreement so long as they do not conflict, interfere with, or impair implementation of the Master Agreement. However, matters that are excluded from Local Supplemental bargaining will be identified within each Article.'') (emphasis added); id. Sec. 2(A) (``The Local Supplemental Agreement may cover all negotiable matters regarding conditions of employment insofar as they do not conflict with the Master Agreement as defined in Section 1. . . . Note: This is not intended to preclude local bargaining of items that are not covered by the Master Agreement, i.e., policies, procedures and directives initiated at the facility level or national level.'') (emphasis added); id. Sec. 2(B) (``In the event either of the national parties determines there exists a conflict with the Master Agreement, they shall forward a written document to the respective local and the other national party identifying the conflict for resolution at the local level.''). These provisions of the Master Agreement and similar provisions in the 1/20/00 MOU make clear that contrary to the assertions in the Memorandum (see Mem. at 2-3 para.para. 5(a)-(c)), the VA's belief that it has met its bargaining obligations at the national level do not relieve it of the obligation to bargain locally as to the matters put at issue by Local 2823's Grievance. Yet once again, as set forth below, Local 2823 has reason to believe that the Cleveland VARO is not honoring its training and bargaining obligations under the Master Agreement, inasmuch as the Cleveland VARO: (1) has not provided training and career development opportunities to all employees; (2) has not ensured that all employees have received the training necessary for the performance of their assigned duties; (3) has not used fair criteria, equitably applied, to assure that training is appropriately distributed among and between employees; (4) has not advised individual employees who request such information of training opportunities that meet identified educational or career objectives, including, but not limited to, VSR and/or RVSR certification; and (5) has refused to bargain with Local 2823 to ensure that fair and equitable training opportunities are made available to all employees. Moreover, as regards training related to certification of VSR and RVSR personnel, the parties 1/20/00 MOU provides that the Department has a number of obligations, relating to training, reciting in relevant part the following: ``VBA commits to a standard of excellence in the quality and quantity of training for all employees. We will ensure training programs, which are the core and prerequisite to certification, are complete and sufficient to provide employees the necessary tools to become certified. There will be a direct relationship between the training program and certification.'' (1/20/00 MOU para. 2 (emphasis added)). ``Employees will proceed through standardized training such as the TPSS program, which may include pre and post tests for the purpose of determining the efficacy of training. Training will include mentoring, on-the-job-training and ongoing feedback.'' (1/20/00 MOU para. 3 (emphasis added).) ``Where incumbent employees have not been performing the full range of duties due to specialization or for other reasons, at the employees request, we will ensure that they are provided training sufficient for them to participate in the certification program.'' (1/ 20/00 MOU para. 6 (emphasis added).) ``A copy of this MOU will be furnished to the Local President of all VBA facilities represented by AFGE. Local unions will be given the opportunity to bargain over appropriate issues not otherwise in conflict with this or other national level agreements, prior to local implementation.'' (1/20/00 MOU para. 10 (emphasis added).) However, once again, as set forth below the Cleveland VARO has violated its obligations under the 1/20/00 MOU, because: (1) the quality and quantity of training made available to employees does not rise to the promised standard of excellence; (2) training programs are not complete and sufficient to provide employees the necessary tools to become certified; (3) employees have not proceeded through standardized TPSS training, and the training generally has not included mentoring, on-the-job training or ongoing feedback; (4) incumbent employees who have not been performing the full range of duties due to specialization or other reasons have not been provided training sufficient for them to participate in the certification program; and (5) management has failed and refused to bargain over these issues. Finally, it is important to note that while your office's Memorandum asserts that it is not obligated to bargain with Local 2823 regarding these training matters relating to VSR and RVSR certification in light of certain alleged positions taken by the national Union in negotiations with the Department, those assertions ignore the facts that: (1) the matters for which Local 2823 has sought to bargain are not the same as those put at issue by the dispute at the national level (and the Memorandum makes no attempt to explain why they are); (2) regardless of the resolution of VA's dispute with the national Union, that will not resolve the training issues for which Local 2823 has sought to bargain locally; (3) the Memorandum's assertions regarding the exclusive recognition of the national Union ignore that the Department agreed with the national Union that local bargaining would still occur and the appropriate VARO would still be obligated to engage in such local bargaining pursuant to the applicable provisions of the Master Agreement.\11\ --------------------------------------------------------------------------- \11\ We further note in this regard that the Memorandum's citation to U.S. Food & Drug Adm., Northeast & Mid-Atl. Regions v. AFGE Council No. 242, 53 F.L.R.A. 1269 (1998) is misplaced and in fact supports Local 2823's position here. Notably, in relevant part U.S. Food & Drug states, ``[a] representative with a collective bargaining relationship in a consolidated bargaining unit is not required to bargain locally with individual components that make up the consolidated unit unless such bargaining has been agreed to at the consolidated level.'' Id. at 1274 (emphasis added); id. (``Parties to a national, consolidated bargaining unit may, and often do, authorize local components to bargain supplemental and other agreements over particular subjects or in particular circumstances.''). And here, it is plain that in the Master Agreement and 1/20/00 MOU, the Department and national Union agreed that local bargaining would continue to be available unless expressly proscribed by the Master Agreement. See supra n.9. --------------------------------------------------------------------------- C. The Training Claim Has Evidentiary Support Once again, there is ample evidentiary support for Local 2823's Training Claim, both nationally and locally. First, both the Office of the Inspector General and third-party studies have found that disparities in training across offices account for at least some of the differences across offices with respect to the average amount of compensation awarded to a claimant. For example, the Deputy Inspector General, in a statement to Congress, represented that ``the degree of rater subjectivity can be influenced by . . . the amount of training and rater experience.'' October 16, 2007 Statement Of Jon A. Wooditch, Deputy Inspector General, Before The Subcommittee On Oversight And Investigations Committee On Veterans' Affairs--U.S. House Of Reps. Hrng On Disability Claims Ratings & Benefits Disparities Within The VBA [hereafter, ``10/16/07 Wooditch Testimony'']. Moreover, variations in training across offices also were identified by the Institute for Defense Analyses (``IDA'') in its study, Analysis of Differences in VA Disability Compensation (hereafter, the ``IDA Study''), as a significant factor responsible for differences in compensation awarded across offices. See, e.g., Hope Yen, AP, Washington Post (July 19, 2007) (``But the study released to AP found that roughly one-third of the problems could be blamed on poor VA standards and inadequate training.'').\12\ Indeed, one of the recommendations of the IDA Study was to ``[s]tandardize initial and on- going training for rating specialists'' 10/16/07 Aument Testimony. Thus, this national evidence suggests that the Cleveland VARO is not receiving training commensurate with that made available to raters in other offices. --------------------------------------------------------------------------- \12\ The IDA study does not appear to be publicly available at this juncture. --------------------------------------------------------------------------- Moreover, the current insufficiency of training programs and policies has been confirmed by the recent report of the Veterans' Disability Benefits Commission, which states: VBA regional office staff must receive adequate education and training. Quality reviews should be performed to ensure these frontline workers are well versed to rate claims. Adequate resources must be appropriated to hire and train these workers to achieve a manageable claims backlog. (10/07 VDBC Report 338 (Recommendation 9.5).) Second, as set forth in the Grievance, there are numerous instances of the Cleveland VARO failing to implement equitably, promptly, and appropriately training. For example, as a result of the Claims Process Improvement teams that have been formed to focus on specific problem areas, employees are not cycled through all areas and do not receive on-the-job training in many areas before they are required to sit for certification exams covering all areas. (See GRV at 2 Sec. 5(3).) Moreover, training that has been provided has not been comprehensive, but instead has largely been addressed to the specific tasks assigned an employee at the time of hire. (See id.) The initial training for VSRs in the Cleveland VARO is focused on the tasks performed by the team to which the VSRs have been assigned, either pre-development or post-determination. Then, 85 percent-90 percent of the training that the VSRs receive on the job consists of overview courses that also cover tasks they perform in their current jobs. This training regimen leaves VSRs unprepared for their new duties when they switch teams and the quality of their work (and their production) suffers. In addition to these problems with VSR training, the Cleveland VARO also fails to train RVSRs appropriately. At the time the grievance was filed, few RVSRs in the office had completed the TPSS training modules that are required for first-year rating specialists. Instead of addressing this deficiency, the Cleveland VARO ignores it: three rating specialists were promoted in October 2007, even though they had failed to complete the basic compensation training module. D. Local 2823's Clarified and Revised Information Requests Addressed to the Training Claim are Proper In addition, as noted in the Grievance, Local 2823 requests information from the Cleveland VARO that is normally maintained, reasonably available, and necessary for Local 2823 to fulfill its representational functions and responsibilities with respect to the prosecution of the Training Claim, and, to that end clarifies and revises its requests for information, hereby requesting production of the following information created on or after January 20, 2000 (except as otherwise indicated): Documents sufficient to show the number of employees (including VSRs and RVSRs) who have received training in the Cleveland VARO, the amount of training that they have received, and the types of training that they have received in each of the fiscal years 2002, 2003, 2004, 2005, 2006, 2007, and 2008 (to date); Documents sufficient to show all plans and programs for training employees (including VSRs and RVSRs) at the Cleveland VARO or elsewhere for the fiscal years 2002, 2003, 2004, 2005, 2006, 2007, and 2008 (to date); All communications from employees complaining, praising or otherwise evaluating the amount, content, type, availability, or other aspects of training provided or made available to employees; All studies, audits, and/or investigations by the VA and/ or third parties (including, but not limited to, the IDA Study) addressing the adequacy of training made available to and/or required of employees at the Cleveland VARO and nationally (including, but not limited to, VSRs and RVSRs); Documents sufficient to show all training that is required and/or suggested for VSR and/or RVSR certification, including, but not limited to, copies of instructions for RVSR and/or VSR national training requirements; Documents sufficient to show all on-the-job training accomplished and its relationship to certification of VSRs and/or RVSRs; A copy of the curriculum formally identified as VSR ``Readiness Training''; A copy of any training materials respecting ``the Candidate Guide''; A copy of any ``Boot Camp'' test or similar Cleveland VARO practice test; A copy of all materials found at the following website address: http://cptraining.vba.va.gov/C&PTraining/VSR/VSRCerTng/ VSRCertCurriculum.htm; A copy of the link to the VSR Certification Training Guide that has been provided to each employee in the Cleveland VARO who is eligible for certification; and A copy of all materials found at the following website address: http://cptraining.vba.va.gov/C&PTraining/VSR/VSRCertTng/ Documents/VSRCertTrainingGuide.pdf. Further, it is plain that production of the requested training information is relevant and necessary to Local 2823's prosecution of its Training Claim, as the information will be used to establish the specific departures of the Cleveland VARO from applicable training requirements, agreements, and policies. 4. The Performance Appraisal Claim In light of the foregoing, and as set forth in the Grievance, ``management has abrogated its duty to maintain a fair and impartial performance appraisal system under multiple sections of Master Agreement 26,'' and has otherwise abrogated VA policy and guidance in its application of the existing performance appraisal system. (GRV at 4 para. 6(a).) A. The Performance Appraisal Claim is Sufficiently Particular For the same reasons that Local 2823's Culture and Training Claims have been stated with sufficient particularity in the Grievance, Local 2823's Performance Appraisal Claim has been stated in a sufficiently particular manner. Again, your office's Memorandum does not identify any authority requiring a more particularized statement of the facts underlying the Performance Appraisal Claim at this juncture, and we are aware of none. B. The Performance Appraisal Claim is Based on Violations of Applicable VA Policies and/or Agreements Local 2823's Performance Appraisal Claim is based on violations of applicable VA policies and/or agreements. First, as set forth in VA Directive 5023, ``[t]he public interest demands the highest standards of employee performance and the continued development and implementation of modern and progressive work practices to facilitate and improve employee performance and the efficient accomplishment of the operation of the Government.'' See VA Dir. 5023 Sec. 2(b). Further, as VA Directive 5023 also states, ``VA management shall carry out its duties in a manner consistent with the terms and spirit of human resources policies, principles and procedures that encourage the highest standard of employee performance and the most efficient accomplishment of VA operations.'' See id. Sec. 2(c) (emphasis added). Management has abrogated both of these VA imperatives, however, by implementing a performance appraisal system that, for the reasons set forth above and below: (1) does not promote the ``highest standards of employee performance and the continued development and implementation of modern and progressive work practices to facilitate and improve employee performance and the efficient accomplishment of'' VA operations; and (2) is fundamentally inconsistent ``with the terms and spirit of human resources policies, principles and procedures that encourage the highest standard of employee performance and the most efficient accomplishment of VA operations.'' Second, as noted in the Grievance, management's implementation of the existing performance appraisal system also contravenes applicable provisions of Article 26 of the Master Agreement. Specifically, the relevant provisions of Article 26 (as amended) that are abrogated by management's implementation and application of the existing performance appraisal system include the following: ``In its entirety and application, the performance appraisal process will to the maximum extent feasible, be fair, equitable, and strictly related to job performance as described by the employee's job description.'' [Am. Art. 26 Sec. 3(A) (emphasis added).] ``Performance appraisals shall be fair and objective.'' [Id. Sec. 3(C) (emphasis added)] ``The union may provide input into any changes to performance standards and/or establishment of new performance standards.'' [Id. Sec. 5(A).] ``Performance standards and elements to the maximum extent feasible shall be reasonable, realistic, attainable, and sufficient under the circumstances to permit accurate measurement of an employee's performance, and adequate to inform the employee of what is necessary to achieve a `Fully Successful' level of achievement.'' [Id. Sec. 5(C) (emphasis added).] ``The Union shall be given reasonable written advance notice . . . when Management changes, adds to, or establishes new elements and performance standards. Prior to implementation of the above changes to performance standards, management shall meet all bargaining obligations.'' [Id. Sec. 5(E).] ``Normally, elements are not weighted or assigned different priorities. However, the Department will inform the employee, at the time the elements and standards are communicated, whether aspects of any job elements are to be accorded different priority. If the elements, standards, or priority changes, that change(s) will be communicated to the employee when it becomes effective.'' [Id. Sec. 5(H).] Notwithstanding these obligations, however, management has breached the foregoing provisions of Article 26 of the Master Agreement by, inter alia: (1) applying the existing performance appraisal system in a manner that is unfair, inequitable, and almost exclusively related to the achievement of quantitative production targets; (2) establishing certification and related requirements that are now part of the performance appraisal while refusing any input from Local 2823 respecting the content and implementation of the same; (3) failing to give reasonable written advance notice to Local 2823 respecting the implementation of newly established certification and related training requirements; (4) weighting quantitative production measures more highly than accuracy measures without formally apprising employees of the same; (5) utilizing statistical production data to evaluate individual performance in a manner that is unreliable, invalid, unfair and inequitable because it fails to take into account the number of issues raised by claims and the complexity of the claims process. C. The Performance Appraisal Claim has Evidentiary Support Like its other claims, Local 2823's Performance Appraisal Claim also is amply supported both nationally and locally. First, as adverted to in Deputy Under Secretary Wolcoff's recent testimony to Congress this past February, even independent third-party consultants have found problematic the existing performance appraisal system's use of quantitative production criteria that fail to account for the numbers of issues raised and complexity of claims. (See 2/14/08 Walcoff Testimony.) And, perhaps more importantly, in testifying before Congress, the Veterans' Benefits Administration (``VBA'') has stated that it ``agrees with the idea of adding an issue-based performance measurement system to our current reporting structure.'' (Id.) Further, as noted above, the Veterans' Disability Benefits Commission similarly found that greater attention to quality should be paid in assessing the performance of employees. (See 10/07 VDBC Report 338.) Second, in terms of the Cleveland VARO, the existing performance appraisal system applied by management again penalizes raters who take the time to assure accurate ratings of multi-issue and complex claims and favors those who are less accurate but meet or exceed applicable production quotas. For example, a case with a minimal amount of evidence, such as a well-documented knee injury, is given the same production credit as a multi-volume multi-issue case for PTSD. Raters are therefore tacitly (and sometimes overtly) encouraged to avoid processing difficult claims. A different problem is the older case whose record has been developed in several stages and requires follow- up with several treating physicians. If an RVSR spends 1 hour with the case and discovers that a private treatment record identified by the veteran is missing, the RVSR should defer a decision on that case. The RSVR, however, does not receive any production credit under the performance plan for deferring this decision. Instead, the performance plan gives the RVSR the stark choice of ignoring the missing evidence and deciding the case or (correctly) deferring the case and absorbing the entire time spent with that case as lost. D. Local 2823's Information Requests Addressed to the Performance Appraisal Claim are Proper In addition, as noted in the Grievance, Local 2823 requests information from the Cleveland VARO that is normally maintained, reasonably available, and necessary for Local 2823 to fulfill its representational functions and responsibilities with respect to the prosecution of the Performance Appraisal Claim, which requests are encompassed by the requests set forth above with respect to the Culture and Training Claims. Again, the requested information is both relevant and necessary, as it will be used by Local 2823 to assess the manner in which the existing performance appraisal system has been applied by the Cleveland VARO and to demonstrate the ways in which the current system violates VA policy and provisions in Article 26 of the Master Agreement. 5. The Veterans' Disability Benefits Claims Modernization Act of 2008, H.R. 5892 Lastly, before closing, we wish to note that one of the reasons that Local 2823 believes a meeting to discuss the parties' differences is advisable at this juncture is that pending legislation may--at least in Local 2823's view--moot many of these disputes. In particular, the Veterans' Disability Benefits Claims Modernization Act Of 2008, H.R. 5892, 110th Cong. (2d Sess. 2008) (the ``Proposed Legislation'') was referred to the House Committee on Veterans' Affairs in April 2008, on July 29, 2008, an amended bill was favorably reported out of Committee to the House floor, and on July 30, 2008, the amended bill was passed by the House on a roll-call vote of 429-0. See 154 Cong. Rec. H7256-H7263 (daily ed. July 29, 2008); 154 Cong. Rec. H7518 (daily ed. July 30, 2008). If enacted in its current form, the Proposed Legislation would: (1) require the Department to engage a third-party to annually assess the quality of claims processing across the VAROs (see Proposed Legislation Sec. 106), thereby potentially addressing concerns giving rise to Local 2823's Culture Claim; (2) require the Department to study and develop new certification standards and programs after obtaining appropriate input from employees and their representatives (see id. Sec. 105), thereby potentially addressing concerns giving rise to a portion of Local 2823's Training Claim; (3) require the Department to study and evaluate training made available to employees (see id. Sec. 106(a)(3)), thereby potentially addressing concerns giving rise to another portion of Local 2823's Training Claim; and (4) study and implement new performance standards that would place greater emphasis on the quality of ratings decisions and less emphasis on the quantity of them (see id. Sec. 103), thereby potentially addressing the concerns giving rise to Local 2823's Performance Appraisal Claim.\13\ --------------------------------------------------------------------------- \13\ Of course, the introduction of the Proposed Legislation merely confirms that Local 2823's Culture, Training and Performance Appraisal Claims are well-founded and raise issues of concern that are shared by employees in other regional offices and Members of Congress. See, e.g., Statement of Congressman Hare In Support Of H.R. 5892, 154 Cong. Rec. H7262 (daily ed. July 29, 2008) (``The largest factors contributing to the claims backlog are the broken culture and processes at the VA. There is a lack of accountability on raters, poor quality assurance measures, a broken work credit system, virtually no training for the VBA personnel, and an outdated information technology system. [para.] H.R. 5892 squarely addresses these problems by creating a more accountable and accurate system that rewards raters for the quality of their work, and it holds them accountable for their mistakes, ensuring that claims are processed correctly the very first time.'') (emphasis added). --------------------------------------------------------------------------- Accordingly, any information that management of the Cleveland VARO could provide to Local 2823 respecting the Department's position with respect to the Proposed Legislation--including whether the Department opposes, supports, or partially opposes and partially supports the Proposed Legislation--would be beneficial. * * * Thank you for your prompt attention to the foregoing matters, and I look forward to a response from your office on or before September 15, 2008, so that the parties may quickly reach agreement on a date when representatives of management and Local 2823 may sit down together in Cleveland to further discuss (and hopefully resolve) the matters put at issue by Local 2823's Grievance. Should I be able to answer any questions in the interim, please do not hesitate to contact me. Sincerely, Jeffrey K. Lamb for Drew G.A. Peel cc: John A. Limposte, Assistant Director Prepared Statement of Patricia A. Keenan, Ph.D., Program Manager, Human Resources Research Organization Good afternoon. I am Patricia Keenan, a Program Manager at the Human Resources Research Organization, known less formally as HumRRO. HumRRO is a non-profit, 501(c)3 research and development organization, established in 1951, that works with Government agencies and other organizations to improve their effectiveness through improved human capital development and management. I will comment today about the Compensation and Pension (C&P) Service's training program, as well as on methods to increase accountability and reduce rating variance. I am the project leader for HumRRO's work with VBA's Skills Certification program. Skills Certification Testing Program HumRRO's has worked closely with C&P Service on Skills Certification program for Veterans Service Representatives (VSRs) and Rating VSRs (RVSRs). We began assisting VBA with the program in 2001, by developing the multiple-choice VSR test that is administered to GS- 10s seeking promotion to GS-11. In 2006 our work expanded to include VSRs at Pension Maintenance Centers (PMC) and RVSRs who have just completed training. Last year we began developing the Skills Certification test for journey-level RVSRs. The VSR test is completely operational with two administrations planned per year. Development of the RVSR end-of-training test is complete and the test is expected to become operational in December of this year. Development of the PMC VSR test is also complete and is expected to become operational next spring. In the future, all Skills Certification tests will be administered via the Internet. This has the advantage of allowing us to expand the types of items on the test to include completion and short essay items. The tests were developed using a content-validation strategy, which requires that the tests reflect important job-related content. The first step in creating that link was to conduct job analyses for each of the four target positions. Subject matter experts linked important tasks to the knowledges, skills and abilities required to perform them. The number of tasks linked to a knowledge area is reflected in the weight given to that knowledge area of the test blueprint and to the number of items in that area that are on the test. To maintain the content validity of a test, it is necessary that job analyses be repeated periodically to ensure that the test still captures the important job requirements. Because the original VSR job analysis was conducted in 2001 and many characteristics of the job have changed since then, we conducted a new job analysis this year. We are in the final stages of revising the test blueprint for the VSR test. C&P Training Programs Since the Skills Certification program began in 2001, C&P Service has initiated several training programs. Newly hired or promoted VSRs and RVSRs are required to take centralized Challenge training. There is an RVSR Challenge course and separate VSR Challenge courses for Pre- Determination and Post-Determination. All three Challenge courses include several Training Performance Support System (TPSS) modules. These are web-based training modules and electronic job aids with accompanying case-based performance practice and performance testing. The content of the modules are content tailored to the specific position. Experienced decision-makers also use the modules for refresher training. VSRs and RVSRs are also required to take 80 hours of refresher training per year. Central Office decides the appropriate content for 60 percent of this training (48 hours); the other 40 percent (32 hours) is determined by each Regional Office on an ``ad hoc'' basis. This mix ensures that training addresses areas recognized as requiring additional training both nationally and locally. Additional training may be required for several reasons--to disseminate information about new regulations or court decisions, to address areas that are commonly appealed, or to correct problems identified during Systematic Technical Accuracy Review (STAR) review. This refresher training is provided in several ways--via classroom training, satellite broadcasts, net meetings, or online courses. VSRs who are preparing to take the Skills Certification Test also receive additional training time to prepare for the test. Please note that HumRRO has no role in preparing or delivering these training courses; our job is to help VBA by developing an independent assessment of knowledge necessary for the job itself. Training and Test Results As of August, 2008, 1,227 participants have passed the VSR Skill Certification test. The passing rates have risen steadily from 25 percent in the 2003 test to 49.58 percent in August of this year. These rising passing rates for the VSR test indicates to us that training is having a positive effect on test performance. We asked participants in the May test which TPSS modules they had completed or used as reference. The three modules with the highest percentage of completion were Original Claim for Compensation, which 157 VSRs reported completing (33.12 percent), Original Claim for Pension, which 127 respondents (26.7 percent), reported completing, and Dependency Benefits, which 118 VSRs (24.9 percent) reported completing. The other modules were completed by 21 percent of test takers or less. The average number of modules completed was three. The general trend in the data shows that those who completed the TPSS modules had the highest scores, followed by those who did not use them at all (i.e., neither completed them nor used them as a reference), followed by those who used them only as a reference. Newer employees were more likely to have completed the TPSS modules whereas employees with longer tenure were more likely to have used the TPSS modules as references. There was no correlation between the number of TPSS modules completed and total test score. Similarly, we asked participants in this summer's RVSR end-of- training test which TPSS modules they completed. Candidates reported completing an average of 6.7 out of 10 modules. Eighty-seven percent of respondents (n=334) said they had completed Rate an Original Claim for Disability Compensation; 72.4 percent (n=278) reported completing Rate an Original Claim for Disability Pension. Other modules were completed by 71 percent of respondents or fewer. There correlation between the number of TPSS modules completed and test score was significant (r(384)=.17, p=.001). Slightly more than 92 percent of respondents (n=354) reported completing Challenge training. The field test for the PMC VSRs was conducted last week. The two TPSS modules completed by the largest percentage of the 60 participants were: Original Claim for Pension (n=24; 40 percent) and Income Adjustments (n=19; 31.7 percent). We will not be able to further analyze the data until scoring is completed in early October. Impediments to Rating As part of the job analyses required for test development, HumRRO staff conducted a series of visits to regional offices to learn more about the VSR and RVSR jobs. The main purpose of these site visits was to identify the critical job elements for incumbents. However, participants in focus groups and interviews also provided information about other aspects of their jobs. We talked with incumbent RVSRs, Decision Review Officers, coaches and Service Center Managers. The rating process is cognitively complex, requiring the RVSR to compare the facts and medical evidence presented in the claim folder to the descriptions of level of disability found in 38 CFR, encompassing two widely differing bodies of knowledge. When the RVSR has completely reviewed the file, established that the veteran has a service-connected condition and all pertinent evidence has been included, the rater begins evaluation process. This is done by comparing the relevant facts presented in the claims folder to the rating schedule, which is organized around 15 body systems (e.g., endocrine, musculoskeletal, respiratory, mental disorders). Participants in the job analysis site visits cited being able to do this with relative ease is the major difference between trainee RVSRs and their journey-level counterparts. They indicated that trainee RVSRs often struggle with anatomy and understanding medical terms at first, then realize that applying the regulations, via the rating schedule, is actually the more challenging part of the job. The RVSR reviews the medical evidence for each separate condition being claimed and matches the condition to a diagnostic code in the rating schedule. These diagnostic codes, in turn, are associated with descriptions for varying levels of severity of impairment. These levels are either assigned percentages in increments of 10 on a scale from 0 to 100 or, for some disabilities such as a muscle injury, they are evaluated on more general descriptions such as severe, moderately severe, moderate, and slight. Thus the RVSR is first required to understand the medical condition(s) (e.g., body system affected, symptoms, severity, limitations thereof) and then to match that information to the correct section(s) of the rating schedule to determine the level of disability for each condition. In addition, there are several factors that make the task even more demanding. These fall broadly into two categories: workload and ambiguity. Workload Factors One of the recurring problems discussed by incumbents was that of incomplete cases and files. When the RVSR begins the adjudication process, the file should be ready for rating. However, they report that often they find the case requires additional development, which the RVSR can do or defer back to the VSR. The problems are varied and range from an incomplete exam, missing justification for diagnosis, unclear information from the veteran that must be clarified, to service records that do not show sufficient information on which to establish service- connection. While these problems can all be resolved, doing the required additional development adds significantly to the time it takes to process a case. The RVSR has spent time reviewing a case that was not ready for a rating decision and the decision must wait until the information can be collected. A second problem is the sheer volume of cases awaiting adjudication. The backlog of work has been growing for years and is increasing more rapidly than ever with the influx of veterans from OEF/ OIF. Veterans today file claims at the time of separation and a large proportion of cases contain multiple issues. A related factor is that often a veteran files additional claims before the first has been decided, with the result of holding up all of the veteran's claims until they can be rated at once. VBA has addressed the workload problem by hiring several thousand VSRs and RVSRs in the past year. One result of this has been that newly hired RVSRs (and VSRs) do not understand the development process well, and often spend much of their time learning what makes a case ``ready to rate.'' While they will eventually become proficient rating specialists, they are less efficient than they might be due to lack of understanding of this vital component of claims processing. RVSRs also face conflicting demands for prioritizing their work. OIF/OEF cases are given priority, as are old cases, and those in which the veteran is facing financial hardship or a terminal illness. Journey-level RVSRs often mentor less experienced RVSRs, reviewing their work and providing feedback, an additional duty in addition to their regular workload. Ambiguity Another common theme heard during the focus groups was that it is critical for RVSRs to understand that what they are rating is most often not black-and-white. There are gray areas in both the medical and legal aspects of the job. A good deal of research is often required to establish service-connection, verify stressors, and understand the nature and severity of a medical condition. RVSRs have a large number and variety of resources available to them to help gather this information, which they then compare to the regulations. However, even their best efforts often result in having to make evaluations based on incomplete medical and legal information. Inexperienced RVSRs usually take longer to rate a case than experienced raters. They are less comfortable making decisions without complete information, and comparing the medical information to the regulations is not an exact science. RVSRs become more comfortable with this ambiguity over time. They also become more familiar with the rating schedule, so they are able to use it with more ease and become more savvy in how to use the schedule. Comments from the focus group respondents indicate that over time, RVSRs develop individual ``rules'' for how to match the medical evidence to the rating schedule, many areas of which leave room for interpretation. This is likely one of the factors that allows them to process cases more quickly; they do not have to spend as much time deciding between evaluation levels. This is also a source of rating variance. VBA raters select the diagnostic code so the correctness of the rating decision depends on the level of knowledge and understanding each RVSR has about the medical descriptions in the rating schedule. The rating schedule contains over 700 diagnostic codes representing distinct physical and mental impairments that are grouped by body systems or like symptoms. Although 700 diagnostic codes sounds like a large number, compared to the several thousands of codes contained in the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM), used by the medical profession, it can be assumed that the rating schedule is less detailed, and thus provides less information to guide evaluations. Increased detail in the rating schedule would likely reduce the amount of individual interpretation that currently results in inconsistencies in rating decisions. It would also require extensive training to learn the revised schedule and additional job aids would need to be developed to improve use of the revised schedule. These factors make matching the medical evidence to the criteria provided in the rating schedule a challenge. But an additional challenge is that the rating schedule does not have diagnostic codes for certain specific conditions, such as carpal tunnel syndrome, Parkinson's disease, pacemakers, or pulmonary embolus. When a claim includes an unlisted condition, the RVSR rates it by analogy to a closely related disease or injury. By their nature, these ``analogous codes'' lack criteria for rating, so raters have to research different body systems to make the evaluation and exercise a wide range of judgments to assign analogous codes. A final opinion raised during the site visits was that many appeals were the result of rating decisions that did not include sufficient detail or explanation of why a claim was denied. As just described, much individual judgment is required in the rating process. This makes it even more important that all evidence be addressed in the rating decision. The reasons and bases section should include all subjective and objective evidence. It is important that veterans feel that their case has been clearly understood and evaluated. The letter should tell them what evidence was in their service records, what the VA medical examination provided (or why a VA examination was not ordered), and all medical evidence that was submitted (e.g., private medical records). The letter has to establish the nexus between the medical evidence and the regulations that determined the outcome, describing how service connection was established, the regulations that applied to each issue, and what evidence is needed to establish service-connection or to receive a higher level of evaluation. Reducing Impediments The workload of RVSRs is not going to become lighter in the near future, so easing the workload and reducing ambiguity could go a long way toward improving ratings. We propose some suggestions to help reduce existing impediments. First, newly hired RVSRs should work Pre-Development for several weeks to learn the system, why different types of evidence are needed, and how to determine that a case is ready to be rated. This would have at least two benefits. The trainee RVSR would not spend time working a case that has insufficient evidence and the mentoring RVSR would not have to do as much explanation about the types and need for different types of evidence. The obvious drawback to this is that it would take longer for new RVSRs to begin rating cases, but we believe having this additional knowledge would pay off in the longer term. Second, the rating schedule is being updated, so it is probably not feasible to develop formal training for each new or revised diagnostic code. But it is important to address the problem caused by individual interpretation of diagnostic codes that are not detailed or specific results. A job aid that includes more specific information about the medical evidence would reduce the level of individual interpretation in ratings. In particular, a job aid that included specific codes and descriptions for disabilities that are relatively frequent and that currently fall under analogous codes, would do a great deal for improving accuracy in these ratings. It is much easier to identify the appropriate evaluation level when the criteria describe specific levels of disability (e.g., range of motion) rather than a more general description such as mild, moderate, and severe. This would allow the rater to match the code to the diagnosis provided in the medical evidence, again reducing errors and variance in the award level. One of the commonly cited reasons for appeals is that the reasons and bases section of the decision does not provide sufficient information or an easily understandable explanation that tells veterans why a claim was denied and what they must do to have the decision reconsidered. RVSRs use templates or scripts to outline the letter and ensure that required information for each section is included. Merely including all required information is not the same in terms of customer service or meeting the spirit of VCAA as is a well-written letter, and these templates cannot help with the complicated problem of presenting technical information in a manner that is well organized and that is easy for the veteran to understand. In this section of the letter, the RVSR describes the material evidence received and the level of evidence required to meet the legal standard as prescribed in the regulations. The ability to understand, organize and clearly present all this information is difficult to train. One remedy is to provide multiple examples of well-written letters that RVSRs can use to guide their own efforts. A more structured approach would be to take the ability to synthesize information and present it in a well-structured, easily comprehensible document into consideration when hiring or promoting RVSRs. A final overall recommendation to reduce variance in ratings is to ensure that all RVSRs receive standardized training, both in content and delivery. Some refresher training is delivered via online tools, broadcasts, or in net meetings, which do provide standardization. A good proportion of refresher training is determined by the local office and is most often provided by Decision Review Officers. It is important that they receive comprehensive training in the technical area being addressed, but also understand how to deliver training; a train-the- trainer workshop that teaches basic training principles as well as how to work with adult learners should be required. Summary It has been HumRRO's pleasure to work with C&P Service for the past 7 years. We are honored to be even a small part of the valuable work the Veterans Benefits Administration does for America's veterans. We have watched both the Skills Certification program and C&P Services Training grow over this time. The resources and effort devoted to training have been reflected in steadily improving pass rates for the VSR Skills Certification test and in the very good pass rate for the RVSR end-of-training test. The greatest impediments to rating accuracy are the pressure to produce, the need for large amounts of medical knowledge and understanding, and ambiguity in interpreting the legal requirements. Being able to spend time in Pre-Development would increase a newly hired RVSR's understanding of the overall claims process as well as the variety and depth of development that is required to rate a case. Job aids can neither reduce the ever increasing number of claims nor reduce their complexity, but by providing increased detail they can make the rating schedule easier to interpret and provide standardization that is currently lacking. Writing and analytical ability were identified as key attributes of good RVSRs, but there is at present no systematic evaluation of these abilities when an individual is hired or promoted to the RVSR position. Finally, we provide a reminder of the importance of standardized training and delivery in ensuring that all rating specialists have a common understanding and method of working. Prepared Statement of Nicholas T. Bartzis, Cleveland, OH (Veteran) Good morning, my name is Nicholas T. Bartzis. I'm here today as a private citizen, concerned veteran, and employee of the Veterans Benefits Administration. I have served in the position of Rating Veterans Service Representative (RVSR) for approximately 8 years in the Cleveland VARO. My relevant past experience included being a training officer in the naval reserves. I possess a joint Law Degree and Master's Degree in Public Administration. As an employee, I am frustrated that I am forced to choose between (1) doing the right thing for the veteran's claim before me by ordering all of the needed intermediate work, or (2) meeting arbitrary production quotas imposed by my supervisors. First, I would like to thank each and every Member who voted for H.R. 5892. I believe that enactment of this bill into law could go far to correct many of the problems we now face. Thank you! Rapid and accurate VA compensation is critical to the quick and efficient transition of former servicemembers and spouses to civilian life. In my opinion, VA managers and adjudicators can do a better job adjudicating claims for VA disability benefits. In 2007, claims processing time increased to an average 183 days. It is my experience that most VA adjudicators endure a constant and mounting pressure to increase their processing of claims in spite of the fact that initial training and follow up training has not kept pace with the needs of VA adjudicators. In general, I and my coworkers do not feel that more decisions equal better individual decisions. The problems with the VA adjudication process identified by Congress, stakeholders and claimants seeking VA benefits are multi- factored and not exclusively the fault of the Veterans Benefits Administration (VBA) management. However, the problems created by the adjudication climate fostered by the VBA are the primary problem. These VBA issues are correctable, and they should be addressed immediately. In my testimony I will focus on the following four topics: VBA training, both qualitative and quantitative issues. Performance management, and a culture that emphasizes quantity of claims processed at the expense of the quality of the decisions made. Accountability, who should be held accountable and why. Potential solutions, from the perspective of one responsible for ratings claims. Training is Insufficient in Quantity and Quality I have been employed as an RVSR for 8 years. I was initially promised formal, centralized training. I have never completed this training because VA management stressed production and did not give raters, such as myself, the time to complete this training. At each juncture, case production requirements for the station trumped my individual training needs. I want to stress that one reason cases are not consistently decided by the over 50 VA regional offices (ROs) is that, as I understand it (based on discussions with employees from other ROs and published studies), training varies widely from RO to RO. Therefore, benefits may be awarded to some veterans but are denied to other veterans who are similarly situated. Nationwide, for the period of January 1, 2007 through September 5, 2008, the Training and Performance Support System (``TPSS'') Basic Rating Completion Report lists 2115 RVSR employees. Only 124 have completed the TPSS portion of the training. This is a completion rate of 17 percent. TPSS training is geared toward newly hired employees and is seldom completed as assigned. Instead, what I see is that, depending on the production demands of the station, those employees are quickly placed in a production capacity. Meaning they are assigned to quickly complete work for the station instead of undertaking further training under penalty of removal should they insist on spending time training instead of processing claims. These new employees are charged with making a correct and timely decision on the claim before them without first completing the proper training. For example, VARO Cleveland has approximately 70 RVSRs. However, in the period January 1, 2007 through September 5, 2008, the TPSS Basic Ratings Completion Report listed 31 RVSRs scheduled to complete training. Of those 31 identified, none have completed the assigned training. This fact clearly shows a lack of commitment to training by local management. Poor initial training is only half of the problem. The job itself requires frequent and detailed updates and discussion about various changes in the law or court decisions. For example, the Veterans Claims Assistance Act of 2000 (VCAA) spawned numerous lawsuits and significantly changed the way VA operates. Also, upon issuance of a court decision or law change, VA must provide rapid instructions to its regional offices. Unfortunately, this is not the case. The U.S. Court of Appeals for Veterans Claims (CAVC) decision in Vazquez v. Peake, 05- 0355, was decided in January 2008, but the VA did not provide instructions to its regional offices respecting this decision until June 2008. Performance Management is Primarily Geared Toward Quantity of Claims Processed and not Quality of Benefits Decisions Performance management may be defined as: ``a system for relating the individual performance on the part of the employee, to the organizational objectives and performance of the agency.'' For VA employees, their performance is controlled by two primary documents. First, is the employee appraisal system, and second, is the employee's performance plan. The Performance Appraisal Program lists the general criteria for how all Title 5 employees will be evaluated. A subset of the Performance Appraisal Program is the individual's performance plan. The individual's performance plan describes the work expected from employees and measures that are expected of them. Presently for RVSRs, there are five stated performance measures: productivity, timeliness, customer service, quality of work, and organizational support. A supervisor assigns values to each of these five stated performance measures. With the exception of productivity, I have seldom seen numerical values assigned to any of the other measures for RVSRs. As such, neither the employee nor any person who reviews their accomplishments after the fact has an accurate description of how much work the employee really did. In general, RVSRs do not obtain work credit for work such as: deferring the rating for additional development by other VA employees, instructional time for the VSR, or sufficient time for reviewing a claims file and ordering a VA exam or reordering the VA exam if it is insufficient. This fact is noted in recommendation number two of the Institute for Defense Analysis study entitled Analysis of Differences in Disability Compensation in the Department of Veterans Affairs. Good, knowledgeable RVSRs are, in a sense, punished when they do work that is not credited by VA management. For example, an RVSR does not get meaningful work credit for analyzing all the evidence in a case and ordering a VA examination, or for reviewing the file and determining that the present VA exam is inadequate, or for reviewing the file and correctly asking a medical question. Careful review of a VA claims file takes time. If, however, employees are not rewarded for careful work and are instead rewarded for processing large volumes of claims poorly, then inevitably claims will be improperly adjudicated and veterans will suffer. For example, if some VA employees are behind in their production, they may go through the file quickly (called top sheeting) and hope that somehow the claim is rated correctly or any error isn't caught. Supervisors, who focus on production, have, in my experience, punished RVSRs who attempt to obtain additional needed development. This is one way in which quantity is emphasized to the detriment of quality. Recently our post determination team had a 68.4 percent FYTD production quantity failure rate for all journey level employees. This means that 68.4 percent of all the journey level--the most experienced employees in that team--have been unable to meet their cumulative production quotas for that year. When brought to their attention, local VA management was highly critical and dismissive of those who identified this issue. Of course, while an individual employee may be unfairly punished by management, the real loser is the veteran whose claim is not adjudicated properly. It is my understanding that VBA has not shared with representatives or employees their method for determining how long each discrete step for evaluating a claim should take. More importantly, they have failed to quantify and provide fair work credit for accomplishing critical intermediate work prior to a claim being decided. Therefore what is needed is an accurate work credit system, within the performance plan, that addresses each discrete type of work and allows the employee sufficient time to accomplish the work. Under the current system only one small portion of the work is measured, the final product, and that has not worked. Performance measures are intended to gauge the employee's contribution to the agency's mission. Instead, what I see is that the metrics as applied to the relevant VA employees do not adequately address the time it takes to really do the work properly. Accountability I believe that Congress wants to know why and who is accountable for our present dilemma of disparate decisions between states and untimely decisions on claims nationwide. As I indicated above, the problem has several causes. Many parties shoulder the burden of our current dilemma. However, the primary burden for accountability for the disparate decisions and long waiting times lies squarely with VBA. Should VBA undertake and scientifically measure the discrete steps it really takes to decide a claim correctly and seriously address the problem with concerned stakeholders, much would be done to alleviate the current problem. Instead, what I see is that the science not being applied and the results shared. Without the science, the number of employees needed to do the work is only an approximate guess based on historical precedent. Without the science, how long it takes to actually complete a typical claim and all its discrete steps is an approximate guess. I believe that most of the problems stem from one issue and one issue only: Station performance goals that are not established based on the time it takes to do the work. Instead what I see is that station performance goals are set by the VA Central office using historical and suspect data. A station's Director and service center manager will sacrifice every activity that interferes with the employee deciding cases, including training, in order to make stations' monthly performance goals. Potential Solutions I believe that Congress is well on the way to correcting many of the problems faced by veterans and the VBA employees who decide their claims. While the process takes time, unnecessary delays pose real hardships for veterans who are losing their houses or unable to feed themselves and their families while solutions are devised and adopted. With that in mind: 1. I strongly encourage the passage of H.R. 5892. This wide- ranging bill is critical to addressing the root cause of the disparity in disability compensation. Specifically, the study on employee work credit and the agency work management system. 2. Increased oversight on VBA programs by Congress to insure compliance with the letter and spirit of programs by the VA managers assigned to accomplish the task. 3. Immediate implementation of a mandatory interim training program for all employees. Within the next 12 months, each employee must complete all identified but unaccomplished formal training per job category. 4. Voluntary increased training by VBA to interested stakeholders like County Veteran Service Officers and service organizations. Thank you for your invitation to testify. I am available to answer any questions of the Committee. Prepared Statement of Michael Walcoff, Deputy Under Secretary for Benefits, Veterans Benefits Administration, U.S. Department of Veterans Affairs Mr. Chairman and Members of the Committee: Thank you for the opportunity today to speak on the important topic of employee training within the Department of Veterans Affairs (VA), Veterans Benefits Administration (VBA). I am pleased to be accompanied by Ms. Dorothy Mackay, VBA's Director of Employee Development and Training, and Mr. Brad Mayes, VBA's Director of Compensation and Pension Service. Today, my testimony will focus on initial and on-going training for Compensation and Pension (C&P) employees. I also will describe employee performance standards, certification requirements, training oversight, and methods used to monitor and enhance the quality and consistency of claims decisions. It is critical that our employees receive the essential guidance, materials, and tools to enable them to learn and develop the knowledge, skills, and abilities required to be successful in their positions. To that end, VBA has deployed training tools and centralized training programs to provide a consistent approach to training. New Employee Training VBA has developed and implemented a standardized training curriculum for new claims processing employees, referred to as the Challenge Training Program. In fiscal year 2006, VBA provided Challenge Training to 678 Veterans Service Representatives (VSRs) and Rating Veterans Service Representatives (RVSRs). That number more than doubled in fiscal year 2007 (VBA trained 1,447) and by the end of fiscal year 2008, VBA will have trained more than 2,150 new C&P employees. In the Challenge Training Program, new hires receive consistent instruction over the course of 2 years. Employees spend 3 weeks in centralized training, generally at the Veterans Benefits Academy in Baltimore, Maryland. The Academy can accommodate up to 240 students at any one time. The centralized training brings together new hires from different regional offices and provides consistent training and networking opportunities. The shared learning experience enables employees with diverse backgrounds who work in different regional offices across the country to develop a shared sense of mission. Instruction at the Academy is provided by experienced VBA staff who are subject matter experts and, who have completed an instructor development course. At the end of each daily session, trainees are invited to complete an anonymous questionnaire to describe what value they gained from the lessons and to evaluate the instructional methods. This information is used to continually adjust and improve the quality of training sessions. When employees return to their home stations, they continue to learn through additional on-the-job training. When classroom instruction is used to develop knowledge on a particular topic, it is generally followed by work on cases involving that topic. As the new hires begin working on actual cases, they spend part of their day developing skills through interactive use of the computer-based Training and Performance Support System (TPSS). This system provides topic modules with a mixed media approach to learning that includes case studies and performance-based testing. It can be utilized by an individual or accessed by a group of individuals to promote discussion of a topic. Currently there are 11 TPSS modules for VSRs, and 13 TPSS modules for RVSRs. VBA is continually striving to enhance the quality of the Challenge Training Program. An independent contractor, hired to evaluate the program, surveyed 1,405 employees and 183 key personnel. We are assessing the results from that evaluation and will use the information to improve the Challenge Training Program. Annual 80-Hour Training Requirement All VSRs and RVSRs are required to complete 80 hours of training each year, although new employees will complete more than this during their first year. Admiral Cooper, the former Under Secretary for Benefits, introduced this requirement. Its continuation is supported by Admiral Dunne, the current Acting Under Secretary for Benefits, and the rest of VBA leadership. The training has been mandated to ensure that claims processing personnel are provided with the most current C&P policies and procedures, as well as the latest decisions from the Court of Appeals for Veterans Claims and precedent-setting opinions from the VA General Counsel. This training also provides for refresher courses in complex claims issues that are not seen on a regular basis, such as special monthly compensation rating codes and benefits available for permanently and totally disabled veterans. Each regional office develops a general training plan for its employees at the beginning of the fiscal year. At least 60 of the required 80 training hours need to come from a list of core technical training topics identified by the C&P Service. For the remaining 20 hours, regional offices are given flexibility to establish training topics based on local needs. This allows regional offices to focus on emerging issues or claims processing areas where errors have been noted. Regional offices are also given flexibility in choosing the instructional method used to deliver the total 80 hours of required training. These methods may include classroom training using approved lesson plans or use of the computer-based TPSS training modules. In addition, regional offices can take advantage of nationwide training programs offered on important and timely topics. Over the past year, this training included topics such as researching stressors in claims for PTSD, C&P medical examinations in claims for PTSD and traumatic brain injury, and attorney representation of veterans in appealed claims. Employee Performance Standards and Accountability Another VBA organizational cornerstone initiative to improve the delivery of benefits and enhance accountability is our system of individual performance assessment. All VSRs and RVSRs are subject to national performance standards measuring the critical elements of quality, productivity, customer service, and workload management, as well as a non-critical element related to organizational support/ teamwork. Performance standards are commensurate with an employee's experience level in the position. These standards are reviewed periodically and amended as necessary in response to changes in workload and claims processing procedures. Managers use an automated tool, called ASPEN (Access Standardized Performance Elements nationwide), to track work items completed and to measure VSR and RVSR performance. Local accuracy reviews are conducted for all decision- makers using the national quality criteria for the Systematic Technical Accuracy Review (STAR) program. Under VBA's performance appraisal program, employee performance is evaluated annually and linked to the employee's performance standards. An employee's level of achievement in each of the four critical elements and one non-critical element is evaluated by a supervisor as exceptional, fully successful, or less than fully successful. Based on evaluations in each of the five areas, a combined overall performance rating is given. The overall ratings include outstanding, excellent, fully successful, minimally successful, and unsatisfactory. In the event an employee's performance is not meeting the fully successful level of achievement for a critical element, an employee will be placed on a performance improvement plan (PIP). The PIP identifies the employee's specific performance deficiencies, the successful level of performance, the action(s) that must be taken by the employee to improve to the successful level of performance, the methods that will be employed to measure the improvement, and provisions for counseling, additional and focused individualized training, or other appropriate assistance. If the employee's performance does not rise to at least the fully successful level for his/her critical element(s), the employee will be removed from the position. In conjunction with the national performance standards, VBA has developed a certification process to assess job proficiency. After successfully demonstrating job proficiency through the certification process, an employee is promoted to the journey level, thereby linking job proficiency to pay grade. Since 2002, the full performance level for a VSR has been the GS-11 level. Promotion to the GS-11 level is contingent upon successful completion of certification testing. Through successfully passing the certification test, VSRs demonstrate that they have the skills necessary to perform the full range of VSR duties, including the ability to work independently on the most complex cases and to review and approve the work of others. Through the national certification program, VBA is raising the skill levels of our core decision-makers and producing greater consistency in claims decisions. Training Oversight Along with an expanded training agenda to accommodate the hiring initiative, VBA has enhanced its training oversight methods to improve accountability. Managers at all levels are held responsible for ensuring that training goals are set and training requirements are met. Each regional office is accountable for submitting a training plan at the beginning of the fiscal year and following through on the plan. The plan is based on an assessment of local needs and anticipates the content and timing of training to fulfill the annual training requirements for regional office employees. In addition, VBA recently created the staff position of Training Manager for each regional office. The Training Manager is responsible for local training reviews, as well as analyzing performance indicators to determine local training needs and implementing the training necessary to meet those needs. The Training Manager is also the lead administrator for the Learning Management System (LMS). LMS is a computerized learning system that was implemented in 2008 that is designed to present training sessions to individual employees and maintain a record of each completed session. Hyperlinks are available through LMS to access training course materials and curricula from the C&P Intranet training Web site, as well as from TPSS training modules. LMS tracks learning hours planned and completed, and is easily accessible by employees and management. On the national level, VBA has established an Employee Training and Learning Board to establish training priorities, promote accountability, and help ensure that training decisions are coordinated and consistent with long range policy plans. The Board is chaired by one of VBA's area Directors, and members include regional and central office Directors. Quality and Consistency As part of the continued commitment to quality improvement, VBA is expanding its quality assurance program. As part of the expansion, the National Systematic Technical Accuracy Review (STAR) staff was consolidated to Nashville, TN, and ten additional staff members were hired. The quality assurance program expansion supports an increase in the annual case sample size for national accuracy reviews from 120 cases per regional office to 246 cases per regional office. This represents a more statistically sound sample size to measure regional office accuracy levels. In addition, the Quality Assurance Staff conducted several focused case reviews this year. These included a special quality review of radiation cases, an ongoing review of extraordinarily large benefit awards, and a special review of cases completed by the Appeals Management Center. The Quality Assurance Staff is also responsible for conducting on- going quarterly data analysis to identify the most frequently rated disabilities or diagnostic codes; assessing the frequency of the assignment or denial of service-connection for each code by regional office; and assessing the most frequently assigned evaluation mode for each code by regional office. Focused audit-style case reviews are conducted at regional offices where rating results are found to be significantly outside the established national averages in order to identify causes of inconsistency. Through these regular reviews, VBA expects to gain more consistent decision-making across regional offices, as well as a better understanding of underlying causes for variation across geographic boundaries. These quality assurance programs are used to identify where additional guidance and training are needed to improve accuracy and consistency nationwide, as well as to drive VBA procedural and regulatory changes. Mr. Chairman, this concludes my testimony. I will be happy to respond to any questions that you or other Members of the Committee may have. MATERIAL SUBMITTED FOR THE RECORD U.S. Government Accountability Office Report to the Chairman, Committee on Veterans' Affairs, House of Representatives VETERANS' BENEFITS: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors May 2008 Report No. GAO-08-561 GAO Highlights Why GAO Did This Study Faced with an increase in disability claims, the Veterans Benefits Administration (VBA) is hiring a large number of new claims processing staff. We were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has VBA planned this training strategically, and how well is the training designed, implemented, and evaluated? and (3) To what extent is the performance management system for claims processors consistent with generally accepted practices? To answer the questions, GAO reviewed documents including VBA policies and training curricula; interviewed VBA central office officials; visited 4 of VBA's 57 regional offices, which were selected to achieve diversity in geographic location, number of staff, and officewide accuracy in claims processing; and compared VBA's training and performance management to generally accepted practices identified by GAO. What GAO Recommends GAO is recommending that VBA collect feedback on training provided by regional offices and use this feedback to further improve training, and hold staff accountable for meeting their training requirement. GAO also recommends that the VA assess and, if necessary, adjust its process for placing staff in overall performance categories. In its comments, VA agreed with GAO's conclusions and concurred with the recommendations. What GAO Found VBA has a standardized training curriculum for new staff and a training requirement for all staff, but does not hold staff accountable for meeting this requirement. The curriculum for new staff includes what is referred to as centralized training and training at their home offices. All claims processors must complete 80 hours of training annually, which may cover a mix of topics identified centrally and by regional offices. Individual staff members face no consequences for failing to meet the training requirement, however, and VBA has not tracked training completion by individuals. It is implementing a new system that should provide this capacity. Although VBA has taken steps to plan its training strategically, the agency does not adequately evaluate training and may be falling short in training design and implementation. VBA has a training board that assesses its overall training needs. However, the agency does not consistently collect feedback on regional office training, and both new and experienced staff GAO interviewed raised issues with their training. Some new staff raised concerns about the consistency of training provided by different instructors and about the usefulness of an online learning tool. Some experienced staff believe that 80 hours of training annually is not necessary, some training was not relevant for them, and workload pressures impede training. The performance management system for claims processors generally conforms to GAO-identified key practices, but the formula for assigning overall ratings may prevent managers from fully acknowledging and rewarding staff for higher levels of performance. The system aligns individual and organizational performance measures and requires that staff be given feedback throughout the year. However, VBA officials raised concerns about the formula used to assign overall ratings. Almost all staff in the offices GAO visited were placed in only two of five overall rating categories, although managers said greater differentiation would more accurately reflect actual performance differences. The Department of Veterans Affairs (VA) has not examined the ratings distribution, but acknowledges a potential issue with its formula and is considering changes. Fiscal Year 2007 Appraisals for Four Offices Were Concentrated in Two Categories [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: VBA regional offices. __________ U.S. Government Accountability Office Washington, DC May 27, 2008 Hon. Bob Filner Chairman Committee on Veterans' Affairs House of Representatives Dear Mr. Chairman: The Veterans Benefits Administration (VBA) is facing an increased volume of claims for disability benefits related to the current conflicts in Afghanistan and Iraq as well as the aging of veterans from past conflicts. Between fiscal years 2000 and 2006, the number of disability-related claims filed annually with VBA increased by almost 40 percent. As a result, VBA continues to experience challenges in processing veterans' disability claims. As of fiscal year 2007, VBA had approximately 392,000 disability claims pending benefit decisions, and the average time these claims were pending was 132 days. According to VBA, the current conflicts have also produced more claims related to post-traumatic stress disorder and traumatic brain injury, conditions few VBA staff have had much experience evaluating. To process the increased volume of claims, in fiscal year 2007 the agency began hiring a large number of new Veterans Service Representatives (VSR), who collect evidence related to veterans' claims, and Rating Veterans Service Representatives (RVSR), who evaluate claims and determine benefit eligibility. It plans to add 3,100 new claims-processing staff by the end of fiscal year 2008. Given the increased volume of claims, the increased focus on certain types of disabilities, and the large number of new hires, training and performance management systems for VSRs and RVSRs now play an especially critical role in enabling VBA to meet its organizational claims processing goals for accuracy and productivity. Training that is properly designed and implemented is vital both to help new staff learn their jobs and experienced staff to update their knowledge and learn about emerging issues. An effective performance management system would also help VBA manage its staff on a day-to-day basis to achieve its organizational goals. To provide Congress with information on the training and performance management of claims processors, we were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has VBA developed a strategic approach to planning training for claims processors and how well is their training designed, implemented, and evaluated? And (3) To what extent is the performance management system for claims processors consistent with generally accepted performance management practices in the public sector? To address these objectives, we collected documents and data from VBA central office and interviewed central office staff. In addition, GAO experts on training reviewed VBA documents related to training curriculum, lesson plans, and course evaluations. We conducted site visits to 4 of VBA's 57 regional offices--Atlanta, Baltimore, Milwaukee, and Portland, Oregon. These offices were selected to achieve diversity in geographical location, number of staff, and officewide accuracy in claims processing. While we examined VBA-wide policies and requirements, we primarily assessed how the training and performance management systems are implemented at four sites. Therefore, our results may not be representative of how these systems are implemented across all regional offices. We assessed VBA's training and performance management practices by comparing them to certain generally accepted practices for Federal agencies in these areas that have been identified by GAO.\1\ We conducted this performance audit from September 2007 through May 2008 in accordance with generally accepted Government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. (See app. I for more detailed information on our objectives, scope, and methodology.) --------------------------------------------------------------------------- \1\ These practices are laid out primarily in two GAO reports: Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, DC: March 2004) and Results Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success, GAO-03-488 (Washington, DC: Mar. 14, 2003). --------------------------------------------------------------------------- Results in Brief VBA has a standard training curriculum for new claims processors and an 80-hour annual training requirement for all claims processors, but staff are not held accountable for meeting this requirement. VBA's three-stage training program for new staff is intended to deliver training in a consistent manner. First, VBA policy states that new staff are required to complete some orientation training, which is provided in their home offices. Second, they are required to attend a 3-week standardized training session, referred to as centralized training, that provides a basic introduction to their job responsibilities. Third, new staff are required to spend several more months in training at their home offices, which is supposed to include on-the-job training, instructor-led training classes that follow a required curriculum, and use of an online learning tool called the Training and Performance Support System. VBA policy states that all claims processors are required to complete a minimum of 80 hours of training annually, and regional offices have some discretion over what training they provide to meet this requirement. At least 60 hours must be selected from a list of core topics identified by VBA central office. Regional offices may choose the topics for the remaining 20 hours based on local needs, such as to prevent errors identified in processing claims. Each regional office develops an annual training plan listing the courses needed, and VBA central office periodically reviews these plans and provides feedback to regional offices. Although VBA has a training requirement for VSRs and RVSRs, it does not have a policy outlining consequences for individual staff who do not complete their required training. Further, VBA does not maintain data on the training completed by individuals, but agency officials said they are currently implementing a new, online learning management system that should enable them to do so in the future. VBA is taking steps to strategically plan its training, but does not adequately evaluate its training and may be falling short in some areas of training design and implementation. VBA appears to have followed several accepted practices in planning its training, including the establishment of a training board that assesses VBA's overall training needs and makes recommendations to the Undersecretary for Benefits. Also, VBA makes some effort to evaluate its centralized training for new staff, soliciting feedback from students with forms that are well-constructed and well-balanced. However, VBA does not require regional offices to collect feedback on any of the training they provide to new and experienced staff. In fact, claims processors we interviewed raised some issues with the training they received. For example, some new staff reported that different instructors in centralized training sessions sometimes taught different ways of performing the same procedure, and that one of VBA's online learning tools--the Training and Performance Support System--is too theoretical and often out-of-date. More experienced staff had mixed opinions as to whether 80 hours of training annually is appropriate for all staff. Also, many experienced staff indicated that training topics are redundant from year to year, and some told us that courses available to them are not always relevant for their position or experience level because they are often adapted from courses for new staff. Some staff said they struggle to meet the annual 80-hour training requirement because of workload pressures. The Department of Veterans Affairs' performance management system for VSRs and RVSRs generally conforms to accepted practices, including aligning individual and organizational performance measures, but the system may not clearly differentiate among staff's overall performance levels. Several elements of VSRs' and RVSRs' performance are evaluated, and these elements are generally aligned with VBA's organizational performance measures. For example, VSRs and RVSRs are evaluated on their accuracy in claims processing, and one of VBA's organizational performance measures is accuracy in claims processing. VBA's performance management system is also consistent with other accepted practices, such as providing performance feedback throughout the year and emphasizing collaboration. However, the system may not clearly and accurately differentiate among the overall performance levels of VSRs and RVSRs. A VA-wide formula is used to translate an employee's ratings on all individual elements into one of five overall rating categories. Several VBA central and regional office managers raised concerns with this formula, saying that it is difficult for staff to be placed in certain overall performance categories, even if staff's performance truly does fall within one of those categories. In fact, when we reviewed the results of VSR and RVSR appraisals at the regional offices we visited, almost all staff were placed in either the outstanding (highest) or fully successful (middle) categories. To the extent that the performance appraisals do not make meaningful distinctions in performance, staff may lack the constructive feedback they need to improve, and managers may lack the information they need to reward top performers and address performance issues. Although VA acknowledged this issue and indicated that it is considering changes to the system, no formal actions have been taken to date. We are recommending that VBA central office collect feedback on training provided by the regional offices, to determine whether (1) 80 hours is the appropriate amount of annual training for all staff, (2) regional offices are providing training that is relevant for all staff, and (3) whether any changes are needed to improve the Training and Performance Support System. We are also recommending that VBA central office hold individual staff accountable for meeting their training requirement and that VA assess and, if necessary, adjust its performance rating system for staff to make it a more meaningful management tool. In its comments, VA agreed with our conclusions and concurred with our recommendations. Background In fiscal year 2007, the Department of Veterans Affairs (VA) paid about $37\1/2\ billion in disability compensation and pension benefits to more than 3.6 million veterans and their families. Through its disability compensation program, the VBA pays monthly benefits to veterans with service-connected disabilities (injuries or diseases incurred or aggravated while on active military duty). Monthly benefit amounts vary according to the severity of the disability. Through its pension benefit program, VBA pays monthly benefits to wartime veterans with low incomes who are either elderly or permanently and totally disabled for reasons not service-connected. In addition, VBA pays dependency and indemnity compensation to some deceased veterans' spouses, children, and parents and to survivors of servicemembers who died while on active duty. When a veteran submits a benefits claim to any of VBA's 57 regional offices, a Veterans Service Representative (VSR) is responsible for obtaining the relevant evidence to evaluate the claim. For disability compensation benefits, such evidence includes veterans' military service records, medical examinations, and treatment records from VA medical facilities and private providers. Once a claim is developed (i.e., has all the necessary evidence), a Rating Veterans Service Representative (RVSR) evaluates the claim, determines whether the claimant is eligible for benefits, and assigns a disability rating based on degree of impairment. The rating determines the amount of benefits the veteran will receive. For the pension program, claims processing staff review the veteran's military, financial, and other records to determine eligibility. Eligible veterans receive monthly pension benefit payments based on the difference between their countable income, as determined by VA, and the maximum pension amounts as updated annually by statute.\2\ In fiscal year 2007, VBA employed over 4,100 VSRs and about 1,800 RVSRs to administer the disability compensation and pension programs' caseload of almost 3.8 million claims. --------------------------------------------------------------------------- \2\ 38 U.S.C. Sec. 5312(a). --------------------------------------------------------------------------- In 2001 the VA Claims Processing Task Force noted that VSRs were responsible for understanding almost 11,000 separate benefit delivery tasks, such as tasks in claims establishment, claims development, public contacts, and appeals. To improve VBA's workload controls, accuracy rates, and timeliness, the Task Force recommended that VA divide these tasks among a number of claims processing teams with defined functions. To that end, in fiscal year 2002, VBA developed the Claims Processing Improvement model that created six claims processing teams, based on phases of the claims process. (See table 1.) Table 1--VBA's Disability Compensation and Pension Service's Claims Processing Teams ---------------------------------------------------------------------------------------------------------------- Team Summary of claims processing duties ---------------------------------------------------------------------------------------------------------------- Triage Team Establishes the regional office's tracking procedures for all mail as well as processes claims that only require a brief review to determine eligibility. ---------------------------------------------------------------------------------------------------------------- Pre-Determination Team Develops evidence for disability ratings and prepares administrative decisions. ---------------------------------------------------------------------------------------------------------------- Rating Team Makes decisions on claims that require consideration of medical evidence. ---------------------------------------------------------------------------------------------------------------- Post-Determination Team Develops evidence for non-rating issues, processes benefit awards, and notifies veterans of rating decisions. ---------------------------------------------------------------------------------------------------------------- Public Contact Team Conducts personal interviews and handles telephone inquiries, including calls from veterans. ---------------------------------------------------------------------------------------------------------------- Appeals Team Handles requests for reconsideration of claims where veterans have formally disagreed with claim decisions. ---------------------------------------------------------------------------------------------------------------- Source: VBA. Note: The Rating Team is made up of RVSRs, the Post-Determination and Public Contact teams are made up of VSRs, and the Pre-Determination, Triage, and Appeals teams are made up of both RVSRs and VSRs. According to one VA official, new claims processing staff generally begin as VSRs and typically have a probationary period of about 1 year.\3\ After their probationary period ends, staff can either continue to qualify to become senior VSRs or apply for RVSR positions.\4\ VSRs are also given the option to rotate to other VSR claim teams to gain a broader understanding of the claims process. --------------------------------------------------------------------------- \3\ While new claims processors are on probation, 100 percent of the claims work they perform is quality reviewed by a supervisor. After their probationary period, only a small sample of their claims are quality reviewed. \4\ Typically, RVSRs are promoted VSRs, although in some instances, VA hires RVSRs from outside of VA who have medical or legal experience. --------------------------------------------------------------------------- VBA Has a Uniform Training Curriculum for New Claims Processors and an Annual Training Requirement for All Claims Processors, but Staff Are Not Held Accountable for Meeting This Requirement VBA has established a standardized curriculum for training new VSRs and RVSRs on how to process claims, and it has an 80-hour annual training requirement for both new and experienced staff; however, it does not hold individual staff accountable for meeting this requirement. VBA has designed a uniform curriculum for training new VSRs and RVSRs that is implemented in three phases--initial orientation training, a 3-week training session referred to as centralized training, and comprehensive on-the-job and classroom training after centralizing training. It also requires all staff to meet an annual 80- hour training requirement. To ensure that staff meet this requirement, each regional office must develop an annual training plan, which can contain a mix of training topics identified by VBA central office and by the regional office. However, individual staff members are not held accountable for meeting their training requirement. Training for New Staff Is Conducted in Three Stages Using a Uniform Curriculum VBA has a highly structured, three-phased program for all new claims processors designed to deliver standardized training, regardless of training location or individual instructors. (See fig. 1.) For example, each topic included in this training program contains a detailed lesson plan with review exercises, student handouts, and copies of slides used during the instructor's presentation. Each phase in this program is designed to both introduce new material and reinforce material from the previous phase, according to a VBA official. Figure 1--Phases of Training for New VSRs and RVSRs [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: GAO analysis. According to VBA policy, the first phase of training for new VSRs and RVSRs is prerequisite training. New VSRs and RVSRs begin prerequisite training at their home regional office as soon as they begin working. Prerequisite training lays the foundation for future training by introducing new VSRs to topics such as the software applications used to process and track claims, medical terminology, the system for maintaining and filing a case folder, and the process for requesting medical records. Although VBA specifies the topics that must be covered during prerequisite training, regional offices can choose the format for the training and the timeframe. New VSRs and RVSRs typically spend 2 to 3 weeks completing prerequisite training in their home office before they begin the second program phase, centralized training. During what is referred to as centralized training, new VSRs and RVSRs spend 3 weeks in intensive classroom training. Participants from multiple regional offices are typically brought together in centralized training sessions, which may occur at their home regional office, another regional office, or the Veterans Benefits Academy in Baltimore, Maryland. According to VBA officials in three of the four offices we visited, bringing together VSRs and RVSRs from different regional offices helps to promote networking opportunities, while VBA officials from two of these offices also stated that it provides a nationwide perspective on VBA. Centralized training provides an overview of the technical aspects of the VSR and RVSR positions. Training instructors should follow the prescribed schedule and curriculum dictating when and how material is taught. For example, for a particular topic, the instructor's guide explains the length of the lesson, the instructional method, and the materials required; lays out the information that must be covered; and provides exercises to review the material. (See fig. 2 for a sample of an instructor's guide from the centralized training curriculum.) Centralized training classes have at least three instructors, but the actual number can vary depending on the size of the group. VBA's goal is to maintain a minimum ratio of instructors to students. Figure 2--Phases of Training for New VSRs and RVSRs [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: VBA. The first week of centralized training for VSRs focuses on key concepts, such as security, privacy and records management; terminology; and job tools, such as the policy manual and software applications. The final 2 weeks of training focus on the different roles and responsibilities of VSRs on the Pre-determination and Post- determination teams in processing claims. To practice processing different types of claims and processing claims from start to finish, VSRs work on either real claims or hypothetical claims specifically designed for training. Centralized training for new RVSRs--many of whom have been promoted from the VSR position--focuses on topics such as systems of the human body, how to review medical records, and how to interpret a medical exam. According to staff in one site we visited, RVSRs new to VBA also take VSR centralized training or its equivalent to learn the overall procedures for processing claims. To accommodate the influx of new staff it must train, in fiscal year 2007 VBA substantially increased the frequency of centralized training and is increasing student capacity at the Veterans Benefits Academy. During fiscal year 2007, VBA held 67 centralized training sessions for 1,458 new VSRs and RVSRs. Centralized training sessions were conducted at 26 different regional offices during fiscal year 2007, in addition to the Veterans Benefits Academy. By comparison, during fiscal year 2006, VBA held 27 centralized training sessions for 678 new claims processors. To implement centralized training, VBA relies on qualified regional office staff who have received training on how to be an instructor. According to VBA officials, centralized training instructors may be Senior VSRs, RVSRs, supervisors, or other staff identified by regional office managers as having the capability and the right personality to be effective instructors. Potential instructors have certain training requirements. First, they must complete the week-long Instructor Development Course, which covers the ways different adults learn, the process for developing lesson plans, and the use of different training methods and media. During this course, participants are videotaped and given feedback on their presentation style. In addition, each time instructors teach a centralized training session, they are supposed to take the 2\1/2\ day Challenge Curriculum Course, designed to update instructors on changes to the curriculum and general training issues. Between October 2006 and February 2008, about 250 VSRs and RVSRs from regional offices completed the Instructor Development Course, and VBA officials reported that, given the influx of new VSRs and RVSRs, they are increasing the number of times this course is offered in order to train more instructors. Instructors can teach centralized training sessions in their home office, another regional office, or the Veterans Benefits Academy.\5\ --------------------------------------------------------------------------- \5\ Staff who teach classes other than centralized training are not required to take the week-long Instructor Development Course, although they may do so if openings exist. They can also take an 8-hour condensed course for regional instructors. --------------------------------------------------------------------------- When new VSRs and RVSRs return to their home office after centralized training, they are required to begin their third phase of training, which is supposed to include on-the-job, classroom, and computer-based training, all conducted by and at their regional office. In the regional offices we visited, managers indicated that new VSRs and RVSRs typically take about 6 to 12 months after they return from centralized training to complete all the training requirements for new staff. During this final phase, new claims processing staff cover more advanced topics, building on what they learned in centralized training. Under the supervision of experienced claims processors, they work on increasingly complex types of real claims. On-the-job training is supplemented in the offices we visited by regular classroom training that follows a required curriculum of courses developed by VBA's Compensation and Pension Service, specifically for new VSRs and RVSRs. For example, new VSRs might complete a class in processing burial claims and then spend time actually processing such claims. The amount of time spent working on each type of claim varies from a couple of days to a few weeks, depending on the complexity of the claim. On-the- job training is also supposed to be supplemented with modules from the Training and Performance Support System (TPSS), an interactive online system that can be used by staff individually or in a group.\6\ TPSS modules provide detailed lessons, practice cases, and tests for VSRs and RVSRs. Modules for new VSRs cover topics such as burial benefits and medical terminology; RVSR modules cover topics such as the musculoskeletal system, general medical terminology, and introduction to post-traumatic stress disorder. --------------------------------------------------------------------------- \6\ In 2001, we reported that VBA had spent or obligated about $18.6 million of the estimated total TPSS program cost of $32 million. See GAO, Veterans' Benefits: Training for Claims Processors Needs Evaluation, GAO-01-601 (Washington, DC: May 31, 2001). --------------------------------------------------------------------------- New and Experienced Staff Have an Annual Training Requirement, and Regional Offices Develop Training Plans That Cover a Mix of Topics Identified Centrally and Locally A policy established by VBA's Compensation and Pension Service requires both new and experienced VSRs and RVSRs to complete a minimum of 80 hours of technical training annually, double the number VBA requires of its employees in other technical positions.\7\ VBA officials said this higher training requirement for VSRs and RVSRs is justified because their jobs are particularly complex and they must work with constantly changing policies and procedures. --------------------------------------------------------------------------- \7\ VBA defines an experienced VSR or RVSR as one who has been in that position for 1 year or more. --------------------------------------------------------------------------- The 80-hour training requirement has two parts. At least 60 hours must come from a list of core technical training topics identified by the central office of the Compensation and Pension Service. For example, core topics for VSRs in fiscal year 2007 included establishing veteran status and asbestos claims development; topics for RVSRs included due process provisions and eye-vision issues. VBA specifies more core topics than are necessary to meet the 60-hour requirement, so regional offices can choose those topics most relevant to their needs. They can also choose the training method used to address each topic, such as classroom or TPSS training. (See app. II for the list of core technical training topics for fiscal year 2007.) Regional offices determine the training topics that are used to meet the remaining 20 hours, based on local needs and input. Regional offices may select topics from the list of core technical training topics or identify other topics on their own. The four regional offices we visited varied in the extent to which they utilized their discretion to choose topics outside the core technical training topics in fiscal year 2007. Two sites selected the required 60 hours of training from the core requirements and identified their own topics for the remaining 20 hours. In the other two sites, almost all the training provided to staff in fiscal year 2007 was based on topics from the list of core requirements. An official in one regional office, for example, said that his office used its full 20 hours to provide training on new and emerging issues that are not covered by the core technical training topics, as well as training to address error prone areas. An official in another regional office said the core requirements satisfied staff training needs in fiscal year 2007, possibly because this regional office had a large proportion of new staff and the core topics are focused on the needs of new staff. Regional offices must develop training plans each year that indicate which courses will actually be provided to staff to enable them to meet the 80-hour training requirement. The training plan is a list of courses that the regional office plans to offer throughout the year, as well as the expected length and number and types of participants in each course. In the regional offices we visited, when managers develop their training plans, they solicit input from supervisors of VSRs and RVSRs and typically also consider national or local error trend data. Regional offices must submit their plans to the VBA central office at the beginning of each fiscal year for review and feedback. Central office officials review the plans to determine whether (1) the regional office will deliver at least 60 hours of training on the required core topics, (2) the additional topics identified by the regional office are appropriate, and (3) staff in similar positions within an office receive the same level and type of training. According to central office officials, they provide feedback to the regional offices on their current plans as well as guidance on what topics to include in the next year's training plans. Regional offices can adjust their training plans throughout the year to address shifting priorities and unexpected training needs. For example, a regional office may add or remove courses from the plan in response to changing trends in errors or policy changes resulting from legal decisions. (See app. III for excerpts from the fiscal year 2007 training plans from the regional offices we visited.) While regional offices have discretion over the methods they use to provide training, the four offices we visited relied primarily on classroom training in fiscal year 2007. In each of these offices, at least 80 percent of the total fiscal year 2007 training hours completed by all claims processors was in the form of classroom instruction (see fig. 3). Officials in two of the regional offices we visited said they used lesson plans provided by the Compensation and Pension Service and adapted these plans to the needs of their staff; one regional office developed its own courses. An official in one office said they sometimes invite guest speakers, and an official in another regional office said that classroom training is sometimes delivered as part of team meetings. The offices we visited generally made little use of other training methods. Only one office used TPSS for its training more than 1 percent of the time. Two offices used self-instruction--such as reading memos from VBA central office--for about 10 percent of their training, and no office used videos for more than 1 percent of their training. The central office usually communicates immediate policy and regulatory changes through memos called Fast Letters, which may be discussed in team meetings or may just be read by staff individually. Figure 3--Most Fiscal Year 2007 Training Hours Completed by Claims Processors in the Offices We Visited Were in the Form of Classroom Instruction [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Staff Are Not Held Accountable for Meeting Their Training Requirement Because the agency has no policy outlining consequences for individual staff who do not complete their 80 hours of training per year, individual staff are not held accountable for meeting their annual training requirement, and at present, VBA central office lacks the ability to track training completed by individual staff members. According to VBA officials, however, the agency is in the process of implementing an automated system that should allow it to track the training each staff member completes. Officials reported that this system is expected to be implemented during fiscal year 2008. VBA officials reported that this system will be able to record the number of training hours and the courses completed for each individual, staff position, and regional office. One official said the central office and regional office supervisors will have the ability to monitor training completed by individual staff members, but that central office will likely not monitor the training completed by each individual staff member, even though it may monitor the training records for a sample of staff members. Furthermore, despite the absence of a VBA-wide tracking system, managers in two of the regional offices we visited reported using locally developed tracking methods to determine the number of training hours their staff had completed. While individuals are not held accountable, VBA reported taking some steps to ensure that staff complete the required number of training hours. VBA central office periodically reviews the aggregated number of training hours completed at each regional office to determine whether the office is on track to meet the training requirement.\8\ According to a VBA official, managers in offices where the staff is not on track to complete 80 hours of training during the year can be reprimanded by a higher-level manager, and if their staff do not meet the aggregate training hours at the end of the fiscal year, managers could face negative consequences in their performance assessments. --------------------------------------------------------------------------- \8\ To determine if VSRs and RVSRs in a regional office are generally meeting their annual training requirement, the aggregate number of training hours completed in a given year by all staff in that office is divided by the number of staff in that office. --------------------------------------------------------------------------- VBA Is Taking Steps To Strategically Plan Its Training for Staff, But Does Not Adequately Evaluate Training and May Be Falling Short in Design and Implementation VBA is taking steps to strategically plan its training for VSRs and RVSRs including the establishment of a training board to assess VBA's training needs. VBA has also made some effort to evaluate its training for new staff, but does not require regional offices to collect feedback from staff on any of the training they provide. Although some regional offices collect some training feedback, it is not shared with VBA central office. Both new and experienced staff we interviewed did, in fact, report some problems with their training. A number of new staff raised issues with how consistently their training curriculum was implemented. Experienced staff differed in their assessments of the VBA's annual training requirement, with some indicating they struggle to meet this requirement because of workload pressures or that training topics are sometimes redundant or not relevant to their position. VBA Is Taking Steps To Strategically Plan Its Training VBA is taking steps to strategically plan its training for claims processors, in accordance with generally accepted practices identified by GAO. (See app. I for a detailed description of these generally accepted practices.) Aligning Training with the Agency's Mission and Goals VBA has made an effort to align training with the agency's mission and goals. According to VBA documents, in fiscal year 2004 an Employee Training and Learning Board (board) was established to ensure that training decisions within the VBA are coordinated; support the agency's strategic and business plans, goals and objectives; and are in accordance with the policy and vision of VBA.\9\ Some of the board's responsibilities include establishing training priorities and reviewing regional office and annual training plans. --------------------------------------------------------------------------- \9\ According to VBA officials, the board is made up of a mix of regional office and central office staff from different VBA business lines including Employee Development and Training, Human Resources, the Compensation and Pension Service, and the Insurance Service. --------------------------------------------------------------------------- Identifying the Skills and Competencies Needed by the Workforce VBA has identified the skills and competencies needed by VBA's claims processing workforce. VBA developed a decision tree and task analysis of the claims process, which GAO experts in the field of training told us made it possible to understand and map both the claims process and the decisions associated with it that supported the development of VBA's training curriculum. Determining the Appropriate Level of Investment in Training and Prioritizing Funding VBA is taking steps to determine the appropriate level of investment in training and prioritize funding. According to VBA documents, some of the board's responsibilities include developing annual training budget recommendations and identifying and recommending training initiatives to the Under Secretary of Benefits. VBA officials also reported developing several documents that made a business case for different aspects of VBA's training, such as VA's annual budget and the task analysis of the VSR and RVSR job positions. Considering Government Reforms and Initiatives According to one VBA official, the agency identifies regulatory, statutory, and administrative changes as well as any legal or judicial decisions that affect how VBA does business and issues guidance letters, or Fast Letters, which can be sent out several times a year, to notify regional offices of these changes. Also, as a result of Congress authorizing an increase in its number of full-time employees and VBA's succession planning efforts, VBA has increased the number of centralized training sessions for new staff and has also increased the number of Instructor Development Courses offered to potential centralized training instructors. As a result, VBA is taking steps to consider Government reforms and initiatives to improve its management and performance when planning its training. VBA Collects Feedback on Centralized Training, but Regional Offices Do Not Always Collect Feedback on the Training They Provide According to accepted practices, Federal agencies should also evaluate their training programs and demonstrate how these efforts help employees, rather than just focusing on activities or processes (such as number of training participants or hours of training). VBA has made some efforts to evaluate its training for claims processors. During the 3-week centralized training session for new staff, VBA solicits daily feedback from participants using forms that experts in the training field consider well-constructed and well-balanced. According to one GAO expert, the forms generally employ the correct principles to determine the effectiveness of the training and ascertain whether the instructor effectively presented the material (see fig. 4). VBA officials told us that they have used this feedback to improve centralized training for new staff. Management at one regional office cited the decision to separate training curricula for VSRs on Pre-determination teams and VSRs on Post-determination teams as an example of a change based on this feedback. Figure 4--Sample of VBA's Centralized Training Evaluation Form [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Although VBA evaluates centralized training, it does not require regional offices to obtain feedback from participants on any of the training they provide to new and experienced staff. In a previous GAO report, VA staff told us that new training materials they develop are evaluated before being implemented.\10\ However, none of the regional offices we visited consistently collect feedback on the training they conduct. Supervisors from three of the regional offices we visited told us that they collect feedback on some of the training their office conducts, but this feedback largely concerns the performance of the instructor. Participants are generally not asked for feedback on course content. Moreover, regional offices we visited that do, to some degree, collect feedback do not share this information with VBA. --------------------------------------------------------------------------- \10\ GAO, Veterans' Benefits: Improved Management Would Enhance VA's Pension Program, GAO-08-112 (Washington, DC: Feb. 14, 2008). --------------------------------------------------------------------------- VBA's Training Curriculum for New Staff Appears Generally Well Designed, but Some Staff Raised Issues Concerning Its Implementation According to GAO experts in the training field, VBA's training curriculum for new staff appears well designed. VBA's curriculum for new staff conforms to adult learning principles, carefully defining all pertinent terms and concepts, and providing abundant and realistic examples of claims work. GAO experts also determined that VBA's training for those who teach the curriculum for new staff was well designed and would enable experienced claims processors to become competent trainers because they are coached on teaching theory and have multiple opportunities to practice their teaching skills and receive feedback. Many of the new staff at all four sites we visited reported that centralized training provided them with a good foundation of knowledge and prepared them for additional training conducted by their regional office. Also, regional office managers from three offices we visited told us that centralized training affords new staff the opportunity to network with other new staff at different regional offices, which imbues a sense of how their positions fit in the organization. However, some staff reported that VBA's implementation of their centralized training was not always consistent. A number of staff at three regional offices reported that during their centralized training the instructors sometimes taught different ways of performing the same procedures or disagreed on claim procedures. Regional office officials told us that while centralized training instructors attempt to teach consistently through the use of standardized training materials, certain procedures can be done differently in different regional offices while adhering to VBA policy. For example, regional offices may differ on what to include in veteran notification letters. VBA officials also told us that centralized training conducted at the regional offices may not be as consistent as centralized training conducted at the Veterans Benefits Academy. According to these officials, unlike the regional offices, the Veterans Benefits Academy has on-site training experts to guide and ensure that instructors are teaching the curriculum consistently. New staff also gave mixed assessments about how training was conducted at their home office after they returned from centralized training. While some staff at all of the regional offices we visited told us that the additional training better prepared them to perform their jobs, with on-the-job training identified as a useful learning tool, others told us that the training could not always be completed in a timely manner due to regional office priorities. Some management and staff at two of the regional offices we visited reported that, because of workload pressures, some of their RVSRs had to interrupt their training to perform VSR duties. Also, a few new staff indicated that VBA's TPSS was somewhat difficult to use.\11\ Although TPSS was developed to provide consistent technical training designed to improve the accuracy of claims ratings, a number of staff at all of the regional offices we visited reported that TPSS was too theoretical. For example, some staff said it provided too much information and no practical exercises in applying the knowledge. Some staff also noted that certain material in TPSS was out-of-date with policy changes such as how to order medical examinations. Some staff at three of the regional offices also reported that TPSS was not always useful in training staff, in part, because TPSS does not use real cases. Three of the regional offices reported using TPSS for less than 1 percent of their training and VSRs at one regional office were unaware of what TPSS was. --------------------------------------------------------------------------- \11\ In 2001, GAO reported that VBA's TPSS may not fully achieve its objectives of providing standardized training to new employees, reducing the training period required for new employees, or improving claims-processing accuracy and consistency. In the report, we recommended actions the agency should consider in providing timely standardized training and providing indicators of the impact of TPSS on accuracy and consistency. In its technical comments on this report, VA indicated it accomplished the first recommendation. See GAO-01-601. --------------------------------------------------------------------------- Experienced Staff Expressed Mixed Views of the Design and Implementation of Their Training At all of the regional offices we visited, staff we spoke with generally noted that training enables them to keep up-to-date on changes in laws and regulations as well as provides opportunities for obtaining refresher training on claims procedures they perform infrequently. However, regional office staff we spoke with differed in their assessment of the 80-hour requirement. Some regional office staff said the number of training hours required was appropriate, while others suggested that VBA adopt a graduated approach, with the most experienced staff being required to complete fewer hours than new staff. VBA officials told us that, in 2007, the Compensation and Pension Service reviewed their annual training requirements and determined the 80-hour annual training requirement was appropriate. However, the officials we spoke with could not identify the criteria that were used to make these determinations. Furthermore, VBA management does not systematically collect feedback from staff evaluating the usefulness of the training they must receive to meet this requirement. Consequently, when determining the appropriateness of the 80-hour requirement, VBA has not taken into account the views of staff to gauge the effect the requirement has on them. Experienced staff had mixed views on training provided by the regional office. Staff at three regional offices said the core technical training topics set by the Compensation and Pension Service are really designed for newer staff and do not change much from year to year, and therefore experienced staff end up repeating courses. Also, a number of staff at all of the regional offices we visited told us some regional office training was not relevant for those with more experience. Conversely, other regional office staff note that although training topics may be the same from year to year, a person can learn something new each time the course is covered. Some VBA officials and regional office managers also noted that some repetition of courses is good for several reasons. Staff may not see a particular issue very often in their day-to-day work and can benefit from refreshers. Also, regional office managers at one office told us that the core technical training topics could be modified to reflect changes in policy so that courses are less repetitive for experienced staff. Many experienced staff also reported having difficulty meeting the 80-hour annual training requirement due to workload pressures. Many of the experienced staff we spoke with, at each of the regional offices we visited, told us that there is a constant struggle between office production goals and training goals. For example, office production goals can affect the availability of the regional office's instructors. A number of staff from one regional office noted that instructors were unable to spend time teaching because of their heavy workloads and because instructors' training preparation hours do not count toward the 80-hour training requirement. Staff at another regional office told us that, due to workload pressures, staff may rush through training and may not get as much out of it as they should. Performance Management System for Claims Processors Generally Conforms to Accepted Practices, but May Not Clearly Differentiate Between Performance Levels VA's performance management system for claims processors is consistent with several accepted practices for effective performance management systems in the public sector, but may not clearly differentiate between staff's overall performance levels. VA's performance management system aligns individual performance elements with broader organizational performance measures, provides performance feedback to staff throughout the year, and emphasizes collaboration. However, the system may not clearly differentiate VSRs' and RVSRs' varying levels of performance. While the system has five summary rating categories for VSRs and RVSRs, several VBA managers told us that, because of a problem with the formula used to convert ratings on individual performance elements into an overall performance category, it is more difficult for staff to be placed in certain categories than others. Performance Management System for Claims Processors Is Generally Consistent With Accepted Practices The elements used to evaluate individual VSRs' and RVSRs' performance appear to be generally aligned with VBA's organizational performance measures, something prior GAO work has identified as a well-recognized practice for effective performance management systems (see app. I). Aligning individual and organizational performance measures helps staff see the connection between their daily work activities and their organization's goals and the importance of their roles and responsibilities in helping to achieve these goals. VSRs must be evaluated on four critical elements: quality, productivity, workload management, and customer service. RVSRs are evaluated on quality, productivity, and customer service. In addition, VBA central office requires regional offices to evaluate their staff on at least one non- critical element. The central office has provided a non-critical element called cooperation and organizational support, and although regional offices are not required to use this particular element, all four offices we visited did so (see table 2). For each element, there are three defined levels of performance: exceptional, fully successful, or less than fully successful.\12\ Table 2 refers only to the fully successful level of performance for each element. --------------------------------------------------------------------------- \12\ The central office has set a minimum performance level for each element that defines the fully successful level of performance. Regional offices may set higher fully successful levels for their staff, and three of the offices we visited had set a higher level for at least one element. Regional offices also have discretion to set the level for exceptional performance in each element for their staff. Table 2--Individual Performance Elements for VSRs and RVSRs ---------------------------------------------------------------------------------------------------------------- Standard for minimum Standard for minimum fully successful fully successful Performance element How Performance is evaluated performance (journey- performance (journey- level VSR) level RVSR) ---------------------------------------------------------------------------------------------------------------- Critical ---------------------------------------------------------------------------------------------------------------- Quality A random selection of 5 cases or 85% accuracy 85% accuracy phone calls per month is reviewed for accuracy based on certain criteria, for example whether all necessary evidence was requested, proper notifications were sent to applicants, and accurate information was provided in phone calls. Any case or phone call with one or more errors is counted as one inaccurate case or call. ---------------------------------------------------------------------------------------------------------------- Productivity Number of weighted actions (VSRs) or 8 weighted actions per 3.5 weighted cases per weighted cases (RVSRs) completed per day \a\ day \b\ day. VSRs receive different weights for different actions, such as 0.125 for conducting a telephone interview or 1.50 for developing the evidence for a claim with a special issue such as radiation. RVSRs receive different levels of credit for processing cases with different numbers of issues to be evaluated. ---------------------------------------------------------------------------------------------------------------- Customer service Number of valid complaints about No more than 3 valid complaints or incidents employee's behavior from external customers or internal colleagues. ---------------------------------------------------------------------------------------------------------------- Workload management Completion of designated tasks in a Tasks are completed in Not applicable timely manner, such as obtaining the timely manner 85 results of a medical exam within a percent of the time specified period of time. ---------------------------------------------------------------------------------------------------------------- Non-critical \c\ ---------------------------------------------------------------------------------------------------------------- Cooperation and Understanding of agency goals, Interacts with colleagues professionally. organizational support interaction with colleagues, Follows directions and adheres to guidance contribution to agency goals. conscientiously. Adjusts easily to different working styles and perspectives. ---------------------------------------------------------------------------------------------------------------- Source: GAO analysis of VBA information. Note: This table includes the levels set for journey-level VSRs and RVSRs, who are considered experienced and fully trained in their positions. For some elements VBA sets different performance standards for entry-level and experienced claims processors. For example, VSRs are typically promoted to the journey-level position after about 2 years. VBA has separate, lower performance standards in the accuracy, productivity, and workload management elements for VSRs who are not yet at the journey level. Also, regional offices have the option of setting fully successful levels for their staff that are higher than the national minimum, but not lower. This table indicates instances when the regional offices we visited have set thresholds that are higher than the national minimum. \a\ Milwaukee has set a fully successful level of 10 weighted actions per day. \b\ Baltimore, Milwaukee, and Portland have set fully successful levels of, respectively, 4, 5, and 3.8 weighted cases per day. \c\ Regional offices are required to use at least one non-critical element. VBA central office provided regional offices with the cooperation and organizational support element, but regional offices are not required to use this element in particular. Three critical elements in particular--quality, workload management, and productivity--are aligned with VBA's organizational performance measures (see table 3). According to VA's strategic plan, one key organizational performance measure for VBA is overall accuracy in rating disability claims. This organizational measure is aligned with the quality element for VSRs and RVSRs, which is assessed by measuring the accuracy of their claims-processing work. An individual performance element designed to motivate staff to process claims accurately should, in turn, help VBA meet its overall accuracy goal. Two other key performance measures for VBA are the average number of days that open disability claims have been pending and the average number of days it takes to process disability claims. VSRs are evaluated on their workload management, a measure of whether they complete designated claims-related tasks within specific deadlines. Individual staff performance in this element is linked to the agency's ability to manage its claims workload and process claims within goal timeframes. Finally, a performance measure that VBA uses to evaluate the claims-processing divisions within its regional offices--and that, according to VBA, relates to the organization's overall mission--is production, or the number of compensation and pension claims processed by each office in a given time period. Individual VSRs and RVSRs are evaluated on their productivity, i.e., the number of claims-related tasks they complete per day. Higher productivity by individual staff should result in more claims being processed by each regional office and by VBA overall. Table 3--Performance Elements for VSRs and RVSRs and Corresponding Organizational Performance Measures for VBA ---------------------------------------------------------------------------------------------------------------- Performance element for VSRs and RVSRs Corresponding VBA performance measure(s) ---------------------------------------------------------------------------------------------------------------- Quality Accuracy rate for ratings of compensation claims ---------------------------------------------------------------------------------------------------------------- Productivity Number of compensation and pension claims completed by the claims- processing division within a regional office in a given time period ---------------------------------------------------------------------------------------------------------------- Workload management \a\ Average days pending for compensation and pension claims ---------------------------------------------------------------------------------------------------------------- Source: VBA and GAO analysis. \a\ Workload management element applies only to VSRs, not RVSRs. The performance management system for VSRs and RVSRs also appears to be consistent with several other accepted practices for performance management systems in the public sector: Providing and Routinely Using Performance Information to Track Organizational Priorities Providing objective performance information to individuals helps show progress in achieving organizational goals and allows individuals to manage their performance during the year by identifying performance gaps and improvement opportunities. Regional offices are supposed to use the critical and non-critical performance elements to evaluate and provide feedback to their staff. Supervisors are required to provide at least one progress review to their VSRs and RVSRs each year, indicating how their performance on each element compares to the defined standards for fully successful performance. In the offices we visited, supervisors typically provide some feedback to staff on a monthly basis. For example, VSRs in the Atlanta regional office receive a memo on their performance each month showing their production in terms of average weighted actions per day, their accuracy percentage based on a review of a sample of cases, and how their performance compared to the minimum requirements for production and accuracy. If staff members fall below the fully successful level in a critical element at any time during the year, a performance improvement plan must be implemented to help the staff member improve. Connecting Performance Expectations to Crosscutting Goals Performance elements related to collaboration or teamwork can help reinforce behaviors and actions that support crosscutting goals and provide a consistent message to all employees about how they are expected to achieve results. VSR and RVSR performance related to customer service is evaluated partly based on whether any valid complaints have been received about a staff member's interaction with their colleagues. And performance related to the cooperation and organizational support element is based on whether staff members' interaction with their colleagues is professional and constructive. Using Competencies to Provide a Fuller Assessment of Performance Competencies, which define the skills and supporting behaviors that individuals are expected to exhibit to carry out their work effectively, can provide a fuller assessment of an individual's performance. In addition to elements that are evaluated in purely quantitative terms, VBA uses a cooperation and organizational support element for VSRs and RVSRs that requires supervisors to assess whether their staff are exhibiting a number of behaviors related to performing well as a claims processor. Involving Employees and Stakeholders to Gain Ownership of the Performance Management System Actively involving employees and stakeholders in developing the performance management system and providing ongoing training on the system helps increase their understanding and ownership of the organizational goals and objectives. For example, VA worked with the union representing claims processors to develop an agreement about its basic policies regarding performance management. Also, VBA indicated that it planned to pilot revisions to how productivity is measured for VSRs in a few regional offices, partly so VSRs would have a chance to provide feedback on the changes. VA's System May Not Clearly Differentiate Between Performance Levels Clear differentiation between staff performance levels is also an accepted practice for effective performance management systems. Systems that do not result in meaningful distinctions between different levels of performance fail to give (1) employees the constructive feedback they need to improve, and (2) managers the information they need to reward top performers and address performance issues. GAO has previously reported that, in order to provide meaningful distinctions in performance for experienced staff, agencies should use performance rating scales with at least three levels, and scales with four or five levels are preferable because they allow for even greater differentiation between performance levels.\13\ If staff members are concentrated in just one or two of multiple performance levels, however, the system may not be making meaningful distinctions in performance. --------------------------------------------------------------------------- \13\ See GAO, Human Capital: Preliminary Observations on the Administration's Draft Proposed ``Working for America Act,'' GAO-06- 142T (Washington, DC: Oct. 5, 2005). --------------------------------------------------------------------------- VA's performance appraisal system has the potential to clearly differentiate between staff performance levels. Each fiscal year, regional offices give their staff a rating on each critical and non- critical performance element using a three-point scale--exceptional, fully successful, or less than fully successful. Based on a VA-wide formula, the combination of ratings across these elements is converted into one of VA's five overall performance levels: outstanding, excellent, fully successful, minimally satisfactory, and unsatisfactory (see fig. 5). Regional offices may award financial bonuses to staff on the basis of their end-of-year performance category.\14\ Prior to fiscal year 2006, VA used two performance levels--successful and unacceptable--to characterize each staff member's overall performance. To better differentiate between the overall performance levels of staff, VA abandoned this pass-fail system in that year, choosing instead to use a five-level scale. --------------------------------------------------------------------------- \14\ In three of the four offices we visited, staff members placed in the outstanding and excellent categories receive bonuses, and in one of these offices some staff in the fully successful category also receive bonuses. --------------------------------------------------------------------------- Figure 5--VA Overall Performance Appraisal Formula [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: GAO analysis of VBA information. However, there is evidence to suggest that the performance management system for VSRs and RVSRs may not clearly or accurately differentiate among staff's performance. VBA central office officials and managers in two of the four regional offices we visited raised concerns with VA's formula for translating ratings on individual performance elements into an overall performance rating.\15\ These officials said that under this formula it is more difficult for staff to be placed in certain overall performance categories than others, even if staff's performance truly does fall within one of those categories. Indeed, at least 90 percent of all claims processors in the regional offices we visited were placed in either the outstanding or the fully successful category in fiscal year 2007. (Fig. 6 shows the distribution of overall performance ratings for claims processors in each office.)\16\ --------------------------------------------------------------------------- \15\ Officials in the other two offices we visited reported no problems with the performance appraisal formula. Officials in one of these offices told us the current five-level system provides more flexibility than the previous pass/fail system. \16\ We asked VA for fiscal year 2007 performance appraisal data for VSRs and RVSRs nationally to determine whether the distribution of staff across overall performance categories is similar at the national level. While VA indicated that it collects performance appraisal data for regional office staff, the agency was unable to provide us with appraisal data specifically for VSRs and RVSRs, as these positions are part of a broader job series. --------------------------------------------------------------------------- Figure 6--Fiscal Year 2007 Overall Performance Ratings for Claims Processors in Four Regional Offices Were Concentrated in the Outstanding and Fully Successful Categories [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Note: These data cover VSRs, RVSRs, and some other claims processing staff. Central and regional office managers noted that, in particular, it is difficult for staff to receive an overall rating of excellent. Managers in one office said there are staff whose performance is better than fully successful but not quite outstanding, but under the formula it is difficult for these staff to be placed in the excellent category as the managers feel they should be. An excellent rating requires exceptional ratings in all the critical elements and a fully successful rating in at least one non-critical element. However, according to staff we interviewed, virtually all staff who are exceptional in the critical elements are also exceptional in all non-critical element(s), so they appropriately end up in the outstanding category. On the other hand, the overall rating for staff who receive a fully successful rating on just one of the critical elements--even if they are rated exceptional in all the other elements--drops down to fully successful. Managers in one regional office commented that the system would produce more accurate overall performance ratings if staff were given an overall rating of excellent when they had, for example, exceptional ratings on three of five overall elements and fully successful ratings on the other two. An official in VA's Office of Human Resources Management acknowledged that there may be an issue with the agency's formula. Although neither VBA nor VA central office officials have examined the distribution of VSRs and RVSRs across the five overall performance ratings, VA indicated it is considering changes to the system designed to allow for greater differentiation in performance ratings. For example, one possible change would be to use a five-point scale for rating individual elements--probably mirroring the five overall performance rating categories of outstanding, excellent, fully successful, minimally satisfactory, and unsatisfactory--rather than the current three-point scale. Under the proposed change, a staff member who was generally performing at the excellent but not outstanding level could get excellent ratings in all the elements and receive an overall rating of excellent. This change must still be negotiated with several stakeholder groups, according to the VA official we interviewed. Conclusions In many ways, VBA has developed a training program for its new staff that is consistent with accepted training practices in the Federal Government. However, because VBA does not centrally evaluate or collect feedback on training provided by its regional offices, it lacks the information needed to determine if training provided at regional offices is useful and what improvements, if any, may be needed. Ultimately, this information would help VBA determine if 80 hours of training annually is the right amount, particularly for its experienced staff, and whether experienced staff members are receiving training that is relevant for their positions. Identifying the right amount of training is crucial for the agency as it tries to address its claims backlog. An overly burdensome training requirement needlessly may take staff away from claims processing, while too little training could contribute to processing inaccuracies. Also, without collecting feedback on regional office training, VBA may not be aware of issues with the implementation of its TPSS, the online training tool designed to ensure consistency across offices in technical training. Setting aside the issue of how many hours of training should be required, VBA does not hold its staff accountable for fulfilling their training requirement. As a result, VBA is missing an opportunity to clearly convey to staff the importance of managing their time to meet training requirements as well as production and accuracy goals. With the implementation of its new learning management system, VBA should soon have the ability to track training completed by individual staff members, making it possible to hold them accountable for meeting the training requirement. As with its training program for VSRs and RVSRs, the VA is not examining the performance management system for claims processors as closely as it should. VBA is generally using the right elements to evaluate its claims processors' performance, and the performance appraisals have the potential to give managers information they can use to recognize and reward higher levels of performance. However, evidence suggests the formula used to place VSRs and RVSRs into overall performance categories may not clearly and accurately differentiate among staff's performance levels. Absent additional examination of the distribution of claims processors among overall performance categories, VA lacks a clear picture of whether its system is working as intended and whether any adjustments are needed. Recommendations for Executive Action The Secretary of Veterans Affairs should direct VBA to: Collect and review feedback from staff on the training conducted at the regional offices to determine if the 80-hour annual training requirement is appropriate for all VSRs and RVSRs; the extent to which regional offices provide training that is relevant to VSRs' and RVSRs' work, given varying levels of staff experience; and whether regional offices find the TPSS a useful learning tool and, if not, what adjustments are needed to make it more useful; and Use information from its new learning management system to hold individual VSRs and RVSRs accountable for completing whatever annual training requirement it determines is appropriate. The Secretary of Veterans Affairs should also examine the distribution of claims processing staff across overall performance categories to determine if its performance appraisal system clearly differentiates between overall performance levels, and if necessary adjust its system to ensure that it makes clear distinctions. Agency Comments We provided a draft of this report to the Secretary of Veterans Affairs for review and comment. In VA's written comments (see app. IV), the agency agreed with our conclusions and concurred with our recommendations. For example, VBA plans to consult with regional office staff to evaluate its annual 80-hour training requirement and will examine if staff performance ratings clearly differentiate between overall performance levels. VA also provided technical comments that were incorporated as appropriate. We are sending copies of this report to the Secretary of Veterans Affairs, relevant congressional Committees, and others who are interested. We will also provide copies to others on request. The report is also available at no charge on GAO's Web site at http:// www.gao.gov. Please contact me on (202) 512-7215 if you or your staff have any questions about this report. Contact points for the Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors are listed in appendix V. Sincerely, Daniel Bertoni Director, Education, Workforce, and Income Security Issues __________ Appendix I: Objectives, Scope, and Methodology We were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has the Veterans Benefits Administration (VBA) developed a strategic approach to planning training for claims processors and how well is their training designed, implemented, and evaluated? And (3) To what extent is the performance management system for claims processors consistent with generally accepted performance management practices in the public sector? To answer these questions, we reviewed documents and data from the central office of the Department of Veterans Affairs' Veterans Benefits Administration (VBA) and interviewed VBA central office officials. We conducted site visits to and collected data from four VBA regional offices, and visited the Veterans Benefits Academy. We also interviewed officials from the American Federation of Government Employees, the labor union that represents Veterans Service Representatives (VSR) and Rating Veterans Service Representatives (RVSR). We compared VBA's training and performance management systems to accepted human capital principles and criteria compiled by GAO. We conducted this performance audit from September 2007 through May 2008 in accordance with generally accepted Government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Regional Office Site Visits We conducted site visits to 4 of VBA's 57 regional offices-- Atlanta; Baltimore; Milwaukee; and Portland, Oregon. We judgmentally selected these offices to achieve some diversity in geographic location, number of staff, and claims processing accuracy rates, and what we report about these sites may not necessarily be representative of any other regional offices or all regional offices (see fig. 7).\17\ --------------------------------------------------------------------------- \17\ To determine each office's accuracy performance in fiscal year 2007, we used data obtained from VBA's Systematic Technical Accuracy Review (STAR) system. In an earlier GAO report, Veterans' Benefits: Further Changes in VBA's Field Office Structure Could Help Improve Disability Claims Processing, GAO-06-149 (Washington, DC: Dec. 9, 2005), we identified problems that affected the use of the STAR data to make distinctions in accuracy among regional offices. However, for the purposes of site selection for our current review, we judged the STAR data to be sufficiently reliable. We made this determination based on a sensitivity analysis we did on earlier year data that considered sampled cases that were not sent in for STAR review. After this analysis we found that even with the existing limitations in the STAR data, Milwaukee and Baltimore had higher accuracy scores and Atlanta and Portland had lower accuracy scores. Even though the sensitivity analysis was done on earlier year data, the ranking of the four offices was similar in fiscal year 2007, showing that the offices we deemed to have higher accuracy scores in an earlier year still had higher accuracy scores in fiscal year 2007 and the same remained true for the offices with lower accuracy scores. --------------------------------------------------------------------------- Figure 7--Regional Offices Selected for Site Visits [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] \a\ Full-time equivalents as of September 2007. \b\ Rank among all 57 regional offices. \c\ Claims-processing accuracy rate for the period of August 1, 2006 to July 31, 2007. During our site visits, we interviewed regional office managers, supervisors of VSRs and RVSRs, VSRs, and RVSRs about the training and performance management practices in their offices. The VSRs and RVSRs we interviewed at the four regional offices had varying levels of experience at VBA. Regional office managers selected the staff we interviewed. We also observed a demonstration of VBA's online learning tool, the Training and Performance Support System (TPSS), and collected data from the regional offices on, for example, the training they provided during fiscal year 2007.\18\ In conjunction with our visit to the Baltimore regional office, we also visited VBA's Veterans Benefits Academy, where we observed classes for VSRs and RVSRs and interviewed the Director of the Academy. --------------------------------------------------------------------------- \18\ One question we asked the regional offices was whether each course on their fiscal year 2007 training plan addressed a core technical training topic. For three of the offices, the data we received did not cover all training hours provided during the fiscal year, but each office provided data on at least 99 percent of its training hours. --------------------------------------------------------------------------- Assessment of VBA's Training for Claims Processors To determine whether VBA's training program is consistent with accepted training practices in the public sector, we relied partly on a guide developed by GAO that lays out principles that Federal agencies should follow to ensure their training is effective.\19\ This guide was developed in collaboration with Government officials and experts in the private sector, academia, and nonprofit organizations; and in conjunction with a review of laws, regulations and literature on training and development issues, including previous GAO reports. The guide lays out the four broad components of the training and development process (see fig. 8). --------------------------------------------------------------------------- \19\ GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, DC: March 2004). --------------------------------------------------------------------------- Figure 8--Four Components of the Training and Development Process [GRAPHIC NOT AVAILABLE IN TIFF FORMAT] Source: GAO Note: The evaluation component may include the use of participant feedback to ensure continuous improvement, as well as an assessment of the impact of training on organizational performance. We have reported that higher-level evaluations that attempt to measure the return on investment in a training program may not always be appropriate, given the complexity and costs associated with efforts to directly link training programs to improved individual and organizational performance. The guide also provides key questions for Federal agencies to consider in assessing their performance in each component. (See table 4 for a sample of these questions.) Table 4--Selected Key Questions to Consider in Assessing Agency's Training Program ---------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------- Planning/Front End Analysis Does the agency have training goals and related performance measures that are consistent with its overall mission, goals, and culture? How does the agency identify the appropriate investment to provide for training and development efforts and prioritize funding so that the most important training needs are addressed first? ---------------------------------------------------------------------------------------------------------------- Design and Development What criteria does the agency use in determining whether to design training and development programs in-house or obtain these services from a contractor or other external source? Does the agency use the most appropriate mix of centralized and decentralized approaches for its training and development programs? ---------------------------------------------------------------------------------------------------------------- Implementation What steps do agency leaders take to communicate the importance of training and developing employees, and their expectations for training and development programs to achieve results? How does the agency select employees to participate in training and development efforts? ---------------------------------------------------------------------------------------------------------------- Evaluation To what extent does the agency systematically plan for and evaluate the effectiveness of its training and development efforts? How does the agency incorporate evaluation feedback into the planning, design, and implementation of its training and development efforts? ---------------------------------------------------------------------------------------------------------------- Source: GAO. In addition, GAO training experts reviewed VBA materials, including training curricula, lesson plans, and course evaluation forms, to determine if these materials are consistent with accepted training practices. Assessment of VBA's Performance Management System for Claims Processors In assessing the performance management system for VSRs and RVSRs, we relied primarily on a set of accepted practices of effective public sector performance management systems that has been compiled by GAO.\20\ To identify these accepted practices, GAO reviewed its prior reports on performance management that drew on the experiences of public sector organizations both in the United States and abroad. For the purpose of this review, we focused on the six accepted practices most relevant for VBA's claims-processing workforce (see table 5). --------------------------------------------------------------------------- \20\ GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success, GAO-03-488 (Washington, DC: Mar. 14, 2003). Table 5--Selected Accepted Practices for Effective Performance Management Systems ---------------------------------------------------------------------------------------------------------------- Practice Description ---------------------------------------------------------------------------------------------------------------- Aligning individual performance Explicitly aligning individuals' daily activities with broader expectations with organizational goals results helps individuals see the connection between their work and organizational goals and encourages individuals to focus on their roles and responsibilities to help achieve those broader goals. ---------------------------------------------------------------------------------------------------------------- Connecting performance expectations to Fostering collaboration, interaction, and teamwork across crosscutting goals organizational boundaries to achieve results strengthens accountability for these results. ---------------------------------------------------------------------------------------------------------------- Providing and routinely using performance Providing objective performance information to both managers and information to track organizational staff to show progress in achieving organizational results and other priorities priorities helps them manage during the year, identify performance gaps, and pinpoint improvement opportunities. ---------------------------------------------------------------------------------------------------------------- Using competencies to provide a fuller Using competencies, which define the skills and supporting behaviors assessment of performance that individuals need to effectively contribute to organizational results, can provide a fuller picture of an individual's performance. ---------------------------------------------------------------------------------------------------------------- Making meaningful distinctions in Providing individuals with candid and constructive feedback helps performance them maximize their contribution, and providing management with objective and fact-based information that clearly differentiates between different levels of performance enables it to reward top performers and deal with poor performers. ---------------------------------------------------------------------------------------------------------------- Involving employees and stakeholders to Actively involving employees and stakeholders in developing the gain ownership of performance management performance management system and providing ongoing training on the systems system helps increase their understanding and ownership of the organizational goals and objectives. ---------------------------------------------------------------------------------------------------------------- Source: GAO. Appendix II: Fiscal Year 2007 Core Technical Training Requirements for VSRs and RVSRs ------------------------------------------------------------------------ Position Course title or topic Training source ------------------------------------------------------------------------ Decision Review Officers (DRO) GS13/Rating Veterans Service Representative s (RVSR) GS7- 12 (Seasoned) Required: 80 hours Effective Dates C&P Training Website http:// Any DRO or RVSR Diabetes Mellitus cptraining.vba.va.gov/ who conducts a Leishmaniasis C&P_Training/RVSR/ training Original Compensation RVSR_Tng_Curr.htm session will Ratings also be given Original Pension credit for Ratings those training Original DIC Ratings hours as part Rating re-opened of their claims training Claims for Increase requirement. New and Material Evidence Re-opened DIC ratings Routine Future Exams 3.105(e) reductions Paragraph 28/29/30 ratings Due Process Provisions Clear and unmistakable errors (3.105(a)) Ancillary Benefits Accrued Ratings Musculoskeletal issues Eye-Vision Issues Infectious Diseases Ear-Hearing Respiratory Disorders Cardiovascular Issues Digestive Issues Genitourinary System Gynecology Hemic/Lymphatic Endocrine (other than DM) Neurological Mental Disorder (other than PTSD) PTSD Special Monthly Compensation (SMC) The Appeals Process Responsibilities of a DRO Hearings Informal Conferences Resolution of Claims Certifying a case to BVA Processing Remands Preparing a Statement of the Case (SOC) Preparing a Supplemental Statement of the Case (SSOC) Role of the Rating Specialist Benefit of the Doubt Weighing Evidence 60 Hours of the required 80 Hours will be selected from the suggested topics above. The remaining 20 hours will be selected at the Stations discretion based upon their own individual quality review. (Training provided from the above topics can be focused on a particular aspect of the topic; i.e. Cold Injuries and Rating Hypertension from Cardiovascular issues could be separate classes) Participation in Agency Advancement Programs (i.e., LEAD, LVA) does not substitute for Required training requirements. ------------------------------------------------------------------------ Veteran Service Representative (VSR) GS 7-12 (Seasoned) Required: 80 hours Reference Materials: Any Super Manual Senior VSR, Training & WARMS Senior VSR, or C&P Website VSR, who Claims Folder conducts a Maintenance training Records Management session will POA/Service Orgs. also be given Original Compensation credit for Claims those training Re-opened Compensation hours Claims including VA Form 21-526 preparation Establishing Veteran time as part Status of their Claims Recognition training Duty to Assist requirement. Requesting VA Exams Issue Specific Claims Development Asbestos Claims Development Herbicide Claims Development POW Claims Development C&P Training Website Intro. to Ratings Paragraph 29 & 30 Ratings Ratings & BDN BDN 301 Interface PCGL Award Letters Dependents and the BDN Compensation Offsets Drill Pay Waivers Pension Awards Processing & BDN Hospital Reductions Burial Benefits Death Pension Accrued Benefits Accrued Awards & the BDN Apportionments Special Monthly Pension Helpless Child Incompetency/Fiduciary Arrangements Claims Processing Auto Allowance and Adaptive Equipment Special Adapted Housing Special Home Adaptation Grants Incarcerated Veterans Processing Write Outs FOIA/Privacy Act Telephone & Interview Techniques Telephone Development IRIS Introduction to VACOLS Education Benefits Insurance Benefits National Cemetery VR&E Benefits Loan Guaranty Benefits General Benefits--FAQs Suicidal Caller Guidance Non-Receipt of BDN Payments Mail Handling Income & Net Worth Determinations Bootcamp test and review of VSR Readiness Guide (2 HRS Required) Reference Material Training and Navigation (1 HR Required) Appeals and Ancillary Benefits Ready to Rate Development Customer Service FNOD Info and PMC Process Intro. to Appeals Process DRO Selection Letter Income Adjustment Materials Income Adjustments 60 Hours of the required 80 Hours will be selected from the suggested topics above. The remaining 20 hours will be selected at the Stations discretion based upon their own individual quality review. ------------------------------------------------------------------------ Veterans Services Representative (VSR) GS 7-12 (New) Required: Entire Curriculum Curriculum is posted (Follow C&P on C&P Training Prescribed Intranet Site Curriculum for new VSRs, as posted on intranet.) http://cptraining.vba.va.gov/ SHARE (BDN & CEST) COVERS PIES Return with Honor Video MAPD AMIE/CAPRI Medical TPSS (Medical Terminology) Reader Focused Writing Tools -------------------------------------------------------- Pre-Determination Team Training: http://cptraining.vba.va.gov/ C&P Website Claims Folder Maintenance Records Management POA/Service Organizations Compensation Original Compensation Claims Non-Original Compensation Claims VA Form 21-526, App. For Compensation or Pension Establishing Veteran Status Claims Recognition Duty to Assist Selecting the Correct Worksheet for VA Exams Issue Specific Claim Development Asbestos Claim Development Herbicide Claim Development POW Claim Development Radiation Claim Development PTSD Claim Development Undiagnosed Illness Claim Development Dependency Contested Claims Deemed Valid and Common-law Marriage Continuous Cohabitation Pension Intro. To Disability Pension Overview of SHARE (SSA) Administrative Decision Process Character of Discharge Line of Duty--Willful Misconduct Claims Development Workload Management Utilizing WIPP -------------------------------------------------------- Post-Determination Team Training: http://cptraining.vba.va.gov/ Ratings & the BDN BDN 301 Interface Video PCGL Award Letters PCGL Dependents & the BDN Compensation Offsets Drill Pay Waivers Star Reporter Pension Awards Processing & the BDN Hospital Reductions Burial Benefits Disallowance Processing DIC Benefits Death Pension Accrued Benefits Accrued Awards & the BDN Apportionment Special Monthly Pension Helpless Child Incompetency/Fiduciary Arrangements Claims Processing Automobile Allowance and Adaptive Equipment Specially Adapted Housing and Special Home Adaptation Grants Incarceration Processing Computer Write Outs DEA Training (req. added 4/06) -------------------------------------------------------- Public Contact Team Training: http://cptraining.vba.va.gov/ ------------------------------------------------------------------------ Source: VBA. Appendix III: Excerpts From Fiscal Year 2007 Training Plans for Four Regional Offices Each training plan we reviewed contained the same informational categories, some of which were what courses were offered by the regional office, whether or not the course was conducted, and how many employees completed the training. Although the fiscal year 2007 training plans we reviewed include data on whether and when the course was actually completed, the initial training plans submitted at the beginning of the fiscal year of course do not have this information. The lists provided below include the first 25 courses listed on each plan alphabetically, a small sample of the courses that the regional offices reported they completed for the fiscal year. Table 6--Excerpt From Atlanta Regional Office Training Plan ------------------------------------------------------------------------ Number of Total hours Course name employees of training completed completed ------------------------------------------------------------------------ Accrued Benefits 15 150 ------------------------------------------------------------------------ Accrued Ratings (2 sessions conducted) 47 80 ------------------------------------------------------------------------ Administrative Decisions 15 60 ------------------------------------------------------------------------ Ancillary Benefits 14 14 ------------------------------------------------------------------------ Appeals and Ancillary Benefits (2 sessions 26 41 conducted) ------------------------------------------------------------------------ Apportionments (2 sessions conducted) 29 194 ------------------------------------------------------------------------ Asbestos Claims Development 9 9 ------------------------------------------------------------------------ Auto Allowance/Special Adapted Housing/ 15 30 Special Home Adaptation Grant ------------------------------------------------------------------------ Benefits Delivery Network 301 Interface (2 48 48 sessions conducted) ------------------------------------------------------------------------ Beneficiary Identification Records Locator 17 17 Subsystem Update ------------------------------------------------------------------------ Blast Injuries (2 sessions conducted) 20 20 ------------------------------------------------------------------------ Burial Benefits (2 sessions conducted) 36 100 ------------------------------------------------------------------------ Board of Veterans Appeals Examinations 46 69 ------------------------------------------------------------------------ Compensation & Pension Website (2 sessions 108 270 conducted) ------------------------------------------------------------------------ Change of Address/Power of Attorney 17 34 Processing/No Record Mail ------------------------------------------------------------------------ Cardiovascular Issues 38 76 ------------------------------------------------------------------------ Certifying a Case to Board of Veterans 12 12 Appeals ------------------------------------------------------------------------ Character of Discharge 78 78 ------------------------------------------------------------------------ Claims Folder Maintenance (2 sessions 17 28 conducted) ------------------------------------------------------------------------ Claims for Direct Service Connection/ 34 34 Aggravation/Presumptive Service Connection ------------------------------------------------------------------------ Claims for Increase 29 58 ------------------------------------------------------------------------ Claims Processing 139 69.5 ------------------------------------------------------------------------ Claims Recognition 84 336 ------------------------------------------------------------------------ Compensation Offsets (3 sessions conducted) 167 352.5 ------------------------------------------------------------------------ Computer Security and LAN Procedures 6 6 ------------------------------------------------------------------------ Source: VBA. Note: Atlanta's training plan reported the regional office conducted a total of 133 courses for fiscal year 2007. Table 7--Excerpt From Baltimore Regional Office Training Plan ------------------------------------------------------------------------ Number of Total hours Course name employees of training completed completed ------------------------------------------------------------------------ Accrued Benefits 5 10 ------------------------------------------------------------------------ Automated Medical Information Exchange/ 6 48 Compensation and Pension Record Interchange ------------------------------------------------------------------------ Appeals and Ancillary Benefits 3 3 ------------------------------------------------------------------------ Asbestos Claims Development 3 3 ------------------------------------------------------------------------ Access Standardized Performance Elements 2 2 Nationwide ------------------------------------------------------------------------ Auto Allowance and Adaptive Equipment (2 16 8 sessions conducted) ------------------------------------------------------------------------ Benefits Delivery at Discharge Development 14 21 ------------------------------------------------------------------------ Benefits Delivery Network 301 Interface (2 5 7 sessions conducted) ------------------------------------------------------------------------ Benefit of the Doubt 3 12 ------------------------------------------------------------------------ Burial Benefits (2 sessions conducted) 7 14 ------------------------------------------------------------------------ Compensation & Pension Website (3 sessions 15 36.5 conducted) ------------------------------------------------------------------------ Certifying a Case to Board of Veterans 3 12 Appeals ------------------------------------------------------------------------ Character of Discharge 15 7.5 ------------------------------------------------------------------------ Claims Folder Maintenance 7 14 ------------------------------------------------------------------------ Claims Recognition 5 20 ------------------------------------------------------------------------ Communication--Nonverbal Cues 3 1.5 ------------------------------------------------------------------------ Computer Security and LAN Procedures 6 12 ------------------------------------------------------------------------ Conducting a Field Exam 3 1.5 ------------------------------------------------------------------------ Continuous Cohabitation (2 sessions 20 20 conducted) ------------------------------------------------------------------------ Core Values 5 5 ------------------------------------------------------------------------ Control of Veterans Records System (3 10 12.5 sessions conducted) ------------------------------------------------------------------------ Customer Service (5 sessions conducted) 40 416 ------------------------------------------------------------------------ Dealing with Difficult Payee Situations 3 3 ------------------------------------------------------------------------ Deemed Valid and Common Law Marriages (2 20 12.5 sessions conducted) ------------------------------------------------------------------------ Dependency Issues (3 sessions conducted) 22 26.5 ------------------------------------------------------------------------ Source: VBA. Note: Baltimore's training plan reported the regional office conducted a total 191 courses for fiscal year 2007. Table 8--Excerpt From Milwaukee Regional Office Training Plan ------------------------------------------------------------------------ Number of Total hours Course name employees of training completed completed ------------------------------------------------------------------------ 8824e 1 1 ------------------------------------------------------------------------ Administrative Decisions 14 91 ------------------------------------------------------------------------ Advanced Data Manipulation in Excel (VA 1 4 Learning Online) ------------------------------------------------------------------------ All--Litigation Hold Memo 130 32.5 ------------------------------------------------------------------------ All-Encryption Training 1 0.5 ------------------------------------------------------------------------ Ancillary Benefits 21 42 ------------------------------------------------------------------------ Auto Allowance and Adaptive Equipment 28 28 ------------------------------------------------------------------------ Blast Injuries (Video) 33 33 ------------------------------------------------------------------------ Board of Veterans Appeals review 7 14 ------------------------------------------------------------------------ Compensation & Pension Website 41 102.5 ------------------------------------------------------------------------ Claims Assistant--Burials 4 4 ------------------------------------------------------------------------ Claims Assistant/Program Support Clerk-- 24 24 Power of Attorney ------------------------------------------------------------------------ Claims Assistant/Program Support Clerk-- 21 178.5 Share and Cest ------------------------------------------------------------------------ Claims Assistant/Program Support Clerk-- 25 25 Veterans Appeals Control and Locator System ------------------------------------------------------------------------ Cardiovascular Issues 30 180 ------------------------------------------------------------------------ Challenge 07-02 Centralized Training 6 720 ------------------------------------------------------------------------ Challenge 07-02 Post Centralized Training 6 1,440 ------------------------------------------------------------------------ Challenge 07-02 Pre-Req. 6 720 ------------------------------------------------------------------------ Claims Folder Maintenance 41 82 ------------------------------------------------------------------------ Claims Recognition 26 26 ------------------------------------------------------------------------ Character of Discharge Determinations, Line 11 24.75 of Duty Determinations, and Administrative Decisions. ------------------------------------------------------------------------ Compensation Offsets 27 94.5 ------------------------------------------------------------------------ Core Values 2 3.5 ------------------------------------------------------------------------ Control of Veterans Records System (2 2 3 sessions conducted) ------------------------------------------------------------------------ Compensation and Pension Examination Project 1 18 ------------------------------------------------------------------------ Source: VBA. Note: Milwaukee's training plan reported the regional office conducted a total of 323 courses for fiscal year 2007. Table 9--Excerpt From Portland Regional Office Training Plan ------------------------------------------------------------------------ Number of Total hours Course name employees of training completed completed ------------------------------------------------------------------------ 020 Development 3 16.5 ------------------------------------------------------------------------ 2007 Veterans Service Center Management 1 26 Workshop ------------------------------------------------------------------------ 3.105(e) Reductions 15 15 ------------------------------------------------------------------------ 38 CFR 3.14 & Pyramiding 2 0.5 ------------------------------------------------------------------------ 5-Tier Performance Evaluations 7 5.25 ------------------------------------------------------------------------ 8824 Preparation 1 5 ------------------------------------------------------------------------ Absence & Leave Circular Training 13 13 ------------------------------------------------------------------------ Account Analysis 3 6 ------------------------------------------------------------------------ Account Audits 3 6 ------------------------------------------------------------------------ Accrued Awards & the Benefits Delivery 2 2 Network ------------------------------------------------------------------------ Accrued Ratings 16 4 ------------------------------------------------------------------------ Add Dependents/Verifying Service 18 9 ------------------------------------------------------------------------ Admin Decisions/Rebuilt/Special Monthly 4 18 Compensation ------------------------------------------------------------------------ Administrative Decisions 5 2.5 ------------------------------------------------------------------------ Agent Orange development 4 4 ------------------------------------------------------------------------ Amputation Rule 2 0.5 ------------------------------------------------------------------------ Ancillary Benefits 28 28 ------------------------------------------------------------------------ Appeal Procedures--Refresher 3 5.25 ------------------------------------------------------------------------ Appeals 33 33 ------------------------------------------------------------------------ Appeals and Ancillary Benefits (3 sessions 34 13 conducted) ------------------------------------------------------------------------ Appeals--Training and Performance Support 1 16 System modules ------------------------------------------------------------------------ Application/eligibility 1 3.5 ------------------------------------------------------------------------ Apportionments (2 sessions conducted) 4 14.5 ------------------------------------------------------------------------ Asbestos Claims Development 23 23 ------------------------------------------------------------------------ Access Standardized Performance Elements 6 6 Nationwide ------------------------------------------------------------------------ Source: VBA. Note: Portland's training plan reported the regional office conducted a total of 509 courses for fiscal year 2007. Appendix IV: Comments From the Department of Veterans Affairs [GRAPHICS NOT AVAILABLE IN TIFF FORMAT] Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Daniel Bertoni (202) 512-7215 [email protected]. Staff Acknowledgments In addition to the contact named above, Clarita Mrena, Assistant Director; Lorin Obler, Analyst-in-Charge; Carolyn S. Blocker; and David Forgosh made major contributions to this report; Margaret Braley, Peter Del Toro, Chris Dionis, Janice Latimer, and Carol Willett provided guidance; Walter Vance assisted with study design; Charles Willson helped draft the report; and Roger Thomas provided legal advice. Related GAO Products Veterans' Benefits: Improved Management Would Enhance VA's Pension Program. GAO-08-112. Washington, DC: February 14, 2008. Veterans' Disability Benefits: Claims Processing Challenges Persist, while VA Continues to Take Steps to Address Them. GAO-08-473T. Washington, DC: February 14, 2008. Disabled Veterans' Employment: Additional Planning, Monitoring, and Data Collection Efforts Would Improve Assistance. GAO-07-1020. Washington, DC: September 12, 2007. Veterans' Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, DC: September 30, 2003. Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government. GAO-03-893G. Washington, DC: July 2003. Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. GAO-03-488. Washington, DC: March 14, 2003. Major Management Challenges and Program Risks: Department of Veterans Affairs. GAO-03-110. Washington, DC: January 1, 2003. Veterans' Benefits: Claims Processing Timeliness Performance Measures Could Be Improved. GAO-03-282. Washington, DC: December 19, 2002. Veterans' Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. Washington, DC: August 16, 2002. Veterans' Benefits: Training for Claims Processors Needs Evaluation. GAO-01-601. Washington, DC: May 31, 2001. Veterans' Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, DC: March 1, 1999. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 25, 2008 Michael Ratajczak Decision Review Officer Cleveland Veterans Affairs Regional Office 80 F Street, NW Washington, DC 20001 Dear Mr. Ratajczak: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Response of Michael Ratajczak to: Questions From the House Committee on Veteran's Affairs, Subcommittee on Disability Assistance and Memorial Affairs Question 1: How useful is online training as opposed to classroom style? Which do employees prefer? Response: The general experience of VBA employees is that online training is less useful than classroom training. Online training is extremely limited insofar as it is not easily conformed to the constant changes permeating VBA's claims process. A computer based curriculum developed only weeks prior to an employee participating in computer based training might be inaccurate in some aspects due to changes in law or policy. In addition, online training is not generally responsive to questions not considered by its designer. So, a question may be suggested to a VBA employee based upon their unique experiences and some aspect of an online curriculum but that curriculum could not provide any useful guidance on the issue presented. In contrast, properly administered classroom training encourages real-time interaction between the instructor and the participants, and provides immediate feedback for questions presented. Moreover, classroom training can easily incorporate any recent changes in law, policy, or regulations into its curriculum, thereby providing more relevant and up to date information. A classroom setting provides an opportunity for participants to deviate from the prepared curriculum to related topics having direct impact on their ability to perform their duties, and provides a similar opportunity for instructors to identify and correct common misconceptions among participants. Properly administered classroom training provides immediate, consistent, useful and relevant guidance based upon practical concerns expressed by participants. It is the most efficient manner to ensure consistency in processing and resolving complex claims, and ultimately provides the best service to veterans. Of course, the foregoing comments assume that the ``online training'' in question is based upon a static format, as characterized for example by VBA's TPSS program. As was suggested by some questions posed during the September 18, 2008, hearing, some training can be effectively presented electronically through the use of teleconferencing technology. Such an approach has the advantage of providing uniform presentation of the subject matter to a widely dispersed audience by an authoritative instructing staff. However, the utility of such training is limited by the number of participants and the amount of available time. The danger of relying on teleconference training to reach a large audience is that it cannot guarantee effective feedback for the concerns of every participant. The best approach to continuing training at VBA incorporates the most useful aspects of computer based, classroom, and teleconference instruction. Such an approach would dictate that VBA's Central Office training staff create a standard curriculum and an electronic claims file relevant to issues identified as requiring additional training. Central Office training staff would then distribute the electronic materials and curriculum to instructors at each Regional Office who are certified by Central Office to conduct centralized training. After soliciting comments from Regional Office instructors Central Office staff could facilitate a teleconference among the Regional Office instructors, addressing their disparate concerns and providing uniform guidance as to how they should approach the materials and topics at hand. Finally, Regional Office instructors could provide training to adjudicators in a classroom setting at each Regional Office in accordance with the guidance provided by Central Office staff during the teleconference based upon Central Office's approved curriculum and electronic claims file. Such an approach would ensure, to the extent possible, that training throughout VBA was uniform in approach and based upon a common experience and guidance from Central Office. Ultimately, institutionalizing uniform training based upon shared common experience will foster greater consistency in approach to resolving veterans claims across Regional Offices, and eliminate ad hoc approaches which result in disparity of entitlement determinations and disability evaluations. Questions 2: VA reinstituted the certification exam in August. What was the response of employees who took the test? Did AFGE have input on the new test? Response: AFGE does not have any direct input on designing or scoring any certification test promulgated by VBA. AFGE is afforded the opportunity to designate a representative to witness the design of certification tests and scoring workshops in accordance with a provision of the Master Agreement between VA and the Union. AFGE representatives to certification testing activities can informally express concerns to VBA management during design and scoring activities. However, in the absence of any concession by VBA management of the validity of a concern raised by AFGE at a certification design or scoring workshop, AFGE has no effective means of directly influencing the certification process prior to its implementation. In the absence of an immediate concession by VBA management to a point raised by an AFGE representative in a certification design or scoring activity, any information gathered by AFGE representatives through their participation in the certification test design or scoring activities can only be utilized to effect certification testing through formal labor-management proceedings. AFGE members do serve as subject matter experts who write the questions for certification tests, and help determine essential knowledge and skills of positions that need to be tested for successful certification. However, AFGE does not attempt to influence its membership with regard to their duties as subject matter experts, and does not actively encourage subject matter experts to report to the Union concerning policies or directions expressed by VBA management at any certification testing design or scoring activity. Our experience is that employees who pass certification testing express a general dissatisfaction regarding the relevance of the material tested. Employees who do not pass certification testing express similar dissatisfaction, and also express frustration that they are not provided with any meaningful feedback regarding where they need to improve their understanding in order to be certified. That frustration is magnified when our members consider that they can be otherwise fully successful in all aspects of their position, but subject to adverse action or arrested career development solely because if their inability to pass a certification test. Regardless of their success in certification testing, frustration arises among AFGE members when they contemplate that supervisors who evaluate their job performance on a daily basis do not presently have any similar certification requirement. The certification process, at present, leaves our members with the curious potential to be evaluated by supervisors who are not certified to have the same level of knowledge and skill level as they do. The impact of that potential has an obvious detrimental impact on the morale of our members. Question 3: According to DAV testimony, the Union has objected to the frequency and other requirements DAV suggests for testing at each phase. What are the AFGE objections? Response: Some of our concerns are expressed in the last two paragraphs of our response to the preceding question. While AFGE has no objection in principle to the concept of certification testing, AFGE does have concerns about the practical application of the certification process and the effects that flawed certification testing can have on our members. As currently constituted, certification testing is not designed to provide VBA employees with any specific feedback concerning what knowledge or skills they need to improve. This is a disservice both to employees who have been successfully certified and want to improve their job performance, and to employees who are attempting to be successfully certified and want to improve their job performance. As currently constituted, the certification testing program is not a useful learning tool. Rather, it more akin to a hazing ritual which somewhat arbitrarily determines whether an employee can continue in their current position or ascend the career ladder. Some of our older members with long histories of fully successful performance evaluations who have many years of experience successfully serving veterans have found it difficult to successfully complete their certification requirement because of the nature and format of the test. In contrast, less experienced members with recent experience in environments that regularly utilize standardized testing (e.g. recent college graduates) may perform adequately on certification testing, but not be as astute in serving veterans due to their simple lack of experience. While VBA management has recently expressed an interest in replacing older employees with younger, more technologically savvy employees, AFGE does not believe that interest is legitimate. Nor can AFGE, in service to its full membership, accept that the design of certification tests should favor an employee with the ability to access reference materials quickly over an employee who understands the correct course of action due to long experience. AFGE does not believe that any employee who can and does successfully perform the duties of their position on a daily basis should be subject to any adverse employment action solely because he or she cannot successfully complete a single certification test. Nor does AFGE believe the measure of an employee's timely, successful, respectful and full service to veterans can be made by reference to a single test. Question 4: In your statement you suggest that the productivity requirements are too high. On average, an experienced rater is completing 2-3 claims a day. Are you suggesting that's too many? How many cases a day should a rater be able to adjudicate? Response: Rating Veterans Service Representative may, in fact complete only 2-3 claims per day on average. If raters were only required to complete 2 to 3 cases per day on average, no one could object since that requirement would be based on an empirically verified direct relationship between what can be accomplished and what is required. Unfortunately, VBA's National Performance Standard for Rating Veterans Service Representatives requires that Journey-level RVSRs (i.e. those with two or more years of experience) complete a minimum of 3\1/2\ weighted cases per day. This misfortune is amplified when one considers that the 3\1/2\ weighted case per day requirement is a floor and not a ceiling, and that individual Regional Offices are free to set daily performance standards for RVSRs on an ad hoc basis which exceed 3\1/2\ weighted cases per day. So, for example, Tiger Team RVSRs are required to complete 4 weighted cases per day. This misfortune becomes a tragedy, both for claims adjudicators and for veterans, when RVSRs who are afforded the ``privilege'' of participating in their negotiated Flexiplace agreement and work from their homes are required to produce even more cases in exchange for that ``privilege.'' This Flexiplace tariff is usually an additional completed case per day. Hence, the aforementioned Tiger Team employee would have to complete 5 cases per day in days they work from home in order to maintain successful performance and their eligibility to participate in the Flexiplace program. The misfortunes and tragedies described in the preceding paragraph become absurd when one contemplates how the floor of 3\1/2\ cases per day could be justified, let alone any upward deviation from that productivity requirement. Regarding increased Flexiplace productivity requirements, any such requirement is either tantamount to an admission that daily supervision in the workplace is obtrusive and counterproductive or, that RVSRs are required to work uncompensated hours in addition to their scheduled tour of duty in order to fulfill the requirements of their local Flexiplace agreements. With regard to the floor requirement of 3\1/2\ cases per day, AFGE believes that no valid empirical evidence has ever been collected to suggest such a requirement is attainable with any acceptable level of accuracy. A time-study analysis was referenced by a representative of VBA management at the September 18, 2008, Hearing and a request was made to provide a copy of that study to the Subcommittee. If the report of any time-study analysis has been provided to the Subcommittee, AFGE requests that we be provided with a copy of the document and an opportunity to comment on it. With respect to how many cases a day an RVSR should be able to adjudicate, two points are salient. First, no one can dispute that an RVSR ought, or should be required, to accurately resolve as many cases on any given day as he or she can. No specific number can be assigned as a productivity requirement on any given day given the disparate complexity of the claims inventory. Any average daily productivity requirement must be calculated based upon empirical evidence concerning what trained RVSRs actually produce with an acceptable degree of accuracy over a 1 year period in order to be valid. Question 5: If training is not being offered and is not available online when it can be completed at anytime employees should have a mechanism to notify VA Central Office that such opportunity does not exist at their RO. How can employees notify managers and senior leaders that the training is not being provided or made available? Response: Such a mechanism does exist for newly hired or promoted employees who have recently completed the centralized portion of their Challenge Training. Those employees are provided with access to a website where they can provide feedback to Central Office's training staff regarding the continuing progress of their Challenge Training when they return to their Regional Offices. Such a mechanism could work if all VBA training was centralized and tracked through Central Office as has been suggested. The problem at present is there is no accountability for Regional Office management who fail to ensure that even the Challenge Training curriculum is implemented after completion of the centralized portion of Challenge instruction. Any such requirement would have to be made part of the Regional Office Director's performance plan, and enforced on RO Directors as a performance requirement by the Office of Field Operations. In the absence of any such requirement, the temptation is for Regional Office management to divert employee resources from completing their Central Office approved training program to helping to achieve goals that are measurements of Regional Office management's performance (e.g. inventory reduction). The end result of this dysfunction is that newly hired or promoted employees never complete a uniform course of training, and their approach to claims processing activities is determined by the experiences they receive at their particular Regional Office as opposed to a common, centrally approved experience. The ultimate detrimental consequent is that consistency and equality of adjudication and evaluation of veterans' claims is not achieved. As the situation now stands, employees can only notify their supervisors that their training has been lacking when they are put on notice that they are being subjected to a performance improvement plan or some adverse employment action. At that point, the employee should be given a meaningful opportunity to participate in identifying where their weaknesses are and how those weaknesses can be addressed through training. Unfortunately, the weakness most often identified by management in such circumstances is a lack of productivity, and the only means to increase productivity in the absence of a sound basic understanding of the laws and regulations governing the administration of VA benefits is through personal experience and effective training. Employees do not strive to produce erroneous decisions or to produce fewer decisions than they can. If an employee is making errors or underproducing, it should be incumbent on the party that identifies those traits to provide an effective remedial course of instruction, tailored to the particular deficiencies identified. Question 6: What would you recommend VA do to improve its training techniques for newer employees? Response: A good first step would be to mandate that all Challenge Training requirements are fulfilled within a set period, and to prohibit the attention of employees in Challenge Training from being diverted to other tasks. In addition, it would be beneficial if newer employees were assigned an individual mentor during their training and for a period after they have completed their formal training. To the extent possible, the mentor-trainee relationship should remain intact so that trainees would not have to constantly adjust their approach to the their work to conform with the idiosyncrasies of multiple mentors. It would also be wise to segregate, to the extent possible, centralized Challenge Training participants into classes composed of trainees who have promoted internally (and thus have some knowledge and experience with the VBA claims process) and those who are hired from outside the VBA (and who require some remedial instruction regarding the mundane aspects of claims processing--e.g. how a claim is routed from the triage activity, to pre-development, to the rating board). That action would allow instructors to devote greater attention to the relative strengths and weakness of their classes, without running the risk of being repetitious or boring some participants. Also, after completion of Challenge Training, ongoing or remedial training must be presented through a single and authoritative voice. As I have suggested elsewhere, that voice should emanate from Central Office, through instructors accountable to Central Office who are assigned to each Regional Office. Question 7: What about experienced raters, would you require them to take the same 80 hours of training if their performance ratings were ``outstanding'' or ``excellent''? Response: The protean nature of the legal requirements imposed on the VA claims process mandates that training be continuous in nature. Regardless of how well any RVSR is performing in any given period, without a constant influx of relevant information in the form of training performance will eventually suffer. Moreover, given the real and perceived needs for increased productivity RVSRs are not particularly well-situated to take time out of their day and study an area where they have some confusion. More importantly, without relevant guidance as to proper procedures and accurate information regarding VBA's policies, RVSRs are not capable of identifying any error in their approach to their duties. So, it is vital that RVSRs continue to devote a discrete portion of their work year toward training. Having said that, the 80 hour requirement may be somewhat excessive. Attorneys and medical doctors in most states do not have that onerous a continuing education requirement, and handle issues which are every bit as complex and important as those in VBA's jurisdiction. It may be fruitful, therefore, to reduce the time VBA spends on continuing training by substituting relevant training developed by reference to common adjudicatory errors for ``refresher'' training pertaining to the general rating process. Once again, if such specific training is centrally developed and administered, it should also foster consistency in decision-making across Regional Offices. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 22, 2008 Kerry Baker Associate National Legislative Director Disabled American Veteran 807 Maine Avenue, SW Washington, DC 20024 Dear Mr. Baker: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Post-Hearing Questions for Kerry Baker, Assistant National Legislative Director of the Disabled American Veterans From the Committee on Veterans' Affairs, Subcommittee on Disability Assistance and Memorial Affairs, United State House of Representatives September 18, 2008 Question 1: At the hearing we discussed the training requirements, what do you think of the substance of the training? Response: Although the Disabled American Veterans would like to offer an opinion as the quality and substance of the VA's training program, we are unable to do so as the VA continues not to provide either this organization or any other Veterans Service Organization the opportunity to review their training program. Question 2: What do you think of VBA's STAR program? Is it effective? Response: The VA's quality assurance tool for compensation and pension claims is the Systematic Technical Accuracy Review (STAR) program. The DAV recommended in its testimony of September 18, 2008, that Congress require the Secretary to report how the Department could establish a quality assurance and accountability program that will detect, track, and holds responsible those VA employees who commit egregious errors. Such a report should be generated in consultation with veterans' service organizations most experienced in the VBA claims process. Under the STAR program, the VA reviews a sampling of decisions from regional offices and uses that sampling to base its national accuracy measures regarding decisions affecting entitlements, benefit amounts, and effective dates. Unfortunately, there still exists a gap in quality assurance for purposes of individual accountability in quality decision-making due to the current small sampling size the VA chooses to use. Specifically, in the STAR program, a sample is drawn each month from a regional office workload divided between ratings, authorizations, and fiduciary end- products. A monthly sample of ``rating''-related cases generally requires a STAR review of 10 rating-related end products. Reviewing 10 rating-related cases per month for an average size regional office that could easily employee more than three times that number of raters, is undeniable evidence of a total void and lack of commitment in individual accountability. If an average size regional office produced only 1,000 decisions per month, which we feel is quite conservative, the STAR program would only review 1 percent of the total cases decided by that regional office. Those figures leave no room for trend analysis, much less personal accountability. To put this in better perspective, according to VA's 2007 performance and accountability report, the STAR program reviewed 11,056 compensation and pension (C&P) cases in 2006 for improper payments whereas, the total number of C&P cases actually available for review was 1,540,211. This equals a percentage of cases reviewed of approximately seven tenths of 1 percent, or 0.72 percent. In closing, we find the STAR program does not fare well as an effective management oversight tool for the VA. It offers only minute sampling of cases compared to actual through-put and it has no accountability mechanism. Effective accountability can be engineered in a manner that holds each VBA employee responsible for his/her work as a claim moves through the system while at the same time holding all employees responsible simultaneously. As errors are discovered, the responsible employees must be held accountable by forfeiture of work credit percentage. One employee would be far less likely to cover for errors or look the other way from errors committed by a fellow employee if they knew their performance standards were equally at risk. This type of system would ensure personal accountability at every stage in the claims process, an essential element if STAR is to ever become an effective management oversight system. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 25, 2008 Ronald B. Abrams Joint Executive Director National Veterans Legal Services Program 1600 K Street, NW, Suite 500 Washington, DC 20006-2833 Dear Mr. Abrams: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Response From Ronald Abrams for Question From the House Committee on Veterans' Affairs, Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' September 18, 2008 Question 1: When NVLSP conducts its Quality Reviews along with The American Legion, does it consistently look at the quality of training as well. If so, what is your feedback on the training and what suggestions have you made over the years. Response: The answer is yes. NVLSP attorneys are part of The American Legion (Legion) team that conducts quality reviews at VA regional offices (ROs). Over the past few years, NVLSP attorneys working with Legion Quality Review Teams visited more than 40 VA regional offices for the purpose of assessing overall operation. In general the quality reviews conducted by the Legion/NVLSP team reveal that RO training suffers because at too many regional offices there are too few experienced supervisors that could provide trainee adjudicators and senior adjudicators proper mentoring and quality assurance. Also we learned that many ROs postponed or suspended training so that maximum effort could be expended on production. The Legion/NVLSP teams found that there is a general inconsistency in how regional office employees are trained and how training is implemented. For example, some stations have regular formalized or structured training programs, while others have training programs that are best described as more informal and sporadic. Some stations have well established and structured training for new employees, but ongoing training for experienced staff is very limited. These finding are similar to a 2005 OIG study. The Legion, in past testimony, recommended that VA reduce its reliance on locally developed training materials and on-the-job training by senior raters. The Legion suggested that the ROs develop training packages based on errors noted the national STAR report and from patterns of errors found by the Board of Veterans' Appeals. Allowing the many regional offices to heavily rely on locally developed training initiatives increases the likelihood of balkanization and great variance in rating decisions. Unless regional offices (both managers and individual adjudicators) learn from their mistakes and take corrective action, there will continue to be a high rate of improperly adjudicated claims, resulting in a consistently high appeals rate and subsequent high BVA remand/reversal rate of regional office decisions. Below is a sample of our comments about training based on the Legion/NVLSP quality reviews. HOUSTON VA REGIONAL OFFICE Interviews with RO employees identified several issues that may adversely affect morale, and the quality of RO adjudications. . . . Training is conducted on a weekly basis but several RO employees rated the training as poor. They felt that the trainers were not qualified to give the training, or not enthused about giving the training. A common complaint was that most trainers merely read off overhead projectors or from a manual. Some staff suggested that a ``Train the Trainer'' program would be beneficial. BUFFALO VA REGIONAL OFFICE Training is being completed on a regular basis, approximately two to three times per month. The Team found the RO training plan to be satisfactory but employees are often encouraged not to spend time on training due to workload concerns. RENO VA REGIONAL OFFICE The station is in compliance with the 80 hours of required training mandated by VA Central Office. Both formal and informal training sessions are also conducted on a regular basis as well as continual ``on-the job-training.'' MUSKOGEE VA REGIONAL OFFICE The station's training program is divided into two main categories: initial training and continuous training. The bulk of training is for new hires. After completing pre-requisite training, new hires are sent to ``Challenge Training'' in nearby training hubs such as Milwaukee or Chicago for 3 weeks. After completing ``Challenge Training,'' new hires continue their education through web-based training modules and on-the- job training. Continuous training is made up of both computer-based web-based training and formal classroom training, which is often led by the training coordinator, the Service Center Manager, Decision Review Officers and rating specialists. This training is usually conducted once or twice a month to meet the Central Office standard of 80 hours of training a year. PHOENIX VA REGIONAL OFFICE In addition to a training program for new employees, there is also ongoing training in place for experienced personnel. The DRO compiles, for training purposes, detailed information on trends and reversals on appealed cases. Also, when appealed decisions are reversed by the DRO, the individual responsible for the original decision is informed of the action taken by the DRO and provided specific reasons for such action. This allows management to maintain accountability, identify problem areas, direct training accordingly and, as a result, improve the overall quality of decisions. Prior to the establishment of the Appeals Management Center (AMC), the DRO also prepared a complete list, used in conjunction with training, summarizing reasons for remands broken down by individual cases. Since the establishment of the AMC, the station has not been able to compile detailed information on individual remands nor identify trends for remands sent to the AMC. Although the appeals team receives summaries of reasons for remand, they are only by region and there is nothing specific for the individual stations. Not being able to identify reasons for remand by individual station will adversely impact the station's ability to identify remand trends and take corrective action. Additionally, there is no mechanism in place at the station for tracking BVA allowances. This not only makes it more difficult to respond to or prevent delays in payment authorization, as previously discussed, it negatively impacts training and, in turn, the quality of decisions. By not tracking reasons for BVA allowances, which are reversals of RO denials, the station is missing an important opportunity to identify trends in erroneous denials and take necessary action to avoid such errors in the future. PITTSBURGH VA REGIONAL OFFICE Training for DROs, RVSRs, and VSRs is conducted on a regular basis. Training sessions are video taped for staff who couldn't attend and for those that work at home. Station work performance standards do not exceed standards established by VA Central Office. . . . The station does not have a mechanism in place for tracking BVA allowances. This negatively impacts training and, in turn, quality of decisions. By not tracking reasons for BVA allowances, which are reversals of RO denials, the station is missing an important opportunity to identify trends in erroneous denials and take necessary action to avoid such errors in the future. Additionally, as BVA remands, in most instances, are being sent to the Appeals Management Center (AMC) for development, the station is denied the opportunity to review and track individual remands. Although the appeals team receives summaries of reasons for remand by region, there is nothing specific for individual stations. Not being able to identify reasons for remand by individual station will adversely impact the station's ability to identify remand trends, include in training and take corrective action. ST. PETERSBURG VA REGIONAL OFFICE . . . According to station management, current staffing is now largely made up of individuals with limited training and limited experience. We were informed that once basic training is completed, the only training conducted is that which is specifically mandated by VBA. There is no ongoing training program. With the current emphasis on production quotas and reducing the claims backlog, management is unwilling to take time away from production for training. It was noted that there were too few managers/supervisors and many of them lacked management training. It was also noted that some inexperienced trainers were assigned to train new employees. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 22, 2008 Dr. Patricia Keenan Program Manager Human Resources Research Organization 66 Canal Center Plaza, Suite 400 Alexandria, VA 22314-1591 Washington, DC 20548 Dear Ms. Keenan: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Response to Questions From the House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' Patricia A Keenan, HumRRO Question 1: You mentioned the focus groups that you conducted with raters and noted that they develop their own ``individual rules.'' Can you tell me more about this practice? Do you have examples of what raters are doing when the medical evidence is ambiguous? Response: I would be reluctant to characterize this as a ``practice,'' which makes the activity sound institutionalized. I see it as a coping mechanism that helps individuals reduce the stress associated with working in a situation that involves a high level of decision-making under uncertainty. It probably contributes to the variance in rating decisions as raters develop their own heuristics. My response is based on what we heard in focus groups and what I have learned about the RVSR job over the last few years as we have developed the RVSR and journey-level RVSR skills certification tests. We heard consistently that it takes several years for RVSRs to become comfortable making rating decisions. There are several reasons for this. These are very high stakes decisions, with a large impact on the lives of veterans and their families. Raters would understandably be very wary about making an error that could hurt these people. Many inexperienced raters are afraid of making a mistake that results in the case being appealed. Experienced raters have learned that everyone is going to have a case appealed sooner or later, that there are a variety of reasons for this happening (e.g., new evidence, error, differing interpretation of the regulation), and it should not prevent a decision. A third reason is that the criteria in some parts of the Rating Schedule are not sufficiently detailed to allow the rater to feel confident that an evaluation is exactly correct. For example, when rating a complaint for a disorder of the ulnar nerve, the schedule provides a detailed description of complete paralysis; however, no description is given for lesser evaluations characterized as severe, moderate or mild. The first two problems are overcome through experience; the third is more problematic. Currently, when there is no clear guidance in the Rating Schedule, RVSRs either try to reason the problem through on their own or solicit help from their peers, coaches or Decision Review Officers (who serve as RVSR trainers). Reasoning through on their own is mostly likely where developing individual decision rules is rooted. Trainee RVSRs have a mentor (a journey-level RVSR) with whom they discuss the evidence and compare it to the available guidance (e.g., the Rating Schedule, training materials, FAST letters). Discussing the claim with others allows the RVSR to take advantage of prior experiences of colleagues as well as their knowledge of the Rating Schedule. Some Service Centers (e.g., San Diego) have a medical professional (retired doctor or nurse) on site to help RVSRs with this type of situation. These people are familiar with the rating process and the Rating Schedule and can help the rater interpret the medical information and match it to the appropriate section of the Rating Schedule. Seeking input from others probably means that these individual decision rules become shared within an office. While the process is very understandable, it is important to recognize that this is not solely a training problem. The deeper roots are in the lack of detailed criteria in some areas of the Rating Schedule and the time pressure raters work under in trying to keep up with the ever increasing number of claims. Question 2: The practice of using analogous codes has increasingly permeated VA's rating process over the years and as you observed increases subjectivity and variance. This seems very unfair to the veteran. What would you suggest VA do instead of using these codes and how could they better train employees to reduce this subjectivity? Response: The use of analogous codes is not unfair, in and of itself. There are legitimate reasons for using them. The first is that maintenance of the Rating Schedule has not kept pace with medical discoveries and advancements. Many of these conditions had not been recognized at the time the Schedule was developed, so they were not included. The result is that many relatively common conditions (e.g., Cohn's disease, erectile dysfunction, insomnia, shin splints) are not currently included in the Rating Schedule. As the Schedule is updated, the conditions will likely be added to the appropriate section(s). Updating the Schedule, however, is likely to take years due to the enormity of the effort. It might be more expedient for VA to conduct a study to analyze the use of analogous codes and identify impairments that occur often enough to deserve their own code or for which the criteria in existing codes are not adequate. Conducting such a study is not a simple proposition as the data are not necessarily easy to interpret. For example, raters may use at least two different diagnostic codes to rate Cohn's disease. This use of multiple diagnostic codes makes it difficult to track the number of Cohn's disease claims which, in turn, makes it difficult to recognize when the number of cases would justify establishment of a separate diagnostic code. A second reason for using analogous codes is that there are some symptoms that are not easily diagnosed. Indeed, the ``Persian Gulf War Veterans' Benefits Act '' recognizes that many Gulf War (GW) veterans suffer from chronic disabilities (e.g., fatigue, headache) resulting from an undiagnosed illnesses. The Code of Federal Regulations (38 CFR, Part II, Chapter 11, Subchapter 11, Sec. 1117) specifies the use of analogous codes for evaluating these undiagnosed illnesses. Other times, a medical opinion does not specify a particular condition. The diagnosis may say that the symptoms are ``like'' a recognized, diagnosable condition, but that condition is not exactly applicable to the symptoms. In these cases, the rater should use a code that is analogous to the recognized condition. It is likely that this circumstance will never disappear, so analogous codes will always have a place in the rating process. Some studies have suggested using the same diagnostic categories (International Classification of Diseases (ICD) and Diagnostic and Statistical Manual for Mental Disorders (DSM)) used by VHA and other health-care providers. While there are some advantages to doing so, to do a wholesale change would cause a major upheaval in the rating process. It would take a lot of time to develop new training materials/ classes and job aids. Conducting training would require that RVSRs take a good deal of time away from their job to learn the new material. It might be more feasible to add parts of the ICD to the existing Rating Schedule, making the transition over time. For both situations, C&P Services could update the Rating Schedule and/or develop training and job aids for RVSRs to learn the new diagnostic codes. The problem of working with analogous codes properly is not entirely a training matter, although that is certainly one avenue to pursue. C&P Service could review existing training materials to make sure they provide adequate guidance for raters. It might be informative to gather input from raters as to the type of information they think would be most useful to them. C&P could also develop additional guidelines for identifying analogous codes. For example, raters might find it useful to take the time (or feel they had the time) to do a quick search of the Internet to give them information they might use to find the appropriate diagnostic code. Question 3: You outlined 3 recommendations in your testimony on September 18, 2008. Have you previously presented these ideas to VA and what was their response? Response: No, I had not discussed these ideas with VA. As a contractor working on the skills certification program, it was not my role to advise VA on areas outside the scope of that work. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 22, 2008 Michael Walcoff Deputy Under Secretary for Benefits Veterans Benefits Administration U.S. Department of Veterans Affairs 810 Vermont Avenue, NW Washington, DC 20420 Dear Mr. Walcoff: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Questions for the Record The Honorable John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, House Committee on Veterans' Affairs September 18, 2008 Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability Question 1: What does VBA do about tying training to individual employee performance? Response: Supervisors are required to conduct oversight of employee training plans and are held responsible for ensuring that employees meet their training requirements. Employee training is tracked through the learning management system (LMS). Through LMS, employees complete a detailed 80-hour training plan each year, which contains curricula deemed appropriate to achieve successful performance in their position. In addition to the required course work, employees can add courses to their learning plan. In fiscal 2008, the Veterans Benefits Administration (VBA) hired training managers to provide additional oversight of the scheduling, and completion of training. In the event an employee's performance is not meeting the fully successful level of achievement for a critical element, the employee will be placed on a performance improvement plan (PIP). The PIP identifies the employee's specific performance deficiencies; the action(s) that must be taken by the employee to improve to the successful level of performance; the methods that will be employed to measure the improvement; and assistance to be provided such as counseling, or focused individualized training. Question 2: In your testimony you noted that VBA contracted with an independent consultant to evaluate the Challenge Training. Although, we discussed this briefly, can the Subcommittee staff get a copy of those results? Response: As part of its national level training evaluation program, VBA contracted for an evaluation of the challenge program. The evaluation sought to answer two questions: Does challenge training, as implemented, provide value to VBA? What are the opportunities for improving challenge training? The report indicates that much value has been realized by VBA's efforts to provide quality, timely training to its new claims processing employees. It also noted areas where VBA could improve the overall management and curriculum of challenge. Efforts are already underway to address the recommendations. A copy of the report is attached. [The report is being retained in the Committee files.] Question 3: How does the information gathered from the STAR review get integrated with the information from the Compensation and Pension Evaluation Program (CPEP) reviews regarding the quality of the exams conducted in cooperation with the Veterans Health Administration? Response: There is no mechanism for integration of systematic technical accuracy review (STAR) and CPEP reviews. However, the STAR questions and the CPEP key indicator questions are similar and essentially cover the appropriateness of the exam request. CPEP reviews are focused on the quality of the examination and STAR reviews are based upon the accuracy of the resulting decisions from those examinations. Question 4: At the hearing, the Chairman asked about the time in motion study, can you please share those results with the Subcommittee as well? Response: VBA contracted with SRA International, Inc., to conduct a work measurement study in March 2007 at 15 regional offices. VBA's pension maintenance centers, benefits delivery at discharge rating activity sites, and foreign claims processing offices were among the regional offices selected to participate in the study. The electronic work measurement application final work rate standards report, submitted by SRA International, is attached. [The report is being retained in the Committee files.] The work rates standards provided in SRA's final report contained some unexpected results, such as longer processing times for original disability compensation claims with 7 issues or less than for claims with 8 issues or more, which prompted a thorough review of the methodology used in the study. Critical flaws were uncovered that produced inaccurate results. The primary errors included: Some completed claims were double counted, resulting in artificially low work-rate standards. Brokered work (those claims completed by select regional offices for other regional offices) was not included in SRA's computations. Not including brokered work inflated the work-rate standard at the regional office completing the work and lowered the work-rate standard at the office of original jurisdiction. It also influenced the overall work-rate standard since only a fraction of regional offices were included in the study. Several formula errors were uncovered in the work-rate standard spreadsheet compiled by SRA that affected the outcome of the study. The errors causing the greatest impact on the work-rate standard calculations were observation count computations. Revisions were made by VBA to the work-rate standards spreadsheet in an attempt to remedy the above issues. The resulting data was then re-analyzed and compared to prior work-measurement study findings. The changes from prior findings were substantial and found to be associated with the factors below: Inconsistent adherence to guidance among participating regional offices on clearing appeals-related claims. A reliable number of completed appeals-related claims were unavailable; therefore, no valid conclusion could be drawn from the work-measurement results. Lack of record of the types of brokered claims completed. Without a record of the exact types of brokered claims completed, completed claims counts could not be properly credited or debited, leading to results not reflective of true workloads. Pension-related claim results were questionable. Prior studies ran for 4 months to account for the cyclical nature of the pension workload. The 2007 study was conducted for only 1 month, providing a narrower view of the pension workload cycle. The sample size for several claim types was too small or nonexistent. Because of the small to nonexistent sample size, the results were unreliable. This again was due to the fact that the study only ran for 30 days. The above issues contributed to a wide variation of results between participating regional offices. This disparity led to the conclusion that the results could not be representative of the time to process claims. It was therefore decided to not use the results of the work measurement study. The recently mandated study of the work credit system and work management system will measure and manage the work production of VBA employees who handle claims for compensation and pension benefits and will also evaluate more effective means of improving performance. Lessons learned from the 2007 study will be used to ensure a valid methodology. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 22, 2008 Bradley Mayes Director Compensation and Pension Service Veterans Benefits Administration U.S. Department of Veterans Affairs 810 Vermont Avenue, NW Washington, DC 20420 Dear Mr. Mayes: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Questions for the Record The Honorable John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, House Committee on Veterans' Affairs September 18, 2008 Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability Question 1: When the ROs report their 80 hours of training, do you know what materials they trained on or just that they completed the required hours? Response: Employees are required to complete 60 hours of core technical training. The topics for the core training are identified by Compensation and Pension (C&P) service. The remaining 20 hours are determined by the Regional Office (RO) based on local needs. The topics and numbers of hours each employee completes are recorded in the learning management system. Question 2: What level of supervision does C&P Service provide to the RO regarding training? Response: C&P service reviews the training accomplishments of each RO at the end of the fiscal year to ensure training requirements were met. Feedback from the reviews is provided to the regional offices. Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Washington, DC. September 25, 2008 Dorothy Mackay Director Employee Development and Training Veterans Benefits Administration U.S. Department of Veterans Affairs 810 Vermont Avenue, NW Washington, DC 20420 Dear Ms. Mackay: In reference to our House Committee on Veterans' Affairs Subcommittee on Disability Assistance and Memorial Affairs Hearing on ``Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability'' on September 18, 2008, I would appreciate it if you could answer the enclosed hearing questions as soon as possible. In an effort to reduce printing costs, the Committee on Veterans' Affairs, in cooperation with the Joint Committee on Printing, is implementing some formatting changes for material for all full committee and subcommittee hearings. Therefore, it would be appreciated if you could provide your answers consecutively on letter size paper, single-spaced. In addition, please restate the question in its entirety before the answer. Sincerely, John J. Hall Chairman __________ Questions for the Record The Honorable John J. Hall, Chairman, Subcommittee on Disability Assistance and Memorial Affairs, House Committee on Veterans' Affairs September 18, 2008 Examining the Effectiveness of Veterans Benefits Administration Training and Performance Management and Accountability Question 1: When RO's develop their training strategic plan, how are the gaps identified and incorporated into future training? Response: In fiscal 2008, the Veterans Benefit Administration (VBA) created a training manager position at every regional office (RO) to improve the ability to identify gaps and plan training. The training manager is responsible for local training reviews, analyzing performance indicators to determine local training needs, and implementing the training necessary to meet those needs. For example, local accuracy reviews are conducted by the Compensation and Pension (C&P) service's systematic technical accuracy review (STAR) staff on claims decisions using national quality criteria. The results of the reviews are available to the RO training manager for trend analysis and formulation of remedial training as necessary. Question 1(a): How does joint data collection and root cause analyses with the VBA on cases that have been remanded inform training? Response: The C&P program review staff conducts monthly remand reviews of all pre-certification Board of Veterans Appeals (BVA) remands. The review results are available to all ROs on the C&P Intranet and are used to provide timely feedback on pre-certification remands in support of efforts to reduce avoidable remands. The results are also used for training purposes. As an example, BVA travel board attorneys use the results when they provide training at the regional offices. Recently, analysis was conducted that compared the primary STAR error for Veterans Claims Assistance Act (VCAA) compliant development to the corresponding pre-certification error categories on the remand reasons report. The results are under review. Detailed remand reason reports are run quarterly and are also posted on the C&P Intranet Web site by RO, area, and nationwide. Question 2: The Instructors Training manual is five volumes. That doesn't seem like an effective tool. Have you consulted with private sector organizations that specialize in developing training protocols for Federal and private sector agencies? Response: The instructors' training manual includes training materials for the three challenge training classes, each given over the course of 3 weeks. The three challenge curriculums include rating veterans service representative (RVSR), pre-determination veterans service representative (VSR) and post-determination VSR. The manual is split into five volumes, two for RVSR training, one for pre- determination VSR training, and two for post-determination VSR training. Each volume includes instructor and student guides, presentations, and reference materials, including copies of fast letters, court decisions, flow charts, and sample letters. VBA is currently exploring ways to reduce the size of the training manuals as the curriculum is revised. All training materials are reviewed by internal certified instructional design specialists. VBA also works with a contractor, Camber Corporation, on the materials.