||Guy K. Zimmerman
Assistant Inspector General for Audit
||Cybele K. Daley
Acting Assistant Attorney General
||Draft Audit Report - Audit of the Effectiveness of the Office for Victims of Crime Tribal Victim Assistance Program
This memorandum responds to the Office of the Inspector General's (OIG's) draft audit report issued on November 1, 2005, regarding the effectiveness of the Office for Victims of Crime (OVC) Tribal Victim Assistance (TVA) Program. The draft report contains seven recommendations and no questioned costs. In general, we agree with the draft report recommendations, and are fully committed to implementing corrective actions to strengthen our administration of TVA grant program. The recommendations are restated in bold below, followed by the Office of Justice Programs' (OJP's) response.
The "number of victims served" appears to have been given considerable weight in the ultimate determination of a program's effectiveness (i.e., neither tribe that received a "no" on this measurement was considered effective). Although the Report contains a cautionary footnote in Table 1, the footnote does not convey the seriousness of the problem with this measure.
Data, such as "number of victims served" gathered for internal reporting purposes is not necessarily appropriate for use as a performance measure. Although this seems to be an easy and appropriate measure for a program aimed at assisting crime victims, the outside observer is tempted to use an increase in the number of victims served as a measure of effectiveness, as occurs in Table 1 of the Report. The number of victims served depends on many factors outside the control of the program in question. In fact, to the extent the law enforcement goal of reducing crime is achieved it seems problematic to use an increase in the number of victims served as a performance measure.
Also, although the Report notes that OVC transferred approximately $450,000 to the National Institute of Justice (NIJ) for an assessment of whether TVA grantee programs could be evaluated, the Report appears to question the significance of evaluating two grants valued under $200,000. TVA grantees operate in a complex environment, therefore, it is not surprising that NIJ limited the cost of the study to two tribes with an expectation that the lessons learned would be useful for other programs.
Responses to Recommendations
Establish long-term and annual performance goals for its tribal victim assistance program.
OJP agrees with the recommendation and OVC will enhance the existing procedures. In compliance with the Government Performance and Results Act, O VC requires TV A grantees collect and report data on the results of their individual programs. OVC will continue to work with grantees to establish goals that are specific to their community’s needs and the long-term sustainability of victim services.
The fundamental goal of the TVA program is to establish victim assistance programs in remote areas of Indian Country. We believe the most appropriate measures of program effectiveness are, ultimately, that services are made available to victims of crime, and that communities and victims are aware of the services available. Although the goals of the TVA grant program are general goals, grantees are required to plan for long-term program sustainability, establish goals and objectives, and submit an annual timeline in which to accomplish the goals and objectives. Some of the individual grantee goals may be simple, such as to hire a victim coordinator. Although such a goal might be dismissed as trivial or too easily achieved, for some tribes hiring a victim coordinator is a significant step forward, as no previous services may have been available.
For example, one OVC Program Specialist traveled for a site visit to a remote native village in Alaska and learned that the TVA grant empowered the victim advocate to say the words "child sexual abuse” aloud for the first time to the village Elder Council, a group of esteemed leaders in a village with a high rate of child abuse. This is a significant step forward because traditional ways, geographical isolation, close-knit and small populations, and socio-economic issues present unique challenges for many American Indian and Alaska Native communities addressing elder abuse, child abuse, domestic violence, and other crimes.
Ensure that resource-allocation decisions reflect program effectiveness.
OJP agrees with the recommendation. The TVA grants are made based on a three-year proposal. Although the initial award is based on need, the accomplishments of TVA grantees toward long-term goals are considered when making resource-allocation decisions for continuation funding in the second and third years. OVC will ensure that documentation is maintained to support all of the factors considered when making resource-allocation decisions.
Provide tribal grantees with definitions of terms used for the required performance measures and guidance on tabulating the performance information reported.
OJP agrees with the recommendation. OVC will continue to provide technical assistance and training on measuring performance as presented to grantees during annual conferences, through a technical assistance and training provider, and through on-site monitoring. OVC will increase its efforts to ensure that the terms used for the required performance measures and guidance on tabulating performance information reported are clearly defined.
Establish a standardized progress report that captures required performance measure information.
OJP agrees in part with the recommendation. We agree that some aspects of performance reposting could be standardized for the TVA program. The aspects of performance reporting that captures the diverse needs, cultures, and goals of the individual grantee programs do not lend themselves to a standardized progress report.
Rather than impose a specialized standard report for the TV A program, we believe that it would be more efficient to use the standard that is being incorporated into t he Grants Management System. In addition, OVC will implement or enhance procedures to ensure consistent and accurate reporting.
Ensure that progress reports include required performance measure data.
OJP agrees with the recommendation. OVC will continue to coordinate with the TVA training and technical assistance provider to increase efforts to ensure that progress reports are accurate and complete.
Summarize the performance information reported by tribal grantees to report on the effectiveness of its tribal victim assistance program as a whole.
OJP agrees with the recommendation. OVC will enhance and formalize its current process for analyzing the effectiveness of the TVA program. The annual report will include a summary of the performance information reported by TVA grantees, as well as the other performance analyses prepared to support funding decisions.
Utilize the performance information reported by tribal grantees to evaluate the effectiveness of individual grantee tribal victim assistance programs, and to follow up with tribal grantees demonstrating poor performance.
OJP agrees with the recommendation. OVC uses the information reported by tribal grantees, as well as conducts programmatic monitoring to evaluate the effectiveness of individual grantee programs. Through these efforts, OVC conducts outreach to grantees requiring assistance in implementing their tribal victim assistance programs. OVC will implement or enhance procedures to improve the programmatic monitoring process and how data learned will be used to highlight model programs.
Thank you for the opportunity to review and provide comments on this draft audit report. If you have any questions regarding this response, please feel free to contact me on 202-207-5933, or LeToya Johnson, Director, Program Review Office.
Deputy Assistant Attorney General for Operations
John W. Gillis, Director
Office for Victims of Crime
Jill R. Meldon, Director
Office of Budget and Management Services
LeToya A. Johnson, Director
Program Review Office