Effectiveness of the Office for Victims of Crime Tribal Victim Assistance Program

Audit Report 06-08
February 2006
Office of the Inspector General

We audited the OVC tribal victim assistance program, which was designed to establish, expand, and improve direct-service victim assistance programs in remote, rural Native American communities. In order to evaluate program effectiveness, the objective of our audit was to obtain grant performance information directly from the grantees and evaluate whether the grants were fully implemented and whether program objectives were achieved. We conducted our audit in accordance with the Government Auditing Standards, and included such tests as were necessary to accomplish the audit objective. The audit generally covered, but was not limited to, tribal victim assistance grants awarded between FYs 1999 through 2004. Audit work was conducted at OVC Headquarters, and four selected tribal grantees. We believe that grant program effectiveness starts with the overall structure and design of the program. Therefore, as part of our audit, we reviewed the OVC to determine the adequacy of the tribal victim assistance program purpose and design. To determine whether the OVC tribal victim assistance program had a well‑defined purpose designed to support a specific problem, we obtained the program’s authorizing legislation and other documentation and identified the overall goals of the OVC tribal victim assistance program. We also obtained statistics and other documentation supporting the problems that were to be addressed by the program. To determine whether the OVC tribal victim assistance program was designed to fill a unique role or whether they unnecessarily duplicated, overlapped, or competed with other federal or non-federal programs, we obtained the total funding and a description of the efforts supported by any program that addressed a similar problem in a similar way. In order for grant programs to be effective the granting agency must incorporate adequate oversight and evaluation. For this audit, we also reviewed the OVC to determine if its tribal victim assistance program incorporated adequate strategic planning to evaluate program effectiveness. To determine whether the OVC tribal victim assistance program incorporated adequate strategic planning to evaluate program effectiveness, we obtained the existing agency GPRA performance plan/performance budget and other program documents supporting the measures established for the OVC tribal victim assistance program. Specifically, we used these documents to determine if the OVC implemented: (1) long-term performance measures to guide program management and budgeting, and promote results and accountability; (2) a limited number of annual performance measures that were identified to directly support the long‑term goals obtained; and (3) challenging but realistic quantified targets for the annual measures. Additionally, we interviewed OVC officials to determine whether: (1) the performance data reported by grant recipients was used to evaluate program effectiveness; (2) the OVC conducted evaluations to determine program effectiveness; and (3) the performance-planning and budget‑planning processes were integrated so that resource-allocation decisions reflected desired performance, and the effects of funding and other policy changes on results were clear. To determine if performance information was used to manage the OVC tribal victim assistance program and improve performance, we determined whether: (1) the data reported by grant recipients was used to inform program management, make resource decisions, and evaluate program performance; (2) the OVC held its program managers and tribal grantees accountable for achieving program results; (3) OVC funds were administered efficiently and obligated in accordance with planned schedules; (4) the program had adequate oversight practices that provided sufficient knowledge of grantee activities; and (5) the program collected grantee performance data on an annual basis. Evaluating the Effectiveness of Individual Grantee Programs We attempted to evaluate the effectiveness of individual grantee tribal victim assistance programs by selecting the four tribal grantees who received victim assistance funding, for which financial audits had been conducted previously as part of our audit on the Administration of Department of Justice Grants Awarded to Native American and Alaska Native Tribal Governments, Report No. 05-18, March 2005. Those grantees are listed in the chart on the following page:   •  Sault Ste. Marie Tribe of Chippewa Indians, Sault Ste. Marie, Michigan   •  Oglala Sioux Tribe, Pine Ridge, South Dakota   •  Mississippi Band of Choctaw Indians, Philadelphia, Mississippi   •  Lummi Indian Nation, Bellingham, Washington In order to evaluate the effectiveness of the victim assistance programs implemented by the four individual tribal grantees, we determined whether they: (1) implemented tribal victim assistance grant objectives; (2) reported grant activities accurately in progress reports; (3) maintained statistical data supporting program performance; (4) documented any program accomplishments; (5) coordinated effectively with criminal justice agencies and service providers; and (6) developed plans to sustain the victim assistance program upon the expiration of grant funding. The following sections summarize our findings. Implementing Grant Program Objectives We reviewed the grant application and award documentation to identify the objectives for each of the victim assistance grants awarded to the four individual tribal grantees. We then reviewed each grant objective to determine if it was consistent with the overall goals of the OVC tribal victim assistance program. Additionally, we interviewed grantee officials and reviewed supporting documentation to determine: how the grantee measured the progress towards achieving objectives of its victim assistance program; if the grantee established timelines in its application for accomplishing the objectives of its victim assistance program; if the objectives of its victim assistance program were implemented; the current status toward achieving grant objectives, in relation to the proposed timelines/activities in the original application; whether the grantee’s victim assistance program was on track to accomplish the objectives listed in the grant award documentation; and whether all grant objectives were achieved for any grant programs that had ended. Accurately Reporting Grant Activities We determined if grantees submitted all required progress reports under the OVC tribal victim assistance program.33 We then assessed the adequacy of the submitted reports by determining whether they contained the required program performance data listed below: number of victims served and type of victimization, number of staff supported by victim assistance funds, number of volunteer hours, number of publications produced, number of training workshops for law enforcement, type of services provided, and progress on goals and objectives identified by the program.34 We also verified progress reports to any source documentation maintained by grantees to determine if the reports accurately reflected actual grant activity. Maintaining Statistical Data Supporting Program Performance We determined whether grantees maintained statistics on the same criteria as required under the previous section: Accurately Reporting Grant Activities in Progress Reports. We then calculated the percentage of victims who received assistance through the program based on the information obtained. Finally, we assessed whether the grantee could demonstrate an increase in services and activities as a result of the OVC tribal victim assistance grant funding received. Documenting Program Accomplishments We determined if files were maintained for services provided to individual victims, and selected a sample of files to review in order to determine the types of services provided, and to identify any information related to the impact of the program on individual victims. Additionally, we provided questionnaires to individual victims to determine if the services received were effective in meeting their needs. Finally, we interviewed tribal and federal collaborating agencies to determine whether the grant program was effective in meeting victims’ needs. Coordinating with Criminal Justice Agencies and Service Providers Collaboration is an essential component of the OVC tribal victim assistance program. Therefore, grantees were required to implement strategies that included coordinating with appropriate local agencies involved in assisting victims. We interviewed tribal and federal collaborating agencies to determine if any efforts to promote partnerships within and outside of the tribal community were successful, and in order to effectively provide services to crime victims. Developing Plans to Sustain Programs After Funds Expire Finally, a significant component of program effectiveness is whether or not the grant program continues after grant funding expires. Generally, grant programs are intended to provide initial funding. However, grantees are expected to sustain the program with other funding sources. We interviewed grant program officials to determine if grantees developed plans to sustain the tribal victim assistance program once the grant expired. Footnotes The OVC requires grantees who receive grants under victim assistance programs to submit a progress report every 6 months, for periods ending June 30 and December 31, for the life of the awards. Progress reports must be submitted within 30 days (120 days for a final progress report) after the end of the reporting periods. The OVC requires that progress reports contain information related to: (1) the goals and objectives of the grant awarded; (2) activities conducted during the reporting period; (3) any timeframes for accomplishing the goals and objectives; (4) the status of the goals and objectives; and (5) how the grantee plans to evaluate performance indicators and evaluations.