This memorandum responds to the Office of the Inspector General's (“OIG's”) draft audit report issued on August 23, 2004, regarding the Office of Justice Programs' (“OJP”) administration of technical assistance and training grants. The draft report contains three recommendations and $5.3 million in questioned costs.1 In general, we agree with the draft report recommendations, and are fully committed to implementing corrective actions to strengthen our administration of all grants in our portfolio, which include the technical assistance and training grants you audited. However, the draft report did not accurately describe the status of a few ongoing OJP efforts to improve accountability.
Specifically, on page iii, the draft report notes that OJP's Grants Management System (“GMS”) is not currently in widespread use among the OJP components and is not fully functional. To the contrary, since I mandated the use of GMS in January 2003, GMS has been in widespread use by all bureaus and program offices. All grant solicitations posted since January 31, 2003, were issued and processed in GMS. In addition, a number of other functionalities have been added to GMS, including progress reporting and programmatic reporting modules.
Also, the draft report includes $5.3 million in questioned costs resulting from the OIG's audit of 21 technical assistance and training grants, and references two prior Government Accountability Office (“GAO”) audits that included similar findings. As summarized in Attachment A, of the 15 grants where questioned costs were noted, $197,114 in questioned costs have been closed. The Office of the Comptroller is working with the grant recipients to ensure that appropriate corrective actions are implemented to address the remaining questioned costs and recommendations. Likewise, as summarized in Attachment B, corrective action has been fully or partially implemented for the recommendations identified in the GAO audit reports.
In addition, the draft report concludes that “OJP did not ... have specific requirements that grantees could follow in developing [performance or outcome] measures.” As the audit referenced only grants awarded between 1995 and 2002, the auditors would not have been aware of my requirement, in early Calendar Year 2002, that all solicitations and grant awards require grantees to develop outcome measures. Thus, all grants awarded since that time have included this mandatory requirement.
For ease of review, the three recommendations included in the draft report are restated in bold, followed by our response to the recommendation.
- We recommend that the OJP ensure that grant managers receive annual training on OJP's requirements governing the submission of timely and accurate reports, allowable costs, grant monitoring requirements, and grant closeout procedures.
We agree with the recommendation. The Office of Justice Programs is committed to providing the training necessary to ensure that grant managers are adequately equipped to administer grants. Beginning in October 2004, the Office of the Comptroller (“OC”) will conduct a series of training sessions for all grant managers that will include topics such as grantee reporting, allowable costs, monitoring, and grant closeout. In addition, in May 2004, OJP conducted in-house fraud awareness training for grant managers.
Bureau of Justice Assistance (“BJA”): Over the past 18 months, BJA's monitoring system has been revamped and improved significantly. The BJA Monitoring Guide was revised and updated and includes BJA's new monitoring protocol. An automated risk assessment tool, known as the Priority Monitoring Assessment, is currently being developed and will identify key factors associated with the performance and program compliance of a specific grantee and grant program. These factors describe the areas to consider when determining a grantee's performance and compliance. The tool determines which grants pose the highest risk and may need an on-site monitoring visit. Effective and timely assessments of the status, progress, and the performance of these grants are essential to ensure grant funds are provided to grantees that effectively achieve their goals and provide corrective action to grantees at risk for managing their grants in a manner inconsistent with the goals of BJA and requirements of the grant program.
The Bureau of Justice Assistance will complete these assessments on an annual basis.
The Bureau of Justice Assistance has also developed checklists to capture critical information on performance and program compliance. The checklists capture information such as financial and programmatic compliance, progress and performance, civil rights compliance, and technical assistance needs. The post-monitoring report has been standardized and issues such as findings/recommendations, technical assistance needs, potential best practices, and significant observations must now be reported to the BJA Quality Assurance Manager and the Policy Office. The Policy Office will develop training and technical assistance for BJA's grantees based on the information in the post- monitoring report.
In addition, BJA plans to train all BJA staff and managers on these performance requirements and protocols in early FY 2005. The training will cover topics such as the BJA monitoring documentation policies, roles and responsibilities of BJA staff relative to monitoring, monitoring plan development, and risk assessment. The final training curriculum and tools will be developed by October 31, 2004.
Office of Juvenile Justice and Delinquency Prevention (“OJJDP”): In response to past recommendations concerning grant monitoring, OJJDP conducted an internal assessment of the status of its monitoring activities. As a result of the assessment, in August 2002, OJJDP developed protocols and standard forms for grant administration. These protocols and standard forms are being used now by OJJDP staff; however, OJJDP recognizes the need for ongoing staff training on grant monitoring. In the past year, OJJDP implemented the practice of sending new staff to professional grant monitoring training conducted by Management Concepts, Inc. This training, consisting of a series of classes, covers all aspects of grant monitoring and the staff can move through the series over time. The first course in the series covers the basic elements of grant monitoring. OJJDP will assess whether there are staff members, particularly those monitoring training and technical assistance grants or contracts, who have not attended the grant monitoring classes and would benefit from it. The assessment will be completed by December 2004 and the training provided to those who need it by June 2005.
Eight OJJDP program managers attended the fraud training provided by OJP in FY 2004. OJJDP is committed to providing financial training to all grant managers and will require all grant managers to attend the financial training that will be provided by the OC in FY 2005. The grant managers will also require OJJDP grantees to attend the financial management training provided by OC in FY 2005.
- We recommend that the OJP ensure that the GMS is brought up to full functioning capacity as soon as possible and grant managers are trained to utilize this system.
We agree with the recommendation. Since January 2003, GMS has been in widespread use by all bureaus and program offices. All grant solicitations posted since January 31, 2003, were issued and processed in GMS. The functionalities within GMS have increased dramatically over the last year, in particular the peer review, grant monitoring, progress reporting and subgrant reporting functionalities. In addition, in FY 2004, grantees began to submit their quarterly financial reports on line.
OJP is in the midst of streamlining several grant business processes that have not been automated in GMS. The goal is to automate efficient and uniform processes, rather than automate a cumbersome and outdated work flow. We expect the following remaining GMS modules to be completed by the end of Fiscal Year 2005: Grant Adjustment Notices, Close Out, External User Administration, Financial Monitoring, and Payment Request Support Subsystem. As part of the implementation protocol for the deployment of new GMS modules, grant managers will receive appropriate training to utilize the system.
- We recommend that the OJP develop performance or outcome measures to assess the effectiveness of technical assistance and training grants.
We agree with the recommendation. In January 2002, I required that performance measures be included in every grant award. Development of performance measures for all grants is an ongoing OJP-wide initiative. As part of our efforts, on September 23, 2004, OJP is sponsoring an in-house training on performance measures that will be facilitated by the United States Department of Agriculture Graduate School. The objective of the training is to help OJP staff understand the requirements of the Office of Management and Budget's June 2003 guidance, Performance Measurement Challenges and Strategies.
BJA: In coordination with the Office of Budget and Management Services (“OBMS”) and the Institute for Law and Justice, BJA is developing performance measures for all BJA grants, including training and technical assistance grants. Performance measures were formally adopted for all competitively funded programs beginning with awards in FY 2003. BJA expects to assign appropriate performance measures to all training and technical assistance grants at the time of award in FY 2005.
Because grant progress reports are submitted semi-annually and Government Performance and Results Act (“GPRA”) performance data is collected quarterly, it has been a challenge to incorporate performance measure data in the annual grant progress report. To address this issue, BJA is working on additional strategies for the collection and analysis of data. In FY 2004, BJA added a special condition for awards that requires quarterly reporting on training and technical assistance performance measures.
BJA instituted a training workshop on measuring performance to be given at all five BJA regional conferences for grantees that are scheduled in late FY 2004 and early FY 2005. Topics covered in this workshop will include GPRA, Outcome Measures, and the Program Assessment Rating Tool (“PART”).
At my instruction, the concentration will be on outcome, as opposed to output, measures.
In addition, BJA has an evaluation website that state and local grantees can use to find useful resources for planning and implementing program evaluations and for developing program performance measures. Information on assessing program performance is available and covers topics such as identifying goals and objectives; measuring activities and outputs (Process Evaluation); measuring outcomes (Impact Evaluation); and establishing the "activities-outcomes" connection (Evaluation Experiments). Program specific information is provided for programs such as drug courts, community justice, multijurisdictional task forces, reentry, and tribal courts. Topics covered include the challenges associated with conducting evaluations, commonly used performance measures, and how evaluation findings can be used for program development and improvement.
OJJDP: In coordination with OBMS and Caliber Associates, OJJDP is developing a standardized performance measurement system. While specific measures have been developed for certain OJJDP programs, performance measures will be included in all OJJDP grants at the time of award in FY 2005 and staff will be trained to monitor the performance measures by the close of FY 2005. OJJDP will work with training and technical assistance providers to set up mechanisms to evaluate the effectiveness of services, such as participant surveys and post-service follow-up. These mechanisms will be required as conditions of grant awards.
We appreciate the opportunity to comment on the draft report. If you have any questions, please feel free to contact me on 202-307-5933, or LeToya A. Johnson, OJP Audit Liaison, on 202-514-0692.