Of the Department's five air fleets, the USMS Air Operations Division was the only aviation organization with a performance measurement system. However, we noted some areas for improvement that, when properly addressed, will strengthen the effectiveness of the Division's performance planning and evaluation.
The Government Performance and Results Act of 1993 requires the head of each agency, by September 30, 1997, to set goals, develop strategic plans, and then measure the results of program performance based on whether the goals have been achieved. Within the Department, the Attorney General has taken steps to implement the Act. In May 1993, she announced the development of a Departmental performance measurement system. By August 15, 1994, components were required to submit mission statements, annual goals, and performance indicators to be used to measure progress towards stated goals. The Attorney General said the system will help the Department meet its responsibilities to be more accountable for what it accomplishes with the resources it receives.
Beginning in FY 1994, the USMS Air Operations Division incorporated strategic planning in its operations. The Division developed a performance plan that defined goals, objectives, and performance indicators to use as a basis for evaluating program results against program goals. The USMS Assistant Director for Management and Planning was primarily responsible for implementation and periodic reviews of the Division's performance plan, and the Division Chief was accountable for the successful execution of the plan.
The three goals identified in the Air Operations Division performance plan were as follows:
· "Move all prisoners requested without incident and meet required time tables while providing a reduced cost compared to commercial transportation."
· "Support all requests of WITSEC [Witness Security], Enforcement Operations, and emergency requirements without incident and with increased security while providing service at a reduced cost compared to commercial transportation."
· "Support all requests for administrative support and required mission priorities."
For each goal, the plan included the performance indicators and strategies for accomplishment. (See Appendix VII for the plan.)
In May 1994, the Air Operations Division completed a mid-year review of its operations based on the performance plan and forwarded its accomplishments to the USMS Assistant Director for Management and Planning. We analyzed the Division's reported measures to verify their accuracy and applicability (effectiveness) with regard to meeting established goals. Based on our results, we questioned some of the reported program results as discussed below.
· Goal 1: Move All Prisoners Requested Without Incident in an Efficient Manner
The Division's reported accomplishments were overstated with regard to fulfilling transportation requests; i.e., the report said the Division was 100 percent effective in meeting all transportation requests (26,018) for the period October 1, 1993, through March 31, 1994. The Division did not track the number of requested moves; therefore, it obtained that number from the USMS Prisoner Coordination Branch. We found, however, that the Branch's number of requests was understated; i.e., the number reflected the quantity of requests fulfilled, rather than the actual number of requests. That is, for the requests that the Air Operations Division could not support (aircraft unavailability, a prisoner's medical condition, weather, etc.), the Prisoner Coordination Branch would adjust its records downward to reflect only the number of requests actually fulfilled. As long as the Branch continues to account for requests in such a manner, the Division's use of the Branch's figures will always indicate that the Division is fully achieving this performance standard, even though it may not be.
We also found that the reported cost of aircraft operations was understated. (See Finding 3.) Further, the Division did not perform a documented comparison between its costs and commercial costs. As a result, the Division was not able to accurately access its effectiveness in meeting this goal, or make appropriate adjustments in its operations, if necessary.
· Goal 2: Support All Special Requests Without Incident in an Efficient Manner
Our audit disclosed inconsistencies between the Division's reported results and other USMS records. For example, in a January 1994memorandum to the USMS Director, the Associate Director for Operations wrote:
". . . AOD [Air Operations Division] is unable to fully satisfy its mission responsibilities regarding international extraditions, international witness security, NPTS [National Prisoner Transportation System] requirements . . . and supporting sister organizations of the Department of Justice (DOJ) due to a lack of suitable aircraft." Further, the AOD's ". . . inability to provide aircraft support has, on occasion, forced branches of the DOJ and the USMS to utilize military aircraft . . . at extremely high reimbursement rates or to charter costly civilian aircraft operated by flight personnel without security clearances."
The Division did not address the above issue in its mid-year review. As noted previously, the division reported that it was able to meet all requests, including special requests.
· Goal 3: Support All Requests for Administrative and Mission Priorities
The Air Operations Division did not capture the data for the following two performance indicators:
· number of mission cancellations/number of missions requested, and
· number of missions completed on time/number of missions with time tables.
However, the Division Administrative Officer told us that recent computer enhancements will allow them to collect the information for the second 6-month reporting period in FY 1994.
Conclusion
Despite the above mentioned deficiencies, the USMS Air Operations Division is to be commended for being the only aviation organization in the Department with a performance measurement system. We also recognize that the USMS implemented the system well in advance of even the earliest Departmental deadlines. Further, the performance indicators and strategies identified in the Division's plan appear to have been well thought out and applicable to the achievement of stated goals.
The purpose of performance planning is not to create another layer of paperwork, but to provide managers with accurate information and results on which to base operational decisions. Division management should utilize and refine its performance plan to maintain a system that uses the established goals and measures to improve program effectiveness and efficiency.
We recommend that the Director, USMS:
8. Ensure the collection of necessary data to correctly assess each identified performance measure.
Closed. Based on Air Operations Division documentation received through March 17, 1995, the Division modified its data collection procedures so that it can correctly assess the identified performance measures in the future.
9. Require full and accurate reporting on all goals and measures identified.
Closed. Air Operations Division staff and the documentation mentioned above assured us that future reporting of identified goals and measures will be full and accurate.
STATEMENT ON INTERNAL CONTROL STRUCTURE
In planning and performing our audit of the USMS Management of Aircraft Operations, we considered the USMS' internal control structure for the purpose of determining our auditing procedures. This evaluation was not made for the purpose of providing assurance on the USMS internal control structure. However, we noted certain matters involving the internal control structure that we consider to be reportable conditions under generally accepted government auditing standards.
Reportable conditions involve matters coming to our attention relating to significant deficiencies in the design or operation of the internal control structure that, in our judgment, could adversely affect the USMS' ability to effectively manage its aircraft operations. We identified weaknesses in the following areas:
· aircraft maintenance. (See Finding 1.)
· cost reporting. (See Finding 3.)
Because we are not expressing an opinion on the USMS' internal control structure, this statement is intended solely for the information and use of the USMS in managing its aircraft operations. This restriction is not intended to limit the distribution of this report, which is a matter of public record.
STATEMENT ON COMPLIANCE WITH LAWS AND REGULATIONS
We have audited the USMS Management of Aircraft Operations. The period covered by the audit included FYs 1991, 1992, 1993, and 1994, through May 31, 1994, and included a review of selected activities and transactions. The audit was conducted in accordance with generally accepted government auditing standards.
In connection with the audit, and as required by the standards, we tested transactions and records to obtain reasonable assurance about the USMS' compliance with laws and regulations that, if not complied with, we believe could have a material effect on operations. Compliance with laws and regulations applicable to aircraft operations is the responsibility of USMS management.
An audit includes examining, on a test basis, evidence about laws and regulations. The regulations for which we conducted tests are contained in the relevant portions of 41 Code of Federal Regulations Part 101-37, "Government Aviation Administration and Coordination." The results of our tests indicated that the USMS did not comply with 41 Code of Federal Regulations 101-37.5 in the area of reporting aircraft costs. (See Finding 3.) However, as previously noted, there are inconsistencies among the regulations and GSA will seek appropriate resolution.
With respect to those transactions not tested, nothing came to our attention that caused us to believe that the USMS was not in compliance with the regulations cited above, other than those for which we noted violations in our testing.
#####