Review of the Bureau of Alcohol, Tobacco, Firearms and Explosivesí
Violent Crime Impact Team Initiative
Evaluation and Inspections Report I-2006-005
Office of the Inspector General
In June 2004, the Department of Justice and the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) established the Violent Crime Impact Team (VCIT) initiative as a pilot program designed to reduce homicides and other firearms-related violent crime in 15 cities. The goal of the VCIT initiative was to “decrease, within six months, the number of homicides, number of firearms-related homicides, number of violent crimes, and number of violent firearms crimes” in target areas in those 15 cities.1 In December 2004, ATF extended the initiative indefinitely and, in April 2005, expanded it to include a total of 20 cities.
There were two primary purposes of the VCIT initiative, as articulated by the Office of the Deputy Attorney General (ODAG) and ATF. The first was to reduce homicides and violent firearms crimes in cities where homicide and violent crime rates had not followed the national trend downward. The second was to test the effectiveness of the VCIT strategy in reducing, rather than displacing, the incidents of firearms-related violence in neighborhoods and communities by identifying, targeting, and arresting the “worst of the worst” violent offenders in specific targeted areas referred to as hot spots. The VCIT initiative was designed to build on regular law enforcement operations and the Department’s Project Safe Neighborhoods.2
To assist in implementing the VCIT program, ODAG and ATF sought the participation of the Drug Enforcement Administration (DEA), the United States Marshals Service (USMS), the Federal Bureau of Investigation (FBI), and the Executive Office for United States Attorneys (EOUSA). Each of these components agreed to support the VCIT initiative to the degree that they could, given existing resources. The VCIT pilot program initially was funded by $499,000 in reimbursable funding from the Department’s Justice Management Division (JMD). In FY 2005, ATF operated 20 VCITs using $6.8 million reprogrammed from other ATF activities and general operating funds. In fiscal year (FY) 2006, ATF received its first appropriation of $20 million specifically to continue and expand the VCIT initiative.
Prior to launching the VCIT pilot program in June 2004, ATF completed a document describing the VCIT initiative entitled Violent Crime Impact Teams: A Comprehensive Strategy (VCIT Strategy) and provided it to ATF Special Agents in Charge and VCIT Coordinators as an implementation guide. The VCIT Strategy described the underlying concept of the VCIT initiative – concentrate on small high-crime areas and remove the worst violent offenders. It also described the initiative’s purpose, objectives, intended outcomes, and activities. On January 26, 2006, ATF issued a report entitled Violent Crime Impact Teams: Best Practices. In the report, ATF stated that the VCIT pilot initiative had been a success and that the initiative should continue because the number of homicides committed with firearms was lower in 13 of the 15 VCIT pilot cities’ target areas than during the same 6-month period of the preceding year.
The Office of the Inspector General (OIG) conducted this review to assess ATF’s implementation of the VCIT initiative. We examined planning and implementation documents from ATF, USMS, the Department’s Criminal Division, and EOUSA and conducted in-person and telephone interviews with personnel from ODAG, ATF, DEA, FBI, USMS, and EOUSA. We also conducted an e-mail survey of ATF’s VCIT Coordinators nation-wide and made site visits to five of the VCIT pilot cities, where we interviewed personnel from ATF, DEA, USMS, U.S. Attorneys’ Offices (USAO), and other federal, state, and local criminal justice officials.
We focused our review on five elements described in the VCIT Strategy that ODAG and ATF Headquarters officials consistently referenced in describing the VCIT initiative. Because these officials emphasized these five elements during our interviews, in speeches, in internal documents describing the VCIT initiative, and included them in the January 2006 Best Practices report, we determined that the elements collectively differentiated VCIT from regular ATF law enforcement operations. Briefly stated, these elements were: (1) targeting specific geographic areas such as neighborhoods or communities with a high rate of firearms violence; (2) targeting the worst violent offenders in those areas; (3) building effective working relationships with community leaders; (4) using ATF firearms investigative technology resources; and (5) working in partnership with other Department law enforcement components.
RESULTS IN BRIEF
The OIG’s review of the implementation of the VCIT Strategy found that while every VCIT had implemented some of the elements of the Strategy, no VCIT had implemented all of the five key elements. Further, we found that ATF did not implement an adequate evaluation plan – at either the program or the local level – to identify the activities that each VCIT undertook, assess how effectively each VCIT carried out those activities, and determine whether violent crime declined in the target areas. We concluded that ATF cannot conclusively show that the law enforcement operations it undertook during the VCIT pilot project were effective at reducing firearms crime in the target areas. While the VCIT Strategy itself may be an effective tool to reduce violent crime in target areas, ATF’s incomplete implementation and evaluation of VCIT prevents it, and us, from determining the Strategy’s effectiveness.
The incomplete implementation of the VCIT Strategy was due primarily to problems with the selection of the VCIT locations, inadequate direction to ensure that the key elements of the Strategy were implemented, and ineffective oversight of the VCIT operations by ATF. ATF did not ensure that each VCIT implemented the key elements of the VCIT Strategy, but instead left it to the Field Division managers to develop local strategies.
ATF developed a detailed VCIT Strategy, but local operations did not implement the Strategy’s key elements.
The 32-page VCIT Strategy that ATF issued in June 2004 described six objectives ATF expected each VCIT to meet, while adapting each objective to local conditions.3 Directives to VCITs to use ATF investigative technology resources and to include personnel from other Department law enforcement components were woven throughout the Strategy. In addition to these objectives and directives, the VCIT Strategy included an extensive list of specific law enforcement practices and procedures that should be adapted to local conditions and implemented as part of the overall VCIT operations.
Through interviews with the former Deputy Attorney General and ATF officials, we identified five key elements described in the VCIT Strategy that we deemed essential to implementing the VCIT initiative. These elements also differentiated VCIT from traditional ATF enforcement operations. In interviews, ATF officials said they expected VCITs to integrate these elements into their local operation. The five key elements are described in detail below:
We visited five VCIT pilot cities (Tampa, Tucson, Philadelphia, Los Angeles, and Miami) after the pilot period had ended to assess in more detail how each of the ATF Field Divisions involved had implemented the key elements of the VCIT Strategy. The following sections describe our observations, together with the results that ATF reported for each location.
ATF indicated in its January 2006 Best Practices report that the Tampa VCIT was successful during the pilot period because the number of homicides committed with firearms decreased from 19 during June through November 2003 to 9 during the same period in 2004 (53 percent) in the target area.
ATF indicated in its January 2006 Best Practices report that the Philadelphia VCIT was successful during the pilot period because homicides committed with firearms decreased from 12 during June through November 2003 to 3 during the same period in 2004 (75 percent) in the target area.
Los Angeles VCIT
ATF indicated in its January 2006 Best Practices report that the Los Angeles VCIT had not demonstrated success during the pilot period because homicides committed with firearms increased from 72 during June through November 2003 to 77 during the same period in 2004 (7 percent) in the target area.
ATF indicated in its January 2006 Best Practices report that the Miami VCIT was successful during the pilot period because homicides committed with firearms decreased from 36 from June through November 2003 to 22 during the same period in 2004 (39 percent) in the city itself (not the target area).
Because of the inconsistency in operations that we were observing during our five field visits, we conducted a survey of 19 VCIT Coordinators5 and additional research to assess how the 5 key elements were implemented by the 14 VCITs we did not visit. We summarize below our findings regarding the implementation of the 5 key elements of the VCIT strategy across the 19 VCITs.
Target Small Geographic Areas
Eighteen of the 19 VCITs Coordinators who responded to our survey considered “identifying and working within a defined target area” a core or essential component of their VCIT initiative and stated that their VCIT targeted or operated in a specific geographic area. However, we found that the specific geographic areas varied greatly in size and population.
Overall, we found that8 of the 19 VCITs targeted areas of 10 square miles or less, and 10 VCITs targeted areas ranging from 20 to 2,100 square miles. VCIT Coordinators reported that their VCITs targeted “specific geographic areas” that varied greatly in size and population. For example, two VCITs targeted entire cities, and one targeted an entire county. The population in the VCIT target areas ranged from 25,000 to 3 million residents.
The VCIT Strategy also indicated that, consistent with targeting a small geographic target area, VCITs should be composed of a small number of ATF Special Agents. Specifically, the Strategy recommended four Special Agents and a Supervisory Special Agent be included on the VCIT. However, we found that ATF assigned an average of 10 Special Agents to each of the 19 VCITs, with the actual number of Special Agents ranging from 3 to 34.
Target the Worst Violent Offenders
We found that 6 of the 19 VCITs (32 percent) used a “worst of the worst” list and kept it updated to reflect arrests and newly identified targets. Of the remaining 13 VCITs, 7 used a “worst of the worst” list, but did not keep it updated, while 6 did not use such a list at all.
In addition, the VCIT arrest data suggested that the VCITs might not have focused on targeted individuals. According to the arrest data, targeted individuals represented only about 20 percent of the total number of arrests made by the VCITs. Although a key element of the VCIT initiative was to target the worst offenders, from June 2004 through May 2005, the VCITs reported making 3,592 arrests, of which only 746 were targeted individuals. We could not determine whether these 746 targeted individuals were the “worst of the worst” violent offenders in the target areas because only 6 VCITs used up-to-date “worst of the worst” lists and ATF did not require VCITs to document the definition of “targeted individual” that they used to compile these data.
Build Effective Working Relationships with Community Leaders
In our survey of the 19 VCIT Coordinators, 11 reported engaging in community outreach such as attending community meetings and participating in neighborhood watch activities. Eight VCIT coordinators did not report community relations as an essential part of their VCITs’ strategies.
Three of the sites we visited were among the 11 that reported they engaged in community outreach. However, during our site visits, we found that none of the five VCITs actively participated in any community outreach.
Utilize ATF’s Investigative Technology Resources
We determined that VCITs did not consistently use the services of the NTC, NIBIN, and CGAB to facilitate their investigations. For example, according to NTC officials, six VCITs consistently used the NTC to expedite gun tracings, seven VCITs used the NTC but not consistently, and four VCIT cities’ use of the NTC actually declined after ATF implemented the VCIT program. ATF officials told us that this apparent decline in NTC use could be due to the failure of local VCITs to flag all of their submissions to the NTC.
We compared submissions to the NIBIN – the program that collects and analyzes images of fired cartridge casings and bullets to link specific firearms to criminal activity – by pilot VCIT cities for 6 months before VCIT implementation (January through June 2004) with their submissions during VCIT operations (July through December 2004). We found that six VCIT cities submitted more images after VCIT implementation, seven VCIT cities submitted fewer images, and two cities did not submit any images to the NIBIN during 2004. Although the purpose of VCITs was to intensify the focus on firearms crimes, we concluded that over half of the VCIT cities made fewer submissions to the NIBIN after ATF implemented local VCIT operations than prior to initiation of the VCIT program.
In response to our survey, 17 of the 19 VCIT Coordinators (89 percent) stated that they used the CGAB – which analyzes firearm trace, handgun sale, and firearm theft information to identify “crime gun” patterns – on an ongoing basis as part of their VCIT strategies. However, during our site visits to five VCIT cities, local VCIT personnel stated they did not routinely use the CGAB. In fact, in three cities we visited, ATF personnel and contract staff explained that they were assigned to produce reports similar to those the CGAB provided. We believe this to be an indication that the type of information the CGAB produced was useful to VCITs, but that ATF had not tailored CGAB reports to support VCIT operations.
Work in Partnership with Other Department Law Enforcement Components
Of the 19 VCIT sites, only 1 included representatives from the DEA, FBI, and USMS; another 14 VCITs included a representative from one or two of these agencies; and 4 VCITs had no representative from any other Department law enforcement component. The VCIT Coordinators reported that the DEA provided Special Agents to 7 VCITs, the FBI provided Special Agents to 3 VCITs, and the USMS provided U.S. Deputy Marshals to 14 VCITs. VCIT coordinators reported that the USAOs provided AUSAs to 17 of the VCITs. See Table 1 on the next page. We also found that when the DEA and USMS assigned personnel to VCITs, the assigned personnel were often unaware of their expected roles and did not always receive clear direction from either ATF or their own agency regarding their expected duties. At only one of the five sites we visited did personnel from either the DEA or USMS tell us that they had a clear understanding of their role on the local VCIT.
Table 1: Department of Justice Component Participation
ATF did not address local conditions in the site selection process and, contrary to the VCIT Strategy, placed VCITs in areas with decreasing violent crime and areas without sufficient cooperation with local law enforcement.
We found that ATF and ODAG officials selected cities to participate in the VCIT initiative without seeking input from ATF Field Division personnel who were most familiar with local crime conditions. Without the advice of ATF field personnel, ODAG and ATF officials often selected locations for the VCIT pilot test that did not reflect ATF’s emphasis on targeting violent crime “hot spots.” We found that in some of the selected target areas violent crime was decreasing or already at a low level. We also found that ATF did not ensure that the pilot sites would receive cooperation from local law enforcement. For example, in some areas, local authorities were not willing to give ATF the local crime data it needed to monitor progress and evaluate the impact of the VCIT initiative on violent crime.
ATF did not provide sufficient direction and oversight to its Field Divisions to ensure that local VCITs implemented the key elements of the initiative.
We found that ATF officials did not develop policies and procedures for incorporating the five key elements that differentiated VCIT from routine ATF enforcement operations either before or during the pilot period. ATF also did not provide resource materials or training on implementing the VCIT key elements to ATF Field Division staff or personnel from federal or local partner agencies prior to initiating VCIT Operations. The ATF Field Operations staff who have the authority and responsibility to oversee the Field Divisions’ operations did not provide written operational guidance regarding how the VCIT Strategy should be implemented. All of the VCIT Coordinators we interviewed told us that ATF Headquarters officials left it to Field Division managers to develop their local VCIT strategies. After the pilot period, in August 2005, ATF Headquarters personnel met with VCIT Coordinators to discuss local operations, and in January 2006, ATF published the Best Practices report, which stated that “this analysis will be used to strengthen future VCIT deployments.” However, ATF did not provide specific policies or guidance for implementing the identified practices.
Because of the lack of operational guidance from ATF Headquarters, local VCIT activities were often a continuation of ATF enforcement activities that had been under way prior to VCIT implementation. ATF Field Division managers and Special Agents in two of the cities told us that, except for working in target areas, their VCITs’ activities were indistinguishable from their Field Divisions’ previous routine operations.
ATF did not develop an adequate evaluation plan, leaving it unable to demonstrate the effectiveness of the VCIT initiative.
We found that ATF has not developed an adequate evaluation plan to determine the effectiveness of the VCIT initiative. Although ATF’s VCIT Strategy clearly stated the importance of determining the effectiveness of the initiative and its impact on homicides and all violent crimes committed with firearms, ATF Headquarters officials did not implement a program-wide evaluation plan that could accomplish this objective. ATF collected standard measures of law enforcement activity – such as arrests – that, alone, were not adequate to evaluate the implementation or effectiveness of a specific strategy such as VCIT. ATF did not collect and analyze the data necessary to identify and correct problems in implementing VCIT’s key elements or in evaluating their effectiveness. For example, ATF did not collect data on the criminal histories of the offenders arrested; prosecution and sentencing of VCIT arrestees; VCITs’ use of firearm technologies; or data on personnel and other resources provided to VCITs by other Department components.
ATF Headquarters officials also recognized the importance of conducting local evaluations of VCIT operations in each city, but they did not provide the Field Divisions with guidance on how to do so. When we examined the local evaluations that the VCIT Coordinators submitted to ATF Headquarters, we found that they described in narrative fashion local lessons learned. Few reports provided quantitative or qualitative data documenting the effectiveness of the activities they described. For example, reports did not provide data on the characteristics (e.g., criminal histories) of the individuals VCITs arrested or the investigative technologies VCITs used in investigations. None of the reports had documentation on the number of VCIT arrestees prosecuted or the length of the sentences handed down for those who were convicted. If the Field Divisions had reported these types of data, ATF could have identified and corrected gaps in implementing VCIT’s key elements and in evaluating their effectiveness.
ATF’s Claim of VCIT Success Is Based on Insufficient Data.
In its January 2006 Best Practices report, ATF compared the number of homicides committed with firearms before and after VCIT implementation in the 15 pilot cities. ATF reported that the pilot initiative was a success because the number of homicides committed with firearms in the VCIT target areas decreased compared with the number of homicides in the same 6-month period of the preceding year in 13 of the 15 pilot cities.
However, our analysis of the data in the ATF report does not support this conclusion. We found that ATF based its analysis on insufficient data and faulty comparisons. ATF inappropriately used city-wide data rather than target area data for 8 of the 15 VCITs and, therefore, could not demonstrate the impact of the VCIT on homicides committed with firearms for half of the VCITs. Moreover, the number of homicides committed with firearms in some of the target areas where ATF implemented VCITs was relatively small, generally ranging from 2 to 32, with the majority under 20 during a 6-month period. Therefore, percentage decreases reported by ATF appear dramatic but actually reflect small changes in the number of homicides committed with firearms, making it difficult to draw conclusions about the effectiveness of the VCIT initiative.
For example, ATF reported that in Albuquerque, homicides committed with firearms in the target area decreased by 50 percent – from 4 between June and November 2003 to 2 during the same period in 2004. In addition, the comparison of data from only two points in time, 2003 and 2004, was not consistent with accepted standards for trend analysis.6
We believe that a better measure of the effectiveness of the VCIT initiative would have been to compare the number of violent firearms crimes (instead of only homicides committed with firearms) in the VCIT target areas for the same periods before and after the initiative was implemented. First, the change in the number of violent firearms crimes is a standard, agency-wide ATF performance measure. Second, the number of violent firearms crimes in a VCIT target area is a large number, ranging from 107 to 4,056, making analyses of the data more reliable. Furthermore, according to the VCIT Strategy, ATF intended to measure the effectiveness of VCITs not only by the number of homicides committed with firearms, but also by the total number of homicides, the number of violent crimes, and the number of violent firearms crimes.
Although a primary goal of the VCIT initiative was to reduce violent firearms crime, our analysis of data that ATF provided for target areas in six pilot cities did not show that the VCITs had done so. Tampa had experienced a decrease in violent firearms crimes before VCIT was implemented there – from 278 in 2002 to 257 in 2003 – followed by a sharper decrease to 107 in 2004 after VCIT implementation. Greensboro experienced an increase in violent crimes committed with firearms before VCIT implementation, followed by a slight decrease during VCIT implementation. Los Angeles showed a decrease that began before VCIT was implemented and continued after VCIT implementation, as did Chattanooga. See Table 2. In two VCIT cities, Albuquerque and Philadelphia, violent firearms crime increased after VCIT was implemented. A comparison of high-crime areas with VCITs to similar high-crime areas without VCITs would constitute a stronger evaluation methodology.
Table 2: Violent Crimes Committed with Firearms in Target Areas
CONCLUSION AND RECOMMENDATIONS
Although we believe ATF developed sound objectives for the VCIT Strategy to combat violent firearms crime, ATF did not implement that Strategy. This was primarily due to ATF’s inconsistent direction to and oversight of its Field Divisions on adapting the VCIT Strategy to local operations. Most VCITs inconsistently incorporated the key elements that were to differentiate the initiative from ATF’s day-to-day enforcement operations. Further, ATF did not develop an adequate evaluation strategy to measure the performance or impact of the VCIT initiative, and ATF’s analysis of the data it collected does not support its claims of success. As a result, although the Strategy on paper appears to be a reasonable and effective approach, after 24 months of operations and the allocation of more than $35 million in reprogrammed funds, general operating funds, and funds appropriated for VCIT, ATF cannot definitively show whether the initiative successfully reduced firearms crime in areas with high levels of violent crime.
For FY 2006, ATF received $20 million to continue and expand the VCIT initiative. To maximize the use of these funds and to inform future funding decisions, we believe that ATF should implement the initiative consistent with its VCIT Strategy and in a manner that will allow ATF to determine whether the Strategy is effective in reducing violent firearm crime in specific areas. To accomplish this, we recommend that ATF:
|« Previous||Table of Contents||Next »|