The Terrorist Screening Center (TSC) is a multi-agency organization administered by the Federal Bureau of Investigation (FBI) that consolidates terrorist watchlist information and provides 24-hour, 7-day a week operational support for federal, state, local, and foreign governments.1 The TSC was created by the September 16, 2003, Homeland Security Presidential Directive‑6 (HSPD-6), which directed the TSC to integrate all existing U.S. government terrorist watchlists and assist in the screening of individuals who, for example, apply for a visa, attempt to enter the United States through a port-of-entry, attempt to travel internationally on a commercial airline, or are stopped by a local law enforcement officer for a traffic violation. Prior to the establishment of the TSC, the federal government relied on at least a dozen separate terrorist watchlists maintained by different federal agencies.
In June 2005, the Department of Justice Office of the Inspector General (OIG) issued an audit of the TSC’s operations from the time of its inception in 2003.2 The OIG reported that although the TSC had made significant strides in becoming the government’s single point-of-contact for law enforcement authorities requesting assistance in identifying individuals with possible ties to terrorism and had developed a consolidated terrorist watchlist database, the TSC had not done enough to ensure that the information in that database was complete and accurate. For example, we reported instances where the consolidated database did not contain names that should have been included on the watchlist. Additionally, we found inaccurate or inconsistent information related to persons included in the database. In this prior review, we also found problems with the TSC’s management of its information technology, a crucial facet of the terrorist screening process. Our June 2005 report included 40 recommendations to the TSC addressing areas, such as database improvements, data accuracy and completeness, and staffing.The objectives of this follow-up audit were to: (1) determine if accurate and complete records are disseminated to and from the watchlist database in a timely fashion; (2) review the TSC’s efforts to ensure the quality of the information in the watchlist database; and (3) assess the TSC’s efforts to address complaints raised by individuals who believe they have been incorrectly identified as watchlist subjects.
To accomplish these objectives, we interviewed more than 45 officials and reviewed TSC documents and databases. To evaluate the accuracy and completeness of the consolidated watchlist, we analyzed the consolidated database as a whole, including a review of the number of records in the database, any duplication that existed within those records, and the associated watchlist processes. We also tested individual records within the database for accuracy and completeness, as well as the timeliness of any related quality assurance activities. Finally, we assessed the TSC’s activities related to individuals who raised complaints following their involvement in a screening encounter. This included examining the TSC’s coordination with other participating agencies and reviewing a sample of such cases to determine if the actions taken were timely and conformed to TSC policy.3
Results in Brief
Overall, this follow-up audit found that the TSC has enhanced its efforts to ensure the quality of watchlist data, has increased staff assigned to data quality management, and has developed a process and a separate office to address complaints filed by persons seeking relief from adverse effects related to terrorist watchlist screening. However, we also determined that the TSC’s management of the watchlist continues to have weaknesses. For example, the TSC is relying on two interconnected versions of the watchlist database. As a result of this and other conditions, we identified several known or suspected terrorists who were not watchlisted appropriately. Specifically, we identified 20 watchlist records on suspected or known terrorists that were not made available to the frontline screening agents (such as a border patrol officer, visa application reviewer, or local police officer) for use during watchlist screening encounters (such as at a border crossing, through the visa application process, or during a routine traffic stop). We also found that the number of duplicate records in the database has significantly increased since our last review.
In addition, because of internal FBI watchlisting processes, we found that the FBI bypasses the normal international terrorist watchlist nomination process and enters these nominations directly into a downstream screening system. This process is cumbersome for the TSC and, as a result, the TSC is unable to ensure that consistent, accurate, and complete terrorist information is disseminated to frontline screening agents in a timely manner.
We also concluded that the TSC needs to further improve its efforts for ensuring the accuracy of the watchlist records. We found that, in general, the TSC’s actions to review records as part of a targeted special project successfully ensured the quality of the data. In contrast, our examination of the routine quality assurance reviews revealed continued problems. We examined 105 records subject to the routine quality assurance review and found that 38 percent of the records we tested continued to contain errors or inconsistencies that were not identified through the TSC’s quality assurance efforts. Although the TSC had clearly increased its quality assurance efforts since our last review, it continues to lack important safeguards for ensuring data integrity, including a comprehensive protocol outlining the agency’s quality assurance procedures and a method for regularly reviewing the work of its staff. Additionally, the TSC needs to work with partner agencies to develop clearly defined areas of responsibility and timeframes for quality assurance matters.
A single omission of a terrorist identity or an inaccuracy in the identifying information contained in a watchlist record can have enormous consequences. Deficiencies in the accuracy of watchlist data increase the possibility that reliable information will not be available to frontline screening agents, which could prevent them from successfully identifying a known or suspected terrorist during an encounter or place their safety at greater risk by providing inappropriate handling instructions for a suspected terrorist. Furthermore, inaccurate, incomplete, and obsolete watchlist information increases the chances of innocent persons being stopped or detained during an encounter because of being misidentified as a watchlist identity.
We are also concerned that the TSC’s ongoing review of the watchlist will take longer than projected. At the time of our audit field work in April 2007, the TSC was continuing its efforts to conduct a record-by-record review of the consolidated watchlist and anticipated that all watchlist records would be reviewed by the end of 2007. However, the watchlist database continues to increase by an average of over 20,000 records per month and contained over 700,000 records as of April 2007. Given this growth and the TSC’s weak quality assurance process, we believe the TSC is underestimating the time required to sufficiently review all watchlist records for accuracy.
Our audit further determined that the TSC was following its procedures and reaching appropriate resolutions in its review of complaints filed by individuals seeking redress from further adverse experiences that they believed were the result of terrorist watchlist screening. However, we found that the redress reviews were not always completed in a timely manner, and we recommend that the TSC and partner agencies develop timeliness measures for each phase in the redress process.
Additionally, the TSC’s redress reviews have identified that the database contains records for individuals that should not be watchlisted and that some watchlist records are inaccurate or incomplete. We believe that these results provide a further indicator that watchlist data needs continuous monitoring and attention. We also believe that the TSC should use information related to terrorist watchlist identities that are frequently the subject of watchlist encounters to proactively initiate redress reviews before complaints are filed.
Our report contains detailed information on the full results of our follow-up review of the TSC and contains recommendations to help the TSC carry out its important role in the terrorist watchlisting process.
Summary of Watchlist Nomination, Screening, and Redress Processes
Agencies that conduct counterintelligence, counterterrorism, and law enforcement activities provide information to the FBI and the National Counterterrorism Center (NCTC) on suspected or known terrorists who are nominated for inclusion on the consolidated terrorist watchlist maintained by the TSC.4 The FBI is responsible for submitting to the TSC all domestic terrorist identity nominations, and NCTC is responsible for submitting international terrorist identity nominations.5 These two agencies employ analysts who review the information on the known or suspected terrorist identity and forward an unclassified subset of information to TSC analysts, who then review the information to ensure that all required criteria are met to incorporate the identity record in the TSC’s consolidated terrorist screening database (TSDB). As additional information is obtained that either enhances the identifying information or indicates that the individual has no nexus to terrorism, source agencies must also submit this information through the nominating process to effect watchlist record modifications and deletions, as appropriate.
The review performed by analysts at NCTC, the FBI, and the TSC includes an analysis of information supporting the watchlist nomination, as well as an examination of the quality, accuracy, and sufficiency of the identity information.6 Thus, all identity records undergo a two-stage review before inclusion in the TSDB: (1) at NCTC and then at the TSC for international terrorist identities, or (2) at the FBI and then at the TSC for domestic terrorist identities.
The TSC shares the terrorist information contained in the TSDB by sending it “downstream” to other government screening systems where frontline screening agents can use the information to identify individuals against TSDB records.7 The following are examples of three databases that contain information from the TSC’s consolidated watchlist: (1) an employee of the U.S. Customs and Border Protection (CBP) agency at a U.S. port-of-entry searches the DHS’s Interagency Border Inspection System (IBIS) to determine if a person should be granted access to the United States, (2) a state police officer stops a vehicle for a traffic violation and queries the driver’s name in the FBI’s National Crime Information Center (NCIC) system, and (3) a State Department consular affairs official searches the Consular Lookout and Support System to determine if a foreign national should be granted a visa to visit the United States. The TSC reported that approximately 270 million individuals are screened by frontline screening agents and law enforcement officers each month.8
When a name appears to be a match against the terrorist watchlist, frontline screening and law enforcement personnel contact the TSC’s 24‑hour call center for assistance in confirming the subject’s identity. In responding to such a call, the TSC Call Center staff searches the TSDB and other databases to determine if a terrorist watchlist identity match exists. Between the TSC’s inception in December 2003 and May 2007, the TSC has documented more than 99,000 encounters for which its call center was contacted. TSC data shows that 53.4 percent of these calls were determined to be a positive match to a terrorist watchlist identity in the TSDB. In those cases, the TSC contacted the FBI, which is responsible for initiating any necessary law enforcement action. In 43.4 percent of these calls, it was determined that the encountered individual did not match the watchlisted identity, and the TSC Call Center staff instructed the frontline screening agent of this resolution. In the remaining 3.2 percent of the encounters, the TSC Call Center staff could not definitively determine if the match was positive or negative and therefore forwarded these calls to the FBI.
The nature of the U.S. government’s actions to screen individuals against the consolidated terrorist watchlist can result in individuals being delayed or detained during security screenings. This can range from an individual being subjected to enhanced security screening and slight delays to missing a flight or being detained for a long period of time. Persons stopped may be actual watchlist subjects, individuals misidentified to a terrorist identity, or someone mistakenly included on the watchlist.
In 2005, the TSC created a process for resolving complaints from individuals who were adversely affected by terrorist watchlist-related screenings and who were seeking relief or “redress.” Since the creation of a unit dedicated to processing such complaints in 2005, the TSC Redress Office has received 438 terrorist watchlist-related redress complaints.
Known or Suspected Terrorists Missing from Watchlist
Our review revealed continued instances where known or suspected terrorists were not appropriately watchlisted on screening databases that frontline screening agents use to identify terrorists and obtain instruction on how to appropriately handle the subjects. Even a single omission of a suspected or known terrorist from the watchlist is a serious matter. We found at least 20 watchlist records that were not appropriately watchlisted to downstream screening databases. These watchlisting errors are discussed in detail below.
Due to technological differences and capabilities of the various systems used in the watchlist process, the TSC maintains two interconnected versions of the TSDB to allow for the electronic import and export of data. Although the TSC is developing an upgraded TSDB to eliminate the need for the two systems, in the meantime TSC officials informed us that these two databases should be identical in content and therefore should contain the same number of records. However, we discovered during our review that these two systems had differing record counts. Specifically, on one day that we tested the databases the difference was 18 records, and on a subsequent day the difference was 38 records.
On March 26, 2007, the TSC informed us that the differing record counts were due, in part, to five watchlist records that were missing from the TSC database responsible for exporting watchlist records to most downstream screening databases. Therefore, the associated terrorist identities were not included in downstream databases used to screen individuals against the terrorist watchlist. Further, our testing of a sample of 105 watchlist records revealed 7 additional watchlist identities that were not being exported to all appropriate screening databases. As a result of the TSC’s failure to export all terrorist watchlist records to screening databases, these 12 watchlisted individuals could be inappropriately handled during an encounter. For instance, a suspected or known terrorist could be erroneously issued a U.S. visa or unknowingly allowed to enter the United States through a port-of-entry. We discussed these records with TSC officials who agreed with our findings and began correcting these omissions.
During the course of our review, we were also informed by TSC officials that in September 2006 they had identified 2,682 records in the TSDB that were not being exported to any screening database. Working with NCTC, the TSC determined that 2,118 of these records should not have been watchlisted in any system and needed to be removed from the TSDB.9 TSC officials conducted a manual review of the remaining 564 records and determined that 8 had not been appropriately watchlisted and needed to be renominated to the TSDB.
However, despite being responsible for removing outdated or obsolete data from the TSDB, the TSC did not have a process for regularly reviewing the contents of the TSDB to ensure that only appropriate records were included on the watchlist. TSC officials told us that they intend to begin performing a monthly review of the database to identify any records that are being stored in the TSDB that are not being exported to any downstream systems. We believe it is essential that the TSC regularly review the TSDB to ensure that all outdated information is removed, as well as to affirm that all records are appropriately watchlisted.
Inconsistent FBI Procedure for Processing Watchlist Data
The FBI’s Terrorist Review and Examination Unit (TREX) receives requests from FBI agents to include an individual with known or suspected ties to terrorism on the terrorist watchlist. These requests are provided on nomination forms, which are also used to modify previous submissions or remove records from the watchlist. Analysts at TREX review the nomination information for accuracy and completeness. Once verified, nomination forms for known or suspected domestic terrorists are electronically forwarded to the TSC where a TSC analyst manually enters the information into the TSDB. This information is electronically distributed to the downstream screening agency data systems, including the FBI’s Violent Gang and Terrorist Organization File (VGTOF), which is part of the NCIC system.
By contrast, once the TREX analyst verifies an FBI-generated international terrorist nomination, the analyst enters the information into VGTOF directly and then submits the nomination form to NCTC. Following its review and vetting, the NCTC analyst manually enters the information into its database – the Terrorist Identities Datamart Environment (TIDE) – that in turn feeds the information to the TSDB. Because TREX has already entered the record into VGTOF, it is not necessary for the TSC to export the record it receives from TIDE to VGTOF. Therefore, these records are not exported from the TSDB to VGTOF.10 Because these VGTOF records will not receive electronic modifications or deletions from the TSDB, the TSC and TREX have agreed that TREX will be responsible for ensuring FBI-originated international watchlist records in VGTOF are accurate, complete, and current.
The FBI’s direct entry of international terrorist watchlist nomination data into a downstream screening database bypasses NCTC and the TSC and makes it difficult for the NCTC and the TSC to carry out their responsibilities related to watchlist nominations and records. In our opinion, this process does not comport with the nomination and data flow procedures agreed to by the partner agencies, which requires agencies to provide to NCTC, rather than directly into a downstream database, information related to known or suspected international terrorists. Additionally, we believe the FBI’s practice is cumbersome for the TSC and creates unnecessary data errors, anomalies, and inconsistencies as described below.
To test for data accuracy and consistency, we reviewed a judgmental sample of 50 FBI-originated additions or modifications to the watchlist.11 We found that while the records for the domestic terrorist nominations were generally accurate, the international terrorist nominations were not. We identified 16 records with 28 instances in which the identifying information related to international terrorists was inconsistent between the nomination form, VGTOF, TIDE, TSDB, or other screening systems. According to TSC officials, TREX analysts frequently augment the data on the nomination forms with information they glean from FBI case files and enter this additional information into the VGTOF system. However, this supplemental case information is not forwarded to NCTC and as a result the information is not included in TIDE, not sent to the TSDB, and not made available, if appropriate, to downstream screening systems. Further, because TREX enters the record into VGTOF before the addition of any other existing information from other government databases to which NCTC has access, this additional information is often not included in VGTOF. As a result, vital information on watchlist subjects is not being shared with all appropriate screening agencies.
In addition, we found that the FBI’s procedures for processing international terrorist watchlist nominations are cumbersome for the TSC and can inadvertently create an incomplete watchlist. The difference in procedures between the FBI’s and other agencies’ watchlist nominations requires TSC analysts to review every incoming international terrorist nomination and indicate within the record whether it is an FBI source record. If a terrorist watchlist record is improperly designated as an FBI source record, the subset of terrorist watchlist records in VGTOF will be incomplete because that record will not be exported to or modified within VGTOF. TSC staff told us that they were concerned about this because, when reviewing nominations from NCTC, it is often difficult to distinguish between FBI source records and nominations received from other agencies. In fact, TSC staff stated that there was a period of time (possibly as long as a year) in which many records had been improperly designated as FBI-originated records and vice versa. In March 2007, the TSC and NCTC addressed this problem by developing a method to permit TSC analysts to more easily identify FBI source records.
FBI officials responded to our concerns by stating that they had implemented their nomination procedures to ensure that FBI international terrorist information was entered into VGTOF in a more efficient manner. Yet, our review of 70 FBI record transactions (50 nominations previously mentioned and 20 deletions) revealed that although the transactions were entered into VGTOF in a timely manner, the transactions were not incorporated into the TSDB in a timely fashion.12 Specifically, 18 of the 70 transactions took more than 5 days for TREX to process, with one transaction taking 35 days. In addition, we identified 2 instances in which TREX erroneously delivered nomination forms for domestic terrorists to NCTC that resulted in delays of 6 and 16 days, respectively.
Delays in including terrorist information in the consolidated database present a significant vulnerability to the integrity of the consolidated watchlist. Further, the FBI’s current practice of bypassing NCTC and the TSC and entering international terrorist-related data directly into VGTOF increases the likelihood that watchlist information within the TSDB, TIDE, VGTOF, and other downstream databases is inaccurate and incomplete. As a result, we recommend that the FBI, NCTC, and TSC work together to design a more consistent and reliable process by which FBI-originated international terrorist information is provided to NCTC for inclusion in TIDE and disseminated to the TSDB and downstream screening systems, including VGTOF.
Duplicate Terrorist Watchlist Records
Multiple records containing the same unique combination of basic identifying information can needlessly increase the number of records that a call screener must review when researching a specific individual. In addition, when multiple records for a single identity exist, it is essential that the identifying information and handling instructions for contact with the individual be consistent in each record. Otherwise, the screener may mistakenly rely on one record while a second, more complete record may be ignored. This can result in important information being missed. Further, inconsistent handling instructions contained in duplicate records may pose a significant safety risk for law enforcement officers or screeners.
In reviewing the TSDB for duplicate records, we defined duplicate records as those records that contain the same identifying information for five primary identifying fields – [SENSITIVE INFORMATION REDACTED].13 In our June 2005 TSC report, we identified 31 such instances of duplicate records in the TSDB and recommended that the TSC implement corrective action. For our current audit, we again determined that duplicate records existed within the TSDB and that the occurrence of duplicates had increased significantly. In March 2007, we found that the TSDB contained 2,533 repeated combinations in the 5 core fields involving 6,262 watchlist records. For example, one unique combination of the 5 core fields had 19 associated records. Further, our analysis of the 6,262 duplicate TSDB records indicated that at least 20 percent had some discrepancy in handling instruction, identifying information, or watchlist export designation. For example, we identified one individual with duplicate identity records in the consolidated watchlist. Because both records pertained to the same individual, the instructions for handling the subject should be consistent. However, we identified significant differences between the records regarding handling instructions and additional warnings related to the individual. Specifically, one record noted that the individual was “armed and dangerous with violent tendencies” and also had a valid arrest warrant. The other record did not contain this important information. These types of inconsistencies place screeners and law enforcement officers at undue risk and could potentially result in the admittance of a dangerous individual into the United States.
According to the TSC Chief Information Officer (CIO), the TSC does not have an ongoing process to review the TSDB for duplicate records. Based on our findings, however, the TSC CIO stated that the TSC plans to implement a procedure to conduct weekly reviews of the TSDB for duplicate records and forward any issues to the TSC’s internal quality assurance unit for review.
The TSC’s Watchlist Quality Efforts
Our June 2005 audit report identified weaknesses in the completeness and accuracy of the consolidated watchlist. At that time, TSC management acknowledged that it needed to focus more attention on ensuring the quality of the watchlist. We recommended that the TSC regularly review and test the information contained in the consolidated watchlist database to ensure the data is complete, accurate, and non-duplicative. We also recommended that the TSC coordinate with participating agencies and establish procedures to identify and resolve missing and conflicting record information.
In response to our recommendations, the TSC increased its quality assurance efforts and implemented a data quality improvement plan. Additionally, in November 2006, the TSC’s consolidated terrorist watchlist database was upgraded to incorporate a tracking feature for quality assurance activities. As a result of this upgrade, individual watchlist records in the database now contain a record (referred to as a QA ticket) in which TSC staff can record all quality assurance work that has been performed on that record.
The Nominations and Data Integrity Unit (NDIU) is responsible for performing the TSC’s activities related to ensuring the quality and accuracy of the watchlist. The NDIU’s activities for ensuring the quality of watchlist information can be categorized into three areas: (1) reviewing incoming watchlist data (referred to as the single review queue); (2) performing reviews of historical records following an encounter where the TSC identifies a potential discrepancy in watchlist records; and (3) conducting special quality assurance projects, such as performing a targeted review of the Transportation Security Administration (TSA) No Fly list.14 As of March 2007, the TSC had assigned 34 staff to this unit. In comparison, as of September 2004 the TSC had 12 staff assigned responsibility for nominations and data integrity tasks, including 1 staff member that was dedicated solely to quality assurance matters.
To examine the TSC’s efforts to ensure the quality of the information in the TSDB, we examined 156 TSDB records that had been subjected to the TSC’s quality assurance procedures. Of these 156 records, 36 involved record deletions and we found that each had been handled appropriately. Using the remaining sample of 120 records, we performed tests to determine if the watchlist records were accurate. We found that, in general, the TSC’s actions to review records as part of a targeted special project successfully ensured the quality of the data, and we identified virtually no errors in the 15 records we tested in connection with special project reviews. In contrast, our examination of 105 records subjected to the single review queue or post-encounter quality assurance reviews revealed that 38 percent of these tested records continued to contain errors or inconsistencies that were not identified through the TSC’s quality assurance efforts.
In general, we believe the actions the TSC has taken to improve quality assurance since our last audit are positive steps. We also recognize that it is impossible to completely eliminate the potential for errors. However, the inaccuracies that we identified in TSDB records that had undergone the TSC’s quality assurance processes underscore the need for additional actions to ensure that the TSDB is a reliable source of information about known or suspected terrorists. The results of our testing and analysis of the TSC’s quality assurance efforts are summarized below.
The TSC’s Review of the No Fly List
In July 2006, the Homeland Security Council Deputies Committee issued guidance on how to correctly apply its criteria for including individuals on the No Fly list. Subsequently, the TSC submitted all TSDB records associated with individuals who were on the No Fly list to a comprehensive quality assurance review using this guidance. When the TSC began its review in July 2006, the No Fly list contained 71,872 records. The TSC completed its special review of the No Fly list on January 31, 2007, determining that the No Fly list should be reduced to 34,230 records.15 The TSC recommended 22,412 records for removal from the No Fly list and placement on the TSA’s Selectee list.16 For another 5,086 records, the TSC determined that the individual did not require inclusion on either the No Fly or Selectee list.
We selected and reviewed 15 records that were part of the TSC’s review of the No Fly list. We did not find any data inaccuracies or inconsistencies in these records. Each record’s basic identifiers [SENSITIVE INFORMATION REDACTED] were shown consistently in all of the affected databases and each record remained the same or was downgraded from the No Fly list in accordance with the final recommendation of the TSC.
Data Inaccuracies and Inconsistencies Exist After Quality Assurance Review
Unlike our review of the No Fly list special project, however, our examination of records passed through the TSC’s single review queue or encounter-driven quality assurance processes revealed that records were still likely to contain errors or inconsistencies. We examined 105 records to determine if basic information [SENSITIVE INFORMATION REDACTED] was shown consistently in all of the affected databases. We also verified that correct handling codes were included on watchlist records. In short, our testing revealed that records the TSC reviewed through its routine quality assurance processes frequently continued to contain errors, which indicates weaknesses in the TSC’s practices for verifying the integrity of the original watchlist data.
As previously reported, we found that 7 of the 105 records we tested were not exported to appropriate downstream databases. In addition, our review of the 105 watchlist records that had been subjected to the TSC’s single review queue or encounter-driven quality assurance processes revealed that 35 records had inconsistent identifying information when comparing one or more fields in the TSC’s consolidated watchlist records with the source or screening agencies’ database records. Identifying information related to terrorist watchlist identities must be accurate and consistent across all systems involved in the watchlisting process, namely the TSDB, the downstream systems, and the nominating agencies’ systems. Inconsistent data can confuse or delay TSC Call Center operators in their efforts to determine whether an encountered individual is a positive match to a known or suspected terrorist. Further, inconsistent information among databases involved in terrorism screening indicates that at least one record may be incorrect. Incorrect records can also misinform frontline screening agents and contribute to the misidentification of a person not on the watchlist or the inappropriate release or admittance of a dangerous individual. Finally, our testing of the 105 sample watchlist records also revealed that 5 records contained incorrect handling instructions.
During our review, it became apparent that both the TSC’s quality assurance efforts and our reviews of watchlist records identified errors and inconsistencies in incoming records from the source agencies – NCTC and the FBI. We discussed the watchlist nomination process with NCTC and FBI officials, and both agency representatives stated that records are reviewed for accuracy, completeness, and consistency before the records are forwarded to the TSC. However, these efforts are failing to identify a significant number of deficiencies in the nominated records. The TSC’s quality assurance efforts, therefore, are hampered by the inaccurate and incomplete source material.
Untimely Resolution of Quality Assurance Issues
Delays in the closure of quality assurance matters directly affects the accuracy of the consolidated watchlist database because records can contain inaccurate and incomplete information for extended periods of time while the matter is being resolved. We examined a sample of 51 quality assurance matters opened between February 2006 and February 2007. We found that these matters were open from 0 days (matter was closed the same day as it was opened) to 329 days. On average, the quality assurance matters examined in our sample were open for 80 days.
The TSC has not established a performance measure identifying what it believes to be an acceptable duration for its analysts to complete a quality assurance review. According to TSC personnel, NDIU analysts were supposed to follow up on all quality assurance matters every 30 days. However, the TSC does not have a mechanism such as a standardized report or digital dashboard that catalogs all outstanding quality assurance matters.17 In concert with the development of timeframes for resolving quality assurance matters, we believe the TSC should develop a system, including an aging schedule, to track its quality assurance work.
Weaknesses in TSC Quality Assurance Policy and Oversight
We also found that the TSC has implemented necessary enhancements in its quality assurance resources and processes since our last audit. Our examination of records submitted to the TSC’s No Fly list special project showed that the TSC’s review was generally successful in ensuring the quality of watchlist records. However, the inaccuracies we found in our review of watchlist records that were subjected to the TSC’s single review queue and post-encounter quality assurance reviews – examinations that are less comprehensive than the No Fly list review – indicate a need for further enhancements to these quality assurance processes.
During our audit, we performed a physical observation of TSC analysts conducting quality assurance reviews of watchlist records. We noted that the analysts’ method of performing their reviews was not always consistent. For example, some analysts inspected all of the documents supporting a TSDB record while other analysts relied solely upon summary information. We also found that the analysts were not consistently documenting their quality assurance work.
We believe that this situation was caused by inadequate standard operating procedures (SOP) detailing the TSC’s quality assurance processes and by insufficient training. The TSC has an SOP for its quality assurance efforts, but the document was last revised on August 16, 2005. Moreover, the document provides incomplete guidance to analysts on the processing of quality assurance matters and did not mention the existence of special quality assurance projects and encounter-based quality assurance reviews. Further, while the SOP informs the analysts performing standard quality assurance reviews how to examine watchlist records, it fails to detail what fields, supporting information, and other aspects of the records the analysts should be verifying and comparing. In addition, these procedures do not instruct the analysts on the necessary actions to take when inaccurate or incomplete information is identified.
Additionally, the TSC provides its quality assurance analysts only a few days of training before allowing them to work independently, and no supplemental training is required. Moreover, the TSC does not have a mechanism for regularly evaluating the work of its quality assurance analysts to help ensure that the analysts are performing appropriate reviews and keeping abreast of any process changes. We believe that the TSC should develop a more detailed and comprehensive quality assurance SOP to better guide NDIU analysts through their work. In addition, the TSC should develop a mechanism to routinely review its analysts’ work to identify processing deficiencies and areas requiring additional training.
Insufficient Process to Comprehensively Review Watchlist Data Quality
In response to our previous TSC audit that identified errors and inconsistencies in the watchlist records, the TSC stated that it intended to conduct a record-by-record review of the approximately 400,000 records in the TSDB. The TSC later estimated that this review would not be complete until 2012. In February 2007, TSC officials stated that the review was being performed through its three-pronged quality assurance strategy – the single review queue, encounter-driven quality assurance reviews, and special projects. TSC officials told us that they plan to examine the TSDB following the completion of the ongoing special projects and determine how many TSDB records have not yet been reviewed. The TSC then plans to review any previously unexamined TSDB records.
In February 2007, TSC officials told us that since the inception of the single review queue in March 2006 over 670,000 TSDB records had been reviewed and the agency had revised its estimated completion date. TSC officials now project that the record-by-record review will be complete by the end of 2007. However, we believe that the TSC may have overstated the number of records reviewed and is underestimating the amount of time and effort that it will take to complete its review of the entire TSDB. We base these conclusions on the following factors:
As previously discussed, the TSC’s single review queue and encounter-driven quality assurance processes do not sufficiently ensure the quality of the watchlist records. Therefore, the TSC should reconsider records examined in these processes in its count of records reviewed.
The number of records reviewed is not limited to the review of unique records. Rather, the TSC’s quality assurance process allows for one record to be reviewed multiple times: through the single review queue, following each request to modify or delete the record, in accordance with one or more special projects, and subsequent to each encounter. Therefore, we believe that the TSC’s cumulative tally of records reviewed can include records counted multiple times.
Between September 2006 and April 2007, the TSDB grew at an average rate of over 20,000 records per month. This growth adds to the analysts’ workload. Since April 2004, the TSDB has more than quadrupled in size, growing from 150,000 to 724,442 records in April 2007.
As of February 2007, there were about 3,000 open quality assurance matters that required follow-up.
As part of this review, we obtained TSC data for the number of quality assurance matters identified and resolved between November 2006 and March 2007.18 These data show that the TSC is identifying incomplete or inaccurate information in TSDB records faster than the matters are being resolved by source agencies. As the following graph shows, cumulative differences between identified quality assurance matters and addressed quality assurance matters increased from 177 in November 2006 to 2,514 in March 2007. This differential also shows that the TSC is regularly identifying errors or concerns with known or suspected terrorist records.
Cumulative Growth in Quality Assurance Matters
Source: OIG analysis of Terrorist Screening Center data
We believe that if the number of watchlist records and the associated quality assurance matters in the TSDB continue to grow, the TSC will not complete the record-by-record review of the TSDB by the end of 2007 as anticipated. The TSC needs to accurately determine the magnitude of the unexamined portion of the TSDB so that the TSC can implement a sound plan for examining those records and develop a realistic completion date for the project. Further, the TSC should establish benchmarks against which it can measure its progress.
Watchlist Redress
We found that the TSC’s efforts to resolve terrorist watchlist redress matters since our previous audit have improved. For example, in 2005 the TSC created a dedicated unit for redress matters. The TSC also helped to spearhead the creation of a multi-agency Memorandum of Understanding (MOU) focusing on watchlist redress (Redress MOU) and developed a comprehensive Redress SOP to ensure watchlist information for redress complainants is accurate, complete, and current.
The frontline screening agencies, such as DHS and the State Department, receive complaints from persons seeking relief, or “redress,” related to the terrorist watchlist screening process. Matters believed to be related to a terrorist watchlist identity or an encounter involving the watchlist are forwarded to and reviewed by the TSC.19 The TSC Redress Office conducts an examination of the watchlist records, reviews other screening and intelligence databases, and coordinates with partner agencies for additional information and clarification. The TSC determines if any watchlist records need to be modified or even removed from the watchlist, ensures these identified changes are made, and notifies the referring frontline screening agency of its resolution. The frontline screening agency is then responsible for responding to the complainant. TSC policy requires that responses to complainants neither confirm nor deny the existence of watchlist records relating to the complainant. According to TSC officials, this nondisclosure policy protects U.S. counterterrorism operations and intelligence objectives and safeguards the personnel involved in these sensitive activities.
We judgmentally selected 20 redress complaints received by the TSC between January 2006 and February 2007 and reviewed the corresponding files to determine if the TSC followed its Redress SOP for resolving complaints. We found that in each of the sampled cases the TSC complied with its Redress SOP, including reviewing the applicable screening and intelligence databases, coordinating with partner agencies, and reaching appropriate resolutions. However, the TSC’s redress activities identified a high rate of error in watchlist records. In addition, we believe the TSC needs to address the timeliness of redress complaint resolutions.
Significant Watchlist Record Changes Following TSC Redress Review
As part of our review, we analyzed TSC data on the resolution of terrorist watchlist redress complaints. Between January 2005 and February 2007, the TSC closed 388 of the 438 redress complaints it received. Through its comprehensive redress review process, the TSC concluded that 45 percent of the watchlist records related to redress complaints required modification or deletion from the watchlist. In some instances, the TSC stated that redress resolution may have been simultaneous to current watchlist record updates. We also found instances where the TSC Redress Office found inaccuracies in the watchlist record or discovered additional, relevant information that had not been passed to the TSC.
The Privacy Officer acknowledged that the high percentage of records requiring modification or removal may point to deficiencies in the terrorist watchlist nomination process and with nominating agencies not providing the TSC additional information important for appropriately updating terrorist records. We believe that the results of the TSC’s redress reviews are a further indicator that watchlist data needs continuous monitoring and attention.
Untimely Resolution of Redress Complaints
The TSC is responsible for adjudicating watchlist-related complaints through its review process and working with nominating and screening agencies to resolve the matters in a timely fashion. The Redress MOU states that one of the goals of the redress process is to provide a timely review, which ensures any required changes to the watchlist are implemented efficiently so that watchlist records do not continue to be inaccurate.
We reviewed TSC files and statistics for closed redress matters to determine the efficiency of redress reviews. This data revealed that it took the TSC, on average, 67 days to close its review of a redress inquiry. For redress matters referred to the TSC during the last semiannual period in our review (July through December 2006), it took the TSC an average of 57 days to finalize its review. In addition to these closed matters, we also analyzed the 50 open redress complaints as of February 27, 2007, and determined that these matters had been open an average of 61 days. Of these complaints, 38 percent were open over 60 days, including 2 inquiries that were pending over 180 days.
| Open TSC Redress Matters (as of February 27, 2007) |
||
| Number of Days Open | Number of Open Redress Matters |
Percentage of Total Open Redress Matters |
| 180 days or more | 2 | 4% |
| 90-179 days | 12 | 24% |
| 60-89 days | 5 | 10% |
| 30-59 days | 11 | 22% |
| less than 30 days | 20 | 40% |
| Total | 50 | 100% |
Our review of redress files indicated that delays were primarily caused by three factors: (1) the TSC took a long time to finalize its determination before coordinating with other agencies for additional information or comment, (2) nominating agencies (the FBI and NCTC) did not provide timely feedback to the TSC or did not process watchlist paperwork in a timely manner, and (3) certain screening agencies were slow to update their databases with accurate and current information. For example, our file review found that the State Department and the DHS’s Customs and Border Protection did not revise encounter records in a screening database in a timely fashion to reflect modified or removed terrorist identities.
TSC officials acknowledged that it has not developed response timeframes for redress matters with its partner agencies. While the Redress MOU states that one of the goals of the redress process is to provide a timely review, the MOU does not define what constitutes a reasonable timeframe. We believe that timeliness measures could be used as standards to evaluate the U.S. government’s efficiency in resolving terrorist watchlist redress inquiries and responding to complainants. Because the TSC is central to resolving any complaint regarding the content of the consolidated terrorist watchlist, we encourage the TSC to organize the U.S. government’s effort to develop timeliness measures for the entire watchlist redress process.
More Proactive Efforts Needed to Mitigate Incidence and
Effect of Watchlist Misidentifications
The TSC does not have any policy or procedures to proactively use information from encounters to reduce the incidence and impact of terrorist watchlist misidentifications. For example, the TSC could program its encounter tracking system to automatically generate a quality assurance lead for the TSC to perform a comprehensive review of watchlist records that have been the subject of a certain number of encounters. Moreover, the TSC’s strategic plan does not include goals or actions associated with reducing the incidence of misidentifications or the impact on misidentified persons other than that covered by a formal redress process. Considering that nearly half of all encounters referred to the TSC Call Center are negative for a watchlist match, we believe the TSC should consider misidentifications a priority and develop strategic goals and policy specific to mitigating the adverse impact of the terrorist screening process on non-watchlist subjects, particularly for individuals who are repeatedly misidentified as watchlist identities.
Conclusion and Recommendations
We found that since our June 2005 report the TSC has enhanced its staffing and implemented practices to handle redress matters and help ensure the quality of terrorist watchlist information. Our review also found that the TSC’s processes for examining watchlist records as part of its special project (the No Fly list examination) and redress complaint reviews were comprehensive and improved watchlist data quality.
However, we found continued weaknesses in other watchlist processes and significant deficiencies in watchlist data. We determined that the FBI’s fragmented international terrorism nomination process caused many terrorist identity records to be inaccurate, incomplete, and inconsistent across watchlist systems. Additionally, the TSC’s single review queue and encounter-driven quality assurance processes were not successful in ensuring the quality of watchlist records. We also found that TSC quality assurance analysts employed disparate procedures in their reviews, and the TSC did not have a mechanism for conducting oversight of its quality assurance efforts.
We believe the TSC should consider incorporating elements from its more comprehensive reviews in its other quality assurance processes to help better ensure the quality of watchlist data. Further, the TSC should develop detailed, comprehensive standard operating procedures and an oversight function for quality assurance matters.
In addition to these process deficiencies, we found suspected or known terrorists still missing from the watchlist or downstream screening systems, incorrect terrorist handling codes on watchlist records, and duplicate identity records within the TSDB. Moreover, our testing of specific watchlist records revealed that records submitted to a TSC quality assurance review contained significant errors –38 percent of the records tested contained data that was inaccurate, incomplete, inconsistent, or not current. Further, our examination of TSC redress data for positive-match encounters showed that the TSC determined that 45 percent of the watchlist records referred for review required modification or removal. In addition, watchlist agencies, including the TSC and nominating and screening agencies sometimes caused unnecessary delays in closing redress inquiry reviews.
The results of our testing of watchlist records, as well as the TSC finding that many records involved in its redress reviews required modification or removal, indicate a deficiency in the integrity of watchlist information. We recommend that the TSC resolve current process weaknesses – within the TSC and at nominating agencies – that are contributing to the weaknesses we identified in the watchlist data. The TSC also needs to develop and implement a plan to complete a sufficient analysis of all watchlist records in a timely fashion.
This report contains 18 recommendations to help the TSC improve its operations and the quality of watchlist data. These recommendations include two recommendations to the FBI directly for matters pertaining to its operations outside the TSC.
The participating agencies include the FBI, Central Intelligence Agency (CIA), and the Departments of Homeland Security (DHS) and State (State Department).
Department of Justice, Office of the Inspector General, Review of the Terrorist Screening Center, Audit Report 05-27, June 2005.
Detailed information regarding the audit objectives, scope, and methodology is contained in Appendix I.
NCTC was established on May 1, 2003, to develop comprehensive threat assessments through the integration and analysis of terrorist information collected domestically and abroad by the U.S. government. NCTC is a component of the Office of the Director of National Intelligence and was formerly known as the Terrorist Threat Integration Center.
The FBI is a source agency for domestic and international terrorist information; it forwards relevant information to NCTC on suspected or known international terrorists.
The TSC’s general criterion for including a record in the consolidated watchlist database is that the nominating agency must have provided evidence of a nexus to terrorism. From a data perspective, the minimum criteria for inclusion of a terrorist identity into the TSDB are that the record contains at least a partial name (e.g., given name, surname, or both) and at least one additional piece of identifying information (e.g., date of birth).
A description of each of the downstream screening systems is contained in Appendix II.
The TSC provided data on screening agency encounters from February through April 2007. We reported the average of these 3 months.
On April 27, 2007, the TSC implemented an information technology solution to delete these records.
To alert the TSC of this non-standard entry of records into the TSDB, the TSC implemented a special flag, referred to as “FBI sole source,” for FBI-originated international records. This designation precludes all future electronic transactions, including related modifications and deletions, from being exported from the TSDB to VGTOF.
The total sample of 50 records consisted of 25 each for domestic and internationally known or suspected terrorists. These 50 records were part of our 156 watchlist record sample that we selected for testing.
Officials from NCTC, the TSC, and TREX stated that each agency attempts to process nominations to the watchlist within 1 day.
For each terrorist watchlist record in the consolidated database, only these five fields are exported to downstream systems for use in identifying suspected or known terrorists during the watchlist screening phase.
During its review of the No Fly list, the TSC continued to receive routine No Fly list additions, modifications, and deletions through the watchlist nomination process. As a result, it is not possible to subtract the special project-driven No Fly list changes from the starting point of 71,872 records and obtain the correct number of No Fly records as of January 31, 2007.
A digital dashboard is a business management tool that visually displays the status of a business project. The dashboard can provide warnings, next steps, action notices, and summaries of a project.
The TSC only could provide historical data on quality assurance matters since November 2006 when the latest version of the TSDB was deployed.
On occasion, the TSC receives a redress complaint referral from a screening agency and determines that the complaint does not relate to a terrorist watchlist identity or an encounter involving the watchlist. The TSC returns such complaints to the referring agency for resolution.