Throughout the existence of the TSC, its management has focused much effort in developing and deploying technology capable of consolidating the different watch lists into a single database. A major challenge for the TSC was to integrate different types of information in varying formats from the existing systems into a comprehensive index of watch listed individuals. The new system would also ultimately need to facilitate real-time connectivity to end-users and include evolving technology, such as advanced name-search capability and biometric data.
The Memorandum of Understanding executed following the President’s mandate to create a terrorist screening organization in HSPD 6 provided direction to the TSC regarding its responsibility to develop a consolidated database. Specifically, the MOU required the TSC to consolidate the government’s approach to terrorism screening and to maintain a continuously updated database containing unclassified terrorist information from the FBI and NCTC. The MOU required that the FBI serve as the TSC’s source with regard to purely domestic terrorism information, defined as information about U.S. persons that has been determined to be purely domestic terrorism information with no link to foreign intelligence, counterintelligence, or international terrorism. The MOU also required that NCTC serve as the TSC’s source of terrorist information, with the exception of the domestic information that the FBI was required to provide. In turn, federal agencies were directed to provide all domestic and international terrorist information in their possession, custody, or control to the FBI and NCTC, as appropriate.
Upon its creation in September 2003, TSC officials and partner agency members formed a working group to define existing database structures and determine the basic functionality and future uses of the consolidated database that they were tasked with creating. During this initial planning process, TSC officials identified several barriers to the timely development of the consolidated database. First, TSC officials stated that there was a shortage of knowledgeable IT professionals with the necessary security clearances to work at TSC. Second, a TSC official did not believe that one contractor had sufficient numbers of qualified employees to complete the full design and implementation of the consolidated database. Third, attempting to hire a single contractor to create the ideal database environment would be cost prohibitive. Finally, because of the critical nature of the project, the TSC faced a compressed deployment schedule. As a result, the TSC divided creation of the consolidated database, which was named the Terrorist Screening Database (TSDB), into three phases: 1) TSDB 1A, 2) TSDB 1B, and 3) Advent TSDB.
According to TSC officials, the initial phase of the database, TSDB 1A, was created using proprietary software owned by the contractor. According to TSC officials, this decision was made in an effort to consolidate the watch list information in the most expedient way possible. The TSC understood when entering this contract that it was purchasing a proprietary software application and that all of the contractor’s programming would continue to remain the property of the contractor. In addition, any modifications to the program or running sophisticated queries would require the contractor’s expertise. The limitations of TSDB 1A, according to TSC officials, would be addressed in the second phase, TSDB 1B.
Between September 2003 and February 2004, TSC staff worked with the contractor to examine the system architecture of the source databases and define the system requirements for the TSDB 1A. This included ensuring that the information that would be received from the participating agency databases, such as individual names and dates of birth, would be compatible with the fields being created for the 1A database.
Although the MOU required the TSC to receive all terrorist information from the FBI and NCTC, at the time of the creation of TSDB 1A the infrastructure for this process was not yet established and the FBI and NCTC were not reliably receiving and inputting terrorist information from other agencies. This resulted in the TSC directly obtaining information from other sources to populate the 1A database.
In February 2004, the TSC began the consolidation of terrorist information by conducting a one-time, manual batch acceptance of data from each of the various supporting systems into the TSDB 1A. This effort included obtaining information directly from the No-Fly and Selectee Lists, as well as from the following systems: TIPOFF, VGTOF, and the Treasury Enforcement Communications System (TECS). By populating the TSDB with information from these five sources, the TSC incorporated information from each of the primary watch listing systems discussed in Chapter 2. As noted in Chapter 2, the remaining systems generally were subsets of information contained in the primary systems or were not actual watch listing systems.
TSC personnel believed that this method of obtaining records was an efficient and effective way of initially populating the TSDB with the most comprehensive universe of terrorist information possible. However, they also recognized that some terrorist watch list records would not be received due to differences in system architectures and participating agency missions, processes, and data requirements. According to the TSC, these obstacles could not be addressed quickly and in many cases could not be anticipated. However, the time constraints under which the TSC was operating mandated that a consolidated database be developed and populated expeditiously. TSC management has asserted that the organization is still working to resolve obstacles to the receipt of additional data.36
TSC management further informed us that neither TSC staff nor the participating agencies reviewed the data prior to its transfer to the TSC because of time constraints and the volume of work involved. Further, because these systems were being relied upon as independent systems, TSC officials believed them to be sufficiently reliable for acceptance.
According to TSC officials, the TSDB 1A began operating on March 12, 2004, and was discontinued on April 1, 2005. The database was manually updated daily using diskettes of new or revised information from the participating agencies. The entire TSDB 1A database was overwritten each day when the new data file was loaded. Given the design of the TSDB 1A database, this overwriting was the only method available to update the information. However, overwriting the data on a daily basis eliminated the ability to view the database in historical context. In addition, the TSDB 1A could not automatically export data to the participating agencies. Rather, the system relied on TSC staff to manually send updated files on diskettes to the supporting systems. The updated information was then uploaded into the databases of the participating agencies. (TSDB 1B, as discussed below, sends direct electronic updates to the agencies.)
TSDB 1A Name-Search Capability
When call screeners at the TSC searched a name in the TSDB 1A, the system used a software application to search on the phonetic code of the last name or the last name with a first name initial, as well as the exact month and day of birth and a plus or minus one in the subject’s birth year. The search software recognized when a nickname was being searched and replaced it with the corresponding proper name (e.g., "Bill" would be replaced with "William"). In addition, searches for names beginning with a silent letter, such as "Knight," would result in several corresponding spellings (such as "Night").
TSC officials reported that this search software did not provide consistently good results on names not originating in Europe or the Americas. TSC managers recognized the shortcomings of this search system and attempted to improve search capabilities in subsequent versions of the database. Name-search capabilities are discussed in further detail in the TSDB 1B and Advent TSDB sections of this Chapter.
TSC management opted to use a different contractor for the development of the second phase of the consolidated database – the TSDB 1B. The primary purpose of this phase was to create a system that provided the TSC with more control over the database and its management and to improve connectivity between the TSDB and other databases.
The new contractor created the TSDB 1B using the basic structure of the State Department’s TIPOFF system, which has more flexible and comprehensive search capabilities than the TSDB 1A. In creating the 1B database, the TSC obtained batches of records primarily from the FBI and NCTC, in accordance with HSPD-6 and the resulting MOU, which as noted previously required all federal agencies to provide terrorist information to these two agencies.37
According to TSC management, the TSDB 1B was originally scheduled for full operational capability, including call screening and exporting of records, by June 2004. Despite it coming online at that time, TSC officials had concerns about the completeness of the records in the database and decided to run the TSDB 1A and 1B in parallel until these concerns could be fully addressed. As noted previously, the TSC stopped using TSDB 1A on April 1, 2005, at which time the 1B database became the single consolidated watch list.38
Unlike TSDB 1A, the 1B database can communicate with the participating agencies' IT infrastructures and databases and can provide automatic data exchange, eliminating the need for daily diskette transfers of new and updated information. Because the TSDB 1B system has the capability for automatic data exchange, it has been used since its inception to export records to the databases at the various participating agencies. In addition, unlike the TSDB 1A, the 1B system does not overwrite the entire database of records on a daily basis. Instead, the TSDB 1B is updated only with additions, deletions, and modifications to the existing records in the database. Consequently, the system retains a history of all changes that were made to the records. This is a stronger system control that provides the TSC with the advantage of tracing record changes and allows for a greater ability to review data accuracy and reliability.
As shown in the following chart, information regarding international terrorism from consular offices, Interpol, the intelligence community, the FBI, state and local law enforcement, and foreign governments is now funneled through NCTC for inclusion in the consolidated watch list. In addition, information regarding purely domestic terrorism from the FBI, state and local law enforcement, and the intelligence community is processed through the FBI for inclusion in the consolidated watch list. The TSC then makes the information from the TSDB available to the appropriate end users, such as border patrol agents, consular offices, and state and local law enforcement. For example, new information regarding an airline hijacker obtained from an overseas consular officer travels to NCTC for vetting. The NCTC then transfers the information on the subject to the TSDB, where the data is distributed to pertinent systems such as CLASS, IBIS, No-Fly, VGTOF, and others.
| Domestic and International Terrorist Information Data Flow |
|
[Not Available Electronically]
|
| Source: TSC Management |
TSDB 1B Name-Search Capability
Because the TSDB 1B system architecture is based on the TIPOFF database, the system uses a name-search capability called [SENSITIVE INFORMATION REDACTED] that is more advanced than the 1A system. Specifically, the [SENSITIVE INFORMATION REDACTED] software uses a broader algorithm for searching names that returns query results more precise to all cultures of names within the database. This software has been used by the State Department since the creation of TIPOFF in 1987. According to DOS officials, [SENSITIVE INFORMATION REDACTED] has been an effective tool in the terrorist watch list process.
Advent TSDB and the Future of the Consolidated Database
In the short term, the TSC plans to make improvements to the TSDB 1B that will increase its completeness, functionality, and usability. The database was programmed to contain fields for additional information including: comments, data sources, and biographical information. However, TSC officials did not enable these fields at the time the system initially came on line in June 2004. The TSC has undertaken an initiative called the "wedge project" to enable and populate these fields. In October 2004, the initial database programming for this project was complete. According to the TSC CIO, however, the TSC is not yet receiving much additional information, primarily because of differences in formatting. The CIO stated in December 2004 that the TSC was in formal negotiations with participating agencies as to the format in which the information is to be sent.39
The TSC’s ultimate goal is to create a database called "Advent TSDB" that will establish real time connectivity between the TSDB and all supporting agency databases. TSC officials also noted that Advent TSDB will include a full-range of biometric data. This information will improve the screening process by providing additional descriptive data against which to screen encountered individuals.
Real-time connectivity between the TSDB and the supporting agency databases will permit the rapid transfer of information between these systems, increasing the timeliness and completeness of all participating systems’ databases while requiring less human involvement. However, most of the participating agencies have different computer operating systems and architecture that may not handle this type of connection. As a result, agencies will need to upgrade their systems to facilitate this capability. While the TSC expects that it will take years to fully implement this plan, the first segment is planned for completion in FY 2005. This first phase will automatically connect NCIC to the TSDB through the Criminal Justice Information Services Division (CJIS), therefore allowing all federal, state, and local law enforcement officials with NCIC access to have immediate, direct, real-time connectivity with TSDB.
The TSC expects that during FY 2005 it also will develop the ability and implement procedures to receive biometric data from NCTC and export that data to NCIC. However, this process is not expected to be fully mature for some time. According to the CIO at the TSC, there is no uniform standard of acceptability for biometric data among the supporting systems. Therefore, only text fields for biometric data are planned to be shared in the first phase. TSC officials stated that graphic files containing some of this information can be made available in the TSDB 1B system; however, this information would not be searchable. In essence, a picture of the biometric information can be stored in the database. TSC officials said that they are awaiting action by other entities to establish the uniform standards and did not know when further progress was anticipated.
Currently, TSDB 1B is an independent system that is not directly connected to the supporting databases. Therefore, in order to access available biometric information, TSC staff must query the source databases, which reside on multiple networks or computer terminals. This may involve searching up to five different systems and switching between classified and unclassified environments. These cumbersome procedures increase the likelihood that biometric information will be missed and adds to the amount of time that TSC staff must take to research available information. The TSC, in conjunction with partner agencies, is currently taking steps to accommodate necessary biometric data in its watch listing efforts.
TSC IT officials have indicated that [SENSITIVE INFORMATION REDACTED] will remain as the TSDB’s name-search capability for the foreseeable future. Although the State Department considers this to be a well-operating program, in the long term the TSC hopes to improve upon its name-search capability and is researching other government agencies’ experiences with the effectiveness of various programs. For example, the [SENSITIVE INFORMATION REDACTED] developed by [SENSITIVE INFORMATION REDACTED] has been adopted by NCTC for use in its new Terrorist Identities Datamart Environment (TIDE) database. This program provides for the automatic expansion of names to incorporate phonetic, cultural, and character variations, as well as combinations of these variations. Since one name can be spelled multiple ways, the software manufacturer claims to use search techniques that allow maximum efficiency in query results. While the TSC continues to research the best software for its mission, a TSC official said in October 2004 that the end product probably would be an expansion of the [SENSITIVE INFORMATION REDACTED] software.
In its relatively short existence, the TSC has experienced numerous changes in its Information Technology (IT) Branch. We found that the management of such an integral part of the terrorist screening process has been deficient. The TSC’s IT Branch – staffed with numerous contractors and little consistent management oversight – has not had strong, effective, and focused leadership over the agency’s IT functions. In addition, the TSC has experienced significant difficulty in hiring qualified staff with adequate security clearances to perform IT functions.
Prior to May 2004, the TSC’s IT Branch was led by acting Program Managers, each of whom was a contractor. In June 2004, the first non-acting, non-contractor Program Manager was brought on board, and he immediately began modifying plans for the next phase of the TSDB. However, the TSC did not hire its first CIO until August 2004.
In June 2004, the TSC established the Systems Architecture Board, which serves as the technical advisory group to the TSC Director, Deputy Directors, and TSDB Project Manager. The group, comprised of IT personnel from the TSC and contractors, is responsible for developing the TSDB system’s architecture.
Unfortunately, many major IT decisions were made prior to the arrival of the CIO and the creation of the Systems Architecture Board in June 2004. These include the creation and implementation of TSDB 1A and 1B and other support systems, as well as the establishment of controls and standards for operating and administering these systems. The CIO told us in October 2004 that the TSC has been operating in an immature IT environment since its inception. He explained that systems planning was negatively affected by the need to expeditiously create a consolidated database. He further stated that the IT Branch was understaffed and had not been sufficiently focused on establishing controls to ensure data integrity.
In our meetings with the TSC’s CIO, we found that he has an extensive background in both the information technology and intelligence fields. He also has acknowledged the need for active controls and audit trails within the TSC’s databases and manual processes. Such audit trails and controls are important because our review of the current TSC databases found little tracking and retention of historical transactions within the databases, as well as a shortage of human access controls.40
Content of the Consolidated Watch List
Each record within the consolidated watch list is designed to contain information about the law enforcement action to be taken when encountering an individual on the watch list. This information is conveyed through a "handling code," which provides insight into the level of threat posed by that individual. Generally, handling codes are expressed on a scale of 1 through 4. These handling codes are described in the following exhibit.
| FBI Handling Codes |
[SENSITIVE INFORMATION REDACTED]
|
| Source: TSC Management |
We reviewed a subset of the records in the TSDB 1B to gain an understanding of the characteristics of the individuals on the consolidated watch list. Our review of these records revealed that, as of October 7, 2004, the bulk of the records in the TSDB 1B were designated in handling codes 3 and 4.41 Specifically, 22 percent of the individuals in our sample were categorized by the FBI as handling code 3. In addition, 75 percent of the records in our sample had a handling code 4, the category requiring the lowest possible law enforcement response. Handling codes 1 and 2 were assigned to 193 and 125 records, respectively. Therefore, a total of only 318 records in our sample of 109,849 records were identified at the highest levels. The following chart provides a breakdown of handling codes applied to the subset of TSDB 1B records that we reviewed.
| Watch Listed Persons by Handling Code (Based on the subset of 109,849 records reviewed42) |
|
| Source: TSC Management |
As shown in the preceding chart, we also identified 336 records for which no handling code was assigned. This issue is related to the accuracy and completeness of individual records and is discussed in Chapter 7.
We asked the Director of the TSC about the types of individuals included in the TSC’s consolidated watch list. She informed us that, to err on the side of caution, individuals with any degree of a terrorism nexus were included in the TSDB, as long as minimum criteria was met (at least part of the person’s name was known plus one other identifying piece of information, such as date of birth). The Director further explained that one of the benefits of watch listing individuals who pose a lower threat was that their movement could be monitored through the screening process and this could provide useful intelligence information to investigators. In addition, she stated that watch listing lower-threat individuals that have associations with higher-threat level terrorists may lead to uncovering the location of higher watch listed individuals.
At this early stage in the TSC’s existence, the creation and operation of a single database housing consolidated terrorist information was the most important aspect of its mission. From the outset, TSC management was aware of the obstacles of fully integrating data from myriad, disparate sources as well as the necessity of blending multiple agency processes and data definitions. In response to these challenges, they focused on establishing the best possible database as quickly as possible.
TSC management recognized and we observed weaknesses in the TSC’s efforts to accomplish this endeavor, including limitations related to name-search capabilities, availability of historical information, and the use of audit trails within the TSDB databases. However, the TSC successfully integrated different types of information in varying formats from the existing systems into a comprehensive index of watch listed individuals.
Despite providing a consolidated watch list in a compressed timeframe, we identified significant weaknesses related to IT management and planning. The TSC is working to improve its IT management and create a system that facilitates complete real-time connectivity to the end users and includes advanced name-search capability and searchable biometric data.
We recommend that the TSC:
- Develop a formal IT plan for maturation of the IT environment at the TSC to address: a) IT staffing needs; b) controls to ensure data integrity; c) adequate oversight over IT contracts and contractors, and d) future improvements in the areas of TSDB connectivity, name-search capabilities, acceptance of biometric data, as well as other IT planning issues.
- Enhance the TSDB to add audit trails to track activity within the database, including historical data and detailed transactions by user, as well as to include enhanced human access controls.
Footnotes
- We reviewed the TSDB 1A database for accuracy and completeness. The results of our testing are contained in Chapter 7.
- During a query performed on the 1B database, we identified five records that did not originate from the VGTOF or TIPOFF or databases (the systems used by the FBI and NCTC, respectively). TSC staff could not explain the source of these records. Further details of this matter are explained in Chapter 7.
- TSC officials explained that the delay in achieving full operational capability of the TSDB 1B resulted in part from technical difficulties related to integrating watch list data in a manner that would ensure supporting agency databases receive in return an appropriate level of detailed information. The delay also resulted from the need to prevent duplicate records from being integrated into the TSDB 1B database. TSC officials explained that there was a shortage of available contractors to work on the project because NCTC was using the same company for its database work.
- For example, the complexion of an individual needs to be recorded in one standard format so the information is searchable.
- More details on our review of the accuracy and completeness of the database are provided in Chapter 7 of this report.
- Our sample consisted of the universe of records in the TSDB 1B that were identified for export to the VGTOF database as of October 7, 2004. This universe of 109,849 records represented 53 percent of the 207,553 total records in the TSDB 1B. We selected these records for review in consultation with TSC IT staff.
- The “Other” handling codes refer to one record that was transferred to the TSDB 1B from the TIPOFF database with the non-existent handling code 5. The TSC informed us that this record has been corrected. The remaining 2,990 records [SENSITIVE INFORMATION REDACTED].
