战术情报:陆军需要重新考虑和测试全源分析系统替代方案
(信件报告，03/07/94,GAO/NSIAD-94-49)
全源分析系统是一个陆军项目，用于自动处理和分析来自所有来源的情报数据。陆军正在分几个阶段或区块开发该系统。第一区将在未来两年内部署，而第三区将在1998年左右开始部署，并在2000年后部署。陆军将该系统在其20年生命周期内的开发、采购和运行成本定为约50亿美元。本报告评估了（1）第一区块的部署准备情况；（2） 进一步开发block II是否是提高系统能力的最具成本效益的方式；以及（3）是否有替代开发区II的替代方案。----------------------------索引术语-------------------REPORTNUM:NSIAD-94-49标题：战术情报：陆军需要重新考虑和测试所有来源分析系统替代日期：03/07/94主题：陆军采购军事系统分析战备研究和开发成本ADP采购测试成本效益分析指挥部控制系统国防应急计划计算机软件标识符：陆军全源分析系统陆军鹰眼/战士计算机系统陆军军阀计算机系统陆军战术指挥和控制系统ATCCS C-141飞机沙漠风暴C-130飞机国防部战场开发和目标获取项目陆军移动用户装备索马里海军陆战队情报分析系统海军战术指挥系统海上空军情报关联模块国防部相关作战情报中心欧洲系统**************************************************************************************************************************************************************此文件包含GAO**报告文本的ASCII表示。保留文本中表示章节标题、**标题和项目符号的描述。正文的主要部分和细分**，如章节、章节和附录，**由双线和单线标识。这些行右端**上的数字表示**文件大纲中每个小节的位置。这些数字与印刷品的**页码不一致。***尽管复制了图**标题，但未尝试显示图形图像。包含表格，但可能与**印刷版中的表格不同。***可通过致电（202）512-6000、将**请求传真至（301）258-4066或致函马里兰州盖瑟斯堡**6015号邮政信箱20884-6015，从GAO文件**分发机构获取本报告的打印副本。我们目前无法接受打印文件的电子订单*****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************封面=======================================================================================================================================================================================================给国防部长的封面报告1994年3月战术情报-陆军需要重新考虑和测试全源分析系统替代GAO/NSIAD-94-49战术情报缩写==========================================================================================================================================================================================================================================================================================================================================================全源分析系统ATCCS-陆军战术指挥和控制系统COEA-成本和作战效能分析DOD-国防部FAISS-部队司令部自动化情报支持系统IAS-情报分析系统JPL-喷气推进实验室TENCAP-国家能力战术探索USAREUR-美国。欧洲陆军信函============================================================================================================================信函B-253165 1994年3月7日尊敬的威廉·佩里国防部长尊敬的部长先生：本报告总结了我们对陆军部署全源分析系统（ASAS）第一区计划的审查作为临时情报处理系统，按计划进行区块二开发。我们的目标是评估一号区块的部署准备情况，二号区块的进一步规划开发是否是提高系统能力的最具成本效益的方式，以及是否有替代方案替代二号区块的开发。背景------------------------------------------信：1 ASAS是一个陆军项目，旨在开发一个计算机化系统，处理和分析来自所有来源的情报数据。陆军正在几个街区内开发ASAS。一号区块将在未来两年内部署。第二区块是一项后续工作，从1994财政年度开始，为期6年，到1996财政年度开始有限的部署。第三区块将于1998年左右开始，并在2000年后部署。陆军估计，ASAS在其20年生命周期内的开发、采购和运营成本约为50亿美元。ASAS block I是在过去10年中开发的，旨在提供初始的、有限的临时能力。喷气推进实验室（JPL）是第一区主承包商，负责vari的系统集成ous block I hardware and software components. The system is housed in trucks and truck-mounted shelters, and includes towed electrical generators. The Army said it has spent $1.4 billion on block I, most of which was for research and development. Block I procurement was $345 million for 11 sets to be fielded at corps and divisions, plus 1 set for training. The remaining Army units will continue to use existing systems until block II is fully fielded. In 1991, elements of the original block I were merged with a development project called Hawkeye, sponsored by the Army Intelligence School. According to Army officials, the Intelligence School initiated the Hawkeye effort because it was dissatisfied with the large, cumbersome equipment being developed for block I. The consolidation was intended to field a more capable system 2 years earlier than planned for block I fielding. This hybrid system included the original JPL equipment and software in one of two elements of each ASAS set. Hawkeye hardware and software was inserted in the other element of the set to replace original block I items. Block II is to have new and enhanced capabilities and be based on an open, or commercial, standard computer architecture. It will be designed as stand-alone workstations transported in large suitcase-sized boxes or transit cases and operated on tables or desks. Therefore, it could require fewer dedicated trucks, shelters, and electrical generators than block I. ASAS block III is to increase capabilities of block II. In addition to ASAS, the Army has other automated intelligence data processing systems in development and operation. One effort involves a derivative of the Hawkeye. Army officials said the Hawkeye was developed for $15 million and uses open commercial computer standards similar to those proposed for block II. According to one program official, the U.S. Army Europe (USAREUR) continued to develop Hawkeye and deploy additional intelligence data processing capabilities in a system called Warrior, which cost about another $15 million. Warrior development will continue in a new effort called Warlord, which is scheduled for initial deployment in March 1994. The Army plans to continue development of Warlord as a rapid prototyping program by agreement among USAREUR, the Army Intelligence School, and the Army program acquisition executive office for ASAS. Warlord development products will be retrofitted into block I and integrated into the concurrent ASAS block II development as appropriate. RESULTS IN BRIEF ------------------------------------------------------------ Letter :2 ASAS block I is not ready for fielding because (1) its initial operational testing was limited and flawed and (2) the block I configuration tested did not have the performance, reliability, availability, and deployability needed to support the Army's operations. The Army is making major changes to correct block I software problems identified in the test and is considering other hardware and software upgrades that address size, complexity, and maintenance concerns. With those changes, the system could be more operationally effective than the configuration previously tested. However, block I equipment is limited in quantity and will not be fielded to all Army units. The Army is also developing Warlord, an intelligence processing workstation based on the already deployed Warrior workstation, existing ASAS software, and software from the Army Tactical Command and Control System (ATCCS). Warlord has more advantages in several operational areas than the current block I and could provide an alternative to fielding and upgrading block I. The Army recently began considering Warlord capabilities for incorporation into block I and as a capability for units that will not receive block I. The ASAS block II development contract was awarded on October 29, 1993, before (1) completing block I initial operational tests and approving the system for final fielding, (2) deciding among proposed software and hardware upgrades to block I, (3) conducting a full cost and operational evaluation study on less expensive alternatives, and (4) establishing a proven performance baseline against which the Army could assess costs and benefits of block II development and procurement. The Army, therefore, does not know whether the planned development of block II is the most cost-effective way to provide improved capabilities. In addition, the Army and other services have several systems with various intelligence processing capabilities that could be integrated into one set of hardware and software. However, the Army has not fully examined the potential to integrate the capabilities of these systems to determine if a new block II program is necessary or if some existing software capabilities can be inserted into the block II program to reduce development. Recently, the Army began considering such an effort as part of block II development. BLOCK I: MAJOR PROBLEMS IDENTIFIED DURING LIMITED OPERATIONAL TESTING ------------------------------------------------------------ Letter :3 The ASAS block I initial operational test, conducted in September 1992, was not adequate to determine system operational effectiveness and suitability, and does not support a fielding release of block I, according to a Department of Defense (DOD) Inspector General report.\1 In addition, the Army operational test report concluded that block I was not operationally effective or suitable and required corrections to hardware and software. We found numerous examples of weaknesses in the operational testing. For example, we found that (1) key software capabilities were not tested; (2) a critical segment of the system, called collateral enclave, which provides the means to disseminate the processed intelligence, was not tested; (3) the majority of the performance criteria used to measure effectiveness and suitability were opinion polls (not pass/fail statistical criteria) and unqualified personnel were used to make those subjective judgments; and (4) a major portion of the test data collected was based on garrison operations rather than field operations. The Army has decided to conduct a second operational test in late fiscal year 1994 to verify that prior test problems have been corrected. The second test includes revised software, addition of the collateral enclave, use of numerical criteria, and data collected from field operations. Accuracy of results from the first operational test in September 1992 is also questionable because test personnel used to make subjective judgments on effectiveness and suitability did not meet experience qualifications. Originally, the nine panel members were to each have 15 to 20 years experience in military intelligence or operations. However, only three panel members had a background in military intelligence, with a combined total of 6 years experience. Only two panel members had more than 5 years experience in operations. No panel member had 15 years experience in either field. Although the block I operational test was limited, it demonstrated critical deficiencies in ASAS software and operational effectiveness. The III Corps' 1st Cavalry Division was used in the operational test. Members of the 1st Cavalry Division said they identified 350 software problems during pretest exercises that were not corrected prior to the test. They said the serious nature of many of these problems caused them to shut off key software capabilities during the test. For example, 1st Cavalry Division officials said that the 1st Cavalry bypassed automatic data correlation of unidentified enemy units because the block I software had mistakenly identified three enemy regiments as only one regiment. Therefore, some required capabilities were not tested. -------------------- \1 DOD-Inspector General Report Review of the All Source Analysis System as a part of the Audit of the Effectiveness of the Defense Acquisition Board Review Process - FY 93, April 20, 1993. OTHER BLOCK I DEFICIENCIES LIMIT OPERATIONAL EFFECTIVENESS ---------------------------------------------------------- Letter :3.1 Besides the problems found during the limited operational test, several other deficiencies exist. These include maintenance, operability, and deployability problems. The system is difficult to maintain, in part, because block I is composed of two different sets of hardware and software; thus, the Army must develop and maintain two sets of logistical support, software, and training packages. The two different computer systems also limit operational flexibility. According to an Army official, the original system was designed to pass data between the two components comprised of Hawkeye and original JPL equipment located at different intelligence locations. However, with two incompatible computers, operators can only pass messages between the two computers, rather than exchange data automatically as intended. The Army has designed a procedure to correct these problems. The complexity of the system poses additional problems. A December 1992 joint assessment by the 1st Cavalry Division and its parent, III Corps, said the block I equipment is difficult for soldiers to operate and presents serious training problems. On May 17, 1993, the Commanding General of the 1st Cavalry Division said ASAS, with minor enhancements, appears able to meet current and future intelligence requirements. However, he is also concerned about his ability to maintain the training levels required to operate block I. The General also said other problems with block I requiring immediate attention are (1) inability to conduct split tactical operations, (2) a major constraint in slow processing time, (3) too much difficulty in message retrieval, and (4) lack of capability in battle damage assessment. The December 1992 assessment also noted a need to improve the equipment's reliability. The Army operational concept requires ASAS to be available for operations 24 hours a day. However, Army analysis of block I in the configuration tested shows the system will only be available for operations about 12 hours each day. Army operational testers computed system availability in wartime conditions at between 38 and 69 percent. According to the Army's technical report, block I will only be available for operations about 52 percent of the time. Another Army study noted that block I had an even lower availability ratio of less than 40 percent. Such limits in the design could translate into high maintenance and operating costs as units attempt to keep the system operational. We also found that block I does not meet key Army requirements for deployability. The ASAS requirements document limits ASAS to no more than three C-141 loads per unit. However, an Army airlift study said four C-141 aircraft loads are required to move a block I set. The Army also waived the rail and helicopter transportable requirement for block I just before operational testing without explanation. According to one Army official, this waiver could create problems. For example, during Desert Storm rail transport to ports was the primary means used to move equipment; without the rail transport requirement, block I could not be used in such situations. In addition, to meet its contingency missions, USAREUR stated that an ASAS system must be capable of drive on/off C-130 transport aircraft for both echelons above corps and corps elements and be man portable at divisions and brigades. However, block I is neither drive on/off capable for upper echelons nor man portable for lower echelons. The Army airlift study noted the need to dismount shelters from 5-ton trucks prior to air movement. An Army official questioned whether cranes to place the shelters back on their vehicles would be available at all destination airfields during a contingency. The block I also may not provide much of an increase in capability over systems already available. For example, a December 1992 Army Science Board study said that, after 10 years of research, it is not clear that the capabilities of the ASAS to be fielded in the near future match those demonstrated in a 1981 system called Battlefield Exploitation and Target Acquisition system. This system was fielded by the United States European Command under the title Linked Operational Intelligence Centers Europe. ARMY EFFORTS TO CORRECT BLOCK I DEFICIENCIES: A NEW OPERATIONAL TEST REQUIRED ---------------------------------------------------------- Letter :3.2 After the 1992 test, the Army modified the defective block I software and hardware configuration to address problems identified in the test, as well as those found by the 1st Calvary Division and the III Corps during exercises. Army officials said the changes have corrected the problems, based on a demonstration and an exercise. However, the changes have not been operationally tested. Also, the Army is developing and considering several major configuration changes, for inclusion in block I, which further increases the requirement for testing prior to final fielding. The Army operational tester assessed the revised software that was demonstrated in April 1993. The assessment concluded that improvements have occurred in the software, hardware, and training. However, the assessment said the demonstration was not conducted under realistic operational conditions and an evaluation of operational effectiveness and suitability cannot be made. The report recommended that a follow-on operational test be conducted only after the test unit personnel have had at least 6 months of experience on the modified system. In addition to the new software upgrades to correct testing deficiencies, the Army plans to replace or significantly modify most of the JPL block I equipment and add new equipment not present during operational testing or the April 1993 operational demonstration. For example, the current JPL set of equipment consists of workstations connected to computer and communications processors in truck-mounted shelters. The Army is developing a replacement computer that combines the workstation and computer processor functions, which eliminates trucks and towed generators. The new computer will also function like commercial open architecture computers, and its smaller size could require fewer transit cases for the workstation. Changes and replacements are also underway or planned to the communications equipment that was tested during the first operational test. Although the Army does not yet know the final configuration for block I, the new upgrades being considered are intended to provide a smaller system that (1) is more deployable, (2) has better performance and greater system redundancy, (3) is easier to operate and maintain, and (4) is more reliable and available. The changes under consideration could also result in a system architecture similar to that planned for block II, which provides advantages in retrofitting block II capabilities back into block I. However, as of October 1993, Army officials were not sure which of the upgrades being developed will be implemented and deployed in block I. A follow-on initial operational test is scheduled about September 1994, primarily on the new software developed to correct prior deficiencies. Other major upgrades to computers, communications processors, and software will not be part of this test; a DOD official said these upgrades will be operationally tested later. However, as of August 1993, a program office document showed up to eight sets of block I equipment were scheduled to be deployed before the 1994 test is completed. Army headquarters officials subsequently stated that deployment will be limited to five block I sets before the test is completed. Three sets, which will be used in the upcoming test, are deployed to units at Fort Hood, Texas, and two sets will be deployed to the 82nd Airborne Division and the XVIIIth Airborne Corps at Fort Bragg, North Carolina. WARLORD COULD POTENTIALLY REPLACE OR AUGMENT BLOCK I ------------------------------------------------------------ Letter :4 Warlord, previously called Warrior, is a current development effort by the Army to merge, on a single workstation, the best features of software from ASAS block I, Warrior, the ASAS Collateral Enclave, and the Common ATCCS Support Software. Warlord will be a workstation that can be used with multiple communications equipment, such as the current ASAS block I communications set, the Trojan Spirit system, the Mobile Subscriber Equipment, and others. Although Warrior and its successor, Warlord, already have many key features desired in ASAS, the Army did not adequately consider how Warlord might be used Army-wide. According to Army officials, Warlord is an initial operational prototype with proven software that is robust enough for issuance to operational units. Warrior, the predecessor to Warlord, has already been issued to operational units and has performance characteristics desired by current users. Troops in both Germany and the United States accepted the former Warrior performance as adequate to meet current operational joint and Army missions. For example, troops used Warrior to (1) conduct all-source intelligence data analysis; (2) provide European intelligence data to the Atlantic Command; (3) exchange intelligence data in a seamless architecture from echelons above corps to corps, to division, and to brigade; (4) provide capability to deployed contingency forces; and (5) provide redundancy to prevent catastrophic loss of capability. Assessment by the Commanding General of the 1st Cavalry and III Corps stress the need to include Warrior as a part of ASAS. According to the Commanding General of the 1st Cavalry Division, Warrior was of great value to targeting and situation development, and corrected a limitation of the current ASAS block I configuration by expanding intelligence processing capabilities throughout the Division. A May 1993 III Corps assessment said the block I collateral enclave and the communications van are working well, but the JPL-developed workstation in block I remains a weak link. The Corps assessment concluded that the Corps could (1) abandon ASAS, (2) continue to use the JPL workstation under specific conditions, or (3) replace the JPL workstation with Warrior workstations. A July 1993 III Corps assessment said the Warrior provided more accurate and quicker situation and target development than any system previously used. A Corps official said the Corps considers Warrior an essential element of ASAS block I. Warlord also provides capabilities that the ASAS program manager is trying to include in ASAS block I through major configuration upgrades and/or develop in block II. For example, Army officials stated that Warlord (1) supports split-based, jump, and networked operations and (2) provides communications and data links with national, joint, coalition, and Army battlefield command and control systems. Other Warlord capabilities the Army is seeking in the follow-on block II program include (1) receipt, processing, and display of framed images and live video; (2) electronic connectivity to national intelligence data bases; and (3) open computer architecture. In addition, Warlord does not require downsizing to meet transportability requirements, whereas the Army is modifying the ASAS block I configuration in an effort to downsize the system. Another major benefit of the Warlord alternative over the ASAS block I is the potential to procure enough sets to field throughout the Army and provide redundancy in each unit at a reasonable cost. Lack of enough block I equipment to deploy Army-wide and lack of redundancy in each Army unit with block I equipment are major problems to operating troops, according to Forces Command officials. These officials told us the 11 existing block I sets to be fielded will go to first-priority Army units only; however, they noted that lower-priority Army units went to Somalia and to Operation Desert Storm. Based on data provided by the Army, we estimate that enough Warlord workstations could be bought to equip the entire Army for about $21.6 million. This does not include the cost of communications and supplemental equipment and training. Additional Warlord units allows Army-wide fulfillment of another key ASAS block II requirement to provide intelligence processing capabilities at all Army organizational levels, from echelon above corps, to corps, to divisions, and to brigades. Warlord units are needed at the echelons above corps and brigades levels because the ASAS program manager has only enough block I units to field to corps and divisions. U.S. Army officials in Germany said block I does not meet its requirements because of this limitation. USAREUR has bought enough Warrior units to provide this capability. Warrior and its successor Warlord also have the potential for lower operations and maintenance costs than those for the current ASAS block I, and, at the same time, to meet block II requirements for direct computer-to-computer connectivity and to implement new Army intelligence doctrine. Preliminary Army cost studies--the ASAS Independent Cost Estimate, dated January 1993, and the ASAS Baseline Cost Estimate, dated February 1993--show that each ASAS block I set should cost about twice as much to operate and maintain, as compared to block II and follow-on systems. According to DOD, block I costs are estimated at $2.1 million annually per set. Warrior is similar in design to the ASAS block II workstation. New Army doctrine combines two separate block I intelligence units at both corps and divisions--one unit has original JPL block I equipment and the other has Hawkeye--to provide a single integrated intelligence unit. An all-Warlord system provides operational flexibility from one set of equipment, and meets the block II requirement for direct computer-to-computer connectivity so all analysts can see the same picture. The ASAS program manager is developing a computer-to-computer capability for the current block I configuration by using Warrior as a bridge between the existing JPL equipment and the Hawkeye. Prior to award of the block II development contract, the Army did not adequately consider the Warlord (1) as an alternative interim capability to block I, (2) for inclusion as a part of block I to correct operational deficiencies, or (3) as a capability to provide Army units that will not receive block I. The ASAS program manager objected to Warlord because it does not have all the automatic features of the JPL equipment in block I. However, current users operating the Warrior have found that it performs adequately without the automatic features in the JPL equipment. The Army called the Warlord predecessor--Warrior--a prototype that is not yet fully documented, supported, or tested. However, the Warrior has been widely deployed and used in missions by USAREUR and other military activities, including the Commander-in-Chief, Atlantic Command, and contingency forces in Somalia. As of April 1993, 195 Warrior workstations were in the inventory. Of these, 93 workstations were deployed to Army units in Europe. Another 20 units were deployed in other joint and emergency support operational locations. In June 1993, an official of the Army Intelligence Command also told us that they plan to add Warlord software to another several hundred existing Army computers to provide automated intelligence support to all DOD theater commanders-in-chief. In addition, as described below, the Army now plans to use Warlord as an integral part of ASAS block I. Thus, documentation, logistical support, and training needed for a fielded system will be provided for Warlord users whether or not it replaces block I. According to Army officials, in October 1993, the General Officer Review Board supervising Warlord development considered the problem of providing intelligence processing capabilities to non-ASAS units by integrating Warlord and other capabilities with ASAS. During discussions of our findings with DOD and Army officials in October 1993, these officials said the Army now plans to use Warlord as an integral part of the 11 sets of block I to be fielded and to provide Warrior/Warlord capabilities to units not receiving block I. BLOCK II CONTRACT AWARD IS PREMATURE ------------------------------------------------------------ Letter :5 The $112-million ASAS block II contract awarded in October 1993 is premature. The Army has not established and tested a core of minimum acceptable ASAS capabilities that can be used to determine cost-effectiveness of additional increments of ASAS performance development. In addition, the analysis of block II alternatives performed in 1992 was limited, and upgrades to block I, Warrior, and other systems since then could render the 1992 analysis obsolete. CORE CAPABILITY FOR BLOCK II NOT ESTABLISHED ---------------------------------------------------------- Letter :5.1 Defense guidance states that evolutionary acquisition programs achieve cost control, schedule maintenance, and user satisfaction by establishing and testing a core of minimum acceptable capabilities and then adding increments of capability to that core. Each increment or block of new operational capability is treated as a separate acquisition, with its own requirements, development contract, testing, and funding. One purpose of the operational testing conducted on block I in September 1992 was to establish baseline performance thresholds for block II. Block I did not pass its test and thus did not establish a core baseline of performance thresholds for block II. According to the April 1993 DOD Inspector General report on ASAS, testing conducted and planned was inadequate to support award of the block II development contract. Block I also has not been operationally retested to verify that problems identified in the 1992 test have been corrected. In addition, the Army did not establish a core set of requirements, because the final configuration of block I was not determined before the block II contract award. Thus, block II development is premature until the revised block I configuration is agreed upon and tested. Another block I field test is scheduled for late 1994; if properly designed, the test would provide an opportunity to establish a proven baseline on which to begin block II development. COST-EFFECTIVENESS OF BLOCK II NOT DETERMINED YET ---------------------------------------------------------- Letter :5.2 The Army also does not have a current and accurate assessment of the value added by block II compared with its development and procurement cost of $764 million. This is especially critical in view of the various upgrades planned for ASAS block I, the continued development of Warlord capabilities, and the trend of tighter dollars for defense. The Army cannot rely on the limited Cost and Operational Effectiveness Analysis (COEA) performed in 1992 as an adequate assessment of potential alternatives to block II. DOD did not require a complete evaluation of ASAS requirements and alternatives as required by its own instructions. DOD Instruction 5000.1 requires the services to examine use or modification of existing systems prior to start of a new system development.\2 An Army official said the ASAS program had obtained a waiver of the COEA requirement at the beginning of the program before it came under the Defense Acquisition Board. Subsequently, the program came under the Defense Acquisition Board process, but DOD still permitted the Army to conduct a limited COEA instead of requiring a full evaluation of alternatives. The COEA was limited because the analysis did not consider use of several existing systems or ongoing and planned modification of any systems. The Army COEA initially considered only the U.S. Marine Corps Intelligence Analysis System (IAS) as an automated alternative system to block II. After we questioned the limited scope of the COEA, the Army included Warrior. The COEA team rejected the IAS and Warrior because (1) neither has an all-source correlated database, which the Army said is an essential requirement for ASAS and (2) neither had connectivity with other ATCCS systems. These reasons are no longer valid, because the Army is adding the ATCCS standard to Warlord, thereby providing connectivity with command and control systems. Also, the Army COEA did not include the option of adding the all-source correlated database software function to the Warlord. An Army official stated the addition of the all-source database can be done. DOD comments on our draft report concur since they state on page 30 of this report that the best of ASAS block I will be integrated into Warlord along with the best of Warrior, the ASAS Collateral Enclave, and ATCCS. Further, the COEA excluded other DOD intelligence data processing programs or individual capabilities in those programs that could be transferred to a single system such as the IAS or Warlord. For example, it excluded the Navy Tactical Command System Afloat, the Air Force Intelligence Correlation Module, planned continued expansion of Warrior--now Warlord--capabilities, planned upgrades to ASAS block I, DOD's upgraded Linked Operational Intelligence Centers Europe system, planned upgrades to the Army Forces Command's Automated Intelligence Support System (FAISS), and other Army intelligence open architecture computer workstations. Software from all these systems has not been adequately analyzed to determine what capabilities are available by merging the best approaches from each system into one combined system. As previously mentioned, many capabilities were developed in Warlord and other service systems when ASAS was delayed, and new capabilities are under development in ASAS block I, Warrior/Warlord, and other DOD programs. For example, Warrior/Warlord is an all-source intelligence processor, and its capabilities are being improved each year. This effort will be continued as a rapid prototyping program concurrently with block II development. In addition, the Army is considering, developing, and inserting major upgrades to block I to resolve identified deficiencies. These include a new block of software, new communications processors, downsized workstations, elimination of some vehicles and generators, integration of the two separate elements of each set of block I equipment, conversion from a closed computer architecture to an open one, and incorporation of Warlord workstations in units receiving block I. Because of the changes made and being made to existing systems since the limited COEA was performed, the Army lacks a current and accurate assessment of capabilities in other Army and DOD systems. Therefore, the Army no longer has information to determine whether the $764-million cost of block II is justified. These block I upgrades and Warrior capabilities, combined with ongoing and planned upgrades to other systems such as the Army's FAISS and intelligence workstations decrease the value added by the ASAS block II program. Therefore, justification to begin the block II program has not been demonstrated. Army officials told us they will evaluate software in other Army and service programs to prevent duplication of effort by the block II contractor. However, this step does not constitute an independent evaluation of the cost- effectiveness of other systems or their potential to be upgraded to meet block II requirements. It also comes too late--after the contract is awarded. -------------------- \2 DOD Instruction 5000.1 provides guidance for implementing acquisition programs. RECOMMENDATIONS ------------------------------------------------------------ Letter :6 We recommend that the Secretary of Defense direct the Army to: incorporate and test upgrades necessary to correct operational and suitability problems identified during testing and user evaluations prior to block I fielding; maximize Warlord's potential to increase block I capabilities and to provide capabilities for units not receiving block I; use planned operational tests of block I, including Warlord capabilities, to establish a performance baseline for evaluation of costs and benefits from continued development of ASAS; fully assess the costs and benefits of future development plans in block II; and restrict additional funding for block II development until (1) the proven ASAS baseline is established, (2) the Army minimizes block II software development by identifying and assessing capabilities of other systems, and (3) develops a plan to transfer the appropriate capabilities to ASAS. AGENCY COMMENTS AND OUR EVALUATION ------------------------------------------------------------ Letter :7 DOD did not agree with our recommendation in the draft report not to field block I in its current configuration. According to DOD, the system is sufficiently capable and reliable to support conditional fielding, that is, fielding to two units for further evaluation. DOD believes block I has demonstrated its use, and, based on the results of additional operational testing, will remove restrictions on fielding remaining units of block I. However, the block I configuration now planned for additional testing in 1994 does not include changes needed to address such problems as lack of system redundancy; limited mobility; and absence of an open systems architecture, which is needed to easily incorporate new capabilities from block II development. The Army is currently considering block I upgrades to address these problems. We modified our recommendation to include reference to block I upgrades now being evaluated. DOD partially concurred with our recommendation in the draft report to require the Army to evaluate Warlord's potential to replace block I or to provide capabilities for units not receiving block I. DOD said that Warlord software will be integrated into the ASAS program and that Warlord software is being considered for use in units not receiving block I. DOD did not agree to evaluate Warlord's potential to replace block I. We modified the recommendation to stress the need to capitalize on Warlord capabilities to enhance and augment the limited quantities and capabilities of block I, in light of the upgrades being considered that should make block I a more operationally suitable system. DOD partially concurred with our draft report recommendation that DOD direct further operational testing of Warlord and planned upgrades to ASAS for use as an interim system and to establish a proven baseline for the continued development of ASAS. DOD said it will test block I again in 1994 in support of the final fielding decision. However, DOD said the Army has not decided to make additional hardware or software changes to block I. Instead, according to DOD, block I is the functional baseline for the ASAS program, a baseline that includes Warlord and other prototyping efforts. DOD did not agree to test this baseline as an interim system or to establish a proven baseline prior to the continued development of ASAS. Therefore, DOD does not have a proven performance baseline for evaluation of costs and benefits from continued development of ASAS. Thus, we revised our recommendation to stress that upcoming operational tests should be used to establish such a baseline. DOD did not concur with our recommendation in the draft report that DOD not award the block II contract until a proven baseline is established and the Army has (1) fully assessed potential contributions to ASAS from other programs and (2) determined that a block II program is still necessary at the planned level of development. DOD said the Department is ensuring that intelligence fusion software capabilities in other systems are considered for reuse in ASAS block II, and that sufficient analytical effort has been expended to justify block II as the approach to satisfy the requirement. DOD permitted the Army to award the block II contract without establishing a proven baseline and before the evaluation of other existing software was completed. The contract was therefore premature and could result in unnecessary costs. We revised our recommendation to reflect the award of the contract by recommending DOD restrict additional funding for block II until the baseline is established and the Army has developed a plan to minimize new software development by using existing software where possible. SCOPE AND METHODOLOGY ------------------------------------------------------------ Letter :8 During our review, we identified and analyzed the missions and functions ASAS was designed to perform, as well as existing intelligence processing systems that had similar missions and functions, to determine if the existing systems could provide interim ASAS-like capabilities. We interviewed program office officials and examined agency documents that described the acquisition strategy used in the ASAS program and the functions of each component. We also met with military and civilian officials from the U.S. European Command; the Forces Command; USAREUR; the Army headquarters in Washington, D.C.; and other DOD agencies. We also observed demonstrations and exercises on the ASAS block I and Warlord in the previous Warrior configuration. Our review was conducted from September 1992 through November 1993 in accordance with generally accepted government auditing standards. ---------------------------------------------------------- Letter :8.1 The head of a federal agency is required by 31 U.S.C. 720 to submit a written statement on actions taken on our recommendations to the Senate Committee on Governmental Affairs and the House Committee on Government Operations not later than 60 days after the date of the report and to the House and Senate Committees on Appropriations with the agency's first request for appropriations made more than 60 days after the date of the report. Copies of this report are being sent to the Chairmen of the House and Senate Committees on Armed Services and on Appropriations and the Secretary of the Army. We will also send copies to other interested parties upon request. Please contact me on (202) 512-4841 if you and your staff have any questions concerning this report. The major contributors to this report were Howard Manning, Assistant Director, and Robert Hadley, Evaluator-in-Charge. Sincerely yours, Louis J. Rodrigues Director, Systems Development and Production Issues (See figure in printed edition.)Appendix I COMMENTS FROM THE DEPARTMENT OF DEFENSE ============================================================== Letter (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) The following are GAO's comments on the Department of Defense's (DOD) letter dated November 12, 1993. GAO COMMENTS ------------------------------------------------------------ Letter :9 1. The All-Source Analysis System (ASAS) Operational Requirements Document, dated January 14, 1993, describes block I as an interim ASAS system that provides limited functional capabilities and a baseline to further develop ASAS to its objective requirement. In addition, a 1993 DOD-Inspector General report on ASAS describes block II as a new program start, not a continuation of block I. 2. The report was changed to reflect the Army's current cost position. 3. The report on the March 1 through April 2, 1993, technical testing supported conditional material release with a "get well" plan requirement for full release. The report concluded that the overall system had improved to the point of being considered technically sufficient for the areas tested, subject to several qualifications. For example, manual processing steps were considered necessary in several key areas to avoid incorrect results from automatic processing. In addition, the report noted few quantitative criteria existed and said several extremely complex subtests, such as nodal analysis and other automated processes, were hindered by poor or nonexistent documentation. Army test officials said the April 1993 operational demonstration DOD refers to was unrepresentative of operational conditions. We observed that demonstration and found that (1) the message load used in the demonstration was reduced below that used in the operational test and (2) personnel used in the demonstration were unrepresentative of normal users, in that they were instructors in military intelligence disciplines and soldiers selected to train other soldiers in ASAS operations. 4. We changed the report to reflect that block I was designed to pass data between the two components. The ASAS program manager has designed a bridge between the two components, using Warrior, to permit direct data exchange between the two components. 5. The report has been expanded on page 5 to disclose the full details of this message. We believe the added material places the statement about the system being "on track" in proper context with the continuing concerns about ability to maintain adequate training levels and performance problems that require immediate attention. 6. An Army official told us the Army has not requested an official study of the changed block I configuration since the operational test. A study is needed to determine whether block I hardware with changes under consideration will meet current mobility requirements. For example, the collateral enclave has been added to the block I configuration, which adds some lift requirement. Additional lift requirements could also result from other new equipment considered necessary to improve performance and provide operational redundancy. 7. The report was changed to portray the U.S. Army Europe's (USAREUR) statement as its opinion of required operational characteristics. According to the ASAS Integrated Logistics Support Plan dated April 1993, 5-ton trucks are provided to carry the workstations and support items and equipment. In addition, a May 1993 III Corps position paper stated a 5- ton cargo truck and three 5-ton vans are needed to transport block I equipment. 8. The Army Science Board criticized both the requirements process and the lack of progress in ASAS capabilities. 9. According to the Linked Operational Intelligence Centers Europe system, draft concept of operations it is to be deployed in extreme weather conditions from Northern Norway to Africa. According to U.S. European Command data, the system has been used in real world exercises such as Reforger and Teamwork 92 and has been deployed to Northern Norway. Furthermore, as a result of its performance, the U.S. European Command reports an unanticipated surge in requests for the system's workstations to support NATO and U.S. European Command rapid deployment requirements. 10. Our characterization of tests and demonstrations to date is based on reports of the DOD-Inspector General, the Army operational test agency, our observations of demonstrations, and discussions with soldiers participating in these tests and demonstrations. For example, soldiers said the Jet Propulsion Lab (JPL) software used in the test was extremely poor and was not ready for operational testing. The soldiers said problems in this software were not fixed until face to face meetings were held between the soldiers and JPL and program manager-ASAS personnel after the operational test. We are pleased by the DOD commitment to operationally test whatever changes are subsequently implemented in block I. 11. We modified our report to more clearly state that Warlord is a development effort to merge the best features of existing software from sources noted by DOD and that Warrior, the predecessor to Warlord, was the system deployed in USAREUR and other locations. DOD, however, has not tested the Warrior and does not plan to test Warlord; therefore, it does not have data to determine whether or not the Warrior or its successor, the Warlord, can replace block I. 12. Our draft report correctly cited statements of Army officials that the Warrior is an initial operational prototype with proven software that is robust enough for issuance to operational units. Support for that statement comes from Army documents and our discussions with users of the Warrior in exercises. Our draft report did not say the Warlord was selected to be the baseline for the ASAS block II. However, June 22, 1993, minutes of the General Officer Review Board managing the development of Warlord state that the Warlord configuration of September 1993 becomes the initial baseline for the block II contractor. 13. We changed the term "field" to read "deploy" because the Army has not conducted the formal procedures to "field" the Warrior but has deployed it to European and other Army units. 14. We changed the report to reflect the Army cost position for costs to operate and maintain block I. 15. DOD has no test data to support its position that existing developmental systems or workstations are not adequate to meet current or future requirements. 16. We believe the block II acquisition strategy has not met DOD guidance for evolutionary acquisitions, which requires ASAS to establish a core of minimum requirements and to test that core. No core has been established, as noted by the April 1993 DOD-Inspector General report on ASAS. This report concluded that the ASAS is not a true evolutionary acquisition program because the Army had not established a core set of requirements and did not plan to properly manage the individual blocks as separate acquisitions. The Army also did not establish tested performance parameters before entering block II development. 17. Our concern is that the guidance did not require consideration of upgrades to existing systems, such as block I upgrades and the Warlord, and an adequate evaluation of other relevant service systems. Also, we are concerned that the rapid growth in capabilities of the Warrior and other systems render the 1992 Cost and Operational Effectiveness Analysis (COEA) obsolete. The primary reasons for the rejection of Warrior and the Marine Intelligence Analysis System (IAS)--lack of an all-source correlated data base and connectivity to Army Tactical Command and Control System (ATCCS)--are no longer valid. Warlord will have ATCCS connectivity, and an Army official said the all-source correlated data base can be added to the Warlord. DOD comments also state that the best of block I will be incorporated into Warlord, along with Warrior, the ASAS Collateral Enclave, and ATCCS. This means that Warlord capabilities will be much expanded over what the Warrior had at the time the COEA was conducted in 1992. 18. Concerning the Mitre study, the statement that substantial enhancements in software capability are needed for Tactical Exploration of National Capabilities (TENCAP) to meet the Army's objective ASAS requirements ignores the fact that ASAS block I software also requires substantial enhancements to reach objective system requirements. Mitre also said TENCAP enhancements should be examined to determine whether the benefits to be accrued outweigh the costs to be incurred. However, the Army did not conduct a COEA to determine whether enhancing TENCAP capabilities or developing ASAS would be the most cost-effective alternative. 19. The Warrior is distributed to fighting units in USAREUR, Somalia, Kuwait, and the United States. In addition, the Program Executive Officer/Command and Control Systems is committed to adequately supporting the deployed Warrior. The Hawkeye is also deployed to fighting units in USAREUR. We found that ASAS consists of three functions: (1) a workstation to receive and process data; (2) communications to bring the data to the workstation and disseminate products; and (3) workstations and related equipment to interact with command and control systems. Warrior, and its follow-on Warlord are workstations that functionally replace the JPL workstation (and associated computers in truck mounted shelters) for receiving and processing data. For the communications function, the Warrior interacts with a number of communications systems. The Warlord also will interact with command and control systems. We modified our report to reflect that the Warrior and the Warlord are workstations that can interact with multiple communications systems. 20. The need and cost-effectiveness of the planned block II development was not properly assessed before the block II contract award because the COEA did not include options such as integrating and then upgrading hardware and software of existing and developmental systems. We are pleased that the Army agreed to initiate efforts to evaluate software in other programs to prevent duplication of effort during block II development. 21. Block II is a new program because the block I hardware does not transition to block II, the block II contractor is not required to accept any of the block I software, and block II is new software combining five different software languages in block I. The DOD-Inspector General report also said block II is a new program because it is composed of new hardware and software and will restart acquisition phase II. For this reason, DOD-Inspector General recommended that the Defense Acquisition Board hold a full milestone II review in lieu of a program review.
