Sensor Exploitation Challenges 1. As Jim Carlini described earlier, one of the major challenges facing the Special Projects Office is how to attack the most difficult surface targets. In this presentation, I’ll describe the role that sensor exploitation plays in target engagement, and I’ll highlight some of the challenges that exist. I ask you to consider how sensors, and exploitation of sensor data, can be used to find and kill mobile targets. 2. Our goal is to enable a robust capability to engage all surface targets. To do so, we collect sensor data for a variety of purposes, including surveillance, target identification, and geolocation of military targets. The Defense Department and DARPA continue to invest in improved sensors and collection platforms to better achieve these goals. At the same time, we must continue to improve our methods to interpret that data. The better our exploitation algorithms, the more we can do with a given suite of sensors. While precision attack on fixed targets such as bunkers and bridges is solved well with modern targeting methods, the engagement of tactical targets, particularly mobile targets, is difficult to achieve in an atmosphere that demands zero-collateral damage. As a result, the Special Projects Office is interested in developing novel means to more robustly target mobile tactical targets, even when they are moving, camouflaged, or employing other means to conceal themselves. 3. This chart depicts the sequence of steps involved in the kill chain for mobile target attack. The exploitation technologies that I’ll be talking about are concentrated at the front-end of the kill-chain. How do we find potential targets in the first place? How do we discriminate military targets from other equipment? How do we track them as they move about on the battlefield? How can we get very precise geographic coordinates on targets that move? If we can answer all of these questions, we can hold all potential targets at risk until we are ready and able to engage them. Consider also the value of automation on the back-end of the kill chain. Exploitation of sensor data to assist with real-time battle damage assessment is another area that we are potentially interested in investing in. Automated BDA will enable restrike if the desired effect was not achieved, or diversion of an inbound weapon to a secondary target if it has been achieved. 4. In my remaining time, I will talk about surveillance, identification, and geolocation. First surveillance: Before critical targets can be targeted, they must be found. DARPA is pursuing several programs toward this end, but further advances are necessary to allow continuous surveillance of mobile targets, despite all countermeasures they may employ. The SAIP Program employs a host of image exploitation techniques to automatically detect and locate military vehicles using Synthetic Aperture Radar imagery -- automation that is essential to enable the warfighter to realize the value of the much higher data rates associated with emerging radars being deployed on such platforms as the Global Hawk Unmanned Aerial Vehicle, the U-2 reconnaissance aircraft, and potentially the Discoverer II satellite system. A SAIP Joint Program Office has been established at Fort Belvoir to oversee the transition of this system to the services. Targets that hide in the forest can be difficult to see with microwave radar, so DARPA has been pursuing programs to develop both foliage-penetrating radar technology as well as hyperspectral sensing techniques to see into the forest. The MSET program is exploiting these sensing modalities synergistically to develop a systematic approach to defeat camouflage and concealment. While these programs are developing useful capabilities, what we really need is a comprehensive system that will continuously track all critical mobile targets, so we don’t have to try to find them when they may be hiding. 5. Now, I’d like to spend a few minutes describing the state-of-the-art in Automatic Target Recognition, and suggest some new directions for research. The ATR problem is to locate and identify targets from imagery such as the synthetic aperture radar image shown here. In situations such as this, where targets are in the open, and we have prior imagery to train with, any ATR worth its salt can find and classify these targets. Where the action is today in the field of ATR, is in attempting to recognize targets in realistic conditions involving partial obscuration, articulation of turrets and gun tubes, intraclass variations, radar absorbing materials, and so forth. While in principle, one could approach this problem by collecting image chips on vehicles under all manner of variations, the combinatorics of doing so are prohibitive. In contrast, the DARPA MSTAR program, managed by Bob Hummel, has been using very detailed 3D models of target vehicles to predict image appearances under all these conditions and to reason in 3 dimensions about hypothesized situations. This approach is paying off exceedingly well, as evidenced by recent formal evaluations…. 6. Let me illustrate with a single, but representative example, involving recognition of an M109 self-propelled howitzer. A classical template-based approach to ATR trained on images of the M109 from various aspect angles but always with the gun-tube aimed straight ahead, will show a gradual degradation in performance, as the gun-tube is rotated from the head-on position. That is shown here by the red curve. The MSTAR system was trained with the same set of imagery, but also makes use of a 3D model of the M109, that enables it to predict what the M109 would look like under varying degrees of turret articulation. Combined with a search strategy that avoids the need to do brute force search through all possible articulations, the MSTAR system performs at the level shown in blue, where performance degrades slightly from the head-on position, but then stabilizes above the 70% recognition level for all other angles of rotation. This is a single example, but illustrates the performance advantage of the model-based approach that has been measured under various other conditions including revetments, intraclass variations, configuration changes, blocking obscuration, as well as depression and squint angles deviations. 7. While MSTAR has shown substantial promise in the lab, much remains to be done to realize a target identification capability for non-cooperative targets under all operating conditions. We have not yet realized the full potential of the model-based approach, and are looking forward to tackling even more difficult challenges associated with combinations of these extended operating conditions, and to addressing the issues of scaling up to recognize hundreds if not thousands of target types. MSTAR is beginning to approach a level of performance that approaches what would be needed for a viable battlefield awareness capability. But to achieve positive ID of targets to be attacked, we must have a nearly perfect recognition rate. How can we do that? What combination of sensors and sensor exploitation techniques is needed? Rather than continuing to invest in incremental improvement in ATR, we are interested in considering all degrees of freedom in an attempt to establish a 99% solution. And to do it under all realistic operating conditions! 8. As I mentioned at the outset, DARPA is pushing to develop an effective capability to engage mobile targets with precision weapons. Two programs have been initiated recently: The AMSTE program seeks to employ multiple airborne radars to track a mobile target and guide an in-bound weapon to the target. Bruce Johnson will be describing AMSTE immediately after I leave the stage. The Airborne Video Surveillance program is attempting to perform a similar feat using optical or infrared TV cameras on board aircraft such as the Predator Unmanned Aerial Vehicle. Video is limited by weather, but when employed in clear-weather situations or low-altitude flight, AVS will be able to automatically detect and track multiple vehicles even if they stop-and-go frequently. AVS will provide precision geolocation by registering video to carefully controlled reference imagery. While it is easy to show capabilities that work for some of the targets, some of the time, DARPA is seeking to develop a composite capability that can be relied upon to work all the time. Wouldn’t it be great if we could do this today in Kosovo? 9. Those of you who have followed DARPA for a while, will recognize a shift in motivation of our sensor exploitation technologies. What had been motivated by the need for comprehensive battlefield awareness is now being motivated by the needs of precision attack. Toward this end, I have described some of DARPA’s activities in surveillance, identification, and targeting -- the front-end of the kill-chain. -- We need to find and track targets before we can engage them. -- We need to have positive ID on a target before we can attack it. -- And we need to geolocate mobile targets very precisely under all circumstances Thanks for your attention -- I look forward to meeting with you individually, to discuss how exploitation of sensor data, can be used to find and kill mobile targets.