Dr. Ke-Thia Yao

Computational Sciences Division

Information Sciences Institute

University of Southern California

4676 Admiralty Way, Suite 1001

Marina del Rey, California 90292

Subject: Letter of Appreciation for computer science data management innovations in support of U.S. Joint Forces Command experiment Urban Resolve

Dr. Yao,

This letter is to express appreciation for your innovations that were implemented in order to support the large-scale battlespace simulations undertaken by the Joint Experimentation Directorate (J9) of the United States Joint Forces Command (JFCOM) during the period of August 2002 until December 2010. This series of simulation experiments, including Urban Resolve had two main objectives: first, to enable the Department of Defense, (DoD) to develop new concepts for fighting in and stabilizing urban environments and second to let the department test new capabilities for winning conflicts in those situations. In the simulation community, these two goals are frequently referred to as analysis and evaluation. Both of these require the efficient capture, archiving and retrieval of the information generated by the experiment as well as the enhancing of the efficacy with which this data was analyzed.

The challenge you met was that of developing new and effective methods of managing heretofore unheard of amounts of data from battlespace simulation that could be comprised of up to ten million independent intelligent agent entities. Your initial performance goal was to be able to support simulation systems generating rates of 100 Gigabytes of data per hour and 8 Terabytes for a two-week event, with data distributed across two supercomputing clusters and across multiple trans-continentally distributed sites. .The original data management system had been designed to manage data from a few workstations organizedon local area networks. The new system had to effectively manage thousands of nodes of large High Performance Computing clusters distributed from Virginia to Maui.

To accomplish this, you:

  • Designed and implemented a scalable data logging system that captured all the simulation data generated during the Urban Resolves exercises. The previous loggers had not able to adequately archive the large volume of dataor to efficiently transmitthe data from battlespace simulations at an acceptable velocity.
  • Enabled a data logging system that operated concurrently with the distributed simulation, all without any perceptibledecrement in simulation performance. Your data logging system was implemented within the boundary of the tight timetable of the joint exercises and without disrupting operations.

Provided post-exercise analysis capabilities enabling the analysts to rapidly and easily query the logged data, even during the experiment.

Further you developed a new system for future use with similar issues:

  • A real-time multi-dimensional analysis capability, e.g. visualized data utilities that provided summary overviews for the analysts.
  • Designed and implemented software that is extremely scalable so as to be able to effectively manage very complex graphing.
  • Designed and implemented distributed software that could work across the entire computing environment consisting of single-node systems, multi-node local systems, and networks of large clusters.
  • Delivered your software implementationson schedule and within the context of large collaborative effort, gaining both the admiration and respect of those with whom you worked.

Without your contributions, the experiments at J9 would not have been nearly so successful.The experiments identified planning and readiness shortfalls and the risk of not modernizing the force. Further, they redefined the way the military operates in urban environments. Urban Resolve also changed the way DOD develops concepts, procures technology and conducts training.The superb manner in which you performed is in keeping with the highest traditions of the Joint Forces Command and I extend to you a personal acknowledgement for a job well done.