Improving Pattern Recognition Methods for Space-Time Structures in Climate Data

Identifying Evolving Organized Space-Time Structures (Hurricanes and Droughts) in Climate Fields

PIs: Upmanu Lall, Jennifer Velez, Yochanan Kushnir, Edward R. Cook, Andrew Robertson, Suzanna Camargo, Richard Seager

Lamont Doherty Earth Observatory of Columbia University

Letter of Intent to submit a proposal to NOAA for $ 422,000 over 3 years,

Program Elements: CPPA and/or CVP

Hurricanes and droughts represent rather interesting end points in the climate system, since potentially they both require a Lagrangian rather than Eulerian perspective for modeling and understanding. Hurricanes are manifest as “tracks” with various physical attributes (e.g., wind speed), while droughts are often manifest as coherent spatial areas with moisture deficit gradients. Tracks are of interest for landfall implications as well as for understanding the dynamics of hurricanes. Contiguous drought areas are likely to be diagnostically related to persistent subsidence or large scale atmospheric flow, and their exact location and extent (e.g., covering parts of Kansas, Iowa and N. Oklahoma) has direct societal relevance. The drought area represents a suppression or redirection of the atmospheric moisture transport track. Often how a track or an area evolves through an event (see for instance, the Drought Monitor), or is similar to (classified with) past (or model) tracks or drought areas is of interest for climate diagnostics, prediction and model inter-comparison. In both cases the space and time history or evolution is of interest, and in both cases the definition of the “object” is conditional on some normalized threshold (e.g., wind speed or PDSI threshold). These attributes make the application of traditional statistical methods difficult for pattern identification, classification and regression (including autoregression) of the spatio-temporal aspects of these objects. Specifically, we may be interested in the following types of questions:

1. For a given threshold that defines a hurricane or a drought, can we identify the most similar tracks or droughts in the historical record or from a model based simulation?
2. How does one best cluster or classify hurricane tracks or contiguous drought areas considering location, spatial extent, spatial geometry, prior duration and relevant physical attributes? Such a cluster may be useful to define an ensemble of the track or area, and hence of its uncertainty distribution as well as the canonical drought or hurricane it represents.
3. How does one relate a hydroclimatic field (e.g., SST, vorticity, humidity at a certain level) to a track/area or its corresponding cluster, in a causal or correlative sense?
4. Predictability: How does a track or an area evolve given its current location, duration and extent and relevant climate fields that influence its evolution? For instance, can we say something about the likelihood that the drought covered area will expand/shrink, increase/decrease in severity through an analysis of the canonical areas historically recorded as a time sequence? Are the model projections of the atmospheric fields and the drought areas/hurricane tracks pointed in the “correct” direction?
5. Variability: How has the frequency of occurrence and recurrence of a particular type of track or canonical drought pattern changed over time, and how does it relate to the typical inter-annual decadal climate modes?

Whereas, techniques such as principal components analysis have been effective in answering similar questions for space-time continuous fields, there is a need to develop tools for singularities such as hurricane tracks and contiguous drought areas. Two tracks or two areas could be made similar by a) displacement of origin, b) stretching or contraction, c) distortion, and d) rotation. Clearly, as far as the geometry of objects is concerned, one could develop criteria for similarity using these measures. However, the fact that we have additional hydrometeorological information that describes each track or area could either add to the dimension of the classification problem or if used causally, lead to much more robust classification. Pursuing the latter goal is of interest to us. We intend to develop algorithms that in the beginning seek to address specific questions (similar to those enumerated above) through an analysis of historical and re-analysis data fields. The intention is to be guided by a practical context of drought/hurricane impact mitigation through risk assessment and visualization.

The proposed research derives from our current work on modeling Hurricane Tracks using statistical methods. A variety of nonparametric, stochastic methods have been developed and tested to simulate hurricane tracks, and to explore the dependence of Atlantic Hurricane genesis and landfall conditional on seasonal climate indices. Our group has already developed several models for track classification and clustering and stochastic simulation of likely tracks. This work has been applied to Pacific and Atlantic hurricanes and also to extratropical and tropical cyclones. The extension to consider physical attributes in addition to track geometry and to drought area evolution is being proposed here.

One of our objectives is to revisit how drought indices should be defined particularly given areal drought considerations, and the implications of the geometry of the drought area conditional on a causal mechanism (e.g., in an area dominated by meridional atmospheric moisture transport).

The approach to the problem will be developed using the following building blocks:

1. Mathematics for the definition of track/area geometry including the ancillary physical state.
2. A statistical approach to object clustering and pattern recognition consistent with the definition in 1. We expect to explore methods such as Self-Organizing Maps, Parallel Coordinate Analysis, Hierarchical mixture models, with parameter estimation done using Bayesian Methods.
3. A Bayesian statistical approach for the space-time evolution of the canonical objects identified using specially designed Markov models with climate fields as exogenous predictors.

The methods will be applied with existing hurricane data to explore the space-time structure and historical context of recent hurricanes (e.g., Katrina), and develop measures of predictability for the tracks empirically and also relative to operational forecast models. For droughts, we’ll pay special attention to the region covered by NAME, and develop an analysis of droughts and their causal structure using recently collected data and numerical model output, as well as the longer historical data. The project deliverables will include:

1. Software products corresponding to the techniques described above
2. Maps of canonical droughts and hurricane tracks, including cluster membership information, and corresponding composites of atmospheric and oceanic state at multiple time scales
3. Conditional probability tables (embedded in software) for the time transition or evolution of a track or drought area object into a future time step, conditional in its current state and the current or numerical model predicted ocean-atmosphere state.
4. White papers that lay out and contrast the empirical dynamical connections between drought/hurricane evolution and key hydroclimatic fields in observations and models.
5. A new set of metrics for defining areal drought as a space-time object with attributes.

We anticipate that the results of the research will be of direct use in communicating information as to the probability, severity, duration and location of drought – through a dynamic visualization of potential outcomes, and similarly for plausible hurricane tracks. They will also be of use to numerical modelers as tools for identifying relevant features from model simulations, and for assessing the uncertainty associated with hurricane or drought forecasts.