The Irrelevance of Historical Analysis
“Time is made up of all the things that use to be and aren’t anymore, and the things that aren’t yet and may never be.” (Aristotle)
Historical analysis and reporting dominates today’s management practice. Historical analysis projects future performance from past patterns. Historical reportingfinds meaning inevents that, when reported, are past facts. Historical analysis and reporting on the surface seem very logical. The thinking goes: History is a big book filled with many real stories that provide lessons for the future usingexperience from the past. Past events have shaped what we are today. By knowing what has happened historically, an organizationbelieves they can understand:
- why things are the way they are.
- what past patterns portend to what is likely to happen in the future,
- what causes the next event or actionas a result of a pastevent,
- what we should avoid,
- how we cancreate better results by looking at what has happened.
The famous axiom “if you don’t understand history, then you are bound to repeat its mistakes” best explains why many people believe in the importance of historical analysis. Or, consider that the accounting profession has insisted that historical reporting is the most objective and auditable basis for reporting thus making it the centerpiece of its reporting model. Or, our college management courses teach forecasting techniques that measures the past to extrapolate to the future. Or, how organizations employ statistics to analyze historical data to understand probability distributions and isolate patterns. These are but a few examples of the reliance on historical data that is prevalent in today’s management practices. All the above seems logical but……
Looked at from another perspective, historical analysis is irrelevant. Historical analysis looksat our trailingfootprints to guide us to the future. We can do no more than study history to try and understand its significance—only the process can make history.Why? A process and its interaction with its environment determine the transformation from one state to the next (i.e. results).
Process knowledge describes the transformation process. Process analysis views the world as a collection of natural and manmade processes that are in a constant state of transitions from one state to the next guided by the transformation logic and bounded by the transformation cycle. Time is the key that unlocks the transformation process. Each process transformation state evolves through time.Yet transformation can only occur in the here and now. Each process is influenced by an ever changing environment that impacts the next transitional state. The most straightforward method to project the future is to understand the current state, the current environmental factors, the transformation process and factor in any relevant actions that are planned.
It is argued that it is through historical knowledge that we can understand the here and now and the future. History is useful to the extent that history can provide an improved understanding of the transformation process. History is not useful when historical data—numbers—are used in forward looking projections. Historical analysis is a poor surrogate for process knowledge.
Historical analysis, the focus of this article, incorporates past historical data into an analysis to project future results. Why do we want to look back when the here and now is the result of the vortex of the multitude of transformation process in perpetual cyclical execution? Past process results are a vapor trail of how the transformation process interacted with its environment in the past. Historical analysis searches for past patterns in the vapor trail to project the future. You can look at the sky to see the vapor trails of airplanes. You can mentally follow those vapor trails to project where the airplane is headed. At times you will be correct. But such projections can never be as insightful as direct process knowledge that shapes all outcomes.
The objective of process knowledge is to provide a relevant projection of the likelihood of the next stage of transformation and what changes should be initiated to increase the probability of achieving a better outcome. Every transitional state is, by definition, point in time specific. The starting conditions depend on how the transformation interacted with the environmental forces in previous transformation stages. We must accept the current state for what it is—there is absolutely no action that can be taken to change the past. If we want a different outcome, then we must make necessary change to control the environment or change the transformation process.
We have two options. One option is to use historical knowledge to issue an exhumation order, perform a post-mortem examination, hold an inquest, seek evidence of the experts, issue a verdict of the coroner’s jury, hold a trial, and condemn the guilty parties. In the meantime, time is passing and the process is continuing on producing similar results.
A second option is to employ process knowledge to guide a process to better results. Knowledge of today’s process state and environmental factors is used to project the probable next transitional results. If the probable outcome is not acceptable, then initiate corrective action to control the environment or change the transformation process.
Progress occurs when, as we transverse each stage of transformation, the results bring us closer to the desired outcome than it would otherwise. Progress can only occur when action is taken to control the environment or shape the transformation process to create better results. Progress is about shaping tomorrow’s results through proactive actions. Proper process cultivation will result in future performance being superior then if the actions were not undertaken.Processes explain change over time.
The process approach is summarized as:
Outcome(tomorrow) = Current Conditions(today) * Transformation(process knowledge)
(1)Theorem of process history—the current transitional state (Current Conditions(today))encapsulate all the relevant impact of past transition states. There is nothingfrom the past that has not already impacted the current transitional state.The past is unchangeable soeither a previous event impacted or did not impact the current transitional state. All past factors are enfolded to the current state of being.
(2)Theorem of a future process state—the next future transitional state (Outcome(tomorrow)) is a function of the transformation (Transformation(process knowledge)) process logic, the current transitional state and how it is impacted by unfolding environmental factors.
(3)Theorem of a process outcome—the degree that the future transitional state will approximate the desired process outcome is dependent on how well the environmental conditions are controlled and the transformation process approximates ideal conditions.
Note: there is no history; there is only today, process knowledge and tomorrow.
An important implication of the theorem of process history is that the impact of history is completely enfolded in the current transitional state. At any point in time, the current transitional state is dependent on past transitional states. Transition implies a changing from one state to another. While the process transformation (Transformation(process knowledge)) is independent of the initial conditions (although it can be adapted), the transformation outcome are highly dependent on the initial conditions.
It is critical to determine the relevant information regarding the current transitional state (Current Conditions(today)).I refer to this assessment as process scanning. Process scanning must be robust enough to cope with twodistinct circumstances. The first condition is where the current transitional state is evident. One needs only to look at a flower to determine whether it needs water. Or, a spare part can be tested to determinewhether it is within specifications.
A second condition exists when current conditions are hidden. Lea Patterson of Pilbara Group presented an excellent example of hidden transitional knowledge—playing black jack with a fixed number of cards. As you transition from one hand (state) to another the cards you draw come from a subset of cards that were not previously played. The ability to project future results depends on knowledge of the current state—what cards have and what cards have not been played. As each hand progresses, the dealer conceals the cards that were played in previous rounds. Card counting enables players to better comprehend the current transitional state by passing information from one transitional state to another. The available cards and the previously played cards are immutable facts at each transitional stage.
The challenge to the process scanning model is how to store hidden information. The answer is that the process scanning model must incorporate relevant information from each transitional state to be passed to the next transitional state.
The theorem of a future process state replaces the need to look at the troublesome shadow of historical results with the analysis of processes, and their compound interaction with environmental forces. Process knowledge provides an understanding of how a process transforms an input under a given set of conditions, including environmental factors, into an output.
Process knowledge is timeless. Process knowledge can be used to explain past results;Process knowledge can be used to explain today’s results; Process knowledge can be used to project future results within a range of probability. Process knowledge is the best source of understanding the future since it is the process transformation process that creates the future.
Historical analysis that incorporates historical data (what was) into a projection of what will be creates a bias to past conditions. There is absolutely no need to introduce a bias into a future projection because the impact of previous states, history, has already been completely embedded in current state (theorem of process history). Process knowledge is sufficient.
The theorem of a process outcome encapsulates why the adaptive management systems of the future must be based on the best possible foundation. Decisions matter! No organization in the world fulfills its full potential. There is an untold amount of value lost to society every year. The world would be a much better place if we could convert some of the value lost into value creation. Enter our decision making tools. If our results are less than preeminent, so must the decision making tools we embrace.
Decision making must be perpetual. Decision making must look to the future rather than the past. Decision making must consider the most current set of conditions. Organizations must be proactive rather than reactive. It is today’s weather and a 10 day projection of future weather conditions that is important rather than the past ten days. Why? It is meaningless that the past ten days were cold. Tomorrows it might be warm due to a new set of atmospheric conditions. No action can be taken on past information—the past is unchangeable. A person can influence their actions only for what is coming. For example, a high probability of projected thunderstorms would result in me turning off the sprinkler system to lower my utility costs.
The theorem of a process outcome provides hope. The only important consideration is a relevant projection of the likelihood of the next stage of transformation and what changes should be initiated to increase the probability of achieving a desired outcome. If we want a different outcome, then we must make necessary change to control the environment or change the transformation process.
Process Foundation
Processes describe the transformation cycle. An input is transformed into an output. The process outcome is bounded by its input, the transformation stages, the resources employed in the process, the environment in which a process occurs and its output. Let us start with the process output. Every process has an intended outcome that similar in nature regardless of where the process is located. Consider a rose bush. There are many varieties of rose bushes. Yet rose bushes have similar characteristics that differentiate it from other types of bushes. When you plant a rose bush, you will not get a hydrangea!
Process transformation is constantly adapting to a changing environment. Darwin’s theory of natural selection articulated this observation. As the process transformation evolves, so also must process knowledge constantly evolve. Also, process knowledge must be constantly updated each time it does not adequately explain transitional results.
Process knowledge should lead to process improvement. The root cause of performance problems must be isolated and the transformation process changed or new process controls adopted to improve performance reliability. Process improvement must be based on reasoned process knowledge.
For example, roses have been cultivated for many eons. Rose bushes were grown during Roman times.Throughout the ages, process knowledge of plant breeding practices hasled to improving the transformation process by adding a great deal of disease tolerance and winter hardiness to the ancient rose. Hybridization has created some wonderful colors and bush sizes.We also regularly water and fertilize our rose bushes to ensure healthy growth. We study the rose bush process and control the factors that lead to aesthetically better and healthier roses. Learning about processes is important because it leads touniversal knowledge.
The same bounds apply to manmade processes. A bank audit process will never produce an iPad. Every manmade process has its unique intended outcome, technology constraints, employee experience differences and input requirements. And, just like natural processes, are universal in nature. Auditing a bank is similar in China, Australia, Europe and the United States. As with rose bushes, we apply technology and best practices to improve the bank audit process.
Every process is subject to natural forces (See process laws paper). A rose bush is subject to the natural weather forces—temperature, water amount, diseases and sunlight. Manmade processes are subject to varying input features and quality, resource stability and external environmental conditions. Different conditions result in differential results. The closer the actual process outcomes will approach the intended outcome as process conditions approximate ideal conditions.
Without an understanding of processes, change appears chaotic, unpredictable, and often out of management’s control. Managers see a current crisis as a situationspecific event rather than as part of a never ending need to adapt. Adaptation is a process, not an event. Process management identifies the major forces of change and highlights the need to continuously adapt to changing forces.
Those organizations that accept the importance of process knowledge will create adaptive management systems. Process knowledge will lead to a better understanding of the forces that shape the future and what changes are required to our processes and how to control the environmental forces that shape the future. Better resource allocation will lead to greater value creation.
The Fallacy of Historical Data Analysis
There are several critical nuances that can blur the distinction between process knowledge and historical data analysis.The first nuance is timing. The current state is very fleeting. No sooner does it appear than it becomes old (historical). But there is a very important distinction between focusing analysis on the current state or unashamedly basing it on past events of yesterday, last week, last month or last year. Current state analysis is termed real time analysis. The goal of real time analysis is to juxtaposedecision making with events as they occur, without any jumps in time. Real time analysis shortens the time between the current manifestation of an event and its influence on decision making.
Real time analysis is only theoretically possible if the time lag between manifestation and action is zero. This goal is impractical in practice. However, the intention of real time systems is very different from historical systems. The fundamental difference is the nexus of the analysis period.
A second nuance is whether using history to create process knowledge makes history relevant. Keep in mind the thesis of this article is not whether history is relevant but whether historical analysis is relevant. The importance is not how process knowledge is developed but, once it is developed, whether it is superior to historical analysis in projecting the future.
Paul Juras, a professor at Wake Forest University, presented an example of plotting a hurricane’s path from historical data. We need only to turn on a TV during hurricane season to see this common practice. While plotting a hurricane’s path might make good TV, it should not be the backbone of a meteorologist’s methodology. Natural forces created the hurricane and natural forces dictate its path. The best predictive power of a meteorologist’s projections will come from an understanding of why the hurricane is where it is and what the forces are in play that will shape where it is going. Where it was is unimportant. Where it will be is critical so preparations to minimize its effects can be undertaken. Where it will be is best understood by process knowledge based on an understanding of nature’s laws.