Note: This document is a work in progress. This note will be removed when all the charts produced by the analysis are documented.

1 What is Sample Path Analysis?

Sample path analysis is a deterministic analysis that can be run on any observed flow process history. It lets us reason about the dynamics and stability of flow processes over the long run, and in real-time.

We use sample path analysis to determine whether a process is stable, and if not what the causes of instability are, and what needs to be done to move the process towards operationally acceptable range of stability.

Please see our technical note: Sample path analysis is not a statistical technique.

1.1 The charts.

The results of the analysis are a collection of charts that are written to an output directory.

For input events.csv, output is organized as follows:

<output-dir>/
└── events/
    └── <scenario>/                 # e.g., latest
        ├── input/                  # a snapshot  of the input csv data
        ├── core/                   # core flow metrics & tables
        │   └── panels/             # single-panel charts
        ├── convergence/            # convergence vs divergence
        │   └── panels/             # single-panel charts
        ├── stability/              # boundedness and existence of limits
        │   └── panels/             # single-panel charts
        ├── advanced/               # optional deep-dive charts

This reference describes every chart produced by the samplepath CLI, grouped by chart type.

2 A Worked Example

The example charts in this reference and the underlying narrative are discussed in our post Little’s Law in a Complex Adaptive System

The example charts in each section below are drawn from the Polaris scenario completed-stories-outliers-removed.

Note: All calculations are done in continuous time and all charts report time accumulations in hours.

3 Sample Path Flow Metrics

Time series charts showing the core functions that govern the long run dynamics of a flow process.

To keep things grounded think of the csv file that you are providing as input to the analysis as the flow process under analysis.

The current version of the library only analyzes binary flow processes - where start and end dates represent arrivals/departures events, the effect we are measuring is the presence or absence of items in the process, and the sample path represents the counting process: the number of items present in the process at any moment in time (aka the WIP).

As we will note along the way, every concept in this document generalizes even for the much more general forms of flow processes once you plug in a different sample path. In particular, the set of outputs produced by sample path analysis are identical even after generalization!

This is why this set of charts is the foundation of flow process dynamics.

Conventions

• All charts are defined over a finite observation horizon of length \(T_{\max}\).

• Lower-case \(t\) is a parameter denoting instantaneous time.

  Functions of \(t\) (such as \(N(t)\)) denote sample paths: the values of some measurable property of a process at a specific moment \(t \in [0,\,T_{\max}]\).

• Upper-case \(T\) is a prefix parameter: a scalar value that determines a time interval \([0,\,T]\).

  Functions of \(T\) (such as \(L(T)\), \(\Lambda(T)\), and \(w(T)\) below) are functionals: functions that take a sample path as input and compute a scalar value from the behavior of that function over that interval \([0,T]\). Here \(T\) can range over any value in the observation window \((0,\,T_{\max}]\).

  These functionals are purely deterministic, pathwise calculations (definite integrals in our case) and should not be confused with statistical aggregates.

3.1 The Core Calculations for Little’s Law

The input to the analysis is a sample path \(N(t)\).

There are six core functionals:

• \(H(T)\): the cumulative presence mass under the sample path over \([0,T]\).
• \(L(T)\): the time average of \(N(t)\), defined as \(L(T)=\frac{H(T)}{T}\).
• \(\Lambda(T)\): the cumulative arrival rate, defined as \(\Lambda(T)=\frac{A(T)}{T}\).
• \(w(T)\): the average residence time per arrival, defined as \(w(T)=\frac{H(T)}{A(T)}\).
• \(\Theta(T)\): the cumulative departure rate, defined as \(\Theta(T)=\frac{D(T)}{T}\).
• \(w'(T)\): the average residence time per departure, defined as \(w'(T)=\frac{H(T)}{D(T)}\).

We may write L(T) as

\[ L(T) = \frac{H(T)}{T} = \frac{A(T)}{T}.\frac{H(T)}{A(T)}. \]

Which we may write as,

\[ L(T) = \Lambda(T).w(T). \]

This is a finite version of Little’s Law, which we call the Presence Invariant for arrivals.

It is a deterministic accounting identity that expresses the relationship between the time average of cumulative presence mass and same cumulative presence mass averaged across arrivals when viewed over a consistent observation winodw.

Interestingly we can repeat these calculations with departures instead of arrivals.

\[ L(T) = \frac{H(T)}{T} = \frac{D(T)}{T}.\frac{H(T)}{D(T)}. \]

which we may write as

\[ L(T) = \Theta(T).w'(T). \]

Thus we have the Presence Invariant for departures:

\[ \Lambda(T).w(T) = L(T) = \Theta(T).w'(T) \]

and this holds for all time points over all finite observation windows.

The Presence Invariant defined above are deterministic identities that hold simultaneously for every point in time in every finite observation window \([0,T]\) with \(0 < T < \infty\) and \(A(T) > 0\).

This implies that the three component functions in each invariant always evolve in a way that preserves the identity, even when the arrival and departure focused arrival/departure rates, and residence times are not equal.

These invariants are thus deterministic constraints that governs the global dynamics of any flow process, even those that are not operating in steady state. It is the foundation for adapting Little’s Law for use in processes that operate far from steady state equilibrium like most software processes are.

3.2 Summary chart

This chart can be found at the top level under <scenario>/

In this chart the main thing to pay attention to are the relationships between the changes in each of these component charts over time.

• At any point in time review whether \(L(T)\) the time average of \(N(t)\) is changing or flat.
• If it is changing then the finite version of Little’s Law tells us that this must be because at least one of \(\Lambda(T)\) or \(w(T)\) changed.

It means that these are the only possible explanations:

• \(\Lambda(T)\) changed: more or fewer things are arriving.
• \(w(T)\) changed: things are taking more or less time to finish.
• A combination of the two.

In the last case there are two possibilities.

Since \(L(T)\) is the product of \(\Lambda(T)\) and \(w(T)\):

• If they move in the same direction they lead to proportionally large changes in L(T)
• If they move in opposite directions, the changes cancel each other and \(L(T)\) tends to flatten.

The last situation happens often when there is an internal feedback loop at play in the process.

All these different types of dynamics can be at play at different points in the evolution of the process. This chart is the place where all those long run dynamics will show most directly and clearly.

It helps us answer “how did the process evolve to produce the flow metrics we are seeing at any given moment in time?” These causal relationships allows us to monitor changes in L(T) and then immediately investigate the cause of that change.

See our post The Causal Arrow in Little’s Law for more discussion on what this means.

3.3 Component metrics

Stacks (multi-panel composite charts) are written at the root of core/:

Individual panels are written under core/panels/:

Their detail descriptions follow.

The sample path \(N(t)\)

sample_path_N.png

Since we measure in continuous time, this chart is a step chart.

• The line will go up with each arrival and go down with each departure.
• The line will stay steady if departures and arrivals balance each other at a given instant.
• In between arrivals and departures, the line will stay flat.

This is a real time chart that reveals current congestion, bursts, and idle periods.

If you are familiar with Cumulative Flow Diagrams, \(N(t)\) represents the distance between the cumulative arrival line and the cumulative departure line in the diagram.

The cumulative presence mass

A key quantity in sample path analysis is the cumulative presence mass. This is calculated as the definite integral of the sample path curve over [0,T]

\[ H(T) = \int_0^T N(t) dt \]

Since the mass is a product of the number of items present over time, the units of H(T) are in item-time. In the language of the Presence Calculus, \(H(T)\) is cumulative presence function over the sample path \(N(t)\).

The functions that drive flow-process dynamics are the time and item averages of H(T): L(T) and w(T). These continuous functions of time are the key quantities in the finite version of Little’s Law.

\(L(T)\): Time average of \(N(t)\)

time_average_N_L.png

Time-average WIP:

L(T) is the time average of the cumulative presence mass. May also be viewed as the rate at which the area H(T) grows.

Its units are in items.

L(T) = (1/T) H(T)

Reveals whether long run average WIP diverges or converges. This is the key top level indicator of process stability. A flat line here indicates a stable process.

Please note once again that this is not a statistical average.

\(\Lambda(T)\): Cumulative arrival rate

cumulative_arrival_rate_Lambda.png

Λ(T) is the arrival rate of items that have arrived up to T (may include items that started before the window):

Λ(T) = A(T) / T

It’s units are items/time

If WIP was zero at the beginning of the observation window, then this is the same as the arrival rate, otherwise this over-counts the arrival rate at the start, but as we observe the process for longer periods, those initial end-effects get averaged out.

\(w(T)\): Average residence time per arrival

average_residence_time_w.png

Average time items are observed as spending in the observation window up to time T (clips the time that is spent outside the window and thus not observed).

w(T) =  H(T)/A(T)

Tracks how “observed time in system” evolves over the sample path. Its units are time.

Please see our posts What is Residence Time and How long does it take for an explanation of what this metric means.

Understanding the difference and relationship between residence time and familiar metrics like Lead Time, Cycle Time and Work Item Age is crucial for understanding why sample path analysis works and these posts explain this.

3.4 The finite version of Little’s Law

littles_law_invariant.png

This plot visualizes the finite version of Little’s Law at work.

It states that for all T, L(T)=Λ(T)·w(T).

We verify this by showing that when we plot L(T) vs Λ(T)·w(T) on a scatter plot, all the points will lie on the with y = x.

Notice how points cluster around certain values of L(T). These are significant operating modes for the process as it moves towards stable states.