6.1 Building Markov Models

While most decision trees include a simple notion of time (i.e., left to right chronologically), there are no shortcuts in a standard tree structure for representing events that recur over time. A state transition model, also called a Markov model, is designed to do just this. Markov models are used to simulate both short-term processes (e.g., development of a tumor) and long-term processes (e.g., an individual’s lifespan).

In a State Transition Diagram (see below), where each state is represented using an oval, arrows represent transitions. In TreeAge Pro you can create State Transitions diagrams but they cannot be used for any analysis. You can convert the State Transition Diagram (STD) to a Markov Model and add model inputs before analyzing the model.

TreeAge Pro represents Markov models in a decision tree format, which is more easily integrated into decision trees. Markov models can be appended to paths in a TreeAge Pro decision tree anywhere you might place a terminal node. The Health Care tutorial example tree Markov Cancer Final.trex is presented below. We will build this Cost Effectiveness Markov model in the subsequent sections.

The design of a basic Markov model requires consideration of several components. These are:

  • Health States: The set of distinct health states which describe the current health status of a patient. The cohort is split amongst the health states at the beginning of each cycle.

  • Events/Transition Subtrees: The model structure to the right of each health state that define the events a patient could experience after starting a cycle in that health state. Dead (absorbing) states will have no transition subtree.

  • Cycle length: The length of time represented by a single cycle (or stage) in a Markov model. This value is implicit in all probabilities, all rewards, and the termination condition. That means using the appropriate cycle-length adjusted values in your model dependent on your chosen cycle length.

  • Initial probabilities: A set of probabilities describing the initial distribution of the cohort among the states. These are used only before the first cycle. For all other cycles, the proportion of the cohort in each state will be dependent on the transition subtrees.

  • Event/transition probabilities: The probabilities for all events in the transition subtrees including, but not limited to, transitions between states.

  • Values/Rewards: Per-cycle costs and/or utilities for each outcome measure being calculated (e.g., costs, QALYs). Rewards may be associated with health states or with events in the transition subtrees.

  • Termination condition: A logical test evaluated at the beginning of each new cycle to determine if the process should continue or stop.

Further details: