While most decision trees include a simple notion
of time (i.e., events on the left side of the tree occur
after those on the right), there are no shortcuts in
a standard tree structure for representing events
that recur over time. A state transition model, also
called a Markov model, is designed to do just this.
Markov models are used to simulate both short term
processes (e.g., development of a tumor) and
long-term processes (e.g., an individual’s lifespan).
State Transition Models - Discrete Event Simulation Models
Markov models built in TreeAge Pro often represent
discrete-time state transition models (although
discrete event modeling is also possible). A discrete-
time Markov model usually follows a basic
design, such that:
- The time period of interest (i.e., 10 years) is
divided into equal intervals, or cycles.
- A finite set of mutually exclusive states
is defined such that, in any given cycle, a
member of the cohort is in only one state.
- Initial probabilities determine the
distribution of cohort members among the
possible states at the start of the process
(often, the entire cohort starts in the same
state).
- A matrix of transition probabilities, applied
in each successive cycle, defines the
possible changes in state.
- To calculate an expected value for the
model, (e.g., net cost or quality-adjusted
life expectancy), different cost and/or utility
rewards/tolls are accumulated for each
interval spent in a particular state.
Graphical Representation
In a bubble diagram, like that shown above below,
each state is represented using an oval, arrows represent
transitions, and numbers along the arrows
indicate the transition probabilities. The probabilities
of the transition arrows emanating from any
state must sum to 1.0.
TreeAge Pro does not employ the bubble diagram
representation of a Markov model. Instead,
TreeAge Pro uses a graphical form known as a
cycle tree, which is more flexible and easily integrated
into decision trees. Markov cycle trees can
be appended to paths in a TreeAge Pro decision tree
anywhere you might place a terminal node.
Calculation Basics
There are two commonly-used methods for evaluating
a Markov model: expected value calculation
(called “cohort” analysis), and Monte Carlo
microsimulation (first-order trials). It is important
to understand the difference between the two analysis
methods, and to recognize the terms associated
with them.
In an expected value analysis, the percentage of
a hypothetical cohort in a state during a cycle is
multiplied by the cost or utility associated with that
state, and these products are summed over all states
and all cycles. In TreeAge Pro, expected value calculations
are the basis of most analyses, including
one-way sensitivity analysis, and baseline cost-effectiveness
analysis.
In a microsimulation, on the other hand, a single trial’s
value is simply the sum of the rewards or tolls
from the states traversed by an “individual” taking
a random walk through the model based on transition
probabilities. An expected value is estimated
by averaging many such trials.
In TreeAge Pro, the same Markov model
can be evaluated by either expected value or
microsimulation methods. Generally, deterministic,
expected value analysis is preferred because it
is more computationally efficient; it returns a mean
value much more quickly than simulation, which
often requires thousands of trials to return a mean
value within an acceptable error.
Additional background discussion can be found in:
- Decision Making in Health and Medicine,
Hunink, and Glasziou (2001), Cambridge University.
You are urged to consult these and other publications
dealing with the concepts which underlie
Markov modeling.
Non-Standard Markov Models
In TreeAge Pro, the basic Markov modeling rules
outlined above can be overruled in a variety of
ways, for example:
- Time-dependent Markov models are easily
handled using tables, tunnels, and/or tracker
variables.
- Discrete event simulation models can combine
sampling from event time distributions
and microsimulation features like tracker
variables.
- A Markov model can be analyzed using
the Node() function in such as way that
sensitivity analysis and other cohort type
analyses can be used, while the
Markov model is actually evaluated using
microsimulation trials.
- EV/cohort analysis of a Markov model can
make use of a realistic cohort with a specific
starting size and composition that may
change over time.
Elsewhere in This Section
> Decision Analysis and Decision Trees
> Discrete Event Simulation Models
> Cost-Effectiveness Analysis
> Monte Carlo Simulation
Learn
how TreeAge Pro users are using decision analysis. >