Using TreeAge Pro Healthcare Module

> Markov Processes
> Cost-Effectiveness Analysis
> Monte Carlo Simulation
> Reports
> Graphs

Markov Processes
Many healthcare problems involve multiple transitions between health states and require that the probabilities of state transitions, together with related utility values, vary over time. Neither decision trees nor influence diagrams offer a practical solution. In contrast, Markov models are designed to efficiently represent cyclical, recursive events, whether short-term processes, such as a surgical procedure followed by ICU, or long-term processes, such as management of a chronic disease.

Markov Models
While most decision trees include a simple notion of time - with events to the right of the tree occurring after those to the left - there are no shortcuts in a standard tree structure for representing events that recur over time. A state transition model, also called a Markov model, is designed to efficiently represent recursive events.

Markov models can be used to simulate either short-term processes (e.g., development of a tumor) or long-term processes (e.g., the life cycle of an individual). Markov models, like standard trees, are used to calculate a wide variety of outcomes, including average life expectancy, expected utility, long-term costs of care, survival rate, or number of recurrences.

Discrete Markov models, such as those constructed in TreeAge Pro Healthcare, enumerate a finite set of mutually exclusive possible states so that, in any given time interval (called a cycle or stage), an individual member of the Markov cohort can be in only one of the states. In order to determine a value for the entire process (e.g., a net cost or life expectancy), a value (an incremental cost or utility) is assigned to each interval spent in a particular state. A simple set of initial probabilities is used to specify the distribution of model subjects among the possible states at the start of the process. A matrix of transition probabilities is used to specify the transitions that are possible for the members of each Markov state at the end of each successive stage. Click here to see a Markov Model. >

Two methods can be used to calculate the value of a discrete Markov model: cohort (expected value) calculations, and Monte Carlo simulation trials. In a cohort analysis, TreeAge Pro calculates the expected value of the process by multiplying the percentage of the cohort in a state by the incremental value - a cost or utility - assigned to that state, and summing these products over all states and all stages. In an individual Monte Carlo simulation trial, TreeAge Pro Healthcare calculates the sum of the incremental values of the series of states traversed by the individual.

In TreeAge Pro Healthcare, an assignment of value in a Markov model is called a reward, regardless of whether it refers to a cost, utility or other attribute. A state reward refers to a value that is assigned to the members of the cohort in a particular state during a given stage.

The values used for state rewards depend, of course, on the attribute being calculated in the model (e.g., cost, utility or life expectancy). The basic Markov analysis calculates average life expectancy for the cohort, usually in terms of life years.

TreeAge Pro Healthcare uses a graphical form known as a cycle tree. Since it is based on a node and branch framework, it is easily integrated into standard decision tree structures and can be appended to paths in a TreeAge Pro Healthcare decision tree.

The root node of the Markov cycle tree is called a Markov node. Each of the possible health states is listed on the branches emanating from the Markov node, with one branch for each state. Possible state transitions are graphically displayed on branches to the right. A state from which transitions are not possible, such as the Dead state, is called an absorbing state. No state rewards are given for being in the Dead state. TreeAge Pro automatically assigns zero values to the state rewards of all absorbing states.

Normally, all the states specified in a model are amenable to the same cycle length. However, sometimes it is necessary to utilize one or a series of states having a different duration. TreeAge Pro Healthcare solves this problem through the use of tunnel states.

A tunnel describes a series of temporary states. In some cases, the tunnel represents a group of similar states which always occur in a specific order. In other cases, a tunnel may contain only a single health state that occurs over multiple cycles, where you need to distinguish among the first, second and every subsequent stage that an individual spends in this multi-cycle state. Often, this is required because some of the state's transition probabilities depend on how long an individual has been in the state, rather than on how long the process has been running (i.e., the patient's age).

A cohort analysis is evaluated probabilistically, as though the cohort had an infinite number of members. In most models, the number of living subjects declines exponentially (i.e., the total mortality rate increases over time), but there will never be a time when all members are dead. In other words, the alive portion of the cohort dwindles asymptotically toward, but never reaches, zero.

TreeAge Pro Healthcare uses a termination condition, or stopping rule, specified at the Markov node to determine whether a cohort analysis is complete enough. TreeAge Pro Healthcare evaluates the termination condition at the beginning of each stage; if it evaluates to true, the Markov process ends and the net reward(s) are reported. The termination condition can include multiple conditions, which may be cumulative or alternative.

An expected value (EV) analysis performed at or to the left of a Markov node will run a Markov cohort analysis at the selected node. Two types of EV analysis, roll back and Markov analysis, will generate additional information about the Markov cohort calculations. For example, in a model designed to measure the time spent in a given state, expected value will be average life expectancy for a member of the cohort and additional calculated values will include the amount of time spent, on average, in each of the specified states, and the percentage of the cohort in each state at the end of the process. If the termination condition had been set to continue the process until most of the cohort is absorbed into the Dead state, the final probability of Dead will approach 1.0.

Performing a Markov cohort analysis at the Markov node yields both a detailed text report and several graphic reports:

Text Report - The full trace of the Markov cohort calculations is shown in the text report, including state probabilities and rewards ordered by stage, together with net rewards. All of the calculated values used to generate the graphs described immediately below are contained in this report. Its contents can be exported for further analysis in a spreadsheet. The report is an indispensable tool for evaluating model results, including the important task of debugging the model by making sure that what happens during the cohort analysis appears coherent.

Survival Curve - Similar to the state probabilities graph but shows the sum of state probabilities for survival states. Click here to see a Survival Curve. >

State probabilities graph - This graph illustrates cohort distribution at each cycle. The graph of a simple model with three states - well, sick and dead - would demonstrate that the Dead state's membership grows in an inverse exponential curve, asymptotically approaching 1 and (with an appropriate termination condition) the model runs until the cohort is largely absorbed into the Dead state.

State rewards graph - This graph shows for each state, what reward was received at each stage.
(For cost-effectiveness models, costs and utilities are graphed separately.)

_stage_reward & _total_reward graphs - Each graph contains a single line plotting the value of the specified keyword at each cycle.

Cost-Effectiveness Analysis
In an era of limited resources and ever increasing demands for healthcare spending, pressure is growing to develop and prescribe treatments which are not only highly effective but also cost-effective. With TreeAge Pro Healthcare, both decision trees and Markov models (or a combined model) can be analyzed on the basis of expected costs, expected effectiveness, or combined cost-effectiveness, with the automatic generation of graphs and reports specifying average and marginal values for costs, effectiveness and cost-effectiveness.

Moreover, cost-effectiveness analysis can be performed either on the basis of expected value calculations or using Monte Carlo simulation. This is particularly important in the case of complex state transition models because in order to evaluate individual outcomes - as distinguished from cohort analysis - Markov models must be calculated using Monte Carlo simulation.

Cost-effectiveness analysis, or CEA, is a collection of methods for the evaluation of decisions based on two criteria using different outcome scales. It is of particular interest in situations where resource limitations require balancing the desire to maximize effectiveness and the need to contain costs.

In TreeAge Pro, CEA simultaneously compares the expected costs and expected effectiveness values of the options at a decision node. The cost-effectiveness (C/E) graph and the text report underlying it are the fundamental tools in cost-effectiveness analysis of decision trees, including Markov models. The C/E graph and text report explicitly report key information needed to interpret the results of C/E calculations, including incremental values and the existence of dominance.

Click here to see the CE graph. >
Click here to see the CE Text Report. >

In the resulting graph, a line (or, rather, a series of line segments) will connect all options that are neither eliminated from consideration by absolute dominance, nor subject to extended dominance. Although the line does not include options subject to extended dominance, in some models such options should not be removed from consideration.

Dominance
In the context of CEA, one alternative is said to be dominated by another if the former both costs more and is less effective. When this is the case, the dominated alternative normally may be removed from consideration. The use of relative position to infer dominance is illustrated in the cost-effectiveness graph. Effectiveness increases from left to right; cost increases from bottom to top. The crossing point of the axes represents one alternative. Its comparators can then be placed on the graph: more costly alternatives above, and more effective alternatives to the right. Thus, an alternative is dominated if it lies both above and to the left of another alternative.

Extended dominance
Sometimes, when making certain population-wide policy decisions, two strategies may be used together as a sort of "blended" policy, instead of assigning a single treatment strategy to all patients. Blending strategies only becomes relevant when the most effective strategy is too costly to prescribe for the entire population.

In the example above, assume that option C is not feasible because it exceeds a cost ceiling. Option B now looks like the best option. However, consider option X, which is not itself a treatment, but is instead a strategy that involves prescribing Treatment C for a portion of the population and treatment A for the remainder.

The line connecting A and C represents the average cost and effect for all possible blends between the two treatments. Any option X located on the heavy line section shown in the graph dominates over option B, because it both costs less and is more effective. This is called extended dominance. However, B cannot be eliminated as readily as could an option that is dominated in the standard sense. This is because X actually requires giving some portion of the population treatment A, the least effective of the three treatments being considered. The ethical implications of such a course cannot be ignored.

The option lowest on the cost axis may represent the baseline alternative. Each additional marker included by the line represents an option which offers better effectiveness at some added cost.

The C/E analysis text report
The C/E graph's text report dialog contains all of the relevant numeric data for the alternatives represented in the C/E analysis graph. The top section of the report will always include one table listing each option's cost, incremental cost, effectiveness, incremental effectiveness, C/E ratio and incremental C/E ratio (ICER), as appropriate. In this table, incremental values are calculated relative to the next least costly option.

The top section of the dialog will also include a second table (Table 2) in cases where a decision has three or more options. Table 2 calculates all incrementals relative to the least costly option. Finally, if a tree has three or more options, and any one of them is dominated in either the absolute or extended sense, a third table (Table 3) will be shown with dominated options excluded. The Notes section of the dialog will describe Tables 2 and 3, if they are shown, and include a textual description of each instance of either absolute or extended dominance.

One-way sensitivity analysis in cost-effectiveness models

C/E sensitivity analysis text report
The text report from a one-way C/E sensitivity analysis shows, at every interval, the average and incremental values for each alternative. Essentially, this report repeats the C/E text report described above for each interval of the sensitivity analysis.

C/E sensitivity analysis graphs
The Graph pop-up menu in the intermediate output window for a one-way cost-effectiveness sensitivity analysis, shown below, offers multiple ways to view the sensitivity analysis output graphically. All options are described briefly below.

Net Benefits … The net health benefit (NHB), or net monetary benefit (NMB)

The use of net benefits in the evaluation of health interventions has been suggested as an alternative to incremental cost-effectiveness. Several supporting rationales have been advanced.

One advantage is the ease with which average NHB or NMB values from different studies can be compared, in order to identify the most cost-effective intervention. Given the same WTP, the intervention with the greater average NHB or NMB is more cost-effective. This is the case for comparisons of any number of strategies.

Another advantage of the NHB or NMB framework is the straightforward manner in which the probability distribution of an intervention's NHB or NMB can be presented and analyzed (versus having to deal with a joint distribution of incremental cost and effectiveness values, as in TreeAge Pro's ICE scatter plot). Confidence intervals can be easily derived from the NHB probability distribution, and comparative distributions can be easily set up for multiple interventions. In TreeAge Pro, one-way cost-effectiveness sensitivity analysis and cost-effectiveness Monte Carlo simulation include graphs that utilize NHB calculations.

Click here to see graph >

Monte Carlo Simulation
The TreeAge Pro Healthcare Module employs a flexible algorithm that allows you to run multiple individual trials using each set of distribution samples. A more sophisticated random number generator enables the software to perform and report on up to 5 million trials or samples in a single simulation.

Roll back and Markov analyses evaluate Markov models using cohort calculation methods. Monte Carlo simulation offers another way to analyze Markov models, using individual trials. For some problems, both Markov cohort simulation and Monte Carlo simulation may be relevant. In many complex models, only Monte Carlo trials may be appropriate.

Stage-dependent transition probabilities
A basic form of Markov process, in which the transition probabilities remain constant throughout the analysis, is known as a Markov chain. However, in the kinds of Markov problems built in TreeAge Pro, transition probabilities and certain other values will usually vary with time. To accomplish this, transition probabilities can reference tables of stage-dependent values.

Monte Carlo simulation of Markov models
One of the important assumptions in a Markov cohort (i.e., expected value) analysis is that the model maintains no memory of previous events. Transitions and rewards are assigned to portions of the cohort based only on their state membership in the current stage (cycle) of the Markov process. The portion of the cohort starting a given stage in a given state is treated as a homogenous group, with no regard given to the different paths that are possible prior to entering that state at that time.

While cohort analysis is the standard method of evaluating Markov processes, some models will require a different kind of analysis. For example, consider a clinical problem in which alternative treatments each may be used once during an illness. In the Markov model, as in reality, if one treatment fails, that information should be carried with the patient (as in a patient record) so that the same treatment will not be used again. To track patient history in a Markov process, analysis must be done via Monte Carlo simulation trials. Because one individual is evaluated at a time, TreeAge Pro makes it possible to use tracker variables to recall each individual's path through the process, creating a flexible form of memory for assigning rewards and determining transitions. Having access to detailed patient history within a Markov process can make it possible, for instance, to enable models to more closely simulate complex biological processes, as well as actions of patients and clinicians, that can impact health outcomes.

Tracker variables, and the memory they provide in Markov calculations, are available only during individual Monte Carlo simulation trials. Trackers can be used for reporting additional output quantities (i.e., other than cost and effectiveness), such as the number of stages spent in a particular state. They can be used to manipulate transition probabilities, state rewards, termination conditions, and any other calculation that might depend on a patient's history.

For example, one set of tracker variables could be used in a model to indicate how often a patient has undergone a particular treatment, while another set of tracker variables keeps track of the current size, type, and location of a tumor. A logic node could compare the value of the tumor size tracker variable to some threshold and appropriately transition the individual to a new state. Or, the tumor location tracker variable could be used as a lookup value in retrieving an appropriate Markov reward from a table of surgical costs.

Monte Carlo algorithm allows first- and second-order simulations to be performed independently or together, as needed. In addition, the software enables:

• Grouping of multiple trials per set of distribution sample values; or
• Distribution sampling with expected value calculations (no trials); or
• Multiple trials with no distribution sampling; or
• One trial per set of sample values.
• Production of acceptability curves, scatter plots and net health benefits distributions
• Leveraging of multi-processor resources for handling growing model complexity and longer simulations
• Distribution sampling frequency, sensitivity analysis and the ability to save and reload simulation results
  
  

Monte Carlo output
TreeAge Pro Healthcare can graph and report on simulation results in greater detail and in a
number of new formats. The healthcare module includes histograms as well as cost-effectiveness (CE) and incremental cost-effectiveness (ICE) scatterplots. TreeAge Pro Healthcare also includes ICE isocontour graphs, net benefit graphs, and acceptability curves.

With the TreeAge Pro Healthcare module, you can:

• Create acceptability curves
• Create a scatter plot of cost-effectiveness pairs, with each option shown in a different color.
• Create a scatter plot of incremental cost and effectiveness pairs, given a baseline strategy and a single comparator, also showing confidence ellipse. The text report for this graph will also show the percentage of samples (or trials) in each of the six regions of the graph.
• Convert a CE scatter plot into an isocontour graph, showing the relative concentration of points. An ICE scatter plot can also be converted into a 3-D "mountain" graph, also showing the relative densities of points.
• Display CE simulation output using an acceptability curve. This is a sensitivity analysis on the willingness-to-pay, showing the changing likelihood that a comparator is cost-effective relative to a baseline strategy. You can place multiple curves on one graph by choosing more than one comparator.
• Explore the use of several net health benefits (NHB) graphs. NHB is defined as Effectiveness - Cost/WTP, and is an alternative to standard, ICER analysis for analyzing decision strategies under dual, cost and effectiveness attributes.
• Use a standard CE graph to plot a single point for each option using the average cost and effectiveness values from the simulation.
• Create distribution graphs of tracker variable values.
• Generate graphs of the distributions of DistSamp() sample values.
  
  

Other Monte Carlo simulation tools

Simulation using multiple processors
To support the complex Monte Carlo simulations described above, TreeAge Pro Healthcare can utilize multiple processors when performing a simulation. It will utilize up to eight processors on a single computer - or even, with a low-cost upgrade, across a local network. TreeAge Pro Healthcare will default to running the same number of calculation threads as your computer has processors, using the CPUs to their greatest advantage. A startup flag allows you to override the default number of calculation threads.

Longer simulations
With the TreeAge Pro Healthcare module, you can run up to 5 million iterations at a time. (This is the maximum number of second-order samples; or, if you are not sampling from distributions, the maximum number of first-order trials.)

A robust number generator
A random number generator allows for robust and long simulations on parallel processors. It is called a Mersenne Twister, and has a period of 2^19937, which is a number with over six thousand digits.

Saving to *.MCS files
TreeAge Pro Healthcare allows you to save simulations as *.MCS files, allowing long simulations to be reopened for later inspection of the many graphs and reports described above, or shared with other TreeAge Pro users.

Enhanced Strategy Selection Frequency graph
The Strategy Selection Frequency graph, already present in TreeAge Pro, has been enhanced in the Healthcare Module, with the addition of an indifference threshold. Any iteration in which the incremental value between the top two alternatives is within a specified threshold value will be marked "indifferent," and placed in a separate bar in the final graph.

Reports & Graphs
The TreeAge Pro Healthcare Module includes a number of reporting and graphing options:

Reports
Reporting supports mainly two functions:

• Tracking the variables that led to a decision outcome
• Sensitivity analysis

Other (management) types of reports can be easily generated by importing your results into Excel using the additional functionality found in TreeAge Pro Excel. Simulation results can be viewed as text-based statistical reports and in a variety of graphical formats including, histograms, scatter plots, bar graphs, two- and three-dimensional density plots.

TreeAge Pro Healthcare's built-in reporting features support a number of different decision-making objectives, including:

• Selecting an optimal strategy based on expected value (NPV).
• Examining the risk profile of a strategy.
• Determining the sensitivity of the outcomes and recommendations of a model to parameter uncertainties.
• Determining the value of perfect/imperfect information.
• Tracking the progression of events within cyclical (Markov) models.

TreeAge Pro Healthcare also enables reporting on the parameter inputs to a model, which can be in the thousands.

TreeAge Pro Healthcare includes the following reports and analyses:

• Monte Carlo Simulation -- Text-based statistical reports and a variety of graphical formats (including histograms, scatter plots, bar graphs, two- and three-dimensional density plots).
• Sensitivity Analysis -- One-way sensitivity analysis and tornado diagrams.
  Click here to see graph >
• Probability Distribution (Risk Profile) -- Bar graph (histogram) that displays the relative probabilities of payoffs (that is, scenarios).
  Click here to see graph >
• Rollback -- Expected values and path probabilities for all events/strategies that are visible in the tree window.
  Click here to see graph >
• Markov Cohort Analysis -- Survival curves or the Markov cohort's changing probability distribution over the course of the process; a full "trace" of the process appears in a table.
  Click here to see graph >
• Cost-Effectiveness Analysis -- The "cost-effectiveness frontier" -- visually identifying the available options that are not dominated and that are within the cost-effectiveness threshold.
  Click here to see graph >
• Risk Preference Function -- Graphs the currently active risk-preference function as a line graph. Click here to see graph >
• Threshold Analysis -- Text-based sensitivity analysis report.
• Expected Value -- Displays the expected value of the subtree rooted at the selected node.
• Path Probability -- Displays the probability that the scenario represented by the path between the selected node and the root node will occur.
• Payoff Range -- Determines the highest and lowest potential payoffs in the subtree rooted at the selected node.
• Standard Deviation -- Calculates the standard deviation of the potential outcomes at a selected chance node.
• Rankings -- A text report displaying the alternatives associated with a decision node, ranked in order of optimality, with their incremental values displayed.
• Show Optimal Path -- Specifies the branch emanating from the selected decision node, which represents the best choice that can be made.
• Rollback -- Displays the results all the basic calculations on the active tree.

Graphs
Graphs can be customized and preferences can be stored to control the display of later graphs.

Graph types -- general:

• Bar
• Line
• Tornado diagram
• Probability distributions (risk profiles)
• Scatter plots
  

Graphical output of the sensitivity analysis:

• Region graphs (from two- and three-way sensitivity analysis) and strategy graphs from one-way sensitivity analysis.
• Cost vs. Effectiveness -- displays an animated version of the base-line cost-effectiveness graph.
• Effectiveness vs. Cost -- same as above, with the axes inverted.
• Sensitive Variable vs. Average or Marginal Cost (or Effectiveness or C/E Ratio) -- six graphs, each with the sensitive variable on the x-axis and the y-axis showing the average or marginal cost, effectiveness or C/E ratio. Each of these graphs is a normal line graph showing how, for example, the marginal cost-effectiveness varies with changes in the input variable.
• Display and print complex two-way sensitivity analysis graphs.
• In previous versions of our software, the coordinates of graph location indicated by the mouse cursor could be displayed in the status bar by holding the control key. With our new TreeAge Pro Healthcare module, if your mouse cursor is near a line, the point will snap to that line.
• In line graphs, in addition to changing the shape of the line marker, if you double-click a marker in the legend, select the color for the line independently. This will allow you to remove the icon which is normally drawn at each interval in the line (i.e., use "no mark"), and differentiate lines by color alone
• Line graphs can be merged. This feature is useful for creating a comparative survival curve, for example. With TreeAge Pro Healthcare, you can merge line graphs, change line colors and check the coordinates of any selected threshold points on a line.