Benefits to a Double-Blind Study

A case-control study is an observational study where researchers select a number of subjects with an outcome of interest (cases), and a number of subjects without the outcome of interest (controls). The prevalence of exposure to a risk factor is then compared between the two groups. If the cases have a much higher rate of exposure to the risk factor than the controls, it may mean that the outcome is related to the risk factor Example:
To evaluate if cell phones cause brain cancer, researchers select two groups of subjects Case group. 100 people with brain cancer Control group: 100 people without brain cancer Researchers then compare cell phone use between the two groups If cell phone use is much higher in the case group then there may be a link between brain cancer and cell phone use

Advantages
Much easier and cheaper to perform than a randomized controlled trial With modern medical databases, a large number of subjects can be evaluated with very little effort Outcomes with a low incidence can be evaluated. This is typically not feasible with a randomized controlled trial. Exposures that are not ethical in a randomized controlled trial can be evaluated. For example, no randomized controlled trial is going to randomize patients to smoking or known carcinogens.

Disadvantages
Does not prove causality Data for patient variables may be lacking or missing and therefore must be estimated Difficult or impossible to control for confounding Patients are not randomized so there is no way to control for unmeasured covariates and hidden bias

A cohort study is an observational study that is used to evaluate whether an exposure is associated with a particular outcome In a cohort study, two groups of subjects are formed based on exposure to a factor. One group has been exposed to a factor, and the other group has not been exposed to the factor. For other variables (ex. age, sex, race, etc.), the groups are matched as closely as possible Neither group has the outcome at the beginning of the study The two groups are followed for a length of time, and the incidence of the outcome is compared between the two groups A higher incidence in the exposed group may mean a link exists between the factor and the outcome Example:
Researchers want to evaluate if smoking causes bladder cancer They identify 2 cohorts of people: Exposed cohort: 100 smokers Control cohort: 100 nonsmokers They follow the two cohorts for a number of years and compare the incidence of bladder cancer between them

Advantages
Much easier and cheaper to perform than a randomized controlled trial With modern medical databases, a large number of subjects can be evaluated with very little effort Outcomes with a low incidence can be evaluated. This is typically not feasible with a randomized controlled trial. Exposures that are not ethical in a randomized controlled trial can be evaluated. For example, no randomized controlled trial is going to randomize patients to smoking or known carcinogens.

Disadvantages
Does not prove causality Data for patient variables may be lacking or missing and therefore must be estimated Difficult or impossible to control for confounding Patients are not randomized so there is no way to control for unmeasured covariates and hidden bias

A meta-analysis is a statistical study where the results from a number of related studies are pooled together and analyzed in order to draw conclusions from a broader set of data A meta-analysis may pool the results of cohort studies, case-control studies, randomized controlled trials, or a combination of the three. Cohort and Case-control studies may be combined in a meta-analysis, but it is not common (nor recommended) to combine randomized controlled trials with other types of studies

Advantages
Often times, different studies on the same condition will come to different conclusions. A meta-analysis is a way to combine the results of these studies so that an overall effect can be estimated. If a trial has a small number of subjects, it may not be powerful enough to find a statistically significant result. A meta-analysis allows small trials to be pooled together so that they have sufficient power to find a significant effect if one exists. The overall effect found in meta-analyses is often less than what is seen in individual trials. When done properly, the effect in the meta-analysis is more likely to represent the true effect in a large population of patients.

Disadvantages
Studies differ in length, design, intervention, population characteristics, and outcome measures. This can make it difficult to determine which studies to include. Combining studies that differ in design may lead to conclusions that are invalid Meta-analyses are subject to publication bias. Publication bias occurs when only studies that found a significant effect or a large effect were published. Meta-analyses are only as good as the studies they evaluate. If there are not enough good studies to perform a valid meta-analysis, the authors should be willing to concede this and abandon the meta-analysis. It can be argued that the results from a large, randomized controlled trial are more valid than the results of a meta-analysis on the same topic

A nested case-control study is a case-control study where the subjects for the study are sampled from a cohort of patients that has been observed for a period of time. The study is said to be "nested" within the cohort. Researchers form two groups from the cohort of patients. One group has the outcome of interest (cases), and one group has not developed the outcome (controls). Exposure to a risk factor can then be compared between the two groups. If the cases have a significantly greater exposure to the risk factor, then the risk factor may be related to the outcome. When selecting the control group in a nested case-control study, the researchers typically select 1 - 4 controls per case as opposed to comparing cases to the entire group of controls. Controls are selected that match the cases on a number of measured variables (ex. age, medical conditions, sex, etc). This helps to control for confounding. Example:
Researchers have observed a cohort of 100,000 women for 10 years At the start of the study, all women gave blood samples Over the course of the 10 years, 2000 women developed breast cancer The researchers are interested in finding out if there is a link between mercury exposure and breast cancer The researchers decide to test the blood samples for mercury levels and compare rates of breast cancer between women with high levels and those with normal levels Testing 100,000 blood samples for mercury levels would cost about $2 million dollars To save money, the researchers decide to do a nested case-control study where they select 2 controls for every case. Controls are selected that match the cases for a number of variables. This way they only have to test 6000 blood samples (2000 cases and 4000 controls).

Advantages
In most cases, the nested case-control study is much cheaper to perform than a full case-cohort study By selecting controls that match the cases on a number of covariates . there is some control for confounding and statistical efficiency should be improved

Disadvantages
Because the entire cohort is not used (for the controls), some precision is lost in the statistical measures, and the confidence intervals for the study outcomes will be wider Like regular case-control studies . nested case-control studies are observational studies and have all the same limitations

An observational study is a study where patients and outcomes are observed and no intervention is applied by the investigators. In observational studies, groups of subjects are formed according to specific exposures (cohort study ) or according to specific outcomes (case-control study ). This is in contrast to a randomized controlled trial where subjects are randomly assigned to a specific group and then followed. Observational studies may be prospective (groups are identified and then observed over time) or retrospective (groups are identified and outcomes are compared)

Advantages
Much easier and cheaper to perform than a randomized controlled trial With modern medical databases, a large number of subjects can be evaluated with very little effort Outcomes with a low incidence can be evaluated. This is typically not feasible with a randomized controlled trial. Exposures that are not ethical in a randomized controlled trial can be evaluated. For example, no randomized controlled trial is going to randomize patients to smoking or known carcinogens.

Disadvantages
The main disadvantage of observational studies is that there is no way to control for hidden bias and unmeasured covariates . In randomized controlled trials, the process of randomization helps to alleviate the effect of these issues.

RANDOMIZED CONTROLLED TRIAL (RCT)

A RCT is a prospective study where subjects are randomly assigned to either a treatment group or a control group In most studies, the two groups are matched as closely as possible by variables such as age, medical problems, lab values, blood pressures, etc. If the study is very large (ex. > 500 subjects), then these factors will often balance automatically through randomization. Randomization is very important because it is the only way to control for unmeasured covariates . Measured variables (ex. weight, age, medical history, etc.) can be adjusted for, but there may be variables that are not measured because the researchers are not aware that they can effect the study outcome. If the study is randomized and large enough, then unmeasured covariates should be distributed equally between the groups, and they will not bias the outcome. Randomization also prevents hidden bias. In observational studies . there is no way to prevent or control for hidden bias. In a RCT, the treatment group is given an intervention (drug or procedure) The control group is not given the intervention, or it is given a placebo or sham procedure The effect of the intervention is then measured over a predetermined time period by a predetermined set of outcomes RCTs are considered the gold standard for clinical trials, and if done correctly, their findings are much more consequential than those of other study designs (ex. cohort studies . case-control studies ) The best RCTs are placebo-controlled and double-blinded where possible

Advantages
Able to control for confounding from unmeasured covariates Hidden bias can be prevented Criteria and outcomes can be clearly defined and accurately measured Large RCTs will give the best measure of an intervention's effect in the intended population

Disadvantages
Expensive and time-consuming Can take many years to complete If an outcome has a low incidence, RCTs are typically not feasible because the study size would need to be very large, and it would make the trial very expensive RCT are not ethical in some situations. Example: Randomizing subjects to smoking or other interventions that are known to be harmful.

Certain countries, HMOs, and insurance companies maintain large databases of medical information on their citizens/patients/clients This information can be used to conduct observational studies Observational studies using medical databases as a source of information have become very popular because they allow investigators to examine a very large amount of information with very little effort, expense, and time Information from medical databases has a number of weaknesses that should be considered

Advantages
Cheap and easy to perform Able to evaluate a large amount of information with very little effort

Disadvantages
Diagnostic codes are often used as criteria. Use of these codes is not well-standardized. Pertinent patient information is often missing or not recorded and must be estimated Only can be used for observational studies

In an as-treated analysis, outcomes for subjects in a study are counted toward the treatment they actually receive. If a subject is randomized to one treatment group, but crosses over during the study and receives a competing treatment, then their outcome counts toward the competing treatment. This differs from an intention-to-treat analysis where a participant's outcome counts toward their original assigned group regardless of whether they crossover As-treated analyses are often performed in studies where there is a high amount of crossover between treatment groups On the surface, as-treated analyses seem to make sense, but in reality, they are susceptible to a large amount of bias Example:
A trial enrolls 200 people with sciatic nerve pain 100 people are randomized to physical therapy, and 100 people are randomized to back surgery The subjects are followed for one year. At the end of one year, differences in back pain and disability are compared. During the course of the year, 30 people assigned to physical therapy end up undergoing back surgery, and 10 people assigned to surgery never actually have surgery and instead receive physical therapy (crossovers) The researchers decide to do an as-treated analysis of their data so that the crossovers will be counted toward the therapy they actually received

Bias in as-treated analyses
In this example, all the patients who crossed-over are counted toward the treatment they actually received: 30 people from the physical therapy group are counted toward surgery, and 10 people from the surgery group are counted toward physical therapy This appears to be logical since the outcomes for crossovers will count toward the treatment they actually received. In reality, the analysis will likely be biased. People often enter trials because they hope to receive one of the treatments being offered. Trial protocols often allow for crossover at some point because this makes it easier to get people to participate. If a patient believes the only thing that will make his sciatic pain better is surgery (the perception that an invasive treatment is actually doing something where other treatments are not), and they are assigned to the physical therapy group, then that patient may be disappointed and is already biased toward not improving. The same patient may later crossover and receive surgery and only then get better because "something has been done." If this patient is counted toward the surgery group, then bias has been introduced. Another issue with as-treated analyses is that randomization is compromised. Randomization is one of the most important statistical methods that protects against bias and confounding (see randomization below)

In double-blind trials, neither the subject nor the researcher evaluating the subject knows which intervention the subject is receiving In single-blind trials, only the subject is blinded as to what intervention they are receiving Double-blinding helps prevent bias on the part of the subjects and the researchers When possible, randomized controlled trials should be double-blinded Double-blinding is not always possible. For example, when comparing a surgical intervention to a medical one.

Intention-to-treat is a method in randomized controlled trials where the outcomes for all participants who are randomized to a certain treatment or control group are counted toward that group regardless of whether they complete their assigned treatment protocol or not In an intention-to-treat analysis, outcomes for patients who stop treatment or drop out of treatment are still counted toward their original randomized group. Even outcomes from patients who crossover and receive a competing treatment are still counted toward the original assigned group. Intention-to-treat analysis is the gold standard for analysis, because other forms of analyses (see per-protocol analysis . on-treatment analysis . modified intention-to-treat analysis . and as-treated analysis ) can introduce bias into a study

Example:
A trial enrolls 200 people with sciatic nerve pain 100 people are randomized to physical therapy, and 100 people are randomized to back surgery The subjects are followed for one year. At the end of one year, differences in back pain and disability are compared. During the course of the year, 30 people assigned to physical therapy end up undergoing back surgery, and 10 people assigned to surgery never actually have surgery and instead receive physical therapy (crossovers)

Intention-to-treat
In an intention-to-treat analysis, outcomes for the 30 people who crossed-over from the physical therapy group and received surgery would still be counted toward the physical therapy group Outcomes for the 10 people who never underwent surgery will still be counted toward the surgery group

Advantages
Limits bias - In the above example, if an as-treated analysis was performed where all the patients who crossed-over were counted toward the group they ended up in, then bias would be introduced. People often enter trials because they hope to receive one of the treatments being offered. Trial protocols often allow for crossover at some point because this makes it easier to get people to participate. If a patient believes the only thing that will make his sciatic pain better is surgery (the perception that an invasive treatment is actually doing something where other treatments are not), and they are assigned to the physical therapy group, then that patient may be disappointed and is already biased toward not improving. The same patient may later crossover and receive surgery and only then get better. If this patient is counted toward the surgery group, then bias has been introduced.

Disadvantages
High crossover or dropout rates can bias a study towards the null (no effect) - If a study has a high crossover or dropout rate, then the study may be biased towards the null, or "no effect." In the above example, the 30 patients who crossed over to surgery are still counted toward physical therapy. If a true difference in outcomes exists between the two treatments, it will be harder to discern since positive surgery outcomes may count toward physical therapy. In the end, there is no perfect way to deal with crossovers and dropouts, but the gold standard for evaluating subjects in a trial is intention-to-treat.

Modified intention-to-treat is when investigators in a trial use an intention-to-treat analysis, but they modify who they include in this analysis Example:
A study wants to compare the effects of a new drug on postmenopausal vaginal dryness 100 women are randomized to the drug, and 100 women are randomized to placebo At the start of the study, all women have a vaginal smear performed After 6 months, changes in vaginal symptoms are measured The researchers present their significant findings in a "modified intention-to-treat" analysis where only women who had ≤ 5 superficial cells on their vaginal smear were included in the analysis

In the above example, the researchers decided to only include a subset of the randomized women in their analysis A modified intention-to-treat analysis should raise red flags because it may mean the researchers did not find a significant result with their intention-to-treat analysis, and they are now digging for a significant results by changing the study's criteria. This is bad form because it can introduce bias. A reviewer should also ask themselves if the modified criteria is clinically relevant. In the above example, most doctors are not going to do a vaginal smear before prescribing a menopause drug, so the fact that it may work in this subset of women is irrelevant. In some cases, a modified intention-to-treat analysis has little impact. Example: researchers do a modified intention-to-treat analysis that only includes randomized patients who received at least one dose of a study drug.

On-treatment analysis is when investigators only include data from patients while they are taking their assigned treatment. If patients stop their assigned treatment for a period, data from that time period is excluded. On-treatment analysis is similar to per-protocol analysis On the surface, on-treatment analyses makes sense - if someone stops treatment then their results should not count since the intervention can not help them if they do not take it In reality, on-treatment analysis introduces bias (See Examples below) Example #1:
A study is comparing a new drug to prevent migraines to an older drug that prevents migraines 100 people are randomly assigned the new drug, and 100 people are randomly assigned the old drug The patients are supposed to take the drugs for 6 months, and the number of migraines in that time period are recorded 30 people taking the new drug and 10 people taking the old drug stop taking it before the end of the trial The researchers decide to do a on-treatment analysis that excludes data from people while they stopped taking their drugs In this circumstance, the on-treatment analysis has introduced bias People stop study medications for a variety of reasons (e. g. side effects, lack of perceived efficacy, etc.). If a on-treatment analysis is performed, the study has been biased towards "responders" and it does not reflect the true efficacy of the drug in the intended population. This may also lead to spurious significant results.

Example #2:
A study is comparing aspirin to placebo for the prevention of deep vein thrombosis 1500 people are given aspirin and 1500 people are given placebo. The 2 groups are followed for 2 years and the rate of deep vein thrombosis is compared. 300 people in the aspirin group stopped taking their aspirin during the trial The researchers do a on-treatment analysis that excludes data from people while they stopped their aspirin. The analysis shows that when compared to placebo, patients who actually took their aspirin had better outcomes than the whole aspirin group (takers and non-takers). On the surface, this analysis seems legitimate. The outcome is objective, and for the most part, people tolerate aspirin well, so comparing people who actually took the drug to the controls makes sense. The problem is that past studies have shown that outcomes in the placebo group also differ between compliant and noncompliant patients. Patients who are compliant in taking their placebo are also more likely to have better outcomes than patients who do not (even for objective outcomes like heart attack). This phenomenon likely occurs because of differences in unmeasured confounders between compliant and noncompliant patients. Because of this phenomenon, the on-treatment analysis leads to erroneous results and is biased.

Overview
In a per-protocol analysis, only subjects who completed the protocol they were assigned to are included in the statistical analyses Patients who vary from the protocol (ex. stop medication), drop out of the study, or crossover to another therapy are not counted in the analyses Per-protocol analyses are similar to on-treatment analyses Per-protocol analyses introduce bias in many cases (see Example #2 and Example #3 below) Per-protocol analyses may be more meaningful in conditions that are difficult to treat and/or when there are few treatment options

Noninferiority trials
In noninferiority trials, per-protocol analyses should be reported with intention-to-treat analyses Noninferiority trials often compare new treatments to standard treatments (ex. new anticoagulant drug compared to warfarin for atrial fibrillation) During the trial, patients assigned to the new treatment may be allowed to crossover to the standard treatment for various reasons (e. g. side effects, new indication, etc.). In intention-to-treat analyses, crossovers can bias a study towards the null, or in the case of noninferiority trials, towards noninferiority. Per-protocol analyses should be reported with intention-to-treat analyses to help measure the sensitivity of the results to protocol violations


Example #1:
A study is comparing a new drug + standard therapy to standard therapy alone in a difficult to treat cancer The new drug has many side effects that make it intolerable for a number of patients. These patients stop taking the drug. The researchers decide to to do a per-protocol analysis to see if the new drug improved survival in patients who actually completed therapy In this case, the per-protocol analysis is meaningful because it will show whether people who can tolerate the therapy actually benefit from it

Example #2:
A study is comparing a new drug to prevent migraines to an older drug that prevents migraines 100 people are randomly assigned the new drug, and 100 people are randomly assigned the old drug The patients are supposed to take the drugs for 6 months, and the number of migraines in that time period are recorded 30 people taking the new drug and 10 people taking the old drug stop taking it before the end of the trial The researchers decide to do a per-protocol analysis that would exclude the people who stopped taking their drugs In this circumstance, the per-protocol analysis has introduced bias People stop study medications for a variety of reasons (e. g. side effects, lack of perceived efficacy, etc.). If a per-protocol analysis is performed, the study has been biased towards "responders" and it does not reflect the true efficacy of the drug in the intended population. This may also lead to spurious significant results.

Example #3:
A study is comparing aspirin to placebo for the prevention of deep vein thrombosis 1500 people are given aspirin and 1500 people are given placebo. The 2 groups are followed for 2 years and the rate of deep vein thrombosis is compared. 300 people in the aspirin group stopped taking their aspirin during the trial The researchers do a per-protocol analysis that excludes the people who stopped their aspirin. The analysis shows that when compared to placebo, patients who actually took their aspirin had better outcomes than the whole aspirin group (takers and non-takers). On the surface, this analysis seems legitimate. The outcome is objective, and for the most part, people tolerate aspirin well, so comparing people who actually took the drug to the controls makes sense. The problem is that past studies have shown that outcomes in the placebo group also differ between compliant and noncompliant patients. Patients who are compliant in taking their placebo are also more likely to have better outcomes than patients who do not (even for objective outcomes like heart attack). This phenomenon likely occurs because of differences in unmeasured confounders between compliant and noncompliant patients. Because of this phenomenon, the per-protocol analysis leads to erroneous results.

In a placebo-controlled trial, subjects who are not randomized to an active treatment (controls) are given a fake or sham treatment instead (Ex. an inert pill or a procedure that is faked) Placebo-control prevents subjects in a trial from knowing whether they are receiving the active treatment. This prevents bias that can arise from a subject's expectations that an intervention works. Placebo-control is very important in trials where outcomes are subjective (ex. pain, disability, frequency of events that are not biologically measurable) It should be applied when possible in trials with objective outcomes

Propensity score matching (PSM) is a statistical technique used in observational studies . It helps to control for selection bias. In PSM, subjects who received a treatment are matched with subjects who did not receive the treatment based on the probability of receiving the treatment. The probability of receiving the treatment is reflected in the propensity score. The propensity score is calculated by combining the probability of receiving treatment for a number of measured covariates . When subjects with the same propensity score are compared, selection bias is limited Procedure:
Determine the probability of receiving the treatment for each measured covariate in the study (ex. age, sex, medical conditions, lab values, etc) For each subject, combine the probabilities of receiving the treatment based on their individual covariates. Derive the propensity score from this combination (usually with logistic regression) Create groups of subjects with the same propensity score. Subjects with the same propensity score will typically match closely on a number of covariates. Compare treated subjects to untreated subjects within these groups. There are a number of ways of doing this. See Propensity score techniques below. Perform multivariate analysis on the propensity-matched sample

Propensity score techniques
Direct comparison
In direct comparisons, treated and untreated individuals with the same score are matched and compared. Matching can be one-to-one, one-to-as many that match, and so on. Groups can also be stratified by propensity score and then strata can be compared directly A major disadvantage of direct matching is that depending on the method used, all of the available data may not be utilized. In addition, if strata are compared, residual confounding may occur.

Propensity score as a covariate
The propensity score can be used as a covariate in a multivariate model In this case, the outcome is the dependent variable, and the independent variables are the exposure/treatment and the propensity score If the probability of receiving the treatment (propensity score) is strongly associated with the outcome, then covariate adjustment for the propensity score will decrease the strength of association between the treatment and the outcome

Propensity score weighting
Propensity score weighting is a method where each individual in the sample is weighted by the inverse of their propensity score Using this method, subjects who are more likely to receive the treatment carry less weight, and subjects who are less likely to receive the treatment receive the most weight With inverse weighting, covariates that are associated with receiving the treatment/exposure (higher propensity score) are neutralized while covariates that are weakly associated with receiving the treatment (low propensity score) gain value. This helps to neutralize selection bias, creating a sample where covariate distribution is independent of treatment selection. [5,6]



Advantages
PSM is helpful when a large number of covariates are measured because it creates a comparison that minimizes selection bias regardless of whether patients match exactly on all of the covariates PSM simulates randomization of treatment for measured covariates (not for unmeasured covariates)

Disadvantages
PSM cannot control for unmeasured covariates and hidden bias

TERMINOLOGY:
Measured variables - patient variables (ex. age, sex, medical problems) that are measured before a trial begins Unmeasured covariates - patient variables that are not measured, but may be related to (and have an influence on) the outcome being evaluated. There may be several reasons these variables are not measured. Researchers may not be aware that they can affect the outcome, or the variables may be expensive to obtain (ex. genetic profiles on every patient enrolled) Hidden bias - any form of bias that may influence the outcome of a trial (ex. researchers assigning healthier patients to the treatment group (a form of selection bias))

RANDOMIZATION
Randomization is a process in clinical trials where patients are randomly assigned to different groups (ex. treatment and placebo) Ideally, the person enrolling the patients in the trial should have no idea which group the patient will be assigned to Randomization is a critical component of controlled trials because it is the only way to control for unmeasured covariates. Randomization also prevents hidden bias. If the trial is large (> 500 subjects), then randomization will typically create groups that are equal for a number of measured variables (ex. age, sex, medical problems, etc) Unmeasured covariates can also affect the trial outcome. If the trial is large enough, randomization will automatically balance unmeasured covariates between groups so that they will not affect the outcome. Randomization prevents hidden bias from occurring Randomization is the main reason randomized controlled trials are the gold standard for clinical studies. Observational studies cannot randomize participants, therefore they are always subject to unmeasured covariates and hidden bias.

In single-blinded trials, only the patient is blinded as to what treatment they are receiving. The investigators or treating doctors are aware of the treatment given. Contrast with double-blinded trials where the investigator and subject are blinded to the treatment allocation Double-blinding is preferred when possible because it prevents investigator bias

When discussing absolute risk and relative risk, it helps to consider the two measures together Absolute risk is the overall incidence of an outcome in an entire group or population Relative risk is the ratio of the incidence of an outcome between two groups Relative risk only considers the proportion of the participants who have the outcome in question where absolute risk considers the entire population being studied. This can lead to some large difference between the two measures.

As one can see from the example above, the relative risk reduction and the absolute risk reduction can be quite different (46% vs 2.3%) It's important to understand the difference, because medical literature will often emphasize a large relative risk reduction when the absolute risk reduction may be quite small and insignificant. From the example above, the drug company for Livelong will likely say that they "reduce the risk of heart attack by 46%." While this may be true, someone taking Livelong will only reduce their overall risk of heart attack by 2.3%.

INCIDENCE
Incidence is the proportion of a group that develops a disease (or experiences an event) over a specified period of time


Example:
100 people without diabetes are followed for 5 years At the end of 5 years, 10 people have developed diabetes The incidence of diabetes over the 5 years is 10%



PREVALENCE
Prevalence is the proportion of a group that has the disease (or has experienced an event) at a single time point


Example:
100 people are selected randomly 10 of the people in the group have diabetes The prevalence of diabetes in the group is 10%

A likelihood ratio is a statistical ratio used to quantify the predictive value of a medical test (typically a screening test) The likelihood ratio is the factor by which the post-test odds of a condition being present (positive likelihood ratio) or absent (negative likelihood ratio) are increased or decreased based on the results of a test The likelihood ratio can be multiplied by the pre-test odds of the condition being present to determine the post-test odds Post-test odds of disease = Pre-test odds X Likelihood ratio Mathematically, the likelihood ratio is calculated using the sensitivity and specificity of the test:
Example:
Test A is a screening test for heart disease Paul is a 60 year-old patient who based on his family history, age, and medical conditions has a probability of 25% of having heart disease
Convert 25% probability to odds = 25/75 = 0.33

Test A has a negative likelihood ratio of 0.1 and a positive likelihood ratio of 1.9
Equivalent to a Sensitivity of 95% and a specificity of 50%

If Paul has a negative test A, then his post-test odds of heart disease are 0.1 X 0.33 = 0.033 If Paul has a positive test A, then his post-test odds of heart disease are 1.9 X 0.33 = 0.627 Converting back to probability:
negative test = 0.033/1.033 = 0.032 or 3.2% probability positive test = 0.627/1.627 = 0.385 or 38.5% probability

In Paul's case, a positive test increases his probability of disease from 25% to 38.5% - not very helpful A negative test decreased his probability of disease from 25% to 3.2% - more helpful This is true because the test had a high sensitivity (low false negatives) and low specificity (high false positives)

NET RECLASSIFICATION INDEX/IMPROVEMENT (NRI)

The Net Reclassification Improvement (NRI). (also called "Index") is a statistical measure used to evaluate whether adding a measure to a predictive model will improve the predictive accuracy of the model. An example would be adding Coronary Artery Calcium Scores (CACS) to the Framingham risk model to see if the addition of the CACS improves the Framingham's ability to predict a heart attack. The NRI takes into account both correct and incorrect reclassifications.

The NRI is calculated in the following manner: In a study, take the people who had an event or outcome (events) Calculate the number of events who were reclassified into a higher risk category when the new measure was included Subtract the number of events who were reclassified into a lower risk category when the new measure was added Divide this number by the total number of people who had an event Now take the number of people who did not have an event or outcome (nonevents) Calculate the number of nonevents who were reclassified into a lower risk category when the new measure was added Subtract the number of nonevents who were reclassified into a higher risk category when the new measure was added Divide this number by the total number of people who did not have an event Add the two proportions together and you have the NRI

The formula for the NRI is as follows:

Example
The Framingham risk model uses age, gender, total and HDL cholesterol, smoking status, and systolic blood pressure to predict a person's 10-year risk of heart attack Researchers want to know if adding Coronary Artery Calcium Scores (CACS) to the model improves the risk prediction of the original Framingham model A cohort of patients is formed and their Framingham 10-year risk is calculated using the original risk model The patients also have their CACS measured at baseline The patients are followed for 10 years and the incidence of heart attacks is measured Each patient's predicted risk of heart attack is calculated using the original Framingham model. The risk is also calculated using a model that incorporates the CACS. The NRI is then calculated by comparing the two models

Interpretation of the NRI
It's important to understand what the NRI actually means because it is commonly misinterpreted


In the example above, let's assume the addition of the CACS to the Framingham model had the following effect:
71 patients with a heart attack are reclassified as higher risk 24 patients with a heart attack are reclassified as lower risk 790 patients without a heart attack are reclassified as lower risk 657 patients without a heart attack are reclassified as higher risk Total number of patients who had a heart attack - 209 Total number of patients who did not have a heart attack - 5669

The calculation of the NRI is as follows: Some people mistakenly interpret this as the addition of CACS correctly reclassifies 25% of the population studied This is not correct. The addition of the CACS correctly reclassified 22.5% of the population who had an event (0.225 X 209 = 47) and 2.3% of the population that did not have an event (0.023 X 5669 = 130) The percent of the total population correctly reclassified is 177/5878 = .03 or 3% Whether or not 3% is clinically significant will depend on the cost and convenience of the additional measure
The number needed to treat (often abbreviated "NNT") is the number of patients who must be treated with an intervention in order for one patient to benefit If the intervention is a screening test, it is the number of people who would have to be screened in order for one person to benefit from screening The NNT can also be referred to as the "Number Needed to Harm" (NNH) when talking about a side effect or adverse event The NNT is calculated by taking the reciprocal of the absolute risk reduction The NNH is calculated by taking the reciprocal of the absolute increase in adverse event Example:
The recent NLST trial (detailed here ) found that yearly CT scan screening reduced the absolute risk of lung cancer death by 0.33% in heavy smokers The number needed to screen is = 1/0.0033 = 303 patients This means 303 patients would have to be screened in order to prevent 1 lung cancer death






OVERVIEW
When most people think of comparing two outcomes, they think in terms of relative risk Relative risk is a comparison that uses fractions. It is the most intuitive way to compare two outcomes.




Example:
Drug A has been shown to reduce the risk of heart attack by 20% Most people understand that people taking Drug A will have 20% fewer heart attacks than people not taking Drug A



ODDS
When discussing odds ratios, it helps to review what odds are Odds are ratios. Odds are not fractions. This can be confusing.


Odds = the number of times that an event will occur for the number of times that it will not occur

Example:
3:1 odds means the event is likely to occur 3 times for every one time it does not occur So for every 4 tries, there will be 3 events and 1 non-event. This equals a probability of 75% (3/4).

To convert between odds and probability:

ODDS RATIO

An odds ratio is a ratio of two odds. Relative risk (sometimes called hazard ratios) is a ratio of two fractions. Because of this distinction, the odds ratio and the relative risk ratio do not always approximate each other. This can be confusing because many people mistakenly interpret odds ratios as relative risk ratios. As the prevalence of an outcome increases above 10%, the odds ratio and the relative risk start to diverge. This occurs because odds above 1:1 (0.50 probability) can run to infinity where a fraction (of a whole) is always between 0 and 1. This can lead to wide discrepancies between the odds ratio and the relative risk.

Example:
Let's assume a group of diabetics are followed for a year and 80% of them suffer a heart attack A group of nondiabetics is followed for a year and 20% of them suffer a heart attack We want to compare the risk of heart attack between the two groups The relative risk of heart attack in the diabetic group compared to the nondiabetics is (0.8)/(0.2) = 4. Diabetics have 4 times the risk of heart attack when compared to nondiabetics. Most people understand this. The odds ratio of a heart attack when comparing the diabetics to nondiabetics is (4)/(0.25) = 16. This differs from the relative risk. It means diabetics have 16 times the odds of a heart attack than nondiabetics. This comparison is not as intuitive for most people and will often be misinterpreted as 16 times the risk of heart attack.

Rules for interpreting odds ratios:
When the incidence of the outcome of interest is < 10% in the study population, then the odds ratio and the relative risk can be considered equivalent When the incidence of the outcome of interest is > 10% in the study population, and the odds ratio is > 2.5 or < 0.5, then the odds ratio will tend to exaggerate the magnitude of the association. In some circumstances the odds ratio can be converted to a relative risk (see below).

ODDS RATIO IN MEDICAL STUDIES

Case-control studies
Odds ratios are always reported in case-control studies because risk is not measured in these studies. The case group has 100% probability of the disease and the control group has 0% probability. The odds of exposure to a risk factor is compared between the two groups, therefore the results are reported as an odds ratio.

Cohort studies
Cohort studies often utilize logistic regression to control for covariates . Logistic regression yields an odds ratio. In some cases, the odds ratio can be converted to a relative risk. See formula below. [2]

CONVERTING AN ODDS RATIO TO RELATIVE RISK

If a cohort study uses logistic regression and reports an odds ratio, it can be converted to relative risk if the incidence of the outcome in the unexposed (control) group is known

OUTCOME MEASURES (SIGNIFICANT CLINICAL OUTCOMES VS SURROGATE ENDPOINTS)

SIGNIFICANT CLINICAL OUTCOMES
A significant clinical outcome is an outcome that has a significant effect on a person's health, functioning, or well-being Examples of significant clinical outcomes include: death, heart attack, stroke, blindness, paralysis, cancer

SURROGATE ENDPOINTS
Surrogate endpoints are measures that have been shown to be related to a significant clinical outcome