Understanding the Reliability of a Test
Download PDF Version What is PDF?
The Science of Testing
It is imperative for a woman to know if she is pregnant. With the proliferation of home pregnancy tests and many other types of medical testing, it is useful to consider how to correctly interpret a given test result. Although this is a complicated subject well beyond the scope of this review, basic concepts can be mastered with relative ease.
Once studied and understood, readers will be better able to address questions such as:
My home pregnancy test is positive. What is the likelihood I am really pregnant?
My home pregnancy test is negative. What is the likelihood I am really not pregnant?
My favorite brand of home pregnancy test claims 98% sensitivity. What does this mean? Does the test lose accuracy when performed by a lay person as opposed to a lab technician?
What are the pitfalls of performing universal screening tests for diseases such as HIV?
Screening Tests
The ideal screening test is low cost,
easy to administer and interpret, noninvasive, and
widely available. Equally important, the ideal screening
test gives a positive result for each and every individual
with the condition under study (sensitivity = 100%)
and a negative result for each and every individual
without the condition under study (specificity 100%).
No test is ideal however and it is important to understand
the limitations of a given test and a given test result.
It
should be noted that screening tests are most commonly
used to detect disease states but in fact can be used
to detect any condition or state. A screening test
may seek to identify individuals with an IQ exceeding
160 or individuals who are pregnant or children with
red hair. Possessing the IQ of a genius or being pregnant
or having red hair have nothing to do with disease
states but the principles of screening for these conditions
apply nonetheless.
Let's begin with some definitions and then some examples.
The following table groups individuals into one of four categories. The two columns divide people into groups according to whether they do or do not have the condition under study. The two rows divide people according to whether they have a positive or negative screening test result.
| Screening Test Results: |
Actually Have Condition? | Totals | |
|---|---|---|---|
| Yes (Column 1) | No (Column 2) | ||
| Positive (Row 1) | a | b | a + b |
| Negative (Row 2) | c | d | c + d |
| Totals | a + c | b + d | a + b + c + d |
Let a = the number of people with the condition under study and with a positive screening test
Let b = the number of people without the condition under study and with a positive screening test
Let c = the number of people with the condition under study and with a negative screening test
Let d = the number of people without the condition under study and with a negative screening test
Sensitivity is the measure used to report how effective a test is in identifying individuals with a disease. The higher the sensitivity the better.
Specificity is the measure used to report how effective a test is in identifying individuals without the disease. The higher the specificity the better.
The real questions to be answered are the following:
If an individual has a positive test, what is the likelihood of having the disease?
If an individual has a negative test, what is the likelihood of not having the disease?
Questions such as these refer to what’s called the "positive predictive value" and "negative predictive value".
Using the variables in the table above, we can define these quantities as follows:
Positive Predictive Value (PPV) = [a/(a + b)] × 100%]
Negative Predictive Value (NPV) = [d/(c + d)] × 100%]
The last important variable in evaluating tests and test results is the prevalence of the disease in question. Prevalence is defined as the percent of individuals in the population under study that have the disease. (This differs from incidence which refers to how many new individuals acquire the disease in a given time period.)
Prevalence = number of individuals with disease/total population
Let’s try to understand these concepts with a few examples:
Problem A
Given: We are employing a new screening test designed to detect asthma in a population of 100,000 people.
Given: Ten percent of this population suffer from asthma.
Given: This new asthma screening test has a sensitivity of 98% and a specificity of 90%.
We now have enough information to evaluate the usefulness of this new test.
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | a | b | a + b |
| Negative | c | d | c + d |
| Totals | a + c Total number of people with disease |
b + d Total number of people without disease |
a + b + c + d Total number of people |
Our first task is to insert numbers into the nine boxes below from the information above. The total population is given as 100,000. This by definition is the total number of people and is placed in the lower right box.
Because the prevalence of asthma is 10%, we can easily conclude .10 × 100,000 = 10,000 people have asthma and 90,000 do not. These values assume the totals of the first and second columns respectively.
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | a | b | a + b |
| Negative | c | d | c + d |
| Totals | 10,000 Total number of people with disease |
90,000 Total number of people without disease |
100,000 Total number of people |
We now use the given values for sensitivity and specificity.
Given a sensitivity of 98% or .98, we can safely conclude that 98% of the individuals with the disease will have a positive screening test so a = .98 × 10,000 = 9,800. This is because by definition, sensitivity = a/(a + c) so .98 = a/10,000 a = 9,800
If a = 9,800, c must equal 200 because a + c =10,000 c= 10,000 – 9,800 = 200
Similarly, given a specificity of 90% or .90, we can safely conclude that 90 Percent of the individuals without the disease will test negative so d = .90 × 90,000 = 81,000. This is because by definition, specificity = d/(b + d) so .90 = d/90,000 d = .90 × 90,000 = 81,000
If d = 81,000, b must equal 9,000 because b + d = 90,000 d = 90,000 – 81,000 = 9,000
We can fill in this additional information as below.
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | 9,800 | 9,000 | a + b |
| Negative | 200 | 81,000 | c + d |
| Totals | 10,000 Total number of people with disease |
90,000 Total number of people without disease |
100,000 Total number of people |
The last two boxes are computed with simple addition to finally complete the table:
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | 9,800 | 9,000 | 18,800 |
| Negative | 200 | 81,000 | 81,200 |
| Totals | 10,000 Total number of people with disease |
90,000 Total number of people without disease |
100,000 Total number of people |
As an internal check on our work, please notice:
18,800 plus 81,200 totals 100,000 as we would expect.
We now have four groups of people with distinct characteristics:
There are 9,800 people that have asthma and have a positive test. They have been correctly diagnosed with asthma. They are called "True Positives (TP)."
There are 200 people who have asthma but have a negative test. They have been incorrectly diagnosed as being free of asthma. They are called "False Negatives (FN)."
There are 9,000 people without asthma but with a positive test. They have been incorrectly diagnosed with asthma. They are called "False Positives (FP)."
There are 81,000 people without asthma and with a negative test. They have been correctly diagnosed as being free of asthma. They are called "True Negatives (TN)."
Prevalence = 10%, Sensitivity = 98, Specificity = 90%
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | 9,800 TP (True Positives) |
9,000 FP (False Positives) |
18,800 |
| Negative | 200 FN (False Negatives) |
81,000 TN (True Negatives) |
81,200 |
| Totals | 10,000 Total number of people with disease |
90,000 Total number of people without disease |
100,000 Total number of people |
Third, we have two groups of people with a serious problem. The 200 people in column 1 have asthma but the test was reported negative. Furthermore, 9,000 people have a positive test for asthma but in fact are asthma free.
Now let’s determine the positive and negative predictive values. (PPV and NPV)
PPV = [a/(a + b)] × 100%] so 9,800/18,800 × 100%= 52.1%
NPV = [d/(d + c)] × 100%] so 81,000/81,200 × 100% = 99.75 %
What does all this mean?
A positive predictive value of 52.1% means that only 52.1% of the people with a positive test actually have the disease. In other words, individuals with a positive test have a 52.1% chance of having asthma. Although this test correctly identified most (9,800 out of 10,000) of the people with asthma, in the process it incorrectly assigned a suspicion of asthma to an almost equal number (9,000) of individuals. This is very problematic.
A negative predictive value of 99.75% means that 99.75% of individuals with a negative test are in fact, asthma free. In other words, individuals with a negative test have a 99.75% chance of being asthma free. This provides helpful, reliable information. The small number (200 out of the 100,000 people tested) of individuals with false negative tests would have a false sense of security.
For almost all those testing negative, this test is an accurate assessment. For those testing positive, the test is almost meaningless as individuals are almost as likely to be asthma free as have asthma.
If this was an inexpensive, noninvasive, widely available test, its primary value would be identifying individuals who are asthma free. Those with a positive test would require additional, more expensive testing in order to answer the question.
Now let’s look at this same test but let’s change the disease prevalence to 2% and 25% and see what happens.
Prevalence = 2%, Sensitivity = 98, Specificity = 90%
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | 1,960 | 9,800 | 11,760 |
| Negative | 40 | 88,200 | 88,240 |
| Totals | 2,000 Total number of people with disease |
98,000 Total number of people without disease |
100,000 Total number of people |
PPV = 1,960/11,760 = 16.67% NPV = 88,200/88,240 = 99.95%
Prevalence = 25%, Sensitivity = 98, Specificity = 90%
| Test Results: | Have Disease? | Totals | |
|---|---|---|---|
| Yes | No | ||
| Positive | 24,500 | 7,500 | 32,000 |
| Negative | 500 | 67,500 | 68,000 |
| Totals | 25,000 Total number of people with disease |
75,000 Total number of people without disease |
100,000 Total number of people |
PPV = 24,500/32,000 = 76.56% NPV = 67,500/68,000 = 99.26%
Summary of Data with Varying Prevalence
| Prevalence (%) | PPV (%) | NPV (%) |
|---|---|---|
| 2 | 16.67 | 99.95 |
| 10 | 52.1 | 99.75 |
| 25 | 76.56 | 99.26 |
With this test, the lower the disease prevalence, the lower the PPV and the higher the NPV.
This example highlights the problems raised when performing mass screenings in populations where a low prevalence of disease is present. Each individual who has a positive test result but is actually disease free must endure additional unnecessary, often expensive, and sometimes painful testing, as well endure the anxiety of being diagnosed with a potentially serious condition. These screening tests establish disease-free individuals with a high degree of reliability. Individuals testing positive require further testing. As disease prevalence falls, the positive predictive value plummets.
Interpreting Home Pregnancy Tests
To correctly interpret home pregnancy tests, it is essential to know the sensitivity, specificity, and positive and negative predictive values for the test when performed by individuals without any medical or laboratory medicine training. Most home pregnancy test manufacturers have this data and will make it available upon request. Even knowing these facts and performing these tests as carefully as you can doesn’t guarantee the reliability of any individual test result.
If you suspect pregnancy, please see an appropriate physician to answer this vitally important question.
Pregnant or Not? That is the Question
From a health perspective, "tis more nobler" (not to mention vitally important) for a woman to know if she is pregnant. If you may be pregnant, please see an appropriate physician for pregnancy testing and make sure your physician is aware of any and all symptoms you may be experiencing. Any abdominal pain, bleeding, cramping, lightheadedness, fainting or a host of other symptoms may suggest the presence of an ectopic pregnancy which may be life-threatening if not detected and treated early. An ectopic pregnancy occurs when the early embryo implants in a location other than the inner lining of the uterus. In these instances, the embryo has no chance of survival but its presence can cause a woman to suffer severe internal bleeding which may lead to hemorrhagic shock and death. Most ectopic pregnancies occur in the Fallopian or uterine tubes. Rarely, implantation can occur within the abdominal cavity. Every instance of suspected ectopic pregnancy should be considered a medical emergency and all pregnant women should be evaluated by an appropriate physician to ensure ectopic pregnancy is not present.
Women with a history of damaged Fallopian tubes such as that caused by Pelvic Inflammatory Disease (from Chlamydia or Gonorrhea infection), previous abdominal surgery or ectopic pregnancy, or tubal ligation (a surgical procedure intended to permanently disable the Fallopian tubes to prevent pregnancy) are at significantly higher risk of ectopic pregnancy. In fact, a woman with a history of previous tubal ligation surgery who now has a positive pregnancy test is assumed to have an ectopic pregnancy until proven otherwise. The risk also increases with the use of certain types of contraception and with increasing age.1 However, ectopic pregnancy can and does happen to women without any known risk factors and the possibility of its presence must be considered with every pregnancy.
There are other important reasons to establish pregnancy as soon as possible. Early diagnosis allows the maximum benefit to be derived from prenatal vitamins and prenatal care and allows a woman to avoid unsafe medications, alcohol, tobacco and all other potentially harmful exposures.
1 Cunningham, et al., William's Obstetrics, 21st Edition (New York: McGraw-Hill, 2001), 884-885.