EURURO Vol. 71 No. 4

Platinum Priority – Editorial

Referring to the article published on pp. 511–514 of this issue

Generalizability of Clinical Trials: Why It Matters for

Patients and Public Policy

Scott Eggener

Department of Surgery, University of Chicago, Chicago, IL, USA

Any mention of generalizability during a presentation is

likely to induce narcolepsy amongst most clinicians.

Randomized trials are invaluable and customarily super-

sede other study designs in guiding clinical decision-making.

A timely, relevant, well-designed, properly executed ran-

domized trial is golden.

Which is exactly why internal validity and generaliz-

ability are more than semantic or statistical nerd chatter.

They are critical to guide whether and howwe should apply

trial findings to patients and populations.

Internal validity represents whether the trial was a fair

test of the hypothesis versus being affected by study design,

study conduct, bias, or random error. Consequently, trialists

strive to create a fair comparison of experimental and

control groups (eg, randomization, reduce contamination),

minimize confounding (eg, no previous cancer treatment

allowed, exclude patients with other active medications,

centralized expert pathology/radiology review), and estab-

lish strict rules (eg, standardized monitoring, validated

questionnaires, clear definition of endpoints). Due to severe

contamination in the control arm, the Prostate, Lung,

Colorectal, and Ovarian prostate cancer screening trial is a

textbook example of poor internal validity

[1,2]

. Regrettably,

it has helped inform public policy and population health

[3] .

External validity, also known as generalizability or

applicability, is the appropriateness of applying the trial

findings to other populations, either generally or specifi-

cally. Would the findings among the studied population

likely be replicable in other cohorts that may differ by

species (eg, animal studies), sex, race, ethnicity, germline

DNA, age, comorbidities, geography, healthcare system, or

socioeconomic status? For example, evaluation of a surgical

checklist within eight diverse hospitals from the World

Health Organization Safe Surgery Saves Lives Program

reduced postoperative complications and mortality

[4] ;

however, a similar large-scale intervention within Ontario,

Canada did not show a similar impact

[5] .

The original study

had favorable internal validity but likely has suboptimal

external validity. Similarly, study of a complex surgery

conducted on a highly select population at a tertiary referral

center would likely have limited generalizability.

Prostate Cancer Intervention Versus Observation Trial

(PIVOT) was a randomized controlled trial evaluating

surgery versus observation for men with clinically localized

prostate cancer within the USA Veteran’s Administration. It

was a high-profile study, higher-profile publication

[6] ,

and

highly influential in forging public policy

[7]

Soon after publication, internal and external validity

were called into question

[8]

. In this month’s issue of

European Urology

, Dalela and colleagues

[9]

use a database

which includes 70% of all cancer diagnoses in the USA

(National Cancer Database [NCDB]) to provide a detailed

and harsh criticism of PIVOT’s generalizability for a typical

population considering radical prostatectomy.

Importantly, PIVOT inclusion criteria specified an esti-

mated minimum 10-yr life expectancy, yet nearly 40% of

men died within 10 yr of entering the study. This represents

a gross violation of the inclusion criteria and threatens

internal validity since the signal (death from prostate

cancer) is obscured by noise (death from other causes).

Additionally, external validity was compromised since

93% of men undergoing surgery within NCDB had a Charlson

Comorbidity index of 0 (highest level of overall health)

versus 56% in PIVOT. Secondly, men aged 70 yr or greater

E U R O P E A N U R O L O G Y 7 1 ( 2 0 1 7 ) 5 1 5 – 5 1 6

ava ilable at

www.sciencedirect.com

journal homepage:

www.eu ropeanurology.com

DOI of original article:

http://dx.doi.org/10.1016/j.eururo.2016.08.048

* Department of Surgery, University of Chicago, 5841 South Maryland, Mail Code 6038, Chicago, IL 60637, USA. Tel. +1-773-702-5195;

Fax: +1-773-702-1001.

E-mail address:

seggener@surgery.bsd.uchicago.edu

http://dx.doi.org/10.1016/j.eururo.2016.09.049

0302-2838/