Quantitative Evaluation of the Trade-off of Strengthened Instruments and Sample Size in Observational Studies

Ashkan Ertefaie, Dylan S. Small & Paul R. Rosenbaum

Weak instruments produce causal inferences that are sensitive to small failures of the assumptions underlying an instrumental variable, so strong instruments are preferred. The possibility of strengthening an instrument at the price of a reduced sample size has been proposed in the statistical literature and used in the medical literature, but there has not been a theoretical study of the trade-off of instrument strength and sample size. This trade-off and related questions are examined using the Bahadur efficiency of a test or a sensitivity analysis. A moderate increase in instrument strength is worth more than an enormous increase in sample size. This is true with a flawless instrument, and the difference is more pronounced when allowance is made for small unmeasured biases in the instrument. A new method of strengthening an instrument is proposed: it discards half the sample to learn empirically where the instrument is strong, then discards part of the remaining half to avoid areas where the instrument is weak; however, the gains in instrument strength can more than compensate for the loss of sample size. The example is drawn from a study of the effectiveness of high-level neonatal intensive care units in saving the lives of premature infants.