We present a theoretical model predicting racially biased policing produces 1) more use of potentially lethal force by firearms against Black civilians than against White civilians and 2) lower fatality rates for Black civilians than White civilians. We empirically evaluate this second prediction with original officer-involved shooting data from eight local police jurisdictions from 2010 to 2017, finding that Black fatality rates are significantly lower than White fatality rates and that this significance would survive an omitted covariate three times as strong as any of our observed covariates. Furthermore, using outcome test methodology and a comparability assumption, we estimate that at least 30% of Black civilians shot by the police would not have been shot had they been White. An omitted covariate would need to be at least three times as strong as any of our observed covariates to eliminate this finding. Finally, any omitted covariate would have to affect Black fatality rates and not Hispanic fatality rates in order to be consistent with the data.
Outcome tests are a method for detecting bias in selection procedures using data on selected units. We use a principal stratification approach to establish lower bounds on this bias for general outcomes. We show that the Knox, Lowe, and Mummolo  bound is sharp for a binary outcome, and weaken the assumptions required for that bound. We also show that the analogous bound for non-binary outcomes is not sharp and provide sharper lower bounds. We illustrate these methods with a re-analysis of the data in Anzia and Berry  on the delivery of federal spending by male and female members of congress. Using Anzia and Berry  data and our assumptions, we find that at least 19.9% of men elected, would not have been elected, had they been women.
For the three selection stages into the U.S. House– candidate emergence, party nomination and winning the general – we have years of research that evaluate each stage separately, with different data, different approaches and attempting to answer different questions. While this gives us a breadth of knowledge it limits our ability to compare the bias women are facing at each stage to understand where are the greatest leaks in the pipeline. Further, given incomplete attention to gender-party gaps during these stages we may be incorrectly averaging the experience across all women concealing that Democratic women and Republican women face different amounts of gender bias at different stages in the process. In order to better understand and compare the amount of gender bias Democratic and Republican women face in the process of getting elected to the U.S. House we need a single parameter that can be used at every selection stage. With an outcome test I am able to estimate the percentage of men who would not have been selected had they been women at all three stages in the process. Using Federal Elections Committee (FEC) data on campaign money raised, data from the Federal Assistance Awards Data System (FAADS) combined with data on primary and general elections I find that there is significant bias against women at every stage. While the candidate emergence stage remains a strong barrier in electing more women it is matched by significant but different within party bias in later stages. Democratic women in particular face a large amount of gender bias in the primaries while Republican women face the most bias in the general election..
Outcome tests are a method comparing rates of observed outcomes across selected groups to evaluate bias in decision making processes. Building on the lower bound estimand from Knox, Lowe and Mummolo , I derive a lower bound in terms of relative risks and develop a sensitivity analysis to weaken the selection-on-observables assumption. Additionally I develop a covariate adjusted sensitivity analysis to assess sensitivity to unmeasured covariates. I am able to estimate a bias adjusted outcome test robust to both measured and unmeasured confounders. Applying this outcome test and sensitivity analysis to data from the Chicago Police Department (1985-2016), I find evidence for gender bias in hiring. I estimate at least 7.4% of men would not have been hired had they been women.