Chapter 3 Implementation
(Draft 2023-05-30)
In our implementation of a random assignment, we follow a set of checking procedures along the way to the final randomization, and we do a final check of critical design and data features immediately before actual assignment.
3.1 Along the Way
- Develop a correct participant flow diagram.
- Formally review randomization code.
- Develop and implement a comprehensive plan for identifying and addressing any duplicates. Depending on the application, this may include duplicate households, vehicles, schools, etc. In individual-level applications or assignments, we often need to ensure that larger households or households that generate multiple applications do not have a higher probability of being selected than smaller households, or households that only submitted a single application or otherwise only appear in the data once.
- Confirm that assignment sample is correct. It should include everyone who is eligible, and no one who is ineligible.
- Confirm that assignment probabilities are correct.
3.2 Immediately Before the Final Randomization
An overarching goal is to only run the final assignment once.
- Check for and remove duplicates.
- Run lottery with a sampled seed, as described in Section 2.4.
- Check participant flow against various tables generated by randomization code.
Once everything in this section has been implemented, in order, and is satisfactory, proceed to run the final randomization.
3.3 The Final Randomization
- Run final assignments using the final asssignment seed, which is set using the procedure in Section 2.4.