OCR: Schools Should Be Wary of Discriminatory AI Outputs
Artificial intelligence-enabled tools can do all sorts of helpful tasks. But AI is trained on existing (human-generated) data and language and can therefore include biases. And its methods rely on probability and can therefore include mistakes. The Department of Education’s Office of Ed Tech has useful explanations and resources in this blog post. Given these risks and the increasing use of AI in education, the Office for Civil Rights issued guidance on Avoiding the Discriminatory Use of Artificial Intelligence. The guidance includes multiple scenarios that OCR advises would likely cause it to open an investigation related to potential discrimination based on disability, sex, race, national origin, or a combination of bases.
