AI Oversight, Accountability and Protecting Human Rights: Comments on Canada’s Proposed Artificial Intelligence and Data Act

Executive Summary

This report is a collaboration of interdisciplinary researchers from the Cybersecure Policy Exchange at Toronto Metropolitan University, McGill University's Centre for Media, Technology and Democracy, and the Center for Information Technology Policy at Princeton University. Canada's investment in developing AI systems has not been matched by a comparable effort to regulate the technology, and while the authors are encouraged by initial efforts to regulate AI in Canada, they share several key concerns and corresponding recommendations to improve the proposed framework, particularly the Artificial Intelligence and Data Act (AIDA) within the newly tabled Bill C-27: Digital Charter Implementation Act, 2022.

On public consultation, Innovation, Science and Economic Development Canada should formally consult on AIDA with community advocates, researchers, lawyers, and groups representing BIPOC, 2SLGBTQIA+, economically disadvantaged, disabled and other equity-deserving populations. On oversight, the AIDA Commissioner needs to be an independent agent of Parliament, with an independent tribunal empowered to administer penalties for contravention, outline auditing best practices, and enforce the law. AIDA currently applies only to the federal private sector, exempting government institutions, so its framework should be broadened to include government institutions. Both the Consumer Privacy Protection Act (CPPA) and AIDA within Bill C-27 should provide a cohesive, technologically neutral and future-proof definition of AI or algorithmic systems across both laws, potentially by defining such systems based on their applications rather than the specific techniques involved.

Bill C-27 also needs to comprehensively address the human rights risks of algorithmic systems, including prohibitions on processing biometric data such as facial images through automated means for unique identification, especially in public settings, subject to only a very limited set of exceptions. AIDA and the CPPA should provide people with recourse to protect fundamental rights, such as the right to object to automated processing of personal data and the right to appeal algorithmic decisions. Certain uses of algorithmic systems, such as those exploiting vulnerable groups based on age or physical or mental disabilities, or systems used by the state for social scoring, must not be allowed given the unacceptable risks they pose to safety, livelihoods, and rights. Bill C-27 and AIDA should also include high levels of protection by default for children.