Risk Mitigation Management

Checklist / Questions to ask your Development team

  • Do you have a risk management plan that consists of the following : identify potential risks at each stage of development, how risks are to be organized, categorized, compared, and consolidated, how risk is to be measure, tasks proposed to mitigate risk. For additional information review ISO31000 Guidelines

  • Do you understand the harms associated with the different types of data including affective data and mapping to emotions?

  • Have you checked for under representation of different stakeholders in the data?

  • Have you identified all sources of potential bias?

  • What is the rating of the implementation team for inclusiveness, diversity, technical AI capabilities ie. skill gaps, rigid mindset, miscommunication, traditional IT practices used?

  • Do you have a plan to strengthen cross-domain, interdisciplinary, and cross-border cooperation and exchange, and solidify an artificial intelligence governance consensus

  • What is your biggest cost driver for the model development/deployment

  • How will you integrate any technologies changes in the model algorithms

  • Have you plans to test the different standards / rules for your industry vertical

  • How did you assess that the application is ethical

  • Is there a way to turn off or roll back the model in production if necessary?

  • Do we test and monitor for concept drift to ensure the model remains fair over time?

  • Are you able to determine the responsible human promptly when harm occurs.

  • Have we taken steps to identify and prevent unintended uses and abuse of the model and do we have a plan to monitor these once the model is deployed?

  • Do you do ethics washing (eg. superficial measures are in place), ethics shopping (ie. choosing ethical frameworks that justify your actions)

2. External Resources - Tools to use

  • AI Incident Database by the Partnership on AI tracking incidents so that developers learn from previous mistakes and reduces the
    likelihood of recurrence.

  • DL Monitoring tool

  • Templates:

    • Risk Management chart

    • Biometrics to emotions mapper eg. e FACS

    • List of risks to consider

      • bias -

      • loss of jobs through automation,

      • human rights impacts of automated decision-making,

      • power imbalance

      • surveillance through mass data collection,

      • environmental damage,

      • Human safety including psychological

      • Manipulating to influence your behaviours, beliefs, and decisions.

3. Case Studies

  • Bias: in hiring,[2–7] health care, [8–17] , societal biases are being perpetuated or amplified [31–46]and criminal justice [18–30].

4. Further Reading