Task Representation Documentation

• Checklist / Questions to ask your Development team

  • Do you capture different versions of the model(s) in a model registry platform? eg. Weights & Biases, Comet.ML and Neptune.ai.

  • Do you capture what features are the most important for decision made

  • Do you have an internal audit plan at the different points in software development life cycle that is relevant

  • Do you have QA/Performance Testing plan with tasks needed to address is continuous regression testing with alerts to human flagged when thresholds are not maintained will require different tasks for the verification, validation, and accreditation eg. competitive model testing - training 2-3 Model algorithms on same data and verification is one method used to get around the intractability of complex systems. IE. individual components are verified, and if they all work on their own, they are assumed to work together.

  • Standardized document helps achiever transparency. How are you documenting your solution

  • Do you have an automated way to change the model if threshold are not met

  • Have you completed a Root cause analysis assessment which includes all components (humans, data, model and software) that are captured in AI

• External Resources - Tools to use

  • AI explainability test tools

  • IBM AI Explainability 360 toolkit aims at tackling some of the technical challenges of ensuring explainable

  • Templates(Sample)s)

    • AI Test and Performance Plan

    • AI Root Cause Assessment form

    • Versioning Information to be filled into sBOM

• Case Studies

  • ING case study

• Further Reading

  • All  Life cycle development models e need to be revised