ML Deployment

• Checklist / Questions to ask your team

  • Which method being used to deploy your model(s) (i) Static deployment or embedded model is where the model is packaged into installable application software and then deployed or Dynamic deployment — where the model is deployed using a web framework like FastAPI or Flask and is offered as an API endpoint that responds to user requests.

  • Are you using a model registry to know what was deployed

  • How are you planning to deploy (i) on a server (ii)container,(iii) serverless deployment or (iv) model streaming — instead of REST APIs, all of the models and application code are registered on a stream processing engine like Apache Spark, Apache Storm, or Apache Flink.

  • Which Model monitoring tools do you use which identifies problems as a model transitions into the real-world. eg of metrics used are: ptime (availability), identifying model drift (loss of performance due to production data characteristics versus training data sets), flagging data quality degradation and more.

  • Do you use any tools associated with AI explainability, ie. to help understand the outputs of their AI/ML models.

  • Is there someone accountable on all decisions made by AI

  • Is there a process for challenging a decision and seeking readdress

  2. External Resources - Tools to use

  • MLCUBE

  • Tools in this space include OctoML, Algorithmia ) and Valoha

  • ML infrastructure and operations tools: Arize, Fiddler and WhyLabs. tools.

  3. Case Studies

  4. Further Readings