09 Sep 2022, Ben Bland

Ethical Innovation in Practice: Our Method for Converting Expert Feedback into Product Specifications for an Ethical AI Explainability Toolkit

To build a brighter future, innovation, technology and entrepreneurship don’t just need ethics; we can also invert this relationship to say that the field of ethics equally needs innovation, technology and entrepreneurship to thrive.

In this post we lay out a summary of the process we have taken to build our product development requirements for xplAInr, by gathering feedback from experts and feeding it into a modern product innovation framework.

Expert Consultation

So far, for our feedback gathering exercise, we have spoken with subject-area specialists via an increasingly diverse range of channels. Besides one-to-one conversations, we have run some focus-group sessions, hosted a workshop at Rightscon, conducted an online survey, and given presentations with follow-up Q&As at relevant interest groups. Each modality brings different advantages and limitations. However, I think that the variety of engagement types is itself an additional source of value, because its inherent diversity helps to round out the package of critical data.

To date, these engagements have put us in direct conversation with over 70 experts, which I feel is an encouraging start. The most popular roles in that group include tech-related legislation/policy, engineering & computing, psychology/sociology, cybersecurity, and philosophy. The majority of these experts are based (although often not from) Europe and North America, which highlights a potential shortfall in cultural diversity but, as I say, it’s a decent start from which to grow.

Data Treatment

Besides the quantitative data from sources such as survey polls, we dump all salient qualitative input from the experts into a comment feed. On its way in, we clean it up for sense, brevity, redundant repetition and suchlike, but try to keep as much as possible of the original tone and intent, so we have one big, ugly, honest set.

Next, we categorise the comments for emerging themes. There is no science applied to this step, it is a case of trying to spot the trends and insights while not projecting too much personal bias on the source data. This is, I feel, a necessarily subjective process. With greater resources available to us, I would expect us to recruit a diverse range of trained people to review and annotate but that’s not currently realistic or necessary.

With our categorised comment feed now in hand, the challenge is to translate it into a source of guidance for specification of the development roadmap for xplAInr. There are many product development frameworks that could be applied to this step to facilitate this translation of information. None are perfect and all require customisation to the specific context of the project in question. I selected one framework with which I have had particular good experiences (in a tech startup-type context) which is called Jobs To Be Done (JTBD).

Here are a couple of sources for more info on JTBD:

JTBD overlaps with other modern innovation frameworks in various ways, particularly with its focus on empathising with the user (or other stakeholder) and basing work on stories about that person’s needs and issues.

So to complete this step, we have taken our (clean and categorised) expert comment feed and written a series of “jobs”, which are stories about what our stakeholders wish to achieve.

Lastly, from those user stories we now list various ways in which we might fulfil the “job”, such as by rewriting xplAInr content for clarity, adding new features, or restructuring the way components of our system are communicated. These emergent requirements can be big or small, and long- or short-term, and indeed it is these dimensions with which we then rank them. 

In Review

Expert, critical and well-timed feedback is as valuable as it is hard to acquire. The people with the deepest knowledge and experience tend to have the least time to spare, and everyone has their own preferences for how and when to deliver input. Then when you finally get that input, there is no perfect or concrete method for making the best use of it.

We have taken a product innovation framework from the entrepreneurial business world, and transposed it onto the somewhat more philosophical space of technology ethics. I hope that this process will prove to be an instructive and important example of a practice that our world greatly needs, and of which I hope we will be seeing a lot more soon – perhaps we can call it Ethical Innovation.

 

31 Aug 2022, Ben Bland

AI Explainability Considerations and how they Relate to Standards

Anyone building or investigating an AI-type system from an ethical perspective could get stuck navigating the mire of guidance and rules that relate to their work. Here we walk through some of the considerations faced in the design or analysis of a specific type of AI close to our own experience: so called “empathic AI” (covering affective computing, emotion AI, emotion recognition and related processes). We then go on to look briefly at how you might link this work to the guidance set out in related standards.

Common Attributes of AI Systems

First, let us briefly recognise the obvious point that there are shared aspects to the development lifecycle of any autonomous or intelligent system (A/IS), including empathic A/IS. xplAInr follows a common set of lifecycle steps through the major common stages, from design to deployment to decommissioning. We currently present 22 such steps, each as a “card” with more detailed information for further analysis by any stakeholder.

Providing a general framework for any A/IS presents an information challenge. If we only include processes and resources that are common to all such systems, it could be difficult for stakeholders to identify important factors that relate to their specific technology of interest. The current, early version of xplAInr that we have published is to some extent focused on empathic AI, which is representative of the framework’s provenance – having been developed by fellow members of a group dedicated to empathic AI ethics (IEEE P7014).

We have created a hierarchy of informational layers within the framework, to allow for every more granular examination. Following this hierarchy, stakeholders should be able to start with a view of a common AI system lifecycle, then dig down to lower layers to explore resources appropriate to the particular requirements of their system types.

Breakdown of an Empathic AI System

Now let us look at the specifics of a system that uses affective computing or similar emotion recognition or analysis – in other words empathic AI.

It is worth noting here that the IEEE P7014 working group, despite having a relatively diverse and expert population, continues at this time to struggle with unpicking the attributes of empathic technology that are truly unique and differentiated from general A/IS. We contend that there are indeed unique considerations. But unique or not, surely it is easy to agree that there are a handful of issues that are heightened in the empathic context. We will briefly examine some of these below, but for a deeper dive into the standardisation process in empathic AI, you could read my post on the IEEE SA Beyond Standards blog, 5 Issues at the Heart of Empathic AI.

Common Rights and Ethics

Starting with the fundamentals, empathic AI raises special concerns with respect to widely recognised human rights and basic principles of common ethical frameworks. For instance, we must question if the system impinges on a person’s right to keep their own thoughts and feelings private, to be protected from emotional manipulation, and to be safe from systems that could trigger psychological harm.

These kinds of fundamental concerns should obviously be addressed at a very early stage, before the system makes contact with a real audience. Some of the earliest cards in the xplAInr lifecycle point in the right direction, e.g:

C1: Resource Planning – urges the system designer to “Create ethical ‘red lines’ around types of system capacity.”

C3: Values-Based Procurement – includes “Establish a Human Rights plan.”

Contentious Foundations of Emotion Science

With empathic AI being built on a relatively young field of science, and dealing with highly subjective, subtle and culturally-variable attributes such as emotions, any stakeholder examining such a system needs to be sensitive to this contentious foundation.

C1: Resource Planning – asks for “Cultural training on behavioral science for all design & dev teams.”

Intimate Data, Inherent Bias and the Privacy Problem

Systems that measure or emulate human emotions and cognitive states come into contact with personal data that is arguably at a whole new level of sensitivity and privacy. As explained in Emotional AI: The Rise of Empathic Media, by fellow P7014 member Andrew McStay, empathic systems “feel into” intimate aspects of our lives – our once-secret moods and states.

Furthermore, within the data sets that underpin empathic AI, and the resulting machine-learning models that are applied to that data, we see a particular risk of biases creeping in. Psychological data sets for machine-learning typically suffer from the common problem of having been built on a population of white, middle-class psychology students. When we start to add assumptions about emotion, which is itself a highly subjective field, these biases can compound.

Such considerations can start to be addressed through xplAInr steps such as:

C2: Stakeholder Analysis – “Analyse cultural variables of each stakeholder (e.g. global user base).”

C12: Input/Output Benchmarking

  • “Dataset benchmarking: bias mitigation for public datasets, novel datasets, and third-party datasets.

  • Verifiable, transparent data underpinning “accuracy” claims.

  • Sensor calibration for diverse user bases (e.g. skin tones, accents, etc.).”

Educating Affected Parties on the Nature of Affect

Empathic AI entails strong, if not unique, considerations with respect to explainability and transparency. Of course, this line of enquiry is what underlies the entire xplAInr project, but let’s look at some specific points at which stakeholders should consider explainability within the system lifecycle.

Special care is required at the point that the system makes first contact with potentially affected parties, not just end users but also possibly others in their wider circle, and even the general public. Most people don’t have a sophisticated understanding of emotion science, or of course the technology and business behind automated and intelligent systems. Even the experts disagree about the nature of feelings and empathy. This problem is not an excuse for empathic AI to be presented to people without due explainability, it should be a spur to take extra steps in this regard.

C18: UX/UI Safety & Controls

  • “On-device user notifications for affective state data capture.

  • Push/serve user notifications to indicate algorithmic processes utilizing affective state data are operating.

  • User controls to toggle on/off collection/capture of affective state data and/or emotion recognition/detection?

  • Transparent display mechanics to avoid using manipulative UX interventions like dark patterns.”

Incorporating Empathic AI and Standards

Staying with our example of empathic AI, there is currently little in the way of published standards that specifically address this field. As with any new subject, the protective infrastructure of standards, law and suchlike are always racing to catch up. But we can work in theory here, to look at how xplAInr can be integrated with standards, and consider what a standard such as the as-yet unpublished IEEE 7014 would say about the system under examination.

As outlined in our recent post on How to Find the Tech Standards You Need, we are developing a library of standards that can be mapped to the xplAInr framework as any stakeholder works their way through it. With standards providing valuable guidance on how to create better, safer systems, our hope is that readers will be able to cross-reference the ethical considerations of the system-in-question against relevant standards, for timely guidance.

At time of writing, IEEE 7014 is still being drafted but we can make some assumptions about how it will map to the ethical analysis process that xplAInr directs. The standard can be expected to address the unique (or heightened) ethical considerations of empathic systems, such as those listed above. In each case, there is likely to be a set of normative statements (e.g. “you shall do X”) or guidance (e.g. “you should do Y”). The reader, whether they are a system designer, a policy-maker, or indeed a member of the public, should be able to check these norms and suggestions against the planned, claimed or realised impacts of the system.

IEEE 7014 is part of a family of standards that specifically tackle the nascent realm of AI ethics (as part of the IEEE Global Initiative on the Ethics of Autonomous and Intelligent Systems). This area is inevitably subjective and lacking in concrete models that we can all agree on. However, even established technical standards, which define very specific boundaries, such as which materials you can or cannot use for a specific type of product, can be interpreted or spoofed in various ways by those attempting to get round them while still claiming conformance.

Despite this room for unethical design or use, systems developers will have to adopt a reasonable level of responsibility for conforming to standards, as those standards can inform laws, as well as other auditing, review or certification activities. Just as the developer can read into the ethical norms set out in standards and other documents, investigating parties such as legislators or journalists can interpret the extent to which they believe the developer has reasonably adhered to them. In this way, even the more philosophical standards, such as those that deal with ethics, can still have teeth.

Looking Forward to Access for All

Going from here, our expectation is that we will continue to generalise xplAInr to the full gamut of autonomous and intelligent systems, while also seeking opportunities to enrich it with resources that are specific to particular niches within the field. These resources will include links to relevant standards – international, regional or specialised – to aid both the creators and interrogators of intelligent systems to adhere to practical ethically-aligned guidance.

All digital systems are in some way socio-technological. They have impacts: foreseen, foreseeable and unforeseeable. Increasingly autonomous and smart systems amplify the scale of these socio-technological impacts, and the need for their examination. And while explainability is just one principle of a many-sided package of ethical guidance, hopefully our framework is a good starting point.

 

How to Find the Tech Standards You Need: Our Method for Mapping Standards to the xplAInr Lifecycle for AI Development

Here we provide a summary of our approach to gathering the world’s technology standards and mapping them to the xplAInr lifecycle for AI development, so anyone can find quickly find the standards that are appropriate to each stage of the lifecycle.

The Place for Standards

Standards hold an important place in the socio-technical landscape, somewhere between guidance and law. Standards can provide value to anyone planning to build, or assess, a technical system. Thus, users of the xplAInr framework should benefit from our library of existing standards.

There are many standards for many different levels of use. They can be geographically limited (e.g. national) or global. They can be directed at general technologies, specific sectors, or a highly specific use such as a particular product. For mapping existing standards to xplAInr, we have started with global standards, which are specifically targeted at autonomous and intelligent systems. If it proves valuable, we aim to grow this library to include ever more specific standards.

This first version of the standards map was built with support from the European Union’s StandICT.eu project. Thus, it focuses on European and global standards, and does not cover other regions (e.g. Asia, the Americas or individual nations). The library currently draws from the following sources:

Global Standards

There are a small number of primary global standards development organizations (SDOs) for technology, including AI. This library covers:

European Standards

As listed by the European Commission as the Key players in European Standardisation, the library covers the European Standardisation Organisations (ESOs), which are:

The Standard Mapping Roadmap

Having listed any AI-related standards from the above organisations, we can start to categorise them for more convenient and advanced access. Ultimately, anyone considering the development of an AI system, whether they are designing one or assessing it from the outside, should be able to discover which published standards apply to each relevant aspect of the system development lifecycle.

But there are challenges in achieving this. Primarily, we are not aware of any consistent naming or categorisation system for any two SDOs, let alone all of them. Even within each SDO there can be significant variance in information architecture and publication strategies. Furthermore, the level of published information on each standard varies across SDOs, with some providing useful overviews or descriptive, plain-English document titles, while others are restricted to their proprietary document codes.

We are exploring how to work with the SDOs to develop such a resource, and are keen to hear your ideas on how to achieve it.