Tag: trust

Posted on

Safe and Trusted Artificial Intelligence 2021

post by Oliver Miles (2018 cohort)

Over three days from 12th-14th July 2021, I attended and participated in the Safe and Trusted Artificial Intelligence (STAI) summer school, hosted by Imperial College and Kings College London. Tutorials were given by leading academics, experts from British Telecom (BT) presented a session on industry applications, and I along with several other PhD students took part in a workshop speculating on AI interventions within the healthcare setting, presenting our work back to the wider group. In the following, I’ll summarise key contributors’ thoughts on what is meant by ‘safe and trusted’ in the context of AI and I’ll outline the themes and applications covered during the school I found to be most relevant to my own work. Two salient lessons for me expanded on contemporary efforts to reconcile accuracy with interpretability in models driving AI systems, and on efforts to systematically gauge human-human/human-machine alignment of values and norms, increasingly seen as critical to societal acceptance or rejection of autonomous systems.

When I read or hear the term ‘Artificial Intelligence’, even in the context of my peers’ research into present-day and familiar technologies such as collaborative robots or conversational agents, despite tangible examples in front of me I still seem to envision a future that leans toward science fiction. AI has always seemed to me to be intrinsically connected to simplistic, polarised visions of utopia or dystopia in which unity with some omnipotent, omniscient technology ultimately liberates or enslaves us. So, when it comes to considering STAI, I perhaps unsurprisingly default to ethical, moral, and philosophical standpoints of what a desirable future might look like. I obsess over a speculative AI’s apparent virtues and vices rather than considering the practical realities of how such futures are currently being realised and what my involvement in the process might mean for both me and the developing AI in question.

STAI began by addressing these big picture speculations as we considered the first theme – ethics of AI. According to AI professor Michael Rovatsos, ethical AI addresses the ‘public debate, impact, and human and social factors’ of technological developments, and the underlying values driving or maintaining interaction’ (2021). In a broad sense there was certainly agreement that ethical AI can and should be thought of as the management of a technology’s impact on contentious issues such as ‘…unemployment, inequality, (a sense of) humanity, racism, security, ‘evil genies’ (unintended consequences), ‘singularity’, ‘robot rights’ and so on (Rovatos, 2021).  An early challenge however was to consider ethics as itself an issue to be solved; a matter of finding agreement on processes and definitions as much as specific outcomes and grand narrative. In short, it felt like we were being challenged to consider ethical AI as simply…doing AI ethically! Think ‘ethics by design’, or perhaps in lay terms, pursuing a ‘means justified end’.

To illustrate this, if my guiding principles when creating an AI technology are present in the process as much as the end product, when I think of ‘safe’ AI; I might consider the extent to which my system gives ‘…assurance about its behavioural correctness’; and when I think of ‘trusted’ AI; I might consider the extent of human confidence in my system and its decision making’ (Luck, M. 2021). A distinction between means and end – or between process and goal – appeared subtle but important in these definitions: While ‘assurance’ or ‘confidence’ appear as end goals synonymous with safety and trustworthiness, they are intrinsically linked to processes of accuracy (behavioural correctness) and explicability (of its system and decision-making rationale).

In her tutorial linking explainability to trustworthiness, Dr Oana Cocarascu, lecturer in AI at King’s College London, gives an example of the inclination to exaggerate the trustworthiness in some types of data-driven modelling that ‘…while mathematically correct, are not human readable’ (Cocarascu, O). Morocho-Cayamcela et al. (2019) demonstrate this difficulty in reconciling accuracy with interpretability within the very processes critical to AI, creating a trade-off between fully attaining the two end goals in practice (Figure 1).

My first lesson for ‘doing AI ethically’ is therefore the imperative to demonstrate accuracy and explainability in tandem and without compromise to either. However, it doesn’t follow that this alone will ensure safe and trusted outcomes. A perfectly accurate and interpretable system may lead to confidence in mechanism, but what about confidence in an AI’s apparent agency?

In her tutorial ‘AI, norms and institutions’, Dr Nardine Osman talked about the ‘how’ of achieving STAI by means of harnessing values themselves. She convincingly demonstrated several approaches employing computational logic (e.g. ‘if-then’ rules) in decision making algorithms deployed to complex social systems. The following example shows values of freedom vs safety as contingent on behavioural norms in routine airport interactions expressed as a ‘norm net’ (Fig.2).

Serramia et al. visualise their linear approach to ethical decision making in autonomous systems, positioning conventionally qualitative phenomena – human values (e.g. safety) – as contingent on and supported by societal norms, e.g. of obligation to provide passports/forms (2018). Efforts to break down and operationalize abstract norms and values quantitatively (e.g. weighting by hypothetical preference, observed occurrence) demonstrate how apparent features of human agency such as situational discernment might become more commonplace in negotiating safe and trusted outcomes.  My second lesson and main takeaway from STAI’21 was therefore the imperative of sensitising AI, and design of AI, to the nuances of social values – distinguishing between value preferences, end-goals, social norms and so forth.

Lastly and significantly, attending and participating in STAI’21 has given me invaluable exposure to the practicalities of achieving desirable AI outcomes. The focus on ‘doing AI ethically’ has challenged me to pursue safety, trustworthiness, and other desirable qualities in my own work – mechanistically in terms of ensuring explainability of my methods and frameworks; and substantively, in terms of novel approaches to conceptualising values and positioning against social norms.

References

Cocarascu, O (2021) XAI/Explainable AI, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/xai-argument-mining/

Luck, M (2021), Introduction, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/event_category/summer-school-2021/

Morocho-Cayamcela, Manuel Eugenio & Lee, Haeyoung & Lim, Wansu. (2019). Machine Learning for 5G/B5G Mobile and Wireless Communications: Potential, Limitations, and Future Directions. IEEE Access. 7. 137184-137206. 10.1109/ACCESS.2019.2942390.

Osman, N (2021) AI, Norms and Institutions, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/norms-and-agent-institutions/

Rovatsos, M (2021) Ethics of AI, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/ethics-of-ai/

Serramia, M., Lopez-Sanchez, M., Rodriguez-Aguilar, J. A., Rodriguez, M., Wooldridge, M., Morales, J., & Ansotegui, C. (2018). Moral Values in Norm Decision Making. IFAAMAS, 9. www.ifaamas.org

Posted on

SoDis

Pepita Barnard is a Research Associate at the Horizon Digital Economy Research and has recently submitted her PhD thesis.

post by Pepita Barnard (2014 cohort)

I am excited to be working with Derek McAuley, James Pinchin and Dominic Price from Horizon on a Social Distancing (SoDis) research project. We aim to understand how individuals act when given information indicating concentrations of people, and thus busyness of places.

We are employing a privacy-preserving approach to the project data collected from mobile device WiFi probe signals’ data. With the permission of buildings’ managers and relevant Heads of Schools, the SoDis Counting Study will deploy WISEBoxes in a limited number of designated University buildings, gather the relevant data from the Cisco DNA Spaces platform, which the University has implemented across its Wi-Fi network, and undertake a gold standard human-count.

What are WISEBoxes? There’s a link for that here

Essentially, WISEBoxes are a sensor platform developed as part of a previous Horizon project, WISEParks. These sensors count the number of Wi-Fi probe requests seen in a time-period (typically 5 minutes) from unique devices (as determined by MAC address). MAC addresses, which could be considered personally identifiable information, are only stored in memory on the WISEBox for the duration of the count (i.e. 5 minutes). The counts, along with some other metadata (signal intensities, timestamp, the WiFi frequency being monitored) are transmitted to a central server hosted on a University of Nottingham virtual machine. No personally identifiable information is permanently stored or recoverable.

We will have ‘safe access’ to Cisco DNA Spaces API, meaning MAC addresses and other identifiers will not be provided to the SoDis research team. The data we gather from Cisco DNA Spaces API will be processed to produce information similar to that gathered by the WISEBoxes, i.e. counts of number of unique users connected to an access point in a period of time.

To develop our ‘busyness’ models, we will also deploy human researchers to count people in designated buildings and spaces. This human-counting element will provide a gold standard for said buildings, at the time of counting. This gold standard can then be modelled against data simultaneously produced from WiFi signal counting methods, producing an estimated level of busyness.

With the help of several research assistants, we will collect 40 hours of human-counting data, illustrating building activity over a typical workweek. We expect to start this human-counting work in the School of Computer Science Building mid-January 2021.

This gold standard human-count will include both a door count and an internal building count. For each designated building, we will have researchers posted at the entrances and exits to undertake door counts. The door counters will tally numbers of people going in and numbers going out within 5-minute intervals using + and – signs. On each floor, researchers will count people occupying rooms and other spaces in the building (e.g., offices, labs, atrium, corridors). Each space will be labelled by room number or name on a tally sheet. Researchers will do two rounds of their assigned floor per hour, checking numbers of people occupying the various spaces. Different buildings will require different arrangements of researchers to enable an accurate count. For example, to cover a school building like Computer Science on Jubilee, we will have 6 researchers counting at any one time.

We expect some of the data collected from the WiFi probes and connections to be spurious (noise), however this is not of concern. Why? Well, to represent busyness, we do not need to worry about exact numbers.

It is accepted that the data may not be accurate, for example, someone’s device may use or send a WiFi probe signal to an access point (AP) or WISEBox in the designated building who is not actually in the building. This potential for inaccuracy is a recognised feature of the privacy-preserving approach we are taking to model busyness for the social distancing tool, SoDis. The researchers undertaking the human-counting study may miss the occasional person roaming the building, but this level of error is not of particular concern. When the human-count is triangulated with the sources of WiFi data, a model of busyness for that space will be produced.

The approach we are testing is relevant not only to our current desire to reduce infection from COVID-19 but may also prove useful to support other health and social causes.