Drivers and Barriers to Digital Inclusion

post by Oliver Miles (2018 cohort)

Interning as an Embedded Research Associate with CityMaaS

 Finding the right internship – introducing CityMaaS

From July-September 2021, I had the privilege of working as an embedded research associate with CityMaaS, a London based start-up in the digital inclusion space. This opportunity arose after pitching my PhD at a Digital Catapult networking event for students and start-ups. I prioritised attending this event as I was especially keen to experience work in a start-up environment. In the weeks following, I was introduced to Rene Perkins – CityMaaS CEO and co-founder. We agreed that at the intersection of her work on digital inclusion, and my work on values-driven personalisation, there was scope and mutual interest for a research project uncovering the drivers and barriers to digital inclusion adoption. Over a series of conversations, we discussed aims and objectives, ultimately formulating some questions and a target population. As I continue to write up research findings, I’ll talk only very briefly about research method and content. The focus here is more on the process of co-creating the ‘right’ internship, doing research work as an intern, and working as an embedded researcher within an external company. After introducing key concepts and CityMaaS products and services, I’ll talk about research participants, rationale and outcomes, reflections on navigating a specifically ‘research orientated’ internship and plans for future work.

Concepts, products, and services

Digital inclusion is a far-reaching domain, but the focus of CityMaaS is specifically on applications of accessibility and mobility. CityMaaS software solutions include ‘Assist Me’ – a web tool for personalising the audio-visual and interactive content on websites; ‘Mobility Map’ – a mapping tool inclusive of machine-learning driven predictions of location accessibility and personalised route planning features; and ‘AWARE’ – an automated compliance checker, scoring and reporting a websites’ alignment with globally recognised web standards[1]. Improving accessibility online and offline is therefore, in a nutshell, the unifying objective for these solutions.

Who are digital inclusion solutions for?

People with additional accessibility and mobility needs – specifically those affected by conditions of visual, audio, physical and cognitive impairment – are ultimately the critical target end-users in terms of product interaction. Crucially though, they are not the clients: As a business to business (B2B) company, CityMaaS market and sell their solutions to public, private, and third-sector organisations with a view to improving their in-house digital inclusion offer; the general incentive being adding socio-economic value.

Why do corporate opinions and practices matter?

While the appeal of improving accessibility and mobility could/should be thought of as self-evident, if companies are to invest in bespoke solutions such as Assist Me, they need to not only be sure of its technical functionality, but confident it thematically aligns with their own conceptions of digital inclusion. Corporate clients therefore, were our population of interest for this work.

Research Activity:

My work combined designing, conducting, and analysing interviews with senior heads of digital from 3rd party organisations known to CityMaaS, with the aim of answering an overarching question, ‘what are the drivers and barriers to digital inclusion?’. Results – from thematic analysis of semi-structured interview transcripts, expressed as 10 value themes driving or hindering accessibility and mobility – would go onto inform the design of an ideation workshop (Fig 1).

Figure 1 – Digital inclusion themes

Interviewees and selected colleagues would then engage in specially designed accessibility and mobility solution ideation workshops, aligning features and functions of CityMaaS products with the 10 emergent themes (Fig 2).

Figure 2 – Software demo, product attributes & values-driven ideation exercise

Research Outcomes:

For CityMaaS, outcomes are aimed here at better defining the qualitative touchpoints for digital inclusion and discovering desirable uses for software, grounded in real-world values. For myself as a researcher of values-driven personalisation in the digital economy, this was a chance to explore emergent drivers and barriers as a values-orientated resource to digital solutions ideation in diverse corporate settings.

Reflections on navigating ‘researching’ and ‘interning’

There were several practical questions which required collaborative discussion with the CDT, most notably the nature of the partnership, data collection and storage, and my potentially conflicting status as both a doctoral research student and CityMaaS intern. We agreed the best framing of my status was that of an ‘embedded research associate’, as while I would be working alongside the CityMaaS team, my research would require university ethical clearance if results were to be useful to me in the wider PhD. As such, data collection and storage were conducted through university systems and protocol. I received no renumeration for my work with CityMaaS, with the prior agreement that research was explorative and not directly connected to business development activity on my part.

In terms of the nature of the work, the biggest challenges were those of project management, resourcing (providing interviewees, access to data) and ultimately ‘scope creep’. In terms of project management, mapping activities to a Gantt chart was personally beneficial, and I ensured that at numerous stages, there were deliverables which kept me accountable. For example, conducting initial requirements gathering sessions with CityMaaS business development and technical colleagues allowed me to hit an early goal of enumerating product features and functions, helping me to learn the product portfolio before interviewing participants later.

Delegation of activities also aided productivity where appropriate. As an example, my colleague in business development had much better access to interested and already connected corporate organisations than I did, meaning the substantive element of the internship wasn’t mostly generating participants.

The biggest danger though remained scope creep. Again, I found that effective project management and having short-term deliverables helped: I was able to complete internal requirements gathering, interview design, participant interviewing, analysis, and workshop design in the allotted 3 months’ time. Completion of ideation workshops though proved to be an ambitious final component; consequently, scheduled to take place in early 2022. While this remains realistic for me due to relatedness of the work to my PhD, if the project had to complete at any of the prior stages, outcomes were designed to be useful as standalone findings.

Conclusion & Future work

On reflection, I found the internship one of the most useful CDT activities for me in terms of both continued professional development and alignment with my own research interests. Moreover, I had never considered working in the digital inclusion sector before or had the opportunity to contribute to research in a start-up environment. As I complete workshops in February 2022 and write-up, I hope my findings are insightful to CityMaaS and useful to furthering my own understanding of values-driven consumption in the digital economy. I would also recommend the Horizon CDT network and partners at Digital Catapult, in terms of networking and finding bespoke internship opportunities.


[1] Based on the w3 (2018) Web Content Accessibility Guidelines (WCAG) 2.1 https://www.w3.org/TR/WCAG21/

 

 

 

 

 

 

 

 

[1] Based on the w3 (2018) Web Content Accessibility Guidelines (WCAG) 2.1 https://www.w3.org/TR/WCAG21/

Digital manufacturing, say what?

Reflection on developing a digital manufacturing toolkit for children

post by Natalie Leesakul (2018 cohort)

What is the first thought that comes to your mind when you think of manufacturing? Assembly lines? Boring and repetitive tasks? Loud noises?

Over the years, the manufacturing sector has evolved and moved more towards the age of digitalization. However, as the sector entered the era of Industry 4.0, skilled-labour shortages started to become a problem. The sector is experiencing the phenomenon where current employees are retiring while failing to attract new skilled employees. The manufacturing sector was not on my radar until I started my PhD at Horizon CDT (Grant No. EP/L015463/1) working in partnership with DigiTOP (Grant No. EP/R032718/1), and that has completely changed my perspective. This sector is filled with innovation. It is about forward thinking. At the core, there is a major element of maximizing productivity and efficiency by utilizing advanced technologies but also a shift towards ensuring seamless human-machine collaboration and contributions to sustainability and the Net Zero movement. After getting more involved in this sector in the past three years, the question that always comes up is how do we make manufacturing sexy? How do we inspire future generations to be interested in manufacturing?

As part of DigiTOP’s work, we are developing a toolkit to help organizations implement and integrate digital technologies. We provide research on the role of human factors and the impact of technology on the workforce along with use cases and recommendations on the technology adoption and design. One of the on-going toolkit developments is an educational set of resources focusing on the introduction to digital manufacturing aimed at a younger audience (primary and secondary children). I had an opportunity to be a part of the team in charge of developing the content from the start of the video production to running our very first workshop with students in collaboration with Speakers for Schools.

At the start, the first question to answer was how to transform a complex topic into something that is easy to comprehend by the children. We see digital manufacturing as “the use of smart, digital, autonomous and intelligent technologies within the manufacturing sector. These technologies include robotics, virtual and augmented reality, sensors and distributed data networks.” Certainly, there was quite a bit to unpack from this definition. Putting ourselves in the shoes of a 7-year-old, a short animated film was chosen as a communicational medium. Acknowledging that there was not any other existing child-focussed videos on manufacturing at the time, we decided to go forward with the plan and the video was produced in collaboration with Cloudifacturing and Digitbrain projects. The video production kicked off with choosing the right supplier. We searched for a video producer who really understood the brief and the video style of our preference and that required reading through multiple proposals as well as extensive discussion on animation style, colour schemes, voiceover artists, etc. Once we decided on the proposal, we provided the producers with an initial script that covered the objectives and key messages of the video including:

        • Introduction to general changes in industry 4.0/digital manufacturing
        • Cloud-based technologies and human factors considerations
        • Digital twin
        • Human physiological sensing
        • Human robot collaboration
        • Virtual reality in manufacturing and design
        • Responsible technology development and adoption

The challenge we had was explaining all of the above topics in a 3-4 minute video. We consulted with experts in child education to translate (boring) academic language into child-friendly and exciting dialogue. After many editions and revisions of the script, here is the final cut of the video:

The project did not stop at the video production. We are continuing to develop further content to support the use of the video as part of a school curriculum. For our first trial, we were invited to run a workshop through Speakers for Schools program under the weekly theme of digitalization. The workshop dove into the topics mentioned in the video in an interactive manner. We wanted to hear from the students while still making it fun and educational. The workshop was divided into three sections: 1. Subject and skills for digital manufacturing; 2. Sensors; 3. Humans & Robots. I was responsible for the third section. Creating workshop content for younger audience required a completely different approach to a preparation for a conference presentation. How do we gain their engagement especially in a virtual environment? What kind of activities needed to ensure positive engagement? How do we keep them entertained instead of dozing off? How do we make it simple but still interesting? We had various brainstorming sessions to find different ways to run this workshop. We found that to keep the same narrative in all three sessions, creating a fictional character called ‘Amy’ who is a digital factory manager was the way to go. Throughout the workshop, Amy faces different scenarios where she needs helps from the students to make decisions. We used tools such as running polls, storytelling, yes/no questions, and lots and lots of pictures!

In my session of human-robot collaboration, I asked the students to help Amy assign tasks to either human operators or robots.

I was very surprised by some of their answers. The answers were mostly robots even for making coffee! To me, this demonstrates the pro-technology nature of the younger generation. They seemed to be quite comfortable with the topic and it was quite refreshing to see positive engagement from the students. Although we are now living in the world of digitalization, the final note that we left the students with was data and ethics. It is important that we are aware of potential issues and impacts that may arise from technology in order to balance the interests of all stakeholders. Special thanks to the team: Dr. Glyn Lawson, Debra Fearnshaw, Dr Adrian Marinescu, Dr Setia Hermawati, and Siobhan Urquhart.

Thank you for taking the time to read my blogpost. I hope this sparks your interests in digital manufacturing and public engagement. My name is Natalie, Horizon 2018 cohort, and my PhD focuses on the adoption of collaborative embodied autonomous systems in the case of digital manufacturing from legal perspective. 😊

 

 

Call for Participants: Testing for Transparency

PhD researcher Ephraim Luwemba (2019 Cohort) is currently recruiting participants to take part in Testing for Transparency Interviews.

About the Interviews

The interviews form an important part of the Testing for Transparency project. By interviewing the people who design digital services, we hope to ensure it is relevant and that we have a realistic impression of the technical and institutional challenges faced when attempting to create transparent privacy policies.

Who can take part

If you currently work on digital services that are likely to be accessed by children or have worked on them in the past, we would appreciate your input. We are looking for professionals involved in all aspects of creating a digital service including but not limited to QA and user testing specialists, UI and UX designers, copywriters, content creators, project managers, project architects, and contract lawyers who have been involved in the drafting of website policies and terms of service.

Amazon voucher

We are offering a £15 Amazon voucher to interviewees as a token of thanks. If you prefer, we can also donate the same amount to a charity of your choice. To do so, please let us know which charity you would like us to send a donation to by e-mailing Ephraim.Luwemba@nottingham.ac.uk prior to your interview.

You can read more information about the project at Testing for Transparency Interviews 

Panel Discussion at the Interrogating Audiences event

post by Kadja Manninen (2018 cohort)

Coming from an arts management background with over a decade’s worth of work experience, for me, one of the main reasons for applying to the Horizon programme, was the course’s strong industry focus. This means that already in the application phase, students are matched with industry partners, with whom they collaborate actively throughout the PhD. My industry partner is The Space, a digital agency funded by the BBC and Arts Council England, whose mission is to help arts and cultural organisations to reach new audiences online. Particularly during the third year of my PhD, I have worked closely with my industry partner on various projects they are partners in. Through these projects and my own PhD research, I have had the opportunity and privilege to talk to around 40-50 arts organisations and independent arts professionals. The conversations have mostly centred around digital transformation, new digital business models, the needs of digital  arts audiences, and more recently, the impact of Covid-19 on the arts and cultural sector.

Therefore, I was thrilled, when the wonderful Helen Kennedy, Professor of Creative and Cultural Industries at University of Nottingham, invited me to take part in a panel discussion at the Interrogating Audiences online event the Institute for Screen Industries Research (ISIR) was organising on July, 9th, 2021.

The event brought together around 60 participants, both academics from different fields as well as representatives of various cultural organisations. The programme was composed of three panel discussions around the following questions:

      1. What questions do we have about audiences?
      2. How do we gather evidence to answer these questions?
      3. How would the answers be used to help design content production and distribution/ marketing strategies?

Below, I highlight some key takeaways from the three panels and my own experience as a panel member.

I was part of the first panel alongside Steward Terry from Broadway Cinema and Jennifer Hessler from University of Huddersfield. Our discussion focused on the many questions we have about audiences. Drawing on the interviews I have recently had with arts and cultural organisations, I decided to approach the topic from arts producers’ perspective.

It is increasingly clear that the Covid-19 pandemic has accelerated digital transformation and organisational decision-making within arts organisations. While the strict social distancing restrictions imposed by the UK government closed cultural venues repeatedly over the pandemic peak year, they also created room for change by allowing arts producers to stop and take time for rethinking their ways of operating and exploring new forms of digital engagement with their audiences.

During my interviews with arts organisations, many described the diverse and often experimental ways of audience engagement they were undertaking under the new circumstances, where in-person encounters were not allowed. The organisations that were already digitally mature at the start of the pandemic, were in the best position to tackle the challenge of reaching digital audiences. They were, for example, able to rapidly repurpose their existing digital content and broadcast it, which enabled them to remain relevant and not to be forgotten by their core audiences. These organisations also tended to be the ones that saw the pandemic as an opportunity to reach new, more geographically dispersed audiences, and thus began creating new types of content specifically designed for digital delivery. There are many examples in the sector, but I think one that is definitely worth mentioning is Opera North’s From Couch to Chorus project, which gathered over 6000 people from all over the world on Zoom to sing opera.

(https://www.operanorth.co.uk/whats-on/from-couch-to-chorus/)

Some arts organisations engaged in unprecedentedly bold experimentation with digital content, and launched new digital products such as online escape rooms or at-home XR experiences. Organisations also upskilled internally, i.e. staff members rapidly learned, for example, the art of online teaching or performing to the camera. More recently, there has been an emergence of digital platforms introduced by arts and cultural organisations, a phenomenon that I think no one could have anticipated pre-pandemic. Another successful example from the sector is Rambert Dance, who have done particularly well with their Home Studio platform, creating an inclusive, global online community around their work, and Darkfield, who through an app-based approach have reached large international audiences for their at-home immersive audio experiences.

Steward Terry from Broadway Cinema agreed that also in the film industry, the engagement with new and existing digital audiences has been something that they have been actively developing, and they as well, launched their own on-demand platform, where selected films can be rented. He underlined that is has been key for Broadway Cinema to determine what makes them unique to motivate audiences to engage with them particularly during the pandemic. Interestingly, he also observed that after the re-opening, their audiences have returned, however, many seem to be somewhat younger than the audiences they had pre-pandemic.

The second panel centred on the different techniques of collecting data from audiences. It was great to see how mixed methods research is now becoming much more frequent in the arts. For example, Bridgette Wessers described the AHRC-funded “Beyond the Multiplex” project, which through a mixed-methods approach aimed to understand how audiences engage with main stream film and how can the cultural value of not-so-known films can be optimised. She emphasised that one dataset does not give you the whole picture, but its importance to approach the phenomenon from different viewpoints.

Richard Broughton from Ampere Analytics shared his knowledge on commercial data collection methods, which they use to evidence audience behaviour. These included financial information on how much audiences are spending, large-scale quantitative surveys and interviews as well as collection of title-level tracking. Listening to Richard’s presentation I couldn’t help wondering that it would be great if there was more collaboration between academics and commercial analytic agencies so that these efficient methods could be also accessible to academic researchers. Erin Sullivan from The Shakespeare Institute described interesting social media research data collection methods such as sentiment analysis and particularly the Netlytic software for collecting and analysing tweets and YouTube posts. This is something that I plan to explore more in detail in the near future.

The final panel focused on the data-driven approaches that inform content production and distribution strategies. Ayesha Taylor-Camara, PhD candidate from University of Nottingham gave a wonderful introduction to her research that explores the value of the BBC. Her presentation helped to understand the role media plays in people’s lives and how a cultural organisation’s value is formed based on the interactions and experiences with their audiences. Matt Locke from Storythings introduced a new (at least to me) concept of “anectdata” that is not necessarily captured by traditional data collection methods, but bears an increasing importance to broadcasters. He also underlined the significance of understanding the churn caused by digital technologies in audience behaviour. Rachel Shaw from the BBC agreed that the pandemic has accelerated change in the TV industry. For instance, the way BBC assesses value of content has changed from vertical to horizontal, since the distribution method has changed from a singular moment to a continuous event.

As the three-hour event approached its end, the closing discussion concluded around the overarching theme of change. Liz Evans from University of Nottingham drew attention to how change and trends impact audience research, and whereas in the industry audience research can be fast-paced, change and trends are often difficult to capture through academic research. Liz also made an important point about cultural audiences not being siloed in one sector, as people who watch television also commonly go to cinema and theatre. Therefore, she called for more inter-sectoral collaboration and understanding that changes in the film and TV sector can also impact other sectors, such as the performing arts.

Indeed, I think there are important lessons we can learn from understanding different audiences from different sectors. There are universal challenges and common issues that Covid has made more apparent, as Liz stated, pushed certain sectors, such as performing arts to a direction they wouldn’t have gone naturally.

To finish, Helen Kennedy pointed out another important question that is pertinent to all cultural sectors and has repeatedly come up in my own research: How can we reach audiences we know nothing about, and how can we create content for them? During the course of the pandemic, it has been easier for audiences to try something new and suddenly become, for example, dance audiences or opera audiences. If new audience demographics keep turning up at online events – whereas they wouldn’t attend this event in a live setting – what does that tell us about the barriers of access to these art forms or cultural institutions?

 

 

Digital Business Model for Performing Arts at BAM2021 Conference

post by Kadja Manninen (2018 cohort)

In September 2021, I presented our paper ‘Digital Business Model for Performing Arts’ at The British Academy of Management (BAM) conference as part of the Creative and Cultural Industries track. I wrote this paper together with my supervisory team; Chris Carter, Andrew Leyshon and Sarah Martindale following the data collection of my first PhD study, which aimed to identify digital business models present in the UK arts and cultural sector, and provide an overview of the stage of digital transformation in the sector.

The interview data was gathered through eighteen semi-structured expert interviews with experienced arts and cultural professionals, who were working for sector support organisations, large and medium-sized cultural organisations and as independent entrepreneurs or consultants.

The goal of the paper was to highlight ‘best practise’ for enabling the adoption of digital business models by UK arts and cultural organisations as well as shed light to most common barriers preventing the adoption of digital business models. I submitted the paper in February 2021 and it was accepted in May 2021. Particularly the second reviewer provided some excellent feedback that I tried to incorporate as much as possible over summer 2021. The conference was hosted by University of Liverpool, but it took place online for the second time this year. My presentation was on September 2nd, and I was also chairing the Creative and Cultural Industries track session.

Overall, the conference experience was very positive. Our session had four papers and ten academics participating from all over the world, e.g. from the UK, France, Brazil and Kazakhstan. I received very positive feedback on the paper and met some interesting academics, with whom I have already been in touch regarding potential future collaborations. I was also asked by the track chair Dr Martha Bernal, whether I wished to be more involved with the track in the future, and this also has already been taken forward in the SIGs annual meeting.

This paper will not feed directly into my PhD thesis, but the analysis of the same dataset in relation to my research questions will feature in my findings. However, perhaps the most valuable outcome from writing the conference paper was that approximately one month after the conference I received an email from a commissioning editor working for a prestigious academic publishing house. He had spotted our paper at the conference and wanted to know whether we thought that the paper had the potential to be developed into a book. While a book project initially sounded somewhat overwhelming, together with my supervisory team, I’m now in the process of working on a proposal, which will be submitted to the publisher, and if accepted after peer-review, can lead to a book contract. This is definitely very exciting news, and hopefully, our book on digital business model for arts and cultural organisations will see daylight at some point in the upcoming years.


 

Not-Equal Summer School

post by Jimiama Mafeni Mase (2018 cohort)

I participated in Not-Equal Summer School, a virtual summer school about social justice and digital economy. The summer school ran from the 7th of June to the 11th of June. It was designed to equip participants with tools to understand and support social justice in this digital economy. Participants were grouped into teams according to their research or career interests (i.e. urban environment, health & care, eco workers & labour, public services, and education & technology), to explore existing and emerging technologies and examine how power and social justice evolve with these technologies.

The first day was simply an introduction with a talk about the evolution of social justice in digital economies and machine learning. The key speakers presented some major topics about the use of Artificial Intelligence (AI) systems for social justice such as the relational structure of AI, the AI modelling pipeline, and the influence of AI and Big Data on human rights. A common issue among the talks was around machine learning model predictions’ interpretability, reliability and bias, and the complexity of the data used to train the models as data are usually collected at different stages in the pipeline.

The second day consisted of workshops that explored the use of gestures to express the consequences of power and social injustice in our work places, research and lives. Later, the identified gestures were used to propose designs of new utopian technologies. My team proposed a recruitment technology that considers hope, transparency and fairness as candidates usually face racial, gender and age bias and discrimination. Our gesture was ‘finger crossed’, which signified our hope for a recruitment technology that will be fair and transparent in its recruitment process. The day ended with an interesting webinar about human and collaborative work-practices of data science to improve social justice in AI.

Digital commons was the topic of Day 3. Commoning is the collective and collaborative governance of material resources and shared knowledge. During the day, we were required to define a common in our areas of interest. My team examined commoning of hygiene and health at the community level. We identified key actors in this space as health care professionals, sanitation workers, and residents. We identified key barriers in the implementation of such a common such as the impact of different jobs and responsibilities, different working schedules, and partnerships with external stakeholders e.g. the city council or NHS. We concluded the workshops by presenting ways of ensuring the success of our proposed digital commons, such as, creating rules and procedures to guide the behaviours of the actors, and emphasised that the rules need to be collectively developed. The day ended with a webinar about making data work for social justice.

The themes of the fourth day were systems change & power dynamics, and working culture. We explored the challenges and opportunities in working cultures and power dynamics to support social justice. Key challenges identified were working with senior stakeholders, managing external partners, limited funds and budget, project deadlines, and resource availability. Later, we discussed methods of improving working cultures and power dynamics such as bringing stakeholders together, confidence to speak up, adopt perspectives that do not necessarily come in research, creating allies, and rapid prototyping. We also proposed that institutions introduce power dynamics and working cultures training courses. The day ended with a webinar about using imagination and storytelling for social transformations and social movements. The speakers emphasised  the importance of visualising the kind of futures we want or imagine.

The summer school finished with two intensive workshops about ‘design fiction’.  That is, research and prototyping design fiction methods for the digital world to envision socially just futures. My team focused on a design fiction for the community, where members of the community could have equal opportunities to care, knowledge, and support with the use of community cobots. The cobots will act upon encrypted information with no personal data, to assist members of the community. We imagined such a cobot will not have access to any personal or individual information, and all members will have equal rights and responses from the cobot. These utopian brainstorming and imagination workshops were a great way to close the summer school. During the last hours of the day, we shared our thoughts about the summer school and each participant was asked to summarise their experience with three words. My words were ‘collaboration’, ‘fairness’ and ‘power’.

It is important to mention that we used Miro throughout the summer school. Miro is a whiteboard and visual collaborative online platform for remote team collaboration. It was my first encounter with the platform but familiarising myself with it was not difficult.

 

 

 

 

 

 

 

 

 

Safe and Trusted Artificial Intelligence 2021

post by Oliver Miles (2018 cohort)

Over three days from 12th-14th July 2021, I attended and participated in the Safe and Trusted Artificial Intelligence (STAI) summer school, hosted by Imperial College and Kings College London. Tutorials were given by leading academics, experts from British Telecom (BT) presented a session on industry applications, and I along with several other PhD students took part in a workshop speculating on AI interventions within the healthcare setting, presenting our work back to the wider group. In the following, I’ll summarise key contributors’ thoughts on what is meant by ‘safe and trusted’ in the context of AI and I’ll outline the themes and applications covered during the school I found to be most relevant to my own work. Two salient lessons for me expanded on contemporary efforts to reconcile accuracy with interpretability in models driving AI systems, and on efforts to systematically gauge human-human/human-machine alignment of values and norms, increasingly seen as critical to societal acceptance or rejection of autonomous systems.

When I read or hear the term ‘Artificial Intelligence’, even in the context of my peers’ research into present-day and familiar technologies such as collaborative robots or conversational agents, despite tangible examples in front of me I still seem to envision a future that leans toward science fiction. AI has always seemed to me to be intrinsically connected to simplistic, polarised visions of utopia or dystopia in which unity with some omnipotent, omniscient technology ultimately liberates or enslaves us. So, when it comes to considering STAI, I perhaps unsurprisingly default to ethical, moral, and philosophical standpoints of what a desirable future might look like. I obsess over a speculative AI’s apparent virtues and vices rather than considering the practical realities of how such futures are currently being realised and what my involvement in the process might mean for both me and the developing AI in question.

STAI began by addressing these big picture speculations as we considered the first theme – ethics of AI. According to AI professor Michael Rovatsos, ethical AI addresses the ‘public debate, impact, and human and social factors’ of technological developments, and the underlying values driving or maintaining interaction’ (2021). In a broad sense there was certainly agreement that ethical AI can and should be thought of as the management of a technology’s impact on contentious issues such as ‘…unemployment, inequality, (a sense of) humanity, racism, security, ‘evil genies’ (unintended consequences), ‘singularity’, ‘robot rights’ and so on (Rovatos, 2021).  An early challenge however was to consider ethics as itself an issue to be solved; a matter of finding agreement on processes and definitions as much as specific outcomes and grand narrative. In short, it felt like we were being challenged to consider ethical AI as simply…doing AI ethically! Think ‘ethics by design’, or perhaps in lay terms, pursuing a ‘means justified end’.

To illustrate this, if my guiding principles when creating an AI technology are present in the process as much as the end product, when I think of ‘safe’ AI; I might consider the extent to which my system gives ‘…assurance about its behavioural correctness’; and when I think of ‘trusted’ AI; I might consider the extent of human confidence in my system and its decision making’ (Luck, M. 2021). A distinction between means and end – or between process and goal – appeared subtle but important in these definitions: While ‘assurance’ or ‘confidence’ appear as end goals synonymous with safety and trustworthiness, they are intrinsically linked to processes of accuracy (behavioural correctness) and explicability (of its system and decision-making rationale).

In her tutorial linking explainability to trustworthiness, Dr Oana Cocarascu, lecturer in AI at King’s College London, gives an example of the inclination to exaggerate the trustworthiness in some types of data-driven modelling that ‘…while mathematically correct, are not human readable’ (Cocarascu, O). Morocho-Cayamcela et al. (2019) demonstrate this difficulty in reconciling accuracy with interpretability within the very processes critical to AI, creating a trade-off between fully attaining the two end goals in practice (Figure 1).

My first lesson for ‘doing AI ethically’ is therefore the imperative to demonstrate accuracy and explainability in tandem and without compromise to either. However, it doesn’t follow that this alone will ensure safe and trusted outcomes. A perfectly accurate and interpretable system may lead to confidence in mechanism, but what about confidence in an AI’s apparent agency?

In her tutorial ‘AI, norms and institutions’, Dr Nardine Osman talked about the ‘how’ of achieving STAI by means of harnessing values themselves. She convincingly demonstrated several approaches employing computational logic (e.g. ‘if-then’ rules) in decision making algorithms deployed to complex social systems. The following example shows values of freedom vs safety as contingent on behavioural norms in routine airport interactions expressed as a ‘norm net’ (Fig.2).

Serramia et al. visualise their linear approach to ethical decision making in autonomous systems, positioning conventionally qualitative phenomena – human values (e.g. safety) – as contingent on and supported by societal norms, e.g. of obligation to provide passports/forms (2018). Efforts to break down and operationalize abstract norms and values quantitatively (e.g. weighting by hypothetical preference, observed occurrence) demonstrate how apparent features of human agency such as situational discernment might become more commonplace in negotiating safe and trusted outcomes.  My second lesson and main takeaway from STAI’21 was therefore the imperative of sensitising AI, and design of AI, to the nuances of social values – distinguishing between value preferences, end-goals, social norms and so forth.

Lastly and significantly, attending and participating in STAI’21 has given me invaluable exposure to the practicalities of achieving desirable AI outcomes. The focus on ‘doing AI ethically’ has challenged me to pursue safety, trustworthiness, and other desirable qualities in my own work – mechanistically in terms of ensuring explainability of my methods and frameworks; and substantively, in terms of novel approaches to conceptualising values and positioning against social norms.


References

Cocarascu, O (2021) XAI/Explainable AI, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/xai-argument-mining/

Luck, M (2021), Introduction, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/event_category/summer-school-2021/

Morocho-Cayamcela, Manuel Eugenio & Lee, Haeyoung & Lim, Wansu. (2019). Machine Learning for 5G/B5G Mobile and Wireless Communications: Potential, Limitations, and Future Directions. IEEE Access. 7. 137184-137206. 10.1109/ACCESS.2019.2942390.

Osman, N (2021) AI, Norms and Institutions, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/norms-and-agent-institutions/

Rovatsos, M (2021) Ethics of AI, Safe and Trusted AI Summer School, 2021 https://safeandtrustedai.org/events/ethics-of-ai/

Serramia, M., Lopez-Sanchez, M., Rodriguez-Aguilar, J. A., Rodriguez, M., Wooldridge, M., Morales, J., & Ansotegui, C. (2018). Moral Values in Norm Decision Making. IFAAMAS, 9. www.ifaamas.org

Reflection of Internship Experience

post by Vanja Ljevar (2017 cohort)

In today’s post I wanted to share with you some of the experiences I had during my internship with Boots, which took place from December 2020, to March 2021. The project I worked on was based on an idea that originated in Boots, a company named ‘winner of the Responsible Business of the Year’ – and for a good reason. The topic of the project is powerful, delicate, stigmatized and close to the heart of everyone involved in the project. It was called: Menstrual pain and prevalence of period poverty in England. The idea was that menstrual pain can be detected through the content of customers’ baskets. I had to classify products that would help us create a definition of menstrual pain and once that was done, I was able to expand the research and go from individual baskets to the level of English districts. This meant that we could measure the prevalence of menstrual pain, the correlations with other factors, such as deprivation and, ultimately – we could get ideas where period poverty exists. Once finished, this project was transformed into a paper that is now going through reviews and final touches.

Even though I didn’t physically go to Boots offices nor did I meet people face to face, I still managed to get a sense of intrinsic cultural values and underlying ideas that exist in Boots and the bottom line was: they really care. They care about their customers, they care about the optimal use of their data and they care about making a change. Considering the relevance of period poverty at any time and especially during an epidemic, it is needless to say this project was important. My results could potentially lead to changes of policies, big claims in the media and it was of utmost importance to get things right.

As it is the case with any internship, it is always one of the most exciting parts of Horizon CDT PhD: for those without real professional experience this is a great chance to learn about ‘the industry’, for those who (like me) already have years of working behind them, this is a chance to expand their horizons and make contacts. Unfortunately, my internship started when we were in the middle of a lockdown, working from home, isolated, worried and overwhelmed with already pressing PhD work. However, through all the ups and downs, there were lessons to be learnt during the 4 months of my internship with Boots. I decided to share these insights, as they came to me: stories I made through learning from mistakes, whilst coding, connecting the dots in my head or even painting my kitchen. 

So, without further ado, here is what I learned through the process.


ABOUT THE START: KILL YOUR DARLINGS

Every project starts with an idea of what can be done. 

However, if you are anything like me, you will start with *many* ideas.

But not everything should be done and nothing can be finished unless we prioritise.

There always comes a moment when we need to stop. Draw a line and say: no more ideas.

As hard as it sounds: sometimes we need to stop working on things that we enjoy, in order to focus on things that are more important.

In other words: no matter how much you love them – kill your darlings, because: no one will know what you do, until you actually do it.


This was the hardest part of my internships: applying data science to a complex topic means that there are many different ways in which we can create insights. However, I had only a limited number of resources at my disposal and in order to create meaningful research, some hypotheses remained just that: hypothesised. 


ABOUT THE BEST KIND OF WORK 

‘Crocodile tears’ come from the ancient Greek anecdote that crocodiles would pretend to weep while luring or devouring their prey.

But here is the thing: even if some tears look the same, research shows that there is a significant difference between tears in terms of their chemical composition.

Rose-Lynn Fisher took a series of microscopic photos that prove: tears of grief are different to tears of joy that are also different to onion tears.

So, here is my point: whatever you do, try to do something you truly believe in.

Whether it is a research project, a hobby or the job in the industry, the difference between people who really enjoy what they do and who do not is real.

Just as they can recognise crocodile tears, people somehow *know* whether your work and efforts are genuine.


I was lucky to work on a project that genuinely meant something to me. Menstrual pain and in particular, period poverty, and the idea that there are still women and girls who in today’s day and age cannot afford menstrual products is something that was very moving and close to the hearts and minds of everyone involved in the project. Personally, I believe that research can be the first step in changing the world’s biggest problems, but it doesn’t always have to be extremely significant. What it does have to be is: relevant to you. Therefore, when choosing (or accepting the internship topic),  be true to yourself, otherwise it will all end up in tears. (The true ones)


ABOUT TWO TYPES OF PROJECTS

Peter Thiel says there are 2 types of progress:

➡Horizontal progress — a type of progress that occurs from being a copycat. 

It can also mean an innovation of existing things, but expanding mostly in scale.

⬆Vertical progress — this is achieved by doing something new. 

Vertical progress is more difficult to achieve because it is an endeavor that was never done before.

Needless to say, we need both.

But vertical success is the one that gets you, well, higher.


I think it is very important to decide what kind of progress this internship should be for you. 

You will create value for a company, but you should also think about what kind of value you will create for yourself and your PhD. I was lucky to work on a project that interested me, however, it was completely unrelated to my PhD research. Therefore, I decided at the very start of the project that I will aim for vertical progress, creating something new, however – in case the project starts to develop and grow, I will (have to) be happy to give it up and help someone else continue the work. This is because I knew that I only had a very limited amount of the time on the project and that my PhD matters more. The truth is, some of us are inherently more prone to vertical thinking, but no matter how hard it is, we sometimes need to focus more on horizontal progress (in my case, finishing my PhD), than vertical progress (creating something new for the company).


ABOUT THE TOPIC

Not too long ago they discovered another painting, allegedly made by Van Gogh.

By many accounts, it wasn’t a pretty painting, not Van Gogh style and not something that would catch any critic’s eye.

But what was interesting about this, was that they really struggled to conclude whether it was a Van Gogh or not.

For the purposes of my story, it actually doesn’t even matter what it was in the end.

What matters though is this: during the investigation, the value of that painting was literally changing from one moment to another – from being worth several million (if it really is him) to virtually – nothing.

But the painting itself was always the same.

And this is the power of our perceptions.

Perceptions make us change our actions from logical to irrational.

And we rarely can estimate the real value of something only based on its functionality.


How important your work (or research) is, often depends on its perceived value by your stakeholders: the company, the government or even your supervisors. The trouble is – some topics can be extremely impactful, but other people might not think that. This is often where the power of communication steps in, and your ability to translate academic, ‘heavy’ content into something that others can perceive as impactful and valuable for them.

Ironically, the research about menstrual pain is so unexplored today because researchers did not perceive it as an important topic.

Many doctors do not perceive menstrual pain as important, because there is not enough research about it.

Many women do not perceive menstrual pain as something they should complain about to their doctors because – even if they do, chances are their doctors won’t perceive this as an issue.

Menstrual pain is not researched enough, because not enough people complain about it and treat it as an important issue.

And the vicious circle continues until we stop it and explain why this topic matters.


ABOUT STUPID QUESTIONS

One of the greatest advice I received recently was this: ‘Whenever you’re in doubt about something and you don’t know how to approach it, ask it as a stupid question.’

There are several reasons for this.

♟️Questions are always less threatening than statements that could be perceived as risky – particularly in front of people we don’t know that well.

♟️90% of the time everyone else is thinking of the same question and is too embarrassed to ask it.

♟️There is a lot of noise in our communication and asking dumb questions makes sure we land on the same page.

This is especially relevant during working from home when it is so easy to misinterpret something or simply – miss the huge part of communication we do not get via Zoom, by observing body language. Therefore, there IS such a thing as stupid questions – we should use them as our superpower.


ABOUT THE POWER OF TEAM

Anton Babinsky syndrome is a rare condition: people are blind or deaf.

But what’s strange about this is that these people are completely unaware of this.

And adamantly deny they are blind.

They confront the evidence of their blindness through lies they say to themselves and others.

I do wonder if we can find a parallel to this syndrome in the class of people who are clueless about their own shortcomings.

Some people are making mistakes, but they are not even aware of it.

Some clever people do stupid things, without even thinking twice about it.

Sometimes we simply don’t know what we don’t know.

But unlike Anton Babinsky patients, who suffer from a rare brain damage, we are able to prevent our own mistakes: by relying on other people in our team.

This is the power of constructive criticism and discussion – we all have a blind spot.

But a second opinion could help us realize it.


I realised the power of this during my internship when I was explaining in one meeting that we could extrapolate someone’s need for painkillers based on their purchases. I was about to make a wrong definition when the Boots project manager said that some painkillers have different packaging sizes and that could mean they simply last longer. Which why, of course, some women might seem like they do not need to buy painkillers, when in reality they have stacks of them at home. It seems obvious now, but I did not think of this before.


ABOUT PROCRASTINATION

During one work day I decided that instead of working on my final report, I would – paint my kitchen.

And there I was, doing a completely unnecessary job for 5 whole hours.

I was so tired in the end that I couldn’t do any other work and I just called it a day.

It got me thinking – why is it that it’s so hard to work sometimes?

Is it because we’re afraid of not meeting our super high standards – so we’d rather not start at all?

Is it because the circumstances are not ideal?

Or is it simply because we’re lazy?


I knew that if I really need a day off, I can just take it. However, I also knew that I needed to stop procrastinating and this was a strategy that I developed during my internship. I created so-called “microgoals”, where for each big task I planned to do on a particular day, I would create much smaller tasks and break them down as much as I could into smaller and smaller goals. Whether it was simply opening the excel sheet, writing a plan of action or even just writing the title – it’s still progress that I can be proud of. This is especially important during the pandemic, when the lines between work and free time can be so blurred that the work could easily take over a whole day and as a result –  seem like a daunting task the next day. This is why I tried to remind myself every day that every journey starts with a small step. And taking a day off to paint the kitchen is also an important step in this journey.


ABOUT THE END

Have you ever felt like you are at the end of your strength, but kept trying?

Whether it was that one more push in the gym or the final hour of your report, you went beyond and above what seems your natural limit and you persevered.

If you did, you’ve experienced something that the Finish call: sisu.

This is a word that does not have a direct translation to English, but represents the mental strength to continue even after you reach the limit of your ability.

The Finish created a whole ideology around sisu and they perceive it as the only thing that got them through the Winter War – and helped them regain their reputation and territory.


I know that many of us feel that 2020 has taken a lot from us. Whatever it is you do: writing your thesis, trying to get the data for that next study, planning your internship, or simply giving it another push to keep it all together in the midst of a pandemic, just remember the words of another war professional who knew the artistry of sisu: ‘if you’re going through hell, keep going.’ However, regardless of your situation, always remind yourself that everything we do during this PhD (including the internship) should be things we enjoy, things that teach us to be more competent and resourceful researchers and that in the end – one day we’ll look back at this short time we spent at CDT and think how lucky we were to have such a great opportunity.


If you would like to read more about my research on menstrual pain or just chat, just drop me a line at vanja.ljevar1@nottingham.ac.uk and if you would like to read more of my stories, visit me at instagram: @tryanglist

 

Reflection of a Colloquium

post by Peter Boyes (2018 cohort)

As part of the programme with my industry partner Ordnance Survey (OS), each year I attend what they call a Research Workshop. It’s a multi-day trip down to their headquarters in Southampton, where they host all their sponsored PhD and Post-Doc students for a colloquium from their partner universities and programmes, both in the UK and a couple from abroad. The days consist of presentation sessions broken into themes of research, these presentations are given by each of the sponsored researchers to an audience of the other colloquium attendees and OS staff who drop in to relevant and interesting themes or talks over the days. In the breaks between presentation sessions there are poster sessions, each student presenting a poster of their work and able to talk with staff or other attendees there. These posters are also displayed over the course of the event to enable staff to drop by and take a look while they may be unable to attend a full presentation session, note questions and get in touch by email or later on in a break when the researcher is free. In addition there’s often a keynote speaker that kicks off the morning session talking around the general theme for each day.

As an annual event I have been able to attend at different stages of my PhD, and see progression across the visits. My view of the purpose of the event changed over appearances, and so did my confidence in my topic and myself. The conference-style event, presenting a poster, giving a talk, handling a Q&A with OS staff and fellow postgraduate researchers gave me a chance to learn from people going through the same process and some advice from them at their different stages of the postgraduate timeline. Over multiple poster sessions I honed the elevator pitch of my research for that year, and developed an understanding of my blind spots, the recurring questions that obviously I hadn’t anticipated or covered well enough in the poster, while developing my communication skills to multidisciplinary audiences. This was an opportunity to see others’ work that was similar to my field in different ways, and to practice communicating the research I was hoping to do or had done at the time of the workshop.

There is something to be said for not having any supervisors there, a little bit of a shock for me in my first year still settling into the doctoral training program at Nottingham. The student-supervisor relationship is a valuable one when navigating a PhD, but at this event I felt truly independent. At similar style events such as our Horizon CDT retreat I feel like even if they don’t contribute in my presentation, my supervisors are there in the background in the room or on the Teams call and may step in with comments or questions to jolt me along or help, but this wasn’t like that. This was more akin to what I expect conferences to feel like as I prepare to attend one and present later this year. Their contribution is there in the work, but I must be able to present and discuss the research as an independent researcher.

The event and this write-up gave me an exercise in reflecting on what stage I am at in my research. My first time attending, I was in the first year of the course, 5 months or so into my PhD and hadn’t exactly done an explicit research activity or carried out a study to talk about, I was still finding my feet. In that year, I talked mostly about my higher education background, my interests in a wide scope, essentially proposing questions I could explore and using the session to gauge some feedback on areas others thought could be interesting. This included areas to explore or advice on going down those paths, suggested literature or studies. Helpfully at this OS workshop there was an industry perspective on the applications and not just the theory or literature side or presentations.

In the next year, I could see for myself when making my presentation that my scope was narrowing, I was settling into an academic area, research questions were emerging less fuzzy, more defined even if not settled on at that point still. With the audience I was more engaged in discussion of conducted or planned studies and details of these, and looking towards potential research output goals and again the applicability to other sectors and industry.

With one of these trips to Southampton left to attend in my final run to thesis submission I will hopefully be in early write-up stages, and will be able to demonstrate some really interesting findings from this last year and my final study, and engage with those in their first years attending the workshop about their experiences in the PhD journey to that point.

To bring this to a conclusion, I would encourage postgraduate research to look for these colloquiums/consortiums even if not offered by your industry partner as they can help you engage with your research in a different way. These are an opportunity to participate without the same pressure or work of preparing a paper and submitting to a journal or conference, those are different experiences, both highly beneficial. I would also recommend in the way writing this has been for me, to engage with reflective exercises for your journey to recognise, even if for just yourself, the work you have been doing, the changes and narrowing of scope, and your understanding of a field or concepts. I would also encourage industry partners with multiple postgraduates across the country to try and organise events like these to support their development, and help to establish academic and industry networks they may be struggling with confidence or opportunities to build beyond their own centre or institution.