Navigating Imaginary Landscapes: My Placement with Makers of Imaginary Worlds

post by Pavlos Panagiotidis (2022 cohort)

My placement with Makers of Imaginary Worlds took place in various locations around Nottingham and remotely.

Starting date: 25/06/2023
End Date: 25/09/2023

During the past summer, I had the opportunity to participate in a three-month placement with Makers of Imaginary Worlds, a company that combines art and technology in innovative ways to create experiences for children and families. I worked on a number of projects during my time there, which helped me gain a better understanding of the practical implications of engaging audiences in mixed reality experiences, as well as the potential for research in the intersection of HCI and performance.

During my placement, I was presented with several opportunities to work on projects that involved immersive technologies, approaching audience engagement, and experimenting with prototype technologies for performance. These projects, located in various parts of Nottingham, posed diverse challenges that made the experience exciting and solidified my interest in the intersection of art and technology. This placement helped me refocus my research objectives towards areas likely to have practical impacts. These areas include developing innovative methods to assess audience engagement through computer vision and creating methodologies to evaluate the aesthetic implications of emerging technologies in performance-making.

One project I worked on involved the qualitative analysis of interviews with visitors to the “Home Zero” art installation. This installation was designed to encourage participants, mainly children and families, to envision a more sustainable future through a playful, interactive experience that used paintings from the National Gallery as a starting point. I cleaned the data and performed a preliminary analysis of interview transcripts to study how audiences interacted with and perceived the installation. My analysis showed evidence that visitors enjoyed engaging with tangible interfaces and hands-on interactions, which made the experience more engaging and effectively supported the learning process. Later that year, I co-authored a paper that transformed some preliminary insights into a study on the significance of tangibility in designing mixed reality experiences about environmental sustainability for children. I also had the opportunity to contribute to another academic paper based on “Home Zero”, which aims to provide bridges between the disciplines of theatre and computer science, exploring how these fields can converge to enhance participatory design.

An example of an interesting field observation was when a child participant in HomeZero used the “Imagination Scanner,” a device that supposedly measured the participant’s imaginative capacity. The child’s excitement was palpable when they scored higher than their parents, and the automated system rewarded them by opening the door to the next part of the installation. This moment highlighted how design and technology could invert typical familial hierarchies, providing a unique and empowering experience for the children involved.

During my placement, I also had the opportunity to engage closely with “The Delights,” an event that blended dance, sensory activities, and interactive installations to captivate its young audience at the Hoopla Festival, which was held in Nottingham’s local parks. My role involved interviewing families to document their experiences and synthesising this information into a detailed report for stakeholders such as the festival committee. This report not only showcased the high level of audience engagement but also underscored the event’s impact on community connection, child development, and the creative transformation of public spaces. I gained valuable experience in the process required by funders to collect and analyse data and report the outcomes of publicly funded events to justify subsidising an art-making company.

Evidently, the event transformed perceptions of local parks from mere recreational spaces to vibrant community hubs that facilitate child development, artistic expression, and community bonding. Interviews with parents revealed significant shifts in how these spaces are viewed and utilised, emphasising the parks’ new roles as sites for creative and interactive family engagement. Notably, parents appreciated how the event encouraged their children’s expressive skills and social interactions, with many noting increased confidence and communication in their children due to participating in the activities offered. The experience showed me the importance of audience insights in designing experiences. Understanding audience behaviour, expectations, and engagement can be crucial in creating successful events. My placement’s most technically challenging aspect was working on a computer vision-based audience participation assessing prototype. This project aimed to collect and analyse data regarding audience behaviour in interactive installations and explore the possibilities of using computer vision technology to refine interactive artistic experiences.

During my placement at MOIW, I gained a deeper understanding of how my backgrounds in theatre, physics, and computer science synergistically apply to mixed reality experiences. The diverse approaches include assessing audience engagement, designing for optimal user experiences, performing qualitative and quantitative data analysis, and exploring the potential of physical and technological prototypes in performance. While being a “jack of all trades and master of none” can pose challenges in pinpointing one’s exact skills, it also allows for unique involvement and contribution to artistic projects.

Further reflecting on interdisciplinary approaches, I recognised that while the potential for convergence between computer science and theatre is evident, the independent evolution of these disciplines has occasionally made collaboration challenging. However, this placement reinforced my belief in the value of interdisciplinary research and the potential to bridge gaps between these fields, making designing each mixed reality performance a valuable step toward this integration.

In general, my placement with Makers of Imaginary Worlds was a valuable experience that enhanced my understanding of immersive technologies and audience engagement in a real-world setting. It solidified my commitment to exploring the intersection of art and technology, paving the way for my future work in the field. Thanks to my placement, I developed a deeper understanding of the intersection of HCI and performance, both academically and practically. I learned that collaboration and interdisciplinary research are crucial in creating and studying mixed reality events. Mixed reality requires a blend of skills and knowledge, including art, technology, and design. Therefore, processes that support interdisciplinary collaboration are essential in creating innovative mixed-reality experiences.

My Placement: Empowering Older Adults with Technology Through the ExtraCare Smart Market Initiative

post by Angela Higgins (2022 cohort)

Technology may help older adults maintain independence and live healthier lives, however there is a belief that this population is completely unable or unwilling to embrace these interventions. My work considers how we may empower people using technology, so when given the placement opportunity to work directly with older people trying out technology, I was eager to accept. My placement with my PhD industry partner, ExtraCare Charitable Trust, was split into several weeklong activities from autumn 2023 to spring 2024.

ExtraCare Charitable Trust is the UK’s leading not-for-profit provider of retirement housing, for people over 55. Their stated mission is “creating sustainable communities that provide homes older people want, lifestyles they can enjoy and care if it’s needed”. As such they encourage active and independent living and provide their residents with opportunities and activities to promote healthy ageing.

The Smart Market scheme is run by ExtraCare to allow their residents to try before they buy a range of technologies, along with set-up support. Older people may benefit from the use of appropriate technology to support independence and manage their well-being. However, there are many perceived barriers to entry, including usability, usefulness, and cost. To negate some of these factors, the Smart Markets allow residents to trial smart devices, including Amazon Echo and Alexa products, Ring doorbells, smart plugs, smart lights, sleep tracking mats, and Fitbits.

I ran Smart Markets at two ExtraCare sites, Lark Hill in Nottingham, and Reeve Court in St Helens, Merseyside (which just so happens to be my hometown). At both locations, the Smart Market was announced at the monthly village meeting, followed by a “Demo Day”, allowing the residents to drop in and ask questions alongside demonstrations. Then, to encourage further uptake, two “Market Days” were conducted where I set up a “market stall” in a common thoroughfare at the villages for two hours on subsequent mornings. At both the Demo and Market days residents could either take the smart device themselves, if they felt comfortable setting it up, or book an installation appointment. After 6-8 weeks I returned to collect the device they’d trialled. I also interviewed some of the residents about their experience with the Smart Market, and their use of technology more generally, to produce a report to assist ExtraCare with their future provision of technology services for their residents.

Through being present in the community, I was able to meet and chat with many of the residents at each location, and met a wide range of people, with different interest levels in technology. Some people already owned a full suite of gadgets, and some had no interest in technology whatsoever. However, quite a few were interested in trying things out but were either concerned about the cost or had no one to help set them up. These people were my main Smart Markets “customers”, and I was able to answer their questions and help them decide on some technology that was useful for them. If they required the help, I then booked them in for an installation appointment where I would come to their property and get them started. Finally, some people stopped by the Smart Market events not to try out a new product, but because they were having issues with technology they already owned. Unfortunately, I didn’t have the time to assist everyone who needed it, but I tried to help wherever I could.

I found installing the technology with residents a joyful and insightful process. Having spent some time in the communities, this gave me a chance to get to know the individuals a little better. Residents all had unique needs, challenges, and aspirations when it came to using technology, and I tried to work with them to set-up the technology. Examples include someone who wanted to try and Fitbit because her doctor had recommended it for health monitoring, someone else wanted a smart plug to avoid having to bend down to switch a light on and off, and another woman wanted to try and Echo Show to display photographs taken by her husband. Before I left, I handed residents a guide to the features of their device and asked them if they had any specific desires from using the device. If possible, I’d set up any services or systems they’d like, such as favourite radio stations, or connections to other apps. I believe that if we are to encourage people to use technology, it should be for purposes they find useful or enjoyable, so having sometime to spend with individuals was invaluable. I was even invited to speak at a retiree’s lunch club by one of the residents, which was a new experience for me, but a very pleasant one!

After the trial period, I returned to collect the devices, and setup the new devices if residents had bought replacements. Some residents agreed to an interview about how they’d found the device, and if they were buying their own. As well as providing direct feedback to ExtraCare about the Smart Markets, I could also provide insight into how residents used technology. Often more technologically competent residents would support their friends and others in the village with their devices. Additionally, many of my interviewees had extensive experience with technology prior to retiring, and did enjoy using it, even if there was the odd difficulty!

The main difficulty I encountered during my placement was with scheduling. As I split this research into separate sections (between sites, and between the different stages of research) often I could not achieve everything I wanted within the allocated time. Sometimes this was due to scheduling conflicts between myself and the residents, sometimes due to conflicts with other responsibilities I had, and sometimes just because life events prevented everything from going to plan. However, this emphasised to me the importance of flexibility and slack time when conducting research.

Overall, I found my time spent at Lark Hill and Reeve Court thought-provoking and inspiring, and will help me develop my future work with older adults. I was able to build relationships within the community, which is especially useful as I will be conducting more research at Lark Hill, and get in-person experience of how older adults were using technology. I especially learnt a lot about outdated and ageist stereotypes at work in the assumptions people have when designing for or researching with this community.  This work has proved valuable to my PhD and I hope ExtraCare have found my work beneficial to facilitate their residents to use technology to stay happy and healthy.

My Placement Experience: Lessons and Triumphs

post by Kuzi Makokoro (2022 cohort)

Reflecting upon my placement, a key lesson around the most important decision to make before starting a placement, was to consider the specific skills and experiences I hoped to gain. This past summer, I had the opportunity to partake in a placement with my industry partner, Co-op, which turned out to be a remarkable and invaluable experience for my professional and academic growth.

Before finalising the arrangements for the placement, including setting the dates, duration, and defining the project, a series of discussions took place between my supervisors and me. We assessed the multitude of opportunities that this placement could offer. It was during these deliberations that the versatility of a placement’s benefits became apparent to me. One option is to align the placement activities with your ongoing PhD research, ensuring that the work is not only relevant to your academic pursuits but also meets the strategic needs of the industry partner. This synergy often results in a mutually beneficial outcome that propels your research forward. Another approach could be to take a break from academic work to gain a breadth of experience in the industry, thereby expanding your professional network and engaging in projects that are also of interest to you.

Having spent the last nine years in commercial roles within various industries and capacities, I was already familiar with the dynamics of industry life. This pre-existing industry experience informed my decision to select a project that complemented my PhD research. Once I made this strategic choice, the focus shifted to pinpointing a suitable project. After numerous consultations, we collectively decided to concentrate on the Healthy Start Scheme—a government- initiative designed to aid low-income families with children under four by providing them with essential foods like milk, fruits, and vegetables. This project was not only crucial to my industry partner but also resonated personally with me, as it underscored the meaningful impact of data-driven initiatives on societal well-being.

The research objectives for the placement were ambitious: to utilise predictive analytics to predict the uptake percentage of the Healthy Start Scheme using food insecurity measures and to apply machine learning techniques to identify and understand the factors that influence uptake significantly. Working in conjunction with an industry partner meant that the practical application of my research findings could potentially aid the partner in supporting and promoting the scheme more effectively.

Entering the placement, I had certain preconceptions about how the experience would unfold, the nature of the work I would engage in, and the interactions I would have with various stakeholders. However, the practical aspects of my placement differed from my initial expectations. I quickly realised that my chosen topic necessitated a more independent working style, with periodic contributions from my industry partner rather than continuous collaboration. This shift led me to a new understanding of the role of a researcher in a consultative capacity, working in partnership with an industry entity. The experience also allowed me to lead a research project autonomously and understand the nuances of impact work. My responsibilities included initiating regular meetings with my contact at Co-op, seeking input and assistance from the wider team when needed, and managing the project’s pace and milestones.

In hindsight, although the timing of the placement originally seemed appropriate, I later reflected on whether doing it later on in my PhD program might have allowed for a richer output. The project demanded proficiency in skills that I had not yet mastered at the time, necessitating a steep and rapid learning process. This included developing an understanding of predictive analytics methodologies, acquiring proficiency in programming languages such as Python, learning about digital data collection techniques, and interpreting complex model results.

Consequently, what was initially set out to be a three-month placement evolved into a five-month project, as additional time was required for me to learn, adapt, and then effectively engage in the research. I adopted various learning strategies, such as the accelerated learning techniques outlined in Jake Knapp’s book “Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days,” which aided me in assimilating new information rapidly, trialling different approaches, and breaking down the project into smaller, more manageable tasks. Ultimately, I was able to enhance my skill set and produce actionable insights from the project, though a better approach to defining deliverables within the given timeframe would have been advantageous.

The research outcome was insightful; we identified several strong predictors within the model, such as income deprivation and language proficiency, as well as intriguing variables like household spending on fish and the caloric density of purchases. We explored various ways in which my industry partner could leverage these insights to better support the Healthy Start Scheme in communities where it is most needed.

In summary, the placement was a journey of adapting to a different work environment, setting pragmatic goals, and scaling up professionally. This learning experience has been instrumental in advancing my PhD work. It reiterates my initial emphasis on the importance of understanding what you seek to achieve from a placement. Although I had not initially set out to acquire these specific skills through the project, they have proven to be of great value as I continue with my PhD journey. Looking ahead, I am excited about the prospect of converting this project into my first published academic paper.

Fantasy Legal Exhibitions

post by Favour Borokini (2022 cohort)

Barely a day after I returned from St Andrews from the three weeks long Diverse Intelligences Summer Institute, I again set out to London for the Fantasy Legal Exhibitions workshop, held on the 18th and 19th of July, 2023 and organised by Victoria Barnes and Amanda Perry-Kessaris. The workshop was funded by the Socio-Legal Studies Association and Kent Law School.

I enjoyed the premise of the workshop and was quite eager to apply for many reasons. One, I enjoy speculative fiction and fantasy especially. I greatly enjoyed Max Gladstone’s portrayal of magician-lawyers in “Three Parts Dead”, partially because it validated a lot of my feelings about the amorality of lawyers and (corporate and commercial) law firms and their often very expansive (and expensive), though invisible, influence on how society and the shaping of culture – I also really like gargoyles, the ghastly grotesques. Rare is the fantasy medium that provides a treatment of the things. It’s almost like no one knows what to do with them. So bravo, Max Gladstone, bravo!

photo of four stone gargoyles
Image credit: Unsplash

With the theme of the workshop being exhibition, I found the paradoxical invisibility of lawyers and the garish nature of legal artefacts: Lady Justicia, the ostentatiousness of lawyers and law firms, and the wig and gown all very fascinating.

Beyond fictive speculation, as part of the application, I reflected on the significance of exhibition in legal research and law and wrote about how the British Museum’s retainership of the face of Queen Idia and other artefacts during a punitive expedition in the 19th century.

The face of Queen Idia is of some significance to me, being the symbol of my alma mater, the University of Benin (UNIBEN), where I received my LLB. Though UNIBEN is a federal university (a type of public university administered by the federal government rather than a state government), it is situate within the territory of the Benin Kingdom, and bearing the face of Idia, the powerful Benin Queen mother, the University represents (as an avatar, ha!) a symbiosis of this plural sort of arrangement.

The British Library’s refusal to release Queen Idia’s mask, even on loan, for the global 1977 Festival of Arts of Culture (FESTAC ‘77), on the other hand, is emblematic of how not to support pluralism.

Ivory mask of Queen Idia
Ivory mask of Queen Idia: Wikipedia

During the workshop, we visited various sites, including the British Museum, where each participant was invited to visit sections of interest, drawing and reflecting on how exhibits reflected law and power. There, I found myself drawn, perhaps inevitably, as I really love anime, to the Japan section high above, where I first happened upon the Kudara Kannon statue. I was drawn to it perhaps because it is literally an avatar, a living embodiment of the deity.

sketch of Kudara Kannon statue
Artistic rendition by clearly, very talented PhD student researching avatars
Kudara Kannon statue
orginal Kudara Kannon statue

The Japan section held many attractions for me and my research, which explores performativity and self-presentation and I found the artefacts extremely fascinating as the numerous ways identity can and is presented held a lot of appeal for me.

label for the Ichikawa Danjuro lineage description

Consider, for instance, the Kabuki of the Ichikawa Danjuro lineage performed by nine generations under the same stage name!

What did this say about fluid identity across different cultures?

After we left the British Museum, we went on to visit the Postal Museum. Here, I found the more archival, custodial nature of the Postal Museum a sharp contrast to the openness of the British Museum.

The Postal Museum was presented as an institution that sought to preserve history. Some of the procedures we undertook in the museum seemed quite ritualistic as well. We had to wash our hands and ensure they were dry, for instance, due to the fragility of the documents. No photographs were allowed within the archives, either.

students looking at archives at the Postal Museum
At the Postal Museum. Photo Credit: Amanda Perry-Kessaris

We had a period of reflection on the differences in exhibition styles between these two institutions and thought about the exhibition of law. How did the presentation of law by its archivists and curates discourage and encourage public perception and interaction with the institution of law?

Workshop brief

Following this (it was a very busy day!), we went to a nearby playground and were encouraged to draw a play item and adapt its display for our own fantasy legal exhibitions.

sketch and notes by Favour
I drew a bench and a man on it under a tree with limited success.

We went on to visit Middle Temple and, finally, the Institute of Advanced Legal Studies (IALS).

display cabinets showing silver cups commemorating first women called to the bar

I think I enjoyed Middle Temple the most, though I did get lost along the way. There were so many stairs and I don’t do well with stairs and lost most of our party, and needed rescuing by Amanda.

There’s something… well, a lot about being a lawyer is quite conservative. Something to do with the nature of law itself and the sort of people typically drawn to it, sticklers for rules and all that. There is also usually a lot of history to preserve. Law provides an in, a mirror and a vantage point to understand society. You can tell a lot about the values of a group of people by studying their law. What is praised and rewarded and what is punished.

Internally, as well, how institutions like Middle Temple, dedicated to the training and development of legal professionals, were formed is rich history too, in and of itself and being a legal practitioner provides belonging. One of my favourite and amusing parts of being a (Nigerian) lawyer is how we address ourselves, “My learned brother.” In a society as conservative as Nigeria, receiving the address of “learned brother” and “gentleman” is quite significant, if also… quite clearly problematic. One may address a female colleague as brother but still treat said “brother” in very unfraternal ways, and female lawyers struggle on many fronts in the legal profession.

In Nigeria, law students wear the same clothes and study together for five years (one year more than most Nigerian undergraduate courses), we then go on to the Nigerian Law School to qualify for the Bar for a year and then spend the rest of our lives working together, dressed the same way. It was that same feeling of camaraderie that I felt at Middle Temple. A space for my kind. It was, on the whole I think, what positive exhibitionism looks like.

At the IALS, we finally got into the more practical side of things. I must preface this by admitting to struggling with visual artistry. Words, I feel quite at home with, and I do like to go on and on with the right audience but drawing and sculpting(!). Nevertheless, she persisted. Our final activities were to design our own fantasy legal exhibits and mould a sort of frontispiece for them.

I chose an eye. An eye is just a ball with long feathers on it, isn’t it?

But more seriously, all performances and exhibitions are done under an eye or for an eye or with an eye or view to achieve some aim and objective. The evil eye, the male gaze, performativeness, theatre, not even when we are alone, especially not if you’re religious, are we ever free from some sort of appraisal.

So I made an eye.

a model of a clay eye by Favour
It has an eyepatch too!

For my fantasy legal exhibition, I decided that I wanted to create an exhibit of self-portraiture avatars with African women.

Favour's sketch of a fantasy legal exhibit

All things come to an end and so too will this piece, very shortly. A few months after the workshop, all participants were invited to create a PowerPoint presentation of their exhibition. Our presentations may be viewed at the Fantasy Legal exhibitions blog post.

My research is socio-legal. In it, I explore how people represent themselves via avatars and what the law is, in response to avatars. These days, I think about the good sides of digital technologies and performance. There’s something there that is law – our norms and cultures and practices, our desires. I plan to take Eugen Ehrlich’s megalomaniac jurisprudence bent to heart as I try to discover what avatar law is.

brief for British Museum

Exploring Children’s Interaction with Robotic Installations: Reflections on Placement with Makers of Imaginary Worlds

post by Victor Ngo (2022 cohort)

My placement ran from July to September 2023, with Makers of Imaginary Worlds (MOIW), and involved planning and running a two-part study that would help inform my PhD research on ‘Artificial Intelligence and Robotics in Live Creative Installations’. More specifically, how children interact and form relationships with the robotic installation, how children understand and shape their interaction in this context, what meaning children attribute to the robot and their interaction, how curious children are during the interaction and what motivates their curiosity.

NED (The Never-Ending Dancer)

To add some background to my placement partner, MOIW are a Nottingham-based art company, who aim to create interactive and sensory experiences where children can play, engage and explore. MOIW’s first live robotics project, Thingamabobas, is aimed at younger audiences and involves the use of a computer vision-equipped robot arm, called NED (The Never-Ending Dancer), which can detect audience members and interact with them autonomously. This installation also includes a series of mechanical circus-like creations that are designed to enable children to interact with them as imaginative, dynamic sculptures that provide novel, enjoyable, and empowering experiences.

Unlike social robots, industrial robot arms typically have a functional design, void of humanoid features or facial expressions. MOIW aims to transform the industrial robot arm into a playful kinetic sculpture that defies expectations, offering an inventive interpretation. By introducing variables such as costume design, musical accompaniment, and contextual storytelling, the artists aim to redefine the perception of the robotic arm. Demonstrating how fiction can facilitate a willing suspension of disbelief among audiences, allowing viewers to trust and immerse themselves in reimagining a new reality.

The Study:

The first half of the study, part 1, was completed at the National Festival of Making, Blackburn, with a tremendous number of people attending and in general a great event! This half of the study explored the audience’s understanding of the initial robotic system, with no changes to the system’s capabilities or functionality. This allowed for a base understanding of its performance, capabilities, and limitations in the wild for a researcher new to the system, this proved extremely useful and provided me with the knowledge necessary to complete part 2. The second half of the study, part 2, was completed at the Mansfield Museum, Mansfield, exploring the same audience understanding, however, the system was altered to allow for 360-degree motion around the robot’s base as well as a different method of detecting the audience, moving to body pose detection from facial recognition. Although a direct comparison between the two studies is not possible, insights into audience responsiveness, engagement, and enjoyment are all possible and valuable to the discussion, and future development of this system or similar systems.

Reflections:

Over the course of three months, this placement has provided me with the opportunity to develop key interpersonal and professional skills, as well as improve my technical aptitude.

My initial discussions with MOIW for the placement were straight away met with enthusiasm and sometimes whimsical imagination from Roma Patel and Rachel Ramchurn, the artists of MOIW. Despite MOIW being a relatively small company, I was fortunate enough to learn the ins and outs of art installation production and how artists like Roma and Rachel turn ideas and imagination into professional productions, and how they deal with issues, changes and unexpected setbacks. Shadowing them allowed me to observe how they manage large-scale projects and interact with professional organisations. This has enabled me to further develop and improve my own understanding of professional engagement and project management.

Throughout all stages of the placement, the artists from MOIW frequently discussed possible alterations and upgrades with me to improve the interaction capabilities of the robot. Here I was able to apply and improve my technical experience, developing solutions to enhance the robot’s interaction capabilities or increase the system’s audience detection accuracy and reliability. Despite the technical success of some of the solutions developed, it is important to highlight that not all of the artist’s ideas were technically feasible or within the scope of the project. Communicating this effectively and managing the artist’s expectations was key to ensuring that both the robot was functional and ready for the study, and the professional relationship with MOIW was maintained, without either party being let down or led to believe the system was any greater or less than what was agreed on.

During the study, I was not only the lead researcher on site but also a range of other roles that sometimes required me to step out of my comfort zone. These included in no particular order of social or imaginative intensity; Thingamabobas Wrangler, Storyteller, Imagination Guide, The Researcher from Nottingham University, Technical Support.

As part 1 of the study at the National Festival of Making was a two-day event over the weekend during the height of the UK summer holiday, I was left with little choice but to quickly adapt to these new roles or suffer being swarmed by the thousands of curious and enthusiastic visitors that attended the event. To my surprise, and with a little help from Roma and Rachel, I was able to help children and adults alike be transported into the whimsical world of the Thingamabobas, for about 20 minutes at a time.

Overall, I thoroughly enjoyed the experience and opportunity to work with MOIW to not only develop an art installation, but to also help run it was a great privilege. The skills I have learned and applied in both professional engagements as well as in the wild will be beneficial to my PhD research and to myself as an individual.

 

Insights from the Oxford Machine Learning Summer School

post by Gift Odoh (2022 cohort)

Between the 13th and 16th of July, 2023, I attended the Oxford Machine Learning Summer School at the Mathematical Institute of the University of Oxford for health applications. The course organised by the AI for Global Goals in partnership with the University of Oxford’s Deep Medicine and CIFAR, was focused on advanced areas in machine learning (ML), ranging from statistical and probabilistic approaches to representation learning (an ML approach based on representations of data that make it easier to extract useful information when building classifiers or other predictors [1]), specialised techniques for complex data structures, computer vision, knowledge representation and reasoning, and the integration of symbolic and neural approaches for enhanced AI capabilities.

My interest in the school was from the opportunity for exposure to valuable exploration into ML’s diverse applications and expectations towards uncovering connections between ML techniques and their relevance to my PhD research, which focuses on robotic teleoperation and human-robot interaction, particularly concerning mental workload indicators and how they can inform robotic assistance schemes in teleoperation. I also saw it as an opportunity to meet people of similar interest in the field and visit the renowned city of Oxford and its University of Oxford colleges, some known to have rich histories.


The first couple of sessions focused on how we understand our environment as humans – covering how we represent the world and its actual truth from different observations. These sessions paved the way for representation learning and how intelligent systems can extract useful information from features present in data, particularly when there are no labels. S. M. Ali Eslami, in his session on representation learning without labels, underscored the importance of labels to effective machine learning but demonstrated how learning can still be achieved when label collections are impossible by showing how different encoders make representations (understanding) of data from inputs as well as how this is reversed though generative models that make real-world estimates of this representations. While most sessions focused on probabilistic models based on generative techniques and casual machine learning, which focus on the learning process, Professor Pietro Lio from Cambridge presented an intriguing session on graph representation learning, which is a form of machine learning useful for organised data in the form of networks or graphs where points of data (nodes) are connected with edges (relationships), making an interesting case for utilising graphs as they are everywhere in research. Although most of the application areas were in molecule generation for proteins and drugs, its application in extracting meaningful insights and predictions from relational data can be applied to model robotic assistance schemes that respond to mental workload within a complex framework where nodes can represent operators’ mental states such as attention levels, stress levels or task demands and the edges can signify the relationship and dependencies between them.

Another key aspect of the course was computer vision for ML. Some of these sessions were on the evolution of computer vision and its techniques and unsupervised visual learning for ML applications, particularly medical imaging. Understanding the progress of computer vision and where it stands today has practical implications for my work, given that vision is integral to teleoperation interaction. Christian Rupprecht presented the stages for understanding a scene, including scene classification, where the general scene is described; object detection, in which various objects in the scene are identified; segmentation, which involves dividing the scene into meaningful, distinct parts and regions; scene graph, which describes the positional relationship between objects; description in which an improved interpretation of the scene is obtained and hierarchy which informs the how scenes are decomposed into objects, parts and materials.

It was, however, useful that the summer school was not just about machine learning techniques in isolation. The segment on Bridging Machine Learning and Collaborative Action Research emphasised the importance of collaboration, especially in areas like digital mental health. For example, the limitations of applying findings from social media data for health states generalisation, methodological issues, challenges understanding other attributes (e.g. offline attributes) and threats of relying on single data sources. This challenge emphasises the indispensability of interdisciplinary collaboration, which resonated deeply with my belief in merging human-robotic interactions with other disciplines for a more holistic approach to tackling the interdepending challenges of robotic assistance in teleoperation. Although it seemed to me that some of the techniques were unique in their approach and application to specific conditions, I see an opportunity for careful examination into how some approaches could come together to enhance robotic autonomy and facilitate better human-robot interaction.

In conclusion, the school has added depth to my understanding by expanding my academic horizon to approach my research through the sessions, including those that felt directly applicable and the seemingly marginally relevant ones. It is also noteworthy that the school was also an opportunity to meet other PhD students from diverse backgrounds and corners of the world. Our interactions provided valuable global perspectives on the various ML applications in health research. I must also add that I had the opportunity to explore the historic city of Oxford and its renowned University of
Oxford colleges both through guided tours and lone walks, which offered a cultural immersion and ignited a sense of academic inspiration. My interactions with students, researchers and industry professionals allowed me to forge meaningful connections in machine learning that broadened my understanding and opened doors to potential future collaborations and opportunities. Overall, it was a transformative learning experience that equipped me with a global network to renew my sense of purpose in my research and professional journey.

References
[1] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Trans Pattern Anal Mach Intell, vol. 35, no. 8, pp. 1798–1828, 2013, doi: 10.1109/TPAMI.2013.50.

Reflections on my placement at the Department for Transport

post by Phuong Anh (Violet) Nguyen (2022 cohort)

I began my placement in the Data Science team of the Analytic Directorate at the Department of Transport in April 2023 to gain access to datasets for my pilot research. However, I feel that my internship officially began in July, when I was able to become familiar with my work and knew precisely what was expected of me. I still remember the rainy afternoon when I went to the warehouse to collect my IT kit. It was quite a funny memory, and now it is quite emotional to pack and return my kit as my internship is over.

My project

My internship was an integral component of my doctoral research on “Using personas in transport policymaking.” I aimed to combine various data sources to investigate the travel behaviour of various demographic groups, and then use this understanding to inform transport planning and policy formulation.

I began by examining multiple DfT data sources, including the National Transport Survey, Telecoms Data, and Transport Data dashboard. I arranged meetings with several data team members to ask them about how they analysed these data in previous projects. I also had opportunities to discuss with members of other teams including System Thinking, Policy, Social Research, and Data administration… to learn about their work and the policy-making process at DfT.

Since the official release of the transport user personas report in July, I have collaborated closely with the personas team. I began with an examination of the methodology for developing personas. I also attempted to apply additional data science techniques to the same dataset (National Transport Survey) to cluster travellers into distinct groups and compare the results of the various methods.

DfT published transport user personas. (https://www.gov.uk/guidance/transport-user-personas-understanding-different-users-and-their-needs).

I worked with the Social Research team to organise workshops introducing the potential of using personas in DfT’s work, such as Road Investment, AI Strategy, and Highway… In addition, as part of my research, I utilised the Social-technical framework theory to structure the transport system and then gathered data to present and analyse the interaction between personas and other transport system components. On the other hand, I learned how policy is formulated and I will continue to work with the Policy team to investigate how personas can support their work.

Some lessons for myself

About work

Working on the Data Science team, which is part of the Advanced Analytics Directorate, was an excellent opportunity for me to improve my statistics, mathematics, and programming skills. Colleagues were very knowledgeable and supportive. Through the team’s regular meetings and project summaries, I got a general understanding of which projects are active and which models and methodologies are used to solve the problems. Sometimes I found myself bewildered by mathematical formulas and technical models. Although I have studied Data Science in the past, which has provided me with a foundation in Data analysis and Programming, in real work data looked more complicated. The assignments in the placement have taught me how to overcome the challenges of dealing with multiple data types.

Working in Civil Service, I had access to numerous training resources, workshops, and presentations, including but not limited to Science and Programming, this course covered user-centric services, artificial intelligence, evidence-based policymaking, and management skills. This is why I regard my civil servant account to be so valuable.

I received numerous perspectives and comments on my research proposal thanks to weekly discussions with my line manager and multiple team members. Importantly, I learned how to present and explain my ideas and academic theories to people from diverse backgrounds, as well as how to make the ideas clear and simple to comprehend. I believe this is a crucial skill in multidisciplinary research, communication, and public engagement.

About working environment

This was also my first time working in civil service, a “very British” working environment. Even though I have more than five years of experience in the airline industry, it took me a while to become accustomed to office work. It could be because the working environment in Vietnam differs from that in the United Kingdom, business differs from civil service, and full-time office work differs from hybrid work.

In addition to the knowledge and expertise I gained, I learned a great deal about time management and how to use Outlook professionally to organise my work, as well as how to spend concentration time between multiple meetings every day. This was extremely helpful when having to divide my time between multiple tasks, such as PhD research, placement, and meetings with supervisors from various institutions. In addition, I believe that working in person in the office is more beneficial than remote working, having access to a larger screen. being able to meet and discuss with multiple people, rather than being limited to 30-minute meetings and a small laptop screen. Thus, I travelled to London every week. These regular catchups with my line manager/industry partner proved helpful because the industry partner was able to provide a realistic perspective and I was able to update them on my work and interact with other DfT employees who supported my research.

About my personal development

Since I began my PhD journey, I have experienced many “first-time experiences”, including my first time working in Civil Service. This placement is not only an integral part of my PhD research, but also provides me with a great deal of experience and lessons for my personal growth. It was so unusual and sometimes difficult, but it forced me to leave my comfort zone. I was confident in my ability to perform well, having had the previous experience of being an airline strategist in the past, but the new experience of being an intern, learning something new, made me humble and enthusiastic as if I had just started school.

My principal lesson is to simply DO IT, JUST DO IT. I believe that the majority of my depression stems from my tendency to overthink. There were times when I examined the data set and had no idea what to do. I was even afraid to send emails or speak with others. However, when I actually did what I should do – WRITE something and ASK some questions – and I saw results, I realised that that work is simpler than I originally thought. Then I learned that sitting in a state of distress and worrying about the future is ineffective in resolving the issue. I must take action and be diligent to see myself become a little bit better and better every day.

The summer was very short, and most colleagues took vacation time. Honestly, the internship was not “comfortable” in the beginning, but now I believe everything is going well. This placement is also assisting me in developing a clearer plan for my PhD project. I am grateful for the support and lessons I have gained from this opportunity, and I am considering another summer placement next year.

‘Outside the Screen’ podcast interviews Szymon Olejarnik

Outside the Screen podcast logo

‘Outside the Screen’ is a podcast about research and policy developments concerning children’s interactions with screen-based content.

Szymon Olejarnik, a first-year Horizon CDT student, was interviewed by ‘Outside the Screen’ about his PhD research, which focuses on youth socialisation in a gaming context with a special focus on autism.

You can listen to the podcast episode at Loneliness, Go! and gaming on the autism spectrum  (Interview starts at 16:40).

 

Diverse Intelligences Summer Institute 2023 Reflective Report

post by Favour Borokini (2022 cohort)

From June 25th – July 15th, 2023, I attended the 6th annual Diverse Intelligences Summer Institute (DISI) Summer School at the University of St Andrews. The institute aims to foster interdisciplinary collaborations about how intelligence is expressed in humans, non-human animals, and artificial intelligence (AI), among others.

I was excited to attend the Summer Institute due to my interest in AI ethics from an African and feminist perspective. My current PhD research focuses on the potential affordances and challenges avatars pose to African women. As AI is now often implicated in the creation of digital images, I thought DISI was a great environment to share ideas and insight into how to conceptualise these challenges and opportunities.

The attendees were divided into two groups: Fellows and Storytellers. Fellows were mostly early career researchers from diverse fields, such as cognitive science, computer science, ethnography,  and philosophy. The Storytellers were artists who created or told stories and had in their number an opera singer, a dancer, a weaver, a sci-fi author, a sound engineer and many others. The Storytellers brought spontaneity and life to what would surely have been a dreary three weeks with their creativity and their ability to spur unselfconscious expression in all the participants.

DISI 2023 began on a rainy evening, the first of several such rainy days, with an icebreaker designed to get Fellows and Storytellers to get to know each other. In the following days, we received a series of engaging lectures on topics as varied as brain evolution in foxes and dogs, extraterrestrial intelligence, psychosis and shared reality and the role of the arts in visualising conservation science. A typical summer school day had two ninety-minute lectures punctuated by two short recesses and a longer lunch break.

The lecture on Psychosis and Shared Reality was given by Professor Paul Fletcher, a Professor of Neuroscience from the University of Cambridge who had advised the development team of Hellblade, a multi-award-winning video game that vividly portrayed mental illness. This game put me in mind of several similar research projects ongoing at the CDT researching gaming and the mind. As a Nigerian, I reflected on the framing of psychosis and mental illness in my culture and the non-Western ways these ailments were treated and addressed. That first week, I was quite startled to find that two people I had spoken casually with at dinner and on my way to St Andrew’s were Faculty members. One of these was Dr. Zoe Sadokierski, an Associate Professor in Visual Communication at the University of Technology, Sydney, Australia, who gave a riveting lecture on visualising the cultural dimensions of conservation science using participatory methods.

In that first week, we were informed that we would all be working on at least one project, two at the most (more unofficially), and there was a pitching session over the course of two afternoons. I pitched two projects: The first project was to explore the aspirations, fears and hopes of my fellow participants using the Story Completion method, a qualitative research method with roots in Psychology, in which a researcher elicits fictional narratives from their participants using a brief prompt called a stem. This method helps participants discuss sensitive, controversial subjects by constructing a story told from the point of view of a stranger.

Many of the stories were entertaining and wildly imaginative, but I was particularly struck by the recurring anxiety that in 2073, the beautiful city of St Andrews would be submerged due to rising water levels. This seemed to me a reflection of how attached we had all become to that historic city, how attachment to places and things can come to help us care more.

For my second project, I and two friends (pictured below) interviewed six of our fellow DISI attendees for a podcast titled A Primatologist, a Cognitive Scientist and a Philosopher Walk into a(n Intergalactic) Bar. The idea was to get artists and researchers to tell an ignorant but curious alien on a flying turtle planet called Edna about their work and the Earth. These interviews sparked amazingly unintended reflective conversations about the nature of life on earth, our relationship with nature and human values, such as honesty. On the final day, we put together an audio trailer for some of the most insightful parts of these conversations as our final presentation.

Photo of our Podcast team. L-R: Antoine Bertin, Favour Borokini, & Matthew Henderson. #TeamEdna

Prone to being critical, I often felt disconcerted by what I perceived as an absence of emphasis on ethics. Having worked in technology ethics and policy, I felt prodded to question the impact and source of a lot of what I heard. In a session on the invisibility of technology, I felt extremely disturbed by the idea that good technology should be invisible. In fact, I felt that invisibility, the sort of melding into perception described as embodiment by postphenomenology, spoke more to efficiency than “good”, bearing in mind use cases such as surveillance.

There were some heated conversations, too, like the one on eugenics and scientific ethics in research. The question was how members of the public were expected to trust scientists if scientists felt ethically compelled not to carry out certain types of research or to withhold sensitive findings obtained during their research.

And the session on questioning the decline in “high-risk, high-return research”, which seemed, unsurprisingly, focused on research within the sciences, led to comments on funding cuts for social sciences, arts and the humanities resulting from the characterisation of these fields as low-risk and low-return, causing me to reflect that, ironically, the precarity of the latter, qualified them more as tagged high-risk, at least, if not high-return.

But the summer school wasn’t all lectures; and there were numerous other activities, including zoo and botanical garden trips, aquarium visits, beach walks, forest bathing and salons. During one such salon, we witnessed rousing performances from the storytellers amongst us in dance, music, literature and other forms of art.

An evening beach bonfire with a Frisbee game
Favour and two “dudes” at the entrance to the Edinburgh Zoo

I also joined a late evening expedition to listen to bats, organised by Antoine, one of my co-podcast partners. There was something sacred about walking in the shoes of the bats that evening as we blindfolded ourselves and relied on our partners to lead us in the dark with only the sense of touch, stumbling, as a small river rushed past.

I think the process of actually speaking with my fellow attendees caused me to feel warm towards them and their research. I believe ethics is always subjective, and our predisposition and social contexts impact what we view as ethical. At DISI, I found that ethics can be a journey, as I discovered unethical twists in my perspective.

It was my first time at the beach!
At the St Andrews Botanical Garden

This thawing made me enjoy DISI more, even as I confirmed that I enjoy solitary, rarefied retreats. As the final day drew near, I felt quite connected to several people and had made a few friends, who I knew, like the rarefied air, I would miss.

The success of DISI is in no small part due to the effort of the admin team, Erica Cartmill, Jacob Foster, Kensy Cooperrider, and Amanda McAlpin-Costa. Our feedback was constantly solicited, and they were quite open about the changes from last year.

I had a secret motive for attending. My research’s central focus is no longer AI, and I felt very out of place not having something I thought was core to the theme. But a conversation with Sofiia Rappe, a postdoctoral Philosophy and Linguistics Fellow, led to the realisation that the ability and desire to shapeshift is itself a manifestation of intelligence – one modelled in many non-human animals, reflecting awareness and cognition about how one fits in and how one should or ought to navigate their physical and social environment.

I look forward to returning someday.

With my friend Khadija, on the last day
After Cèilidh-ing, with Mia and Paty

You can listen to our podcast here: SpaceBar_Podcast – Trailer 

A Reflection on The Connected Everything and Smart Products Beacon Summer School 2020

post by Cecily Pepper (2019 cohort)

My first summer school started with an invite via email. Despite my interest in the topic, my first thought was that robotics was not my area of expertise (coming from a social science background), so maybe I shouldn’t bother applying as I’d be out-of-my-depth. Although after some consideration, I thought it would create some great opportunities to meet new people from diverse backgrounds. So, I stopped worrying about my lack of knowledge in the area and just went for it; and I got a place!

The summer school was held digitally due to COVID-19 restrictions, which had both its benefits and pitfalls. On the first day, we were welcomed by Debra Fearnshaw and Professor Steve Benford, and were then given the opportunity to introduce ourselves. From this it was apparent that there was a wide variety of delegates from several universities, with a range of disciplines including social sciences, robotics, engineering and manufacturing. The first day mostly consisted of talks from experts about the challenges we face in connecting technology and the potential of co-robotics within the fields of agrirobotics, home and healthcare. The main task of the summer school was to create a cobot (collaborative robot) that could overcome some of the issues that COVID-19 has created or exacerbated. The issue that the group chose to address had to fall into one of the categories introduced on the first day: food production (agrirobotics), healthcare or home. Along with this challenge, more details were needed on function, technological components, and four key areas of the cobot design: ethics, communication, learning and safety. These aspects were introduced on the second day. After being split into groups at the end of the first day, I felt happy as my group had a range of experience and expertise between us, which I felt would bode well for the challenge as well as being beneficial for myself as I could learn something from everyone.

Similarly, the second day consisted mostly of talks, this time based on the four themes mentioned previously. The ethics discussion was interesting and included in-depth explanations around aspects to consider when reflecting upon the ethical consequences of our designs, such as privacy, law, security and personal ethics. An online activity followed the ethics talk but was soon interrupted by a technical glitch. Despite this, we were able to engage with alternative resources provided in order to reflect upon the ethics of our cobot design. This was useful both for our eventual design, as well as applying this to our own PhD research.

The other themes then followed, including a discussion around interaction and communication in technology. This was an insightful introduction to voice user interfaces and alike, and what the current research is focusing on in this field. While fascinating on its own, it was also useful in thinking about how to apply this to our cobot design, and which features may be useful or necessary for our cobot’s functionality. A talk on the third theme of learning was then delivered, including details about facial recognition and machine learning, and the applications of these in the field of robotics. Likewise, this was useful in reflecting upon how these features may be applicable in our design. Finally, the theme of safety was considered. This talk provided us with the knowledge and ability to consider safety aspects of our cobot, which was particularly apt when considering COVID safety implications too. Overall, the first two days were quite lengthy in terms of screen time (despite some breaks), and I found myself wilting slightly towards the end. However, I think we could all understand and sympathise in the difficulty of minimising screen time when there is a short space of time to complete all of the summer school activities.

On the final day, we split into our teams to create our cobot. This day was personally my favourite part of the summer school, as it was fantastic to work with such a variety of people who all brought different skills to the group. Together, we developed a cobot design and went through the themes from the previous day, ensuring we met the design brief and covered all bases. Probably the biggest challenge was keeping it simple, as we had so many ideas between us. Despite our abundance of ideas, we were strict with ourselves as a group to focus and keep the design simplistic. Additionally, the five-minute presentation time meant that we had to keep our design simple yet effective. We then presented our home assistant cobot, Squishy. Squishy was an inflatable, soft cobot designed to assist carers in lifting patients who were bed-bound (as occupational injuries are a significant problem within the care industry). Squishy’s soft design enabled comfort for the patient being lifted, while the modular design provided a cost-effective solution and the possibility of added-extras if necessary. Along with this, Squishy was beneficial in that it consisted of wipe-clean surfaces to enable effective cleaning in light of COVID-19, as well as aiding social distancing by reducing the need for carer-patient contact. Other features of Squishy included machine-learned skeletal tracking and thermal cameras to aid safe functionality, and minimal personal data collection to maintain ethical standards. After the presentations and following questions, the judges deliberated. Results were in…my team were the winners! While I was happy to have won with my team, the most fruitful part of the experience for me was meeting and learning from others who had different backgrounds, perceptions and ideas.

Overall, I felt the summer school was well-organised and a fantastic opportunity to work with new people from diverse backgrounds, and I was very glad to be a part of it. I’m also pleased I overcame the ‘Imposter Syndrome’ feeling of not believing I would know enough or have enough experience to be a valuable delegate in the summer school. So, my advice to all students would be: don’t underestimate what you can contribute, don’t overthink it, and just go for it; you might end up winning!

The Summer School was funded by EPSRC through the Connected Everything II network plus (EP/S036113/1)