Robot dancer interacting with children – Feng’s internship

post by Feng Zhou (2017 cohort)

The robot dancer was one of four exhibits of “Thingamabobas,” a playful, sensory experience where participants meet and interact with a circus troupe of performative hybrid mechanized sculptures crafted from sustainable and recycled materials. The installation space is a place of wonder. It draws on the absurdities of British artists Heath Robinson and Rowland Emett’s contraptions, Calder’s Circus (1930’s), automata, object theatre, puppetry, and circus acts.

The robot dancer was shown in Lakeside Arts at the University of Nottingham. It attracted many families with children to attend.

We chose the low-cost robot arm – Ned as our dancer. It is also safe working with humans. The Intel RealSense Depth Camera was used as “dancer”‘s eyes to conduct facial recognization.

The fan-shaped zone is the range of the camera. It is also the zone in which children interact with the “robot dancer.” When the camera “sees” the face of the nearest child in the green area, it will dance following the child’s face. Since the “robot dancer” has a bad eye condition, it will be hard to tell whether there is anyone around when the nearest child shows in the purple zone. Thus the “robot dancer” will try to look around and search for children. And then, when children stand farther than the purple area, the distance will be over the range of the camera’s detection. So the “robot dancer” will fall asleep. During the same time when children go through “interacting,” ” searching,” and “sleeping” areas, there will be corresponding music being played.

There are still many points that could be improved such as increasing the interacting zone by increasing amount of cameras, switching professional robotic arms to achieve more sufficient movements, and decreasing the delay of interaction.

It was an amazing experience to work with artists and children.

My job was primarily on developing the robotic interacting system base on ROS, including developing the robotic interacting mode based on distance from the users, music playing etc. From this internship, I extended my skill set by learning the ROS system, which enables high flexibility in combining multiple devices into an integrated system. This enabled me to extend my six-axis 3D printing system, which is significant for my PhD research. During the process of developing the interaction mode of the robotic arm, I had a chance to work with dancers, who interacted with the system with dancing motion, through which, I got valuable experience working with dancers who are also aimed users for my PhD research.

Paper reflection – Articulating Soma Experiences using Trajectories

post by Feng Zhou (2017 cohort)

Somaesthetics combines the term ‘soma’ with ‘aesthetics’. The concept of ‘soma’ is predicated on the interconnectedness of mind, body, emotion and social engagement, considering all to be inseparable aspects that together form an embodied, holistic subjectivity. Aesthetics here refers to the ways in which we perceive and interact with the world around us. Somaesthetics is a widely used methodology for user study, which plays a significant role in my PhD research. Researchers who have focused on the research of Somaesthetics for many years and have published a number of prominent papers from the Royal Institute of Technology Stockholm visited the Mixed Reality Lab (MRL) of the University of Nottingham (where I am based) and collaborated with researchers here to run workshops on Somaesthetics. It was an excellent chance for me to learn Somaesthetics deeply through the workshop and explore the application of this methodology to my research.

Researchers were split into four groups to explore different applications. The group I was involved in was to explore the boundaries between humans and technology. The skin is traditionally seen as being a critical boundary of the body and one way of defining the bodily self. We can see, i.e. perceive with our eyes, our “external”, fleshy body – our moving limbs and parts, and our skin as the boundary between our “external” and “internal” body –our organs, cells, muscles etc., – which we cannot see, but instead feel or imagine. However, the boundary may be considered malleable. Take the example of a prosthesis – is this a part of our body or a separate piece of technology?

We attached cloth straps to the dancer’s calf and thigh so other members of the team could control them. Participants had to imagine a limb that had a ‘mind of its own’ – an exploration of dance where a part of one’s body was separated from control. The dance experience became one of negotiating control with one’s own body. This could serve as a conceptual stand-in for novice kinaesthetic skills where one’s body is unable to do what is asked – perhaps lacking the range of motion needed. But this was beyond being simply unable to perform the controlled limb; it actually became a separate performer in its own right, creating an intriguing partnership with a part of one’s own body, and encouraging the dancer to question the boundaries of their body and soma.

As we began to dance, our bodies behaved as we expected and we were unfettered. As our group began to take control of our limbs, we lost some agency over our bodies. The external influence started to exert itself in such a way that it restrained us, or actively pulled us. We were no longer ‘at one’ with our own bodies – rather those who controlled our limbs shared control with us. Over time as we learned how to work together, that action could even be considered a part of us (at least as far as the experience goes). It should be noted that the group members pulling on the straps were a stand-in for a ‘disobedient’ prosthesis – so the notion of it becoming part of us, or perhaps beginning as part of us, separating from us and returning might be more tightly aligned to our own body than the group experience – nevertheless the group does have access and licence to control our limbs.

This workshop was one of the user studies to support our final paper. Questioning the boundaries between humans and technology also invites reflection on the boundary between ‘inside’ and ‘outside’: separated by the skin, breathing in and out, ingesting and excreting. Thinking through these boundaries allows designers to redefine them, and thus challenge not only where the soma begins and where it ends, but also where the boundaries of experience lie. This turned out to significantly support my user workshop with disabled dancers to personalise their prostheses.

My job for the final paper was mainly to describe the activity I was involved in during the workshop. This was a precious experience to learn to write a paper collaboratively with many authors. Our final paper has 14 authors from the Royal Institute of Technology Stockholm and Mixed Reality Lab. Each of us wrote a specific part of the paper on Overleaf. We also have regular meetings to discuss writing up issues. This was also the time l started to learn Latex, which helped a lot in my left writing up on papers and thesis.

Paper Accepted in the Machine Learning, Optimization and Data Science Conference

post by Jimiama Mafeni Mase (2018 cohort)

EFI: A Toolbox for Feature Importance Fusion and Interpretation in Python

Divish Rengasamy (a PhD candidate with the Institute for Aerospace Technology at the University of Nottingham, who recently passed his viva) and Dr. Grazziela Figueredo (my supervisor) published a manuscript to Applied Sciences. Their manuscript proposed a solution to address the lack of agreement among feature importance techniques regarding how they quantify the importance of features to machine learning predictions.  Their solution combined the results from multiple feature importance quantifiers to reduce the variance in estimates and to improve the quality of explanations using crisp information fusion techniques, such as mean and majority vote. A few months later, Grazziela scheduled a meeting to discuss opportunities to improve their solution using a fuzzy logic system (FLS), which is one of the main approaches in my PhD due to its capability to capture and model uncertainties and ambiguity in information.  Divish, Dr. Mercedes Torres Torres (who was my second supervisor at that time, but now works for B-Hive Innovations at Lincoln) and I attended the meeting. In the meeting, Divish went through their proposed solution and together we identified some limitations i.e., significant loss of information as their approach reduced several quantifiers to crisp outputs and difficulty to understand the representation of `importance’ as continuous values (also known as importance coefficients). Next, I described the potential benefits of FLSs, such as using human-understandable linguistic terms to define concepts and using fuzzy sets and rules to handle ambiguous information. They loved the idea and provided synthetic data for experiments.

After obtaining the results from the experiments, we observed that FLSs outperformed the previous solution proposed by Divish in capturing increased variation of feature importance caused by increased data dimensionality, complexity and noise. However, the previous solution still showed remarkable performance in situations with less noise and less variability in the importance of features produced by the multiple machine learning models and feature importance techniques.  It was at this point that we decided to develop an open-source Python toolbox called Ensemble Feature Importance (EFI) consisting of the two solutions i.e., crisp information fusion and FLS.  However, we needed an expert in software modularity and testing to handle the development of the toolbox. Fortunately, Aayush Kumar (a software data scientist), who was doing a master’s in data science at the University of Nottingham, agreed to come on board as the developer of the toolbox. Aayush implemented the toolbox in Python programming language as his master’s dissertation. The toolbox consists of packages to automatically optimise ML algorithms, calculate and visualise the importance of features using various feature importance techniques, aggregate the importance of features from multiple ensemble methods, and create fuzzy logic systems for capturing uncertainties and interpreting importance. The audience for the toolbox is machine learning researchers, end-users of machine learning systems, and decision-makers. After developing and testing the toolbox, we approached Professor David Winkler (a Professor of Biochemistry & Chemistry at La Trobe University, Professor of Pharmacy at the University of Nottingham, and Professor of Medicinal Chemistry at Monash University) to assist in the theoretical motivation of the solutions and as a  second pair of eyes in reviewing the paper.

The paper was written in two weeks and the team was divided into two groups. The first group was made up of Divish, Grazziella and I, who developed the paper structure and decided on the content.  We used `Overleaf’, an open-source online collaborative platform for writing and editing documents, to write the paper. The second group was made up of Mercedes, Benjamin Rothwell (Divish’s supervisor) and David, who reviewed and proofread the paper. Comments were made in ‘Overleaf’ using color-coding and the platform’s chat functionality. After several iterations, we all agreed the paper was ready for submission. A month after submission, we received an email titled ‘Final Decision: Notification of Paper’ with delightful content that our paper had been accepted for presentation at the conference. It was indeed a pleasant message because we were aware of the importance of our toolbox to the machine-learning community. There were a few comments from the reviewers. We were required to address all the comments before uploading a camera-ready version of the paper with a maximum of 15 pages including references. I quickly created a google word document for addressing the comments and shared the link with the other authors. We dedicated a week to addressing the comments, as they were minor. Divish and I did a majority of the revision, as we were more familiar with the methodologies. The most recent version of the toolbox is limited to classification tasks; however, we plan to extend the toolbox to deal with regression tasks. In addition, the toolbox is open-source (i.e., accessible to the public for review, modification and extension)  and we encourage other researchers and ML engineers to contribute to its growth, such as improving the structure of the toolbox, implementing additional state-of-the-art ML algorithms and implementing other feature importance techniques.

Finally, I will like to mention the role of this paper in my PhD. To recap, my PhD aims to develop a reliable, comprehensive and interpretable intelligent driving risk assessment system that considers the simultaneous occurrence of driving behaviours and external conditions. The fuzzy approach implemented in the toolbox for aggregating importance coefficients from multiple machine learning models and feature importance techniques, and to make the importance of features easily understandable has also been explored in my thesis. In my thesis, we explore FLSs for capturing uncertainties in driving data and providing meaningful representations of driving behaviours as linguistic terms using fuzzy sets. In addition, we explore human-understandable fuzzy rules to combine the impact of driving behaviours and external conditions on road safety and incorporate expert knowledge into the system.

Gaining another perspective – a reflection on my internship with Derbyshire County Council (DCC)

post by Kathryn Baguley (2020 Cohort)

My internship opportunity arose from my ongoing work with DCC. Whilst working on another project in early January 2022, I was excited to hear the news that DCC was looking to initiate two large data projects. Hearing this made me wonder whether there might be an opportunity for me to experience some different work within DCC and complete my internship. Upon enquiring, I was delighted to hear that DCC and the CDT thought this would be a good match, and I was given the go-ahead to continue.

As you might expect, DCC, like many large organisations, holds vast amounts of data across many systems. The projects I worked on aim to enable more meaningful use of data for DCC, its employees, and service users. The work looks at aspects such as the accuracy of data, data migration, and the interoperability of new systems moving forward. At the end of the project, DCC hopes that having data in a better place can drive key performance indicators (KPIs). KPIs will assist with better planning, which it hopes will result in a better outcome for service users and generate much-needed savings for the public purse.

I completed my internship over three months on a full-time basis in the Information Technology team. I was excited to be a part of a new team, but it was reassuring that I had worked with some colleagues on previous projects. Overall, my role was part of a larger multidisciplinary team to assist DCC in achieving better management of its data, and enable more agile working.

Going into my internship I hoped for further insights into how a large organisation makes system changes, mainly around automated decision-making. I was also looking to dive deeper into understanding Microsoft systems and work more closely with my new colleagues in Information Technology (IT).

My internship plan was to liaise with IT, the project team and internal clients (various departments) to understand both the technology and the intended context for use. However, we quickly discovered that the scope was too broad, so I concentrated on the technology side and laid foundations to assist the context later. I prepared documents showing the work carried out for accountability of which also acted as a living document moving forward. I also gave ongoing feedback to the project and senior management teams. I recorded my work through a data protection impact assessment (DPIA). The document quickly grew to some 100 pages just to lay down the foundations. I feel the learning acquired here is something that I will definitely carry forward and use in my PhD writing. I am also reflecting on how the Council is carrying out these projects. I am thinking about how risks were assessed relating to automated decision-making and how I can expand my Peer Review Module output to integrate into my PhD writing.

My internship was fully remote, but I do not feel this negated my experience in any way. I felt fully integrated into the organisation and into the project teams from the very beginning. I accept that I may feel this way simply because of my pre-existing relationship with the organisation and colleagues within. It was helpful to me that I had some knowledge of the wider Council and basic knowledge of the IT systems. The experience did build my research skills in both a practical and academic way, since I had to use a combination of speaking with skilled colleagues and online research. It also underlined to me that I find research is much more satisfying if it can be done in a collaborative way with others, as it brings the topic to life. Moving forwards, I am now wondering how I might bring in the practical and collaborative approach to my work, so as to limit the usual loneliness a PhD can bring.

I thoroughly enjoyed the internship, and it is always a pleasure to work on projects where the people in the organisation are passionate about what they do and about doing the right thing. At the end of the internship, I was recognised for doing a good job, and this has helped me to build confidence in an area to which was technically unfamiliar.

On reflection of the module, I feel the biggest challenge was obtaining the right internship in the first place. I envisaged that the process might be more accessible in my case being a qualified and practising solicitor with several years of professional practice experience. In reality I found opportunities scarce, and maybe this appeared to be the case, owing to increased remote working in 2020 with the pandemic and less collaboration in general. I also had my doubts about the module and how I might make it useful for me. As someone with a professional career before starting my PhD, I was keen to ensure the internship gave me more than ‘work experience’ and provided insights that I could use in my PhD.

A further reflection I have is that while there is an emphasis on doing the internship early, I think it is more important to conduct it when the right opportunity comes along with the right organisation. I began to feel pressure to find and do the placement module from the start. For me, I did not feel I had enough detail to understand what I wanted from the experience in the first year. By my second year I did know more about what I wanted from the internship and I am very glad I did wait; the experience has definitely benefited me.

In conclusion, while the internship is complete, I am happy that DCC have provided me with more opportunity to continue working with them to progress these projects. The work carried out over the placement underlines for me the need for industry and academia to work together and how it can produce much better results for all. In particular, I am currently liaising with the IT team about building on the systems work, and risk assessments carried out in the internship.

 

 

Summer placement with Campden BRI

post by Melissa Clover (2021 cohort)

This summer (2022) I undertook a 3-month placement with my partner company, Campden BRI. They specialise in food and drink science and innovation in the UK, based largely out of their site at Chipping Campden in the Cotswolds. Deciding to complete this at the end of my first year was a deliberate decision despite being aware that my skills and research are in the very early stages; the hope was that being able to familiarise myself with the company and the different areas in which it works, as well as doing considerable networking, would provide a sound foundation on which to build our working relationship as the PhD progresses. I am pleased to say that this hope was realised and that whilst I did not complete a set project for Campden during the placement, the time spent with them over the summer has greatly developed my understanding of the company, as well as providing my first exposure to Consumer & Sensory Science in industry.

Right from the outset, my partner supervisor and I were determined to create a varied schedule for the duration of the placement. I was eager to express interest in a diverse range of departments and activities, knowing that even where not directly related to my studies, the experience would be useful in some way. My partner supervisor was highly engaged and helpful in facilitating this, producing a 12-week schedule and contacting all of the different people on my behalf to set up initial meetings so that I could then make arrangements independently. This was a highly useful activity and I would recommend it to anyone undertaking a placement of this nature where your activities and duties vary week on week; it created a sense of structure and helped me remain focussed on initial objectives and activities – something that as we all know can be difficult from time to time. The objectives outlined at this early stage were as follows:

      • To better understand Campden BRI as a business, getting to know how different departments are run and interact, what kind of data/databases are available as resources for the PhD research and how the member companies can be involved in the producer side of the research
      • To be exposed to consumer and sensory science in practice
      • To learn new methods of data collection and analysis within sensory and consumer
      • To undertake a mini project that can be shared, potentially with the members in the next Member Interest Group (MIG)

Despite a positive Covid test the day before I was due to begin, the schedule kicked off (2 weeks later than expected) with a visit to the Packaging Pilot Plant on-site in Chipping Campden. On the whole I was working remotely, with an average of 3 visits a month, but where possible (as was the case with this first day), I tried to have initial meetings with people in person and take any opportunities for in-person activities, such as lab tours and consumer testing days. This helped form part of the working activities that enabled the objectives to be achieved; these were as follows:

      • Shadowing/working with both the consumer & sensory team and packaging team
      • Designing a survey to understand trade-offs for consumers when considering certain categories of food products that they buy
      • Receiving training on Compusense and MaxDiff/Conjoint Analysis
      • Being given time within the placement to read and analyse existing work already done by Campden BRI e.g., relevant reports
      • Combining online work with visits to Campden and test centre in Leamington Spa depending on the nature of the work
      • Delivering short presentations on my findings within the next Packaging and Sensory MIGS (tbc)

The experience gained from these placement activities stems further than the networking opportunities gained and professional skills developed (e.g., communication, problem solving, collating and writing up information, teamwork), as much of it will be useful within the context of my PhD research. From designing a survey to learning new statistical skills and acting as notetaker within focus groups, I was fortunate in being able to engage in practical activities that have added to the skillset I will require going forward. I felt that I was able to achieve a sound balance of academic and industry-related skills development; the former came from learning new statistical methods and how to use consumer-specific software and the latter largely from shadowing within different departments. Additionally, I was able to gain useful insights into the production and industrial side of food systems, being reminded of the constant need to think about application of research and ensure that the context within which a certain system functions is carefully considered.

Regarding my take-home messages and/or lessons learnt, one of my most prominent observations was that of the differing pace between academia and industry. In reality, the initial objectives from an academic perspective were highly ambitious, due to the logistics surrounding ethical approval for a survey to be undertaken. This was not an issue as such, rather required adaptations to the schedule which ended up being advantageous (being exposed to another department that is highly related to my area of work – regulatory). It simply highlighted that once again the context of applied research must be considered. This is of course not unique to industry, as similar discrepancies exist between policy and academia relating to pace, funding and overall objectives. It was however useful to be reminded that throughout the duration of the PhD, I need to be mindful of deadlines, pace and expectations when collaborating with Campden.

I am thrilled to be starting my second year with the foundation this placement provided and feel confident that the connections made will be useful as I continue researching. My hope is that as I get more research experience under my belt, I will have more to offer Campden by means of a potential second placement and/or via their bi-annual Member Interest Groups.

Summer Placement with Nottingham City Council (NCC)

post by Torran Semple (2021 cohort)

During summer 2022, I was on placement with my industry partner, Nottingham City Council (NCC). My main responsibility was to draft NCC’s annual Domestic Energy Efficiency Fuel Poverty report, which combines statistics on fuel poverty, energy efficiency, housing quality and energy prices, as well as information about fuel poverty grants (e.g., financial assistance for Nottingham residents). The report was aimed at various groups, including the public, third-sector organisations and local businesses, therefore there was a particular focus on how to effectively communicate information to all audiences.  Given the report’s potentially varied readership, I was determined to use data visualisation techniques to communicate information in a way that was attractive to any reader. Rather strikingly, none of the council’s previous fuel poverty reports included any graphics and were instead populated with (objectively) cumbersome tables.

Fuel poverty is caused by a combination of factors, including housing quality and household income. Given that there is a geographical aspect to housing and income deprivation, intuitively you would expect the same for fuel poverty trends. Figure 1 shows a bivariate (fuel poverty and index of multiple deprivation) map of Nottingham’s Lower Super Output Areas (LSOAs) that I created to illustrate this point. This graphic is similar in nature to the map I generated for NCC’s fuel poverty report and, pending approval from council staffers, it will also be included in the final release. Nevertheless, this was the perfect excuse to hone my GIS skills – something that had lingered on my to-do list for too long.

Figure 1. Bivariate choropleth map of Nottingham, showing fuel poverty (LILEE metric) and Index of Multiple Deprivation (IMD) per LSOA

Academic and Political Collaboration

Since my undergraduate degree, I have been involved in research projects at the interface of academia and politics. These included analysing public perceptions towards pedestrianisation in Edinburgh city centre and exploring mobility patterns during the COVID-19 pandemic in Scotland. In both projects, Edinburgh City Council and Transport Scotland engaged with our research and in the case of the latter provided invaluable data to aid our analysis. Given the vast reach of governmental bodies, the data that can be captured are inevitably more extensive and representative than what would be achievable otherwise. The same applies to my current working relationship with NCC, and the summer placement provided an opportunity to become familiar with the council’s approach to collecting, storing and sharing data. I did, however, begin to realise that some data are so closely guarded, such as household income and employment data, that some aspects of my research questions would need to be tweaked accordingly.

Although my experience working with NCC was overwhelmingly positive, some differences in working style became apparent. For example, one of my research questions explores issues with the current Low-Income Low Energy Efficiency (LILEE) fuel poverty metric. The metric is controversial as it focuses on household efficiency, rather than the ability of occupants to afford energy, and omits ~40% of homes in Nottingham on the basis that they are too efficient (EPC ratings A–C). From a statistical perspective, this is an obvious oversight and is something that should be discussed and corrected where possible. However, the dominant opinion at NCC seemed to be that this issue was best left untouched, as fuel poverty metrics are imposed at a national level. Eventually, we were able to compromise, and I was permitted to add a short discussion to the report regarding the potential for the current metric to underestimate fuel poverty.

I may be wrong, but I suspect that one unresolvable difference between academic and governmental work is that the latter is subject to a greater level of public and media scrutiny. This likely perpetuates the need for councillors or any other elected officials to present things in a positive light and improve their chances of re-election. As a result, a metric that undeniably underestimates poverty may not be politically disadvantageous, whereas, from an academic perspective, this simply looks like inaccuracy. I still consider this to have been a very valuable experience for myself, as I became more acutely aware of the differences in priorities in an academic and political context.

Final Thoughts

In summary, my placement with NCC provided some excellent opportunities to network and become familiar with local policymaking processes, two experiences that will undoubtedly aid my PhD research. Perhaps most saliently, I now feel more comfortable working in the space that bridges academia and politics, which is of course key to ensuring that my research has real-world impact. I think that I was able to bring a fresh perspective to NCC and generally my ideas were well-received by colleagues. I was particularly pleased that I was entrusted to write the first draft of the fuel poverty report and that some of my ideas, such as including visualisations as communication tools, were adopted and will hopefully continue to be used in the future.

Reflections of my placement with Ipsos

post by Dan Heaton (2020 cohort)

Background

My PhD examines how discourses concerning the agency of decision-making algorithms can be understood using a hybrid language analysis framework. This framework consists of a multi-disciplinary methodological approach, involving NLP-based computational linguistic tools, corpus linguistics and critical discourse analysis, all underpinned by social actor representation. At the beginning of my doctoral studies in September 2020, my chosen partner for my PhD was the Trustworthy Autonomous Systems (TAS) Hub. TAS is a group of three universities collaborating on the ‘development of socially beneficial autonomous systems that are both trustworthy in principle and trusted in practice by the public, government, and industry’ – a very suitable partner, given my PhD focus!

However, as TAS is still very much within the ‘academia bubble’, my supervision team and I decided that it would be best to undertake my placement with an external industry-based company. After what seemed like an endless search, we found what we thought might be the right fit: Ipsos, the market research and public opinion business. Their Behavioural Science team, led by Colin Strong, were keen to collaborate on different social and commercial analytical projects, especially ones where I could see how their automated analysis worked and others that were concerned with improving and innovating existing approaches through using different qualitative lenses. As a result, I undertook my placement with Ipsos between July and September 2022.

Disinformation Radar

The main project I was involved with was the Ipsos disinformation radar. This involved two main parts: detecting disinformation using linguistic markers and the exploration of social media users respond to and discuss disinformation online. This was in direct response to a client brief that had been received regarding the detection of disinformation and preventing its online reach.

In the first part, involving disinformation detection, I surveyed many pieces of literature regarding disinformation detection. Beginning with literature felt like a familiar starting point, which was very welcome, but I was very excited to see how this would play out when we applied these detection ideas on pieces of suspected disinformation. Although we anticipated that many pieces of disinformation that were shared would be text based, we found that the vast majority were external video links – something we hadn’t accounted for at all in the application of our proposed detection framework! Nevertheless, an exciting challenge was to see whether this applied to the transcripts of these videos, only proving the difficulty of the task at hand and how fulfilling the original brief would have been bordering on the impossible.

After this, I focused on the second part of the project, which involved using ideas stemming from Linguistic Inquiry and Word Count dictionaries, as suggested in Colin’s favourite academic paper by Humphreys and Wang (2017). We had a great behavioural focus and ended up devising a system that detected the intuitive and reflective processes present in tweets, indicating how a tweet author might have responded to disinformation.

I was delighted to be able to bring corpus linguistics into the picture here in order to analyse collocates (frequently occurring adjacent or near-adjacent words) in discourses. Showing Ipsos staff how corpus linguistics tools work was a real highlight and cemented a feeling that I wasn’t just there to learn, but I was also there to teach. As a former primary school teacher and leader, this was very satisfying!

However, the most exciting part of working on the disinformation radar project was the idea that we were doing something that went against norms. I think I went into this placement with a strange mindset: the translation into industry might end the creative freedoms of academia and impose restrictions on what we can and can’t do due to an increased focus on outputs. I am happy to say that this was not the case at all. The innovative way that we worked on this project was not only refreshing,

Though there is something to be said about people’s ‘readiness for change’ – something that frequently appeared as an item of discussion in our meetings. With social media analytics (including the detection of disinformation) having a heavy focus on what automated detection can do, we anticipate some resistance to our ‘pick-and-mix’ approach. This is a challenge that resonates with my PhD too. Although we didn’t quite find the ultimate answer to tackling this, I now feel well-equipped with strategies to get people on board with a multi-disciplinary approach when arguing the relevance of my doctoral project.

Social Analytics for Clients in Action

I also gained experience of working directly with the social analytics team, supporting on client and company briefs. The brief I worked on was investigating sustainability across social media platforms – what does ‘sustainability’ means in social spaces? What do online authors associate with sustainability? This was a good opportunity to use Ipsos’ social intelligence analytics platform, Synthesio, and delve into their sentiment and topic modelling tools.

Undertaking this work brief helped me realise the needs of clients in Ipsos’ daily routines. It also allowed me to reflect on how NLP-based computational linguistic tools were being used and interpreted. It was interesting to use different software to what I have engaged with so far during my doctoral studies. For example, the topic modelling was a little more sophisticated and even attempted to name the topics for you! Naturally, this often ended with some errors, but the topic groupings seemed reasonable and helped narrow down a data focus. That being said, the outputs from Synthesio weren’t necessarily revolutionary when comparing them to the ‘off-the-shelf’ NLP-based tools I’ve used prior. Obviously, this isn’t a problem unique to Ipsos, but it’s a good reminder of why my PhD is important in tipping that scale and showing that a combination of approaches is beneficial.

Final Reflections

Overall, this placement helped me realise how research like mine translates out of academia. Combining quantitative and qualitative linguistic analytics is one thing, but making sure you’ve got a strong ‘philosophical stance’ (as Colin aptly named it) is another. I was imagining I might see a few things being done for the sake of it at Ipsos; however, when it came to it, it was refreshing to see that the desire for underpinning choices with a strong theoretical rationale was very much there with the disinformation radar project. It was also brilliant to see how things really worked in applied social analytics and a great reminder of the necessity of my PhD work so that the reliance on automated outputs isn’t skewed one way or another. Through undertaking this project, I believe it helped refine my analytical skills in an adjacent area of study: cognitive linguistics. This will, ultimately, be beneficial to my doctoral studies as a potential theoretical or methodological underpinning to my work.

I would like to thank Colin for his time, effort, enthusiasm and guidance during this placement. Duncan Fergusson and Danielle Latreche were also great advisors, too, and I couldn’t have done it without them. As the placement ends, I’m very much looking forward to continuing our working relationship together for the remainder of the PhD and beyond.

Playing between industry and academia: reflections on my placement with Makers of Imaginary Worlds

post by Daniel Swann (2021 cohort)

Introduction

As part of my PhD project at the Horizon Centre for Doctoral Training, I underwent a placement throughout the summer of 2022 with my industry partner Makers of Imaginary Worlds (MOIW)—an arts company that designs interactive installations for young children and families. Interestingly, the company was founded by Horizon CDT alumnus Roma Patel (2013 cohort) and many of its ongoing projects have grown from her own doctoral research. As such, the three months I spent working alongside MOIW offered an invaluable insight into how the central issue of my own PhD—that is, how children’s playful experiences can inform interactive technologies—are engaged within the “real world.”

As is typical within the creative industries, MOIW is a small company for whom no two working days are the same. Furthermore, by creating interactive art installations with innovative technological features, it engages regularly with a wide variety of stakeholders including cultural institutions, local families, and the research community. Fortunately, this meant that my placement involved a range of different activities, and I will share the main experiences from my time spent working with MOIW over the course of this blog post. Ultimately, I hope to identify some key takeaway points of how this experience has shaped my research which gesture towards how industry can productively engage with academia more generally.

TAS HUB All Hands Meeting

After settling in and getting acquainted with the team at MOIW, one of the first major events on the agenda was presenting at the annual Trustworthy Autonomous Systems (TAS) All Hands Meeting in London. The TAS Hub is a UKRI-funded project which aims to encourage collaboration and world-class research into autonomous systems around six main nodes: functionality, governance and regulation, resilience, security, trust, and verifiability. While this covers a vast range of activities involving everyone from senior academics to policymakers, the TAS Hub also works with “artists, musicians, directors, writers, and performers” as part of its Creative Strand to express its research aims in a way that is more accessible to the public.

The All Hands Meeting is an opportunity for all those working under the TAS umbrella to share their projects and, as such, I joined MOIW to present their artistic residency as part of the 2022 TAS programme. This was centred around the installation NED: the Never-Ending Dancer. NED is a robotic arm that has been programmed to dance with children who engage with it through movement and face recognition technology. It was previously part of a larger installation called Thingamabobas but was isolated for a pilot study in April 2022 in which we attempted to understand how children understand and engage with a robot that is behaving in a playful way. More details of this study as part of the TAS artistic residency can be found on the MOIW website.

While representing MOIW with NED beside me in the live exhibition space, I received a lot of interest in the project—particularly from researchers who were working in areas that had little to do with children or the arts. Some of the conversations that occurred, as a result, justified the TAS Hub’s claim that creative applications can often open the space for dialogue better than research or technology alone can. It also provided me with valuable experience in presenting studies that I’m involved with to an entirely new audience.

Alongside the live exhibition space, the All Hands Meeting also featured talks and panel discussions from a number of academics working on TAS projects. I ended the two-day event by attending the TAS Programme Early Career Researcher dinner, which offered a great opportunity to network with fellow PGRs and learn more about how they are engaging with the various nodes that contribute towards the TAS Hub.

Home:Zero Study

The next major phase of my placement was involving the HOME-Zero project. HOME-Zero is “a creative research and development project to help ignite a public conversation about the relationship between household emissions and climate change.” Makers of Imaginary Worlds were commissioned for this project and produced an original interactive installation in collaboration with Lakeside Arts and the Mixed Reality Lab at the University of Nottingham and through a series of co-design sessions involving young families from the local area that took place over several weeks. The installation was exhibited at the National Gallery X studio and Lakeside Arts in May 2022, but an additional exhibition was organised for Strelley Road Library during my placement towards the end of July.

The set-up of the installation itself proved challenging and was therefore a valuable insight into the practicalities of producing interactive experiences that are suitable for a wide variety of contexts. The installation was designed for use in gallery or theatre spaces and had to be adapted to the library, including alterations to the functionality of certain elements and impromptu repurposing of others. There were logistical difficulties also: some of the technology included in the installation required support from various collaborators and getting in touch with them when needed in the short window of time provided for set-up proved challenging. We recognised that it would have been useful to have a detailed technical documentation for the installation beforehand. However, the unique way in which HOME-Zero was (co-)created meant that there were limited resources (namely, time) for producing such documentation.

Despite the difficulties of reproducing the HOME-Zero experience at a public library, the response proved it was more than worthwhile. Families from the local area were able to enjoy this experience—complete with interactive technological features and an engaging actor/facilitator—and begin intergenerational conversations around sustainability and household consumption. Along with the overwhelmingly positive response, I was able to conduct interviews with some of the families involved in the co-design process of HOME-Zero who offered unique perspectives on the installation that are likely to contribute to a future publication.

Theatre Hullabaloo and The Undiscovered Island

Beyond the smaller and more miscellaneous tasks involved with the placement, the final major project involved taking MOIW’s The Undiscovered Island installation to Theatre Hullabaloo. This venue, based in Darlington, is innovative for providing a space dedicated to theatre for younger audiences. Although The Undiscovered Island is one of MOIW’s most toured installations, it required some preparation before going to the theatre to be in exhibition for 6 months in total!

The Undiscovered Island is an immersive experience for young children to learn about the ecological challenges faced by coral reefs through several different interactive technologies. Some of the artefacts included in the exhibition had been developed from prototypes and were therefore identified as needing minor repairs and reinforcement. For a few weeks in August, I occupied a bay in the Mixed Reality Lab to work on some of the artefacts, which mainly involved resoldering, swapping out components, and stress-testing to try to reproduce 6 months of interactions and to ensure that all the artefacts were safe for small children to use. Although not all the repairs were straightforward, I was fortunate to have help from colleagues at MOIW and other researchers working in the Mixed Reality Lab who were happy (or at least willing) to help! With everything ready to go, myself and the MOIW team set off for Darlington to spend a week getting The Undiscovered Island ready.

The set-up itself went well and it was a pleasure to work with friendly colleagues from both MOIW and Theatre Hullabaloo, which truly does fantastic work for the communities it serves. Like at Strelley Road Library, we had to make some decisions on how best to adapt the installation to the space but within a few days, we had everything up and running and ready to open by mid-September. In contrast to my experience at the TAS All Hands Meeting, my time working in Darlington was characterised by collaborating with individuals who were vastly experienced in many aspects of my research although from a largely non-academic perspective. I found this to be particularly beneficial as I could learn more about the concerns of important stakeholders in my research—namely young families and arts practitioners—in an organic and meaningful way. On a more practical level, it also allowed me to work with individuals and institutions with whom I may be able to collaborate further at some point in the future.

Conclusion

As the placement drew to an end and I made my transition back into being a full-time PhD student, I was able to reflect on how these months working with MOIW may inform my research going forward.

    • Seek opportunities to share research with the wider community. Some of the insights offered by researchers from ostensibly unrelated fields at the TAS All Hands Meeting were truly interesting and valuable. While it is obviously important to keep consistently in contact with one’s own research community (the field of child-computer interaction, in my case), I believe taking opportunities to present in different contexts can produce surprising benefits.
    • Never underestimate the value of even the simplest documentation. In the case of HOME-Zero, it wasn’t possible to produce thorough documentation for set-up due to time restrictions. Nevertheless, I learned that making notes along the way can result in massively improve efficiency later. I imagine this to be as true for setting up an installation as it is for writing a conference paper, for example.
    • Get into the field with your “researcher hat” only half-on. Conventional academic forms of gathering information from stakeholders—such as through interviews or focus groups—have their uses but also their limitations. Collaborating directly with someone who works in my field shed light on various issues that I hadn’t previously considered and is the type of opportunity that an industry placement can uniquely offer.

Those who work in the arts and those who work with children often have unconventional working habits when compared with other industries. As a company that does both, this is certainly true for Makers of Imaginary Worlds! But this meant that my placement was a hugely valuable experience in allowing me a deeper understanding of the opportunities available beyond my doctoral studies, both in the sense of what may come after my PhD and the impact my research can be made outside of academia.

Reflection on the British Machine Vision Association summer school

post by Muhammad Suhaib Shahid (2020 cohort)

Over the summer of 2022, I attended the annual British Machine Vision Association summer school at The University of East Anglia. The weeklong summer school provided an opportunity for computer vision researchers, in both academia and industry, to explore a wide range of computer vision topics through intensive lectures and labs. From the 11th to the 15th of July, I had the chance to learn from expert instructors, participate in hands-on activities, and connect with other professionals and students in my field. In this reflection, I will share my key takeaways and insights from the summer school, as well as any challenges or setbacks I faced and how I overcame them. I will also discuss the impact of summer school on my personal and professional development.

Over the course of the week, I had the opportunity to learn from expert researchers from some of the most active research groups in the field of computer vision both in the UK and abroad. Throughout the series of lectures, they provided comprehensive overviews of the fundamental concepts and techniques of image processing and analysis, as well as lectures on unsupervised learning, image segmentation and deep learning for Computer Vision. Though most computer vision researchers are very much adept in these areas, re-covering these subjects proved key for a complete understanding of the advanced topics that we encountered later in the program.

Over the five-day period, there were 16 lectures and one lab with each session being run by an expert lecturer or instructor in that specific field. Most relevant to my research were two afternoon sessions held on days two and three. The first headed by Oscar A. Mendez on “Deep Learning for Computer Vision” and the second by Chris Willcocks on “The concepts and characteristics of different deep generative modelling approaches”. The two sessions addressed the use of Computer Vision in medical image processing and synthesis, a topic key to my research. Deep learning, which is a type of machine learning that uses neural networks to learn from large amounts of data, was a reoccurring theme throughout the week. Many researchers, myself included, encounter issues surrounding finding large enough data sets, a problem prevalent, especially when working with medical imaging.  I found these topics to be particularly interesting, as they are at the forefront of computer vision research and have many practical applications in a variety of fields.

In addition to the lectures and discussions, we also had the opportunity to participate in a few hands-on activities that allowed us to apply what we were learning in a practical setting. The afternoon session of the first day took place in a computer lab. Working on individual tasks, through helping each other along when needed, we implemented various PyTorch functionalities from scratch. It was fascinating to see how the tools and packages used in our everyday work as researchers are implemented, something that often crosses my mind now months later whilst undertaking various programming endeavours.  I appreciated the opportunity to work with my team and receive both instruction and feedback from the instructors.

One of the most memorable experiences I had during the summer school was the opportunity to connect with other professionals and students interested in computer vision. The program brought together a diverse group of individuals from a variety of backgrounds, disciplines, and even countries. It was inspiring to see the different perspectives and approaches that people brought to the table. We had several networking events and social activities that provided us with the opportunity to get to know each other and learn about each other’s interests and goals. I made some lasting connections that I believe will be beneficial in my future academic and professional endeavours.

Overall, the summer school has had a significant impact on my personal and professional development. In addition to learning about the technical aspects of computer vision, I also gained valuable skills in networking, problem-solving, and communication. I feel more confident in my ability to design and implement computer vision systems, and I am excited to explore the many applications of this technology in the future. However, the summer school was not without its challenges and setbacks. One of the biggest challenges I faced was keeping up with the fast-paced nature of the program. There was a lot of information to absorb in a short period of time, and I found myself struggling to keep up at times. However, I was able to overcome this challenge by staying organized, asking for help when I needed it, and making the most of the resources available to me.

I do have a few suggestions for improving the summer school experience for future participants: it would be helpful to have more time for networking and connecting with professionals in the field. While I enjoyed the opportunity to meet and collaborate with other students, I also think it would be valuable to have more structured opportunities to engage with industry professionals and learn about career paths in computer vision. There was only a lecture that briefly touched on how to find the best post-research opportunities. Additionally, the time allocated to lab work was not sufficient. There was only one session, and no opportunity to implement our knowledge in any real-life scenarios; I would suggest that future events take advantage of the wide knowledge base available and allow students to work in groups, fostering greater opportunities to apply their domain-specific knowledge.

In conclusion, the summer school on computer vision was a valuable and enriching experience that has had a lasting impact on my personal and professional development. I am grateful for the opportunity to have learned from expert instructors and to have connected with other like-minded individuals. I highly recommend this summer school to anyone interested in exploring the exciting field of computer vision!

Angela meets Industry Partner to discuss their future strategy

post by Angela Thornton

I’ve just come back from 5 days in Nardo, Southern Italy where I was on retreat with my industry partner the Carboncopies Foundation (CCF). We stayed in Relais Monastero Santa Teresa – a 13th-century baronial palace which was subsequently converted into a cloistered monastery. The bedrooms were all beautifully unique and our meeting rooms were equally stunning even if getting the Wi fi to work and finding enough plug sockets and adapters could be challenging! We had a few hours off on the Friday to explore the historic town with its cobbled streets and beautiful buildings and even managed a short trip to the nearby seaside on Sunday when it was a lovely sunny day.

However, we had an intensive schedule which involved launching new departments and initiatives and also progressing existing projects. I’m on the Board and actively involved in both the main Research and Education offices as well as leading the Ethics department so there was a lot of information to discuss and disseminate. Specifically I led an ideation session on an Ethical Framework for Whole Brain Emulation (WBE) which is one of my key areas of responsibility for 2023 and beyond. This is an active research area but WBE in humans is likely to take many decades so planning to ensure responsible research and innovation is challenging. Thankfully as an ex-industry researcher, I have considerable experience in ethics and codes of conduct and was on the Ethics and Compliance Committee for the British Healthcare Business Intelligence Association. Hence I was able to draw on that experience to moderate a creative session to identify key topics and risks. Following idea generation, we conducted some interim thematic analysis which I will review and develop to provide a draft table of contents for the Ethical Framework. This first iteration is an internal process, but CCF will subsequently consult with other experts in the field with the aim of authoring the first set of Ethical Guidelines for WBE.  It was particularly productive to have members of the CCF who aren’t always directly involved in ethics as they bought a unique perspective.

The event was notable not just in how much we achieved collaboratively but how enjoyable it was to spend time with people that I had only previously met virtually. Our shared memories will include some entertaining moments including star gazing from the roof and spinning LED glow poi.