post by Dan Heaton (2020 cohort)
Background
My PhD examines how discourses concerning the agency of decision-making algorithms can be understood using a hybrid language analysis framework. This framework consists of a multi-disciplinary methodological approach, involving NLP-based computational linguistic tools, corpus linguistics and critical discourse analysis, all underpinned by social actor representation. At the beginning of my doctoral studies in September 2020, my chosen partner for my PhD was the Trustworthy Autonomous Systems (TAS) Hub. TAS is a group of three universities collaborating on the ‘development of socially beneficial autonomous systems that are both trustworthy in principle and trusted in practice by the public, government, and industry’ – a very suitable partner, given my PhD focus!
However, as TAS is still very much within the ‘academia bubble’, my supervision team and I decided that it would be best to undertake my placement with an external industry-based company. After what seemed like an endless search, we found what we thought might be the right fit: Ipsos, the market research and public opinion business. Their Behavioural Science team, led by Colin Strong, were keen to collaborate on different social and commercial analytical projects, especially ones where I could see how their automated analysis worked and others that were concerned with improving and innovating existing approaches through using different qualitative lenses. As a result, I undertook my placement with Ipsos between July and September 2022.
Disinformation Radar
The main project I was involved with was the Ipsos disinformation radar. This involved two main parts: detecting disinformation using linguistic markers and the exploration of social media users respond to and discuss disinformation online. This was in direct response to a client brief that had been received regarding the detection of disinformation and preventing its online reach.
In the first part, involving disinformation detection, I surveyed many pieces of literature regarding disinformation detection. Beginning with literature felt like a familiar starting point, which was very welcome, but I was very excited to see how this would play out when we applied these detection ideas on pieces of suspected disinformation. Although we anticipated that many pieces of disinformation that were shared would be text based, we found that the vast majority were external video links – something we hadn’t accounted for at all in the application of our proposed detection framework! Nevertheless, an exciting challenge was to see whether this applied to the transcripts of these videos, only proving the difficulty of the task at hand and how fulfilling the original brief would have been bordering on the impossible.
After this, I focused on the second part of the project, which involved using ideas stemming from Linguistic Inquiry and Word Count dictionaries, as suggested in Colin’s favourite academic paper by Humphreys and Wang (2017). We had a great behavioural focus and ended up devising a system that detected the intuitive and reflective processes present in tweets, indicating how a tweet author might have responded to disinformation.
I was delighted to be able to bring corpus linguistics into the picture here in order to analyse collocates (frequently occurring adjacent or near-adjacent words) in discourses. Showing Ipsos staff how corpus linguistics tools work was a real highlight and cemented a feeling that I wasn’t just there to learn, but I was also there to teach. As a former primary school teacher and leader, this was very satisfying!
However, the most exciting part of working on the disinformation radar project was the idea that we were doing something that went against norms. I think I went into this placement with a strange mindset: the translation into industry might end the creative freedoms of academia and impose restrictions on what we can and can’t do due to an increased focus on outputs. I am happy to say that this was not the case at all. The innovative way that we worked on this project was not only refreshing,
Though there is something to be said about people’s ‘readiness for change’ – something that frequently appeared as an item of discussion in our meetings. With social media analytics (including the detection of disinformation) having a heavy focus on what automated detection can do, we anticipate some resistance to our ‘pick-and-mix’ approach. This is a challenge that resonates with my PhD too. Although we didn’t quite find the ultimate answer to tackling this, I now feel well-equipped with strategies to get people on board with a multi-disciplinary approach when arguing the relevance of my doctoral project.
Social Analytics for Clients in Action
I also gained experience of working directly with the social analytics team, supporting on client and company briefs. The brief I worked on was investigating sustainability across social media platforms – what does ‘sustainability’ means in social spaces? What do online authors associate with sustainability? This was a good opportunity to use Ipsos’ social intelligence analytics platform, Synthesio, and delve into their sentiment and topic modelling tools.
Undertaking this work brief helped me realise the needs of clients in Ipsos’ daily routines. It also allowed me to reflect on how NLP-based computational linguistic tools were being used and interpreted. It was interesting to use different software to what I have engaged with so far during my doctoral studies. For example, the topic modelling was a little more sophisticated and even attempted to name the topics for you! Naturally, this often ended with some errors, but the topic groupings seemed reasonable and helped narrow down a data focus. That being said, the outputs from Synthesio weren’t necessarily revolutionary when comparing them to the ‘off-the-shelf’ NLP-based tools I’ve used prior. Obviously, this isn’t a problem unique to Ipsos, but it’s a good reminder of why my PhD is important in tipping that scale and showing that a combination of approaches is beneficial.
Final Reflections
Overall, this placement helped me realise how research like mine translates out of academia. Combining quantitative and qualitative linguistic analytics is one thing, but making sure you’ve got a strong ‘philosophical stance’ (as Colin aptly named it) is another. I was imagining I might see a few things being done for the sake of it at Ipsos; however, when it came to it, it was refreshing to see that the desire for underpinning choices with a strong theoretical rationale was very much there with the disinformation radar project. It was also brilliant to see how things really worked in applied social analytics and a great reminder of the necessity of my PhD work so that the reliance on automated outputs isn’t skewed one way or another. Through undertaking this project, I believe it helped refine my analytical skills in an adjacent area of study: cognitive linguistics. This will, ultimately, be beneficial to my doctoral studies as a potential theoretical or methodological underpinning to my work.
I would like to thank Colin for his time, effort, enthusiasm and guidance during this placement. Duncan Fergusson and Danielle Latreche were also great advisors, too, and I couldn’t have done it without them. As the placement ends, I’m very much looking forward to continuing our working relationship together for the remainder of the PhD and beyond.