Mel travels to Sweden for CRITIS2019

Mel Wilson  (2018 cohort) has recently returned from the CRITIS 2019 conference in Linkoping, Sweden where she successfully submitted the paper titled  Exploring How Component Factors and Their Uncertainty Affect Judgements of Risk in Cyber-Security 


Post by Melanie Wilson (2018 Cohort)

This conference paper was submitted following the work done for my PLP. It involved the process of recording and analysing the effects of uncertainty in experts’ ratings of cyber security risks, using an interval range method. This procedure allows capture of a value for uncertainty in a rating, by the use of an elliptical marking, as illustrated below.

A larger ellipse represents a greater uncertainty and a smaller represents greater certainty.

Following the PLP write up my PLP supervisor Josie McCulloch suggested that I might like to present the findings at a conference. Coincidentally I had been talking with an industry colleague who holds a doctorate in cyber security and is a senior figure in the cyber security industry with a large, international company with a particular interest in the industry sector addressed by the PLP work. He had suggested CRITIS as an ideal showcase for the paper.

After passing the details on to Josie, Zack and Christian we all felt that this was a worthwhile submission and after some discussions decided that we would submit a long paper with some adjustments from the original PLP work, to cover a greater range of data and greater depth of statistical analysis. Zack was to work on the statistical adjustment, with Josie and me looking at the general paper presentation and Christian inputting as necessary from his more experienced perspective of conference submissions.

I have been a commercially published author since the late 1980s, but I have not published an academic paper before. I enrolled on the Graduate School online course “ An Introduction to writing for academic Journals” which I found very helpful. It explained clearly the process and ways of dealing with each stage both practically and emotionally, as it recognised that peer review can be a harsh undertaking and hurtful if one’s mindset it not aligned to the process.

In general peer review does not differ too much from that of submitting to a mainstream publication. The biggest difference from my perspective was that you had several reviewers, rather than just a single editor. This meant that there were several different perspectives to address.

As a team I felt that each of us as contributors brought a different perspective and style to the paper. I had wondered how this might be aligned, as this kind of working, on a paper, was new to me. We all contributed to the paper by using overleaf; we also discussed ideas around changes and met to talk over differing aspects before the first submission. This was a really interesting process and one I really valued as it gave me a good insight into the way others could work in academia.

Following submission and peer review we again put our ideas forward on adjustments that could be made, and each contributed in their area. We met a discussed the points and addressed each one that was highlighted by each reviewer taking on board the suggestions and often hypothesising on the perspective of each reviewer and their field of expertise. Most of the points were valid and were useful contributions towards clarity and completeness within the paper.  I feel we addressed all the points we felt had validity for change and we explained our perspective if we felt the point was perhaps unclear, but correct. I felt that working in this way was very helpful and our different ways of looking at the project benefited us all as it brought greater depth of multiple perspectives in to play.

I was impressed with how we all worked together on the project and how well everyone’s skills complemented the others. I’ve worked in a great many industry and charity sector teams and am very aware of the psychological process of team building, but in this case the transitions were smooth and at all times calm and friendly.

CRITIS2019
CRITIS2019

The conference was very interesting and gave me a chance to hear about other work in the area as well as the have many talks with various attendees on a large range of associated subjects. Particularly interesting were those working is gamification on learning strategies which link into my PhD work.

An added and very exciting bonus was that the paper was presented with the Young CRITIS award. The conference process has a rejection rate of 2/3rds of submissions and our paper was stated as a clear winner of the award, which is something I feel proud to have been a part of.

I am hoping to use this method of recording uncertainty in some of the questionnaires for my PhD in terms of capturing the uncertainty of risk, online as experienced by children.  I also want to use the capture of uncertainty from the teachers of the children in terms of the skills changes they perceive that the children have experienced.

From my perspective as an Evolutionary psychologist the capture of uncertainty in risk is a very necessary part of the data needed to improve the industry’s ability to predict and assess how both experts and the general population assess risks and consequently respond to them. Using this uncertainty capture can help us to analyse what biases may be influencing decision making and to find methods to mitigate these as we increase individuals’ abilities to accurately predict the probability of the risks effect.

I am pleased that I undertook the PLP I chose and have progressed the work in this way. I am looking forward to working with this method and team in the future.

LINK TO PAPER https://arxiv.org/abs/1910.00703

Human Aspects of Cyber-crime and Online Fraud Summer school

Post by Melanie Wilson (2018 Cohort)

This summer school was presented by Canterbury University in collaboration with the Leicester Castle Business School of De Montfort University, Leicester.

I attended on the Monday & Tuesday of this three-day event, as I had a family commitment on the Wednesday.

The presentations on the Monday were specifically addressing ideas around cyber-crime, social engineering and fraud. These are particularly relevant to me as my PhD is around increasing children’s abilities to identify and resist activities and approaches whilst online. I am addressing these from the perspective of enabling children to recognise attempts from others to engage in social engineering and to have the confidence and personal autonomy to reject anything they feel uncomfortable with, and to seek help where it is needed.

I am working with the Northamptonshire Police cyber-crime team on this and as such have a valuable insight into the challenges they have seen children facing as well as my own perspective as a psychotherapist working with children.

The summer school was the first of its kind to be run at Canterbury University and was led with great enthusiasm and skill by Jason Nurse. The summer school involved people from academia, social enterprise and industry which allowed a large variety of input and ideas to be expressed. Jason was skilful in accommodating discussions within the topics and I found that this approach, rather than the “talked at” approach, was very beneficial. Cyber-crime is a fast-growing field and the traditional approaches of academic study, which often take years to complete, are at risk of being overtaken as both technology and its associated exploitation by criminals proceeds at a rate far outstripping the slower traditional progress of academic work.

I feel this pace change was reflected well in the way the summer school was run. Some of the input could have been improved with more industry input to increase the pace and knowledge of the current challenges further, but I believe that Jason is aware of this and plans to address this in future events.

The first session explored the basics of cyber-crime reflecting on what forms it can take and highlighting how insidious it can be. It addressed the aspects of this area of criminology that are rapidly expanding and exploiting the tools that are available to enhance crime via technological means. One big take-home from this introduction and discussion was how fast this is developing in criminal circles where there are few restrictions and great financial gains to be made. This is at odds with the crime fighters and honest technological industry where there are checks and balances to be met in all circumstances which often results in a slower response which the criminal can exploit.

The Cyber Protect and Prevent Officer for Kent then gave us her perspective on how cyber-crime was affecting policing and the tensions between businesses which often wanted  just to solve the issue and move on with business as usual and the desire to pursue cyber-crime as a crime with ramifications for the criminals.

The final session of the day looked at profiling cyber criminals and looking at how these criminals might be led into a perpetrator role. This is particularly relevant to my work because vulnerability leads to both perpetrator and victim activities and the two routes often share common factors.

At the end of the sessions we arranged to meet after dinner in a pub in Canterbury. Doing so was valuable as it provided a relaxed atmosphere in which to talk further with other attendees both about their work and that of others and gave a great deal of insight into the varying field that are involved with this ever-expanding and important field.

Tuesday morning looked at how cyber-crime is often underlined by psychological methodologies that criminals have learnt to use in order to perpetrate their crimes. We explored how social engineering uses a number of methods to elicit cooperation from people, utilising their vulnerabilities and often just their normal desires to help others and be nice. Again, this is an area that I focus on a great deal and feel addressing our ability to say “no” is fundamentally decreasing individuals’ vulnerability to such tactics. There is a noticeable difference in the psychological mechanisms that criminals exploit ruthlessly and the non-criminals’ tendency to trust.

The afternoon sessions addressed the cost to businesses from cyber-crime.  It was led by Edward Cartwright from De Montfort University and Anna Cartwright from Coventry University. It addressed business vulnerabilities and the attacks that businesses face daily, and the routes into the enterprise which are often indirect. The conflicts of security and running businesses were highlighted and discussed. The reality of end users often not rejecting companies following a breach and whether reputational damage is as damaging as is often thought.  In Anna’s session we looked at the financial motivation for attacks and at what level the attacks and demands became profitable for criminals.

Finally, we looked at the problems and advantages in cyber security that Small and Medium Sized Enterprises face, addressing the challenges of this sector where often there are just a few individuals trying to complete multiple roles.

This summer school was fun. Where a learning experience is fun, lively and open to discussion, I feel far more is gained than from a situation where there is just one voice with very little interaction.

It greatly benefited from a range of perspectives and allowing those to be expressed and discussed. I feel everyone learned something from the variety and range of participants at the event and very much look forward to taking part next year.

Creativity Sandpit with Imperial, Cambridge, and Nottingham

This past May, 28 PhD students from three CDTs based at Imperial, Cambridge and Nottingham participated in a creativity sandpit.

Post by Natalie Leesakul and Shazmin Majid (2018 Cohort)

One of the perks of being Horizon CDT students is the connection with the network of centres for doctoral training led by the Engineering and Physical Sciences Research Council (EPSRC). Through this connection, Shazmin and I took part in a 3-day creativity sandpit workshop organized by Imperial HiPEDS in collaboration with Nottingham Horizon and Cambridge Sensors CDTs. This workshop was quite a unique summer school experience as it was designed for the students to work in a multidisciplinary environment to solve real world industry based problems using the Creative Problem Solving (CPS) framework to help generate innovative ideas. Industry professionals from companies such as a Formula 1 racing team were also in attendance.

On the first day, the workshop started off with various ice-breaking activities, providing the space for the students and industry professionals to get familiarized with each other and learn about others’ backgrounds. Fortunately, as part of Horizon CDT training, we are prepared to do an elevator speech on the spot; hence, giving a quick snapshot of our PhD research was not too difficult. However, the challenging part was discussing the research in more detail with others who have very different backgrounds and skillsets. While Shazmin and I have a background in Psychology and Law respectively, majority of the students are engineers and computer scientists. It was certainly refreshing to hear from a different perspective. Some of the questions and suggestions we got were something that we have not thought of before. Certainly, one of the crucial skills that we have taken away from this experience is the ability to simplify our PhD projects into something that a mixed audience can understand while maintaining engagement in the discussion. This ability to simplify our work in the wild helped us to connect with our own work and really hone in what we are trying to achieve in terms of aims and objectives.

On the second day, after the ice-breaking activities, we went on to learn about CPS and how it can be applied to different projects from the industry to academic research projects. This framework comes in quite handy for doctoral students as our aim is to find the research gap and produce innovative and original ideas to that field of research. This framework is applicable throughout the whole PhD, from the time when we have writer’s block to the time when we have a lot of ideas and not knowing what to focus on. CPS can help in generating the innovative ideas and also eliminating the ‘not-so-good’ ones; it introduces 4 stages of idea developing: clarify, ideate, develop, and implement. During this activity, we were encouraged to discuss with others and to think about our preferences in the process based on our skills. Following the CPS group work, the next activity was to form a team and explore the real-world challenges provided by the industry ranging from Formula 1 racing to drone solution to ad-tech. This was when we needed to put the learned skills together – maintaining a good rapport in a team consisting of people from different backgrounds and applying the CPS framework.

Natalie’s Experience

For the second day activity, I chose a Formula 1 (F1) racing challenge on how to best make use of the existing data in order to gain more fan base. The problem we received was that F1 fan base started to become stagnant, if not declining. Hence, the goal of this challenge was how to attract the new demographics while maintaining the current fan base.

In term of applying the CPS framework to the sandpit challenge, I’ve learned that the stage of clarifying and ideating was my strongest suit. Coming from the business and law background, I’m very comfortable with unpacking the problem, identifying facts, and generating solutions at the abstract level. In contrast, developing technical solution is not part of my practice. Hence, when forming a group to work on the challenge, I looked for others who had the technical expertise. As mentioned before, working in a multidisciplinary team was certainly a challenge on its own. My team consisted of 5 people – our team dynamic ranged from law to marketing to engineering. Certainly, we had a significant competitive advantage over the other teams given the number of people we had on the team and the wide range of expertise. However, the brainstorming session got very messy and unorganized which was problematic as we had to work in a limited amount of time. We started off with shouting out different ideas. Then, we had to choose the most promising idea to further develop which we struggled with quite a bit at this stage.

Thus, this was when we applied the CPS framework. We revisited the challenge objective to truly understand what was the problem we were trying to solve. From that, we looked at the ideas and started crossing out the ones that did not quite answer the question. Once we narrowed down the ideas to a few that we liked, we took on the big picture approach by bringing different ideas under one main solution – starting a promotional campaign for industry wide with the particular racing team as the major driver of the campaign. We arrived to “Feel the data” campaign which was designed to introduce F1 to a new market segment by playing on the FOMO (fear of missing out) trend. We wanted to create the hype around F1 so the idea was to organize a pop-up event at major attraction in the city and then rotate the event to different cities after a few weeks. The pop-up site would include various activities such as driving simulator, pit stop tire changing, and impersonation of driver’s personal engineer. The activities would be designed based on the data of the drivers and their teams in order to create more immersive experience and also incentivize the diehard fans to join the event given that they would gain more insights into their favourite teams and drivers. There would also be a rewards app for the fans to collect points from visiting different sites to win freebies and to vote for the next location of the pop-up site. We hoped this campaign would increase the fan base as it would expose F1 sport to the new demographics and help the crowd understand this sport better.

I found the creative sandpit challenge quite similar to the PhD journey in a nutshell. First, I needed to understand the problem and the root cause of it. Second, I had to explore various solutions in order to find the most optimal one. Third, it was all about how to convey the findings to a wide range of audience. Similarly, a PhD is about understanding the problem, filling in the knowledge gap, and convincing the community of why this project is worth pursuing. At Horizon, we are trained to go broad and expand our research objectives then strategically narrow down the scope to the specific. With that, the ultimate goal is to provide creative and innovative approach to tackle the challenge – just like the industry R&D projects!

Shazmin’s experience

As part of the challenge we received a short brief on the types of data that the racing team were able to show viewers which ranged from car algorithmic outputs to physiological measures of the driver. However, the company were reluctant to deliberate further about the physiological measures of the driver based on the ethical and legal challenges of showing this data on air. After the brief, all members of the summer school were asked to choose the industry challenge they wanted to tackle and then further split into specific teams which tackled the problem from a specific angle. Naturally, as my PhD looks into health markers, I gravitated towards exploring driver physiological data. We ended up being a small team of three, one other student who also had a background in health markers and another more traditional computer science PhD student.

We had half a day to explore the problem with the ability to ask F1 representatives questions, with the plan to pitch our idea to industry specialists in the morning the next day. We brainstormed our ideas using post-it notes and jointly decided which ideas were most feasible. We came to the decision that showing the following types of physiological metrics of the driver could be of interest to fans: heat levels, hydration, and heart rate. We were told that showing specific metrics would result in an ethical and legal problem so instead of showing specific numbers, we thought we could visually show the data in a less identifiable way. For example for hydration levels, we could have an icon of a water bottle with a categorical (potentially 3 categories) indicator of level of fullness. The company showed interest in our initial idea, especially the Director of Engineering who further formulated that our idea could potentially fit into a database that they are trying to make progress with specifically targeted at the Young Driver Programme. We developed our idea further with this information and prepared our pitch in the morning. The “quantified self” is a term that I learned during my PLP which describes the cultural phenomena of self-knowledge through technological self-tracking. Based on my knowledge of this, I was the one to open the pitch by describing this and asking the audience how many people were wearing some sort of wearable tracking device (a large proportion was!) to describe how we as a generation were gravitating towards becoming the quantified self and how our idea was following this trend.

We received very positive feedback from industry experts on our pitch. I have been in touch with the Director of Engineering since the summer school and hope to arrange a visit to the company in the near future.

Reflection on event

When we were practising how to use CPS, we had to critically reflect on our strengths and weaknesses. Our experiences of group work with Horizon helped us understanding our default behaviours, this allows us to improve ourselves and know what skills that we still lack of and how others can help us out. Because at the end of the day, even though a PhD may feel like a journey of one, in reality, we have a lot of support and it is important to know who and when we should ask for the support. Thus, when forming a team, we were encouraged to look for those who had the skills that we lack to ensure that the team consisted of various skill sets so that everyone had a unique contribution to the team. It was a little stressful at first as we only had the afternoon and the next morning to complete the assignment before presenting it to the industry. Although we were able to generate a lot of ideas, we spent a decent amount of time dwelling on the initial ones that each one of us came up with. This was when the CPS played a big role in helping us filtering out different ideas in order to generate the most optimal solution as a team which we could then develop it further. We were under pressure, however, we had to learn how to work together and liaise with each other in spite of our differences in expertise. Building a good team was definitely a challenge on its own, but it was a very valuable experience. We, as multidisciplinary PhD students, are constantly working with people from different disciplines which it is not always easy. By having more practice working in a diverse team, we have learned to better clarify and communicate our ideas which we can continue to apply throughout our doctoral process.

Last but not least, one of the unplanned bonding activities we (both the students and people from the industry) did together was chasing down the peacocks for pictures – good times!

 

 

 

Second EPSRC Impact award for Richard!

Richard Ramchurn (2015 cohort) won the EPSRC “Telling Tales of Engagement” 2018 Award. This is the second year running that Richard has secured this award which will give up to £10,000 of funding to maximise the reach and impact of his research. The “Telling Tales of Engagement” 2017 Award funded Richard’s mobile touring cinema which travelled the UK in 2018 and 2019.


Post by Richard Ramchurn (2015 Cohort)

The MOMENT is a brain controlled film which has been touring since 2018. It has been primarily screened in a converted cinema caravan which has allowed it to travel outside of the cinema circuit. 

The research project adopts a performance-led research in the wild methodology, through which impact though public performance is an inherent factor – the default practice behind this methodology is that real-world artefacts are professionally made, and performed for the public, as the mode of studying their design and implications.

The MOMENT has had over 300 public screenings across the world, at FACT, Lakeside Arts, Sheffield DOC/FEST, Ars Electronica, Kendal Calling, Blue Dot, Arts By The Sea, Leeds, Geneva and Reykjavik International Film Festivals, and Aesthetica. We were also invited to  international events: SPARK British Council event in Hong Kong, Brain Film Festival in Barcelona; Riverside Film Festival in Padua. Substantial work has been made to make the touring of the caravan self-sustainable and there are plans to further exhibit throughout 2019.

I have been invited to present my research and engage in panel discussions with the film and computer game industry at: Creative England’s Proconnect conference; Continue Conference; Picturehouse London with B3 Media; Broadway Cinema Nottingham; Geneva and Reykjavik International Film Festival; Sheffield DOC/FEST and FACT Liverpool. Organisations and individuals have since contacted me to ask to work on upcoming projects, to collaborate, and screen the film at their festivals.

The project has now secured funding to screen Live Score performances in 2020. In these performances musicians Hallvarður Ásgeirsson and Scrubber Fox perform the score to The MOMENT as the film is created live from the brain data of an audience member.  The performances are followed by a Q and A with the musicians and myself. We previewed the Live Score in Reykjavík and Nottingham last year to engaged audiences. We are now planning a UK tour for the summer of 2020.

A live score accompaniment for an interactive film is an unique, timely, and relevant proposition, capable of capturing both the public’s imagination and commercial interest. Our tour offers an alternative engagement proposition: creative, interactive live performances that large audiences can experience collectively at local arts venues. This model fits with the  industries move towards marketing cinema as a live experience, both through streaming theatre and music performances to screening venues (e.g. NT Live), and by creating immersive environments in which screenings take place (e.g. Secret Cinema).

The Live Score has the potential to reach larger audiences, including both a wider film industry audience, and members of the public who may not usually engage in academic research. The performances are set to be an exciting and dynamic way to share my research.

IEEE FG 2018 paper & conference experience

Post by Siyang Song (2016 Cohort)

Venue: FG 2018, Xi’an, China https://fg2018.cse.sc.edu/

Paper Title: Human Behaviour-based Automatic Depression Analysis using Hand-crafted Statistics and Deep Learned Spectral Features

The work presented in this paper is the extension of my CDT PLP module. During the PLP, we found that current clinical standards for depression assessment are subjective, and requires extensive participation from experienced psychologists. Therefore, my supervisor (Dr. Michel Valstar) and I explored a video-based automatic depression analysis approach, and find our approach beat state of the art systems in a dataset. As a result, we decided to write a paper about this approach.

This bulk of the paper was written by me. Since I didn’t have much experience about writing paper at that point, my supervisor helped and taught me about how to organize a paper, as well as how to implement ablation studies to show the strengths of the proposed approach. In addition, my external partner (Prof. Linlin Shen) also helped me to check the typos and languages. After, several rounds improvement, we eventually submitted the paper to FG conference which focus on the automatic face and gesture analysis.

Three months after the submission, we received the reviewers comments, where two of them gave ‘weak accept’ and two of them gave ‘borderline’. In the rebuttal, we addressed issues one by one. For some real drawbacks, we made some changes. Also, some comments were gave due to the misunderstanding of the paper. To deal with them, we detailed explained in the rebuttal.

Posters:

This study is the basis of my PhD study. Based on this work, while we continuously developing a better model for practical use, we also looking to other temporal modeling methods.

Building several models for not only depression analysis, but also personality traits and anxiety analysis.

https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8373825

–originally posted on Siyang’s blog

VRtefacts Outreach at Derby Museum & Art Gallery

Post by Joseph Hubbard-Bailey (2016 Cohort)

The VRtefVRacts project provides museum and gallery visitors with the opportunity to hold and explore exhibit objects which they would otherwise just look at behind a literal or figurative red rope. Throughout the day, visitors from around the museum were invited to come and put on a VR headset, interact with some 3D-printed VR-augmented models of artefacts, and share their own story or commentary about the objects as they handled them. They then moved into another room for a short interview about the experience, allowing for the next participant to get started with the VR. While previous outreach events I’ve done have felt engaging and productive, none have been as interactive as this VRtefacts trial; others mostly involved having conversations across tables, and the distance and dynamic between researcher and participant felt similar to a campus-based study scenario. Due to the nature of this event, with participants engaging physically and narratively during the session, members of the public seemed much more a part of what was going on, as opposed to passive spectators.

For the visitors who chose to participate in the VRtefacts project, the experience served as both a novel sort of ‘exhibit’ in itself and a novel way to access preexisting materials in the museum’s collection. The latter seemed of particular value in the case of visitors who lived locally and so visited the museum often, offering an unexpected new level of access to familiar objects. The opportunity to contribute or “donate” a story as part of the VRtefacts experience may also have been particularly appealing to those who visit regularly and were keen to ‘give back’ to the museum. Several visitors did fall into this category of ‘regulars’, but there were also plenty of people who were passing through and popped in to pass the time. Visitors across both of these groups commented about how the decision to work with VRtefacts reflected well on Derby Museum, showing its openness to new ideas and resistance to stagnate. For those who were visiting the museum in groups, engaging with the VRtefacts exhibit seemed to provide a great source of interest and conversation, as they emerged and compared experiences. The fact that the corresponding artefacts themselves were available in the museum’s collection also meant that there was a comfortable transition back into the rest of the exhibit, as people could go and find the ‘real thing’ they had just encountered virtually.

Before I left the museum for the day, I sat down on the duct-taped-still chair and had my hairdo sabotaged by the VR headset so that I could have a go at the VRtefacts experience myself. I chose and inspected a small intricate model of a giant jet engine, turning it over and fumbling around the prickly detail of the gaskets while I tried to think of something clever to say for the camera. It reminded me of a frighteningly massive aircraft housed at the RAF Museum in Hendon, where I’d been for relentless school trips as a child due to its proximity to school grounds. I remember cowering through the awful hangar where the scary plane’s wings were so expansive that you had no option but to walk underneath them if you wanted to get out. While this wasn’t a pleasant experience, I think the physicality of being below the Vulcan — which I now know was not just a war plane, but a strategic nuclear bomber — came to mind during VRtefacts because it was a similar example of the power of perspective.

Image credit: Kenneth Griffiths (Ascension Island, 1982)

When an object is in a glass case or on a screen or behind a rope, I think we often instinctively revert to what I can only describe as a ‘flat’ perspective on it. We might press our noses to the glass as children to try and get a closer look, but the glass fogs up and we get told off, so eventually our curiosity wanes and we take a respectful cursory look instead. What this tired perspective gives us is often limited to two-dimensional factual information about the object of interest, without the weight and contour  and color of the object’s life. I’m very glad I decided to have a go with the VRtefacts pilot myself before I left the event, because it made me aware of how cowering under the expanse of the Vulcan’s wings taught me more about the gravity of war than any of my history lessons had. There is a narrative power in an artefact’s physicality which cannot be accessed by simply looking at it — the VRtefacts project has the potential to provide that physicality in a way that protects the original object, which needn’t even be on the same continent as it’s VR counterpart.

Beyond the benefit this technology could offer in enhancing the habitual gallery-goer’s usual experience, there is also potential benefit to those who aren’t so familiar and comfortable with these venues. Having come from a family who didn’t really go to museums or galleries, I still feel quite awkward and out of place in these spaces at times. I don’t think it’s much of a leap to suggest that projects like VRtefacts — which offer more diverse ways of accessing meaning in historical and art objects — have the potential to make galleries and museums not only more engaging for visitors, but more accessible to a diverse range of visitors.

Thanks to Jocelyn Spence and the rest of the VRtefacts team for letting me join in for the day!

VRtefacts is a pilot project developed within the European Union’s Horizon 2020 research and innovation programme under grant agreement No 727040, GIFT: Meaningful Personalization of Hybrid Virtual Museum Experiences Through Gifting and Appropriation.

–originally posted on Joe’s blog

Programming in Unity at the DEN Summer School

Post by Joe Strickland (2017 Cohort)

Back in the summer of 2018 I attended the DEN summer school in Bournemouth. One of the big draws of the summer school for me was the programming in Unity course that was being offered. Having come from a psychology background, I had no programming knowledge but it was becoming clearer and clearer that this was going to be something that held me back during my PhD, especially when it came to prototyping ideas for experiences. The course itself was pretty good, we ran through several different elements of using Unity including the basics of building scenes, game object physics, and exporting our scene onto a smartphone and viewing it with a cardboard header as a VR experience. We also started using Vuforia and making basic AR content. This workshop gave me a good basic understanding of Unity, but more importantly, it showed that what I wanted to learn and eventually make was well within my grasp. This was very important for motivating me to carry on learning how to build Unity experiences, as well as code in general.

Once the summer was over, my supervisors and I sat down and started discussing short-term goals to get me learning everything I’d need to learn in order to build interactive AR experiences myself. The first of these goals was to learn Python and C# in order to understand the logic of coding and be able to write my own Unity scripts to control different elements of the software. My supervisor ran me through all the basics in Unity that I might need for the specific things I was going to make, a welcome refresher after the summer school course, and I was sent off to learn my languages. Personally, I found Python quite easy to learn. The logic of the language made sense to me and the online resource I had been recommended taught it in a very hands on a practical way, with many small assignments to try out new coding knowledge and to keep old knowledge fresh and reinforced in your memory. Also, the course was broken up into bite size chunks and I found doing a lesson a day over the course of a month a very productive way of learning this language.

C# scripting was a little harder for me to grasp. I don’t know whether it was the difference between it and Python throwing me off or knowing that having to learn this was going to be more important for my PhD, but it took a lot more to try and figure out what I was doing with it. Learning this was done through some of the Unity provided tutorials, as well as other user generated tutorials on YouTube. I was also learning how to use Unity to specifically make the first short term goal project I had been assigned; making videos plays in Unity. The Unity video player isn’t completely user friendly and it took a lot of trial and error and searching Unity message boards and community sites to find out how to get it to work in the way I wanted it to. Having got it to work I moved on to controlling it a bit more and building an experience where the audience can press keys to trigger the playing of different video clips. I crafted a game object for each video clip we had and had them generated and destroyed whenever we needed that video playing, depending on the input of the audience. What I ended up with was a functional interactive film about a man trying to find his heart medication, where the audience could decide whether he moved left, right, or had a heart attack at various points in the film. When I showed my supervisors they liked it but found how I had made the film incredibly inefficient, so they tasked me with remaking it so that different videos played on the same game object and not on different ones. This next step proved challenging but eventually I managed to write a functioning Unity script which changed the state of the game object and, once a game object was of a certain state, it would play different videos with different audience responses. It would then change its stage again to allow the experience to progress. This experience pleased my supervisors, but they didn’t like how making decisions at the wrong times messed the game up, so I had to add delays into the script that stopped audiences making decisions at the wrong points in the experience. Fortunately, this wasn’t to difficult to do, although trying to use time as a function while coding with the video player in mind did prove confusing.

I was also asked to build a restaurant scene and fill it with moving virtual characters, but this was very similar to the summer school exercises and the Unity developer tutorials so this didn’t prove too tricky. Characters were downloaded from Adobe Maximo, so came with animation cycles attached and a few YouTube tutorials later I had people looking around and being furious at virtual restaurant tables.

Finally, I was asked to build an AR tester experience. I had to place a virtual character, like those from the restaurant scene, into a real world environment and have them occluded by a real world object, specifically sitting behind and hidden by a real world table. This is something that is surprisingly hard to find official Unity information for. There is lots of help for tracking markers and placing AR content in the real world but not so much for having that content blocked by real objects. I eventually found a YouTube tutorial which addressed a similar problem in a way which allowed me to figure out how to solve my own. They showed how there was a depth occluder material that you could use to create invisible game objects that would block the audience’s vision of the virtual content. Creating a cube the size of a table top and placing it over the lap of my sitting virtual character, then using a placemat as a tracking marker in the real world to position the avatar behind a real table allowed the virtual character to appear as if they were sitting in the real world. The illusion was particularly impressive when the character moved and there arms would disappear and reappear below and above the line of the back of the table. See attached photo for a snapshot of the experience.

If I had any advice to any other researchers looking to get into creating XR experiences, or even just learning to code, it is there’s no time like the present to start learning. There are plenty of great resources online for free that go through everything you’ll need to know step by step, while also allowing you to navigate through lessons to learn the specific things that you need for whatever project you might be working on. Though getting an understanding of the basics is fundamental you can pick or choose what of the more specific stuff to learn to suit your needs fairly easily. Also, just like any skill, you’ll need to keep practising. Find some little challenges to work towards, like I had set out for me. There were a few times I’d not focus on coding for a few weeks and then notice that I had forgotten something I definitely knew before and had to go back over previous lessons or code that I had written to find it. Don’t fall for this like I did, keep it up at a steady pace and you’ll be writing code in no time.

STEM activity at Loughborough Grammar School

Post by Melanie Wilson (2018 Cohort)

We visited the Loughborough Schools STEM activity which was taking place at Loughborough Grammar School which included pupils from several Loughborough schools. STEM stands for Science, Technology, Engineering and Maths and the workshops encourage pupils to design and explore a project based on these criteria. We gave a presentation on the pathway we took when designing, prototyping and manufacturing the Endeavour LED sabre.  We then addressed the need to consider what activities the final product would be used for including any limitations or challenges which might need to be addressed. Finally we talked with the pupils individually and invited them to tell us about their projects and explored options that might be worth thinking about in their design stages and beyond.

More about Mel’s work and activities can be found here.