Deep Dive #4 - Measuring User Experience in Virtual Reality

00:00:00: [MUSIC]

00:00:04: Hello and welcome to another episode of Deep Dive into Applied Science.

00:00:10: Today, we talk about virtual reality and a very special PhD project.

00:00:16: And the PhD candidate can introduce himself.

00:00:21: >> Thank you for having me.

00:00:22: My name is Lukas Künzer.

00:00:24: I'm a research assistant here at Drier University of Applied Sciences.

00:00:28: And I'm doing my PhD in virtual reality and

00:00:32: system development in a cooperation with the University Drier.

00:00:36: So first of all, thanks for coming on the podcast and for introducing your project.

00:00:41: Before we dive deeper, can you describe your project in detail

00:00:46: in one sentence, if even possible?

00:00:49: >> Yeah, sure, sure, I'll try.

00:00:51: So my project is all about developing a system and

00:00:56: I guess a set of guidelines for measuring subjective user experience

00:01:01: in virtual reality scenarios and environments for research.

00:01:05: >> So correct me if I'm wrong, it's basically a questionnaire in virtual reality?

00:01:12: >> Yeah, basically, so when you're measuring user experience,

00:01:16: you can measure it through, for example, subjective self-reports,

00:01:21: which are very often questionnaires, though it can also be something like

00:01:26: people just narrating their experience, in our case, due to us going for

00:01:32: more quantitative approach in our data collection and analysis.

00:01:37: That means questionnaires and how to present them in virtual reality so

00:01:42: that it's very, well, it's very user friendly for

00:01:47: the study participants to rate their experience on scales of one to ten,

00:01:55: one to seven, and we have to consider a lot of usability factors

00:02:01: in order to provide them with a good, neutral experience so

00:02:05: that we are not introducing any kind of bias into our data.

00:02:09: >> We had another PhD student on here in the second episode.

00:02:14: His name is Nikolaj Hebke, and he also does a project with virtual reality.

00:02:21: I think you two know each other?

00:02:23: >> Yeah, we do know each other, though I'll need to correct it a little bit.

00:02:30: He's not doing full on virtual reality, virtual reality at least in academic

00:02:37: research, but also when we're looking at the products that are being put on

00:02:44: the market by various manufacturers, it's kind of like a spectrum,

00:02:47: going from augmented reality through mixed reality and to virtual reality.

00:02:57: It's kind of debatable, every manufacturer also uses their own terms,

00:03:03: as we most recently also saw with someone like Apple who introduced

00:03:08: a spatial computing device, so they kind of don't want to label it virtual reality.

00:03:13: But this reality virtual reality continuum that exists since the 90s in research,

00:03:21: like when trying to describe display technologies that would place

00:03:27: Nikolaj's research kind of like between the center and the virtuality end,

00:03:33: and what I'm doing quite a bit closer towards the virtuality.

00:03:38: Due to the fact that we're using head mounted displays that occlude you from

00:03:45: the reality completely, so your head is covered and you're looking at displays

00:03:51: within that headset, so there is little stimulus from the outside world

00:03:56: reaching you while with Nikolaj's project, they're using a pass-through device.

00:04:01: So either with a clear display, using some kind of holochrome type display,

00:04:10: or they're using a video pass-through, and that's where everything gets muddled,

00:04:15: because those are virtual reality devices taking information from the real world

00:04:20: using the cameras, overlaying virtual content on top of this image of the real world,

00:04:27: and it gets complicated.

00:04:30: But what exactly is it that fascinates you about virtual reality?

00:04:35: I think it's the new opportunities that come from using virtual reality as a medium.

00:04:41: You can simulate a lot of stuff, and so this for example is very interesting

00:04:47: when you're talking about manufacturing processes or prototyping processes

00:04:54: where you have digital design data, digital development data,

00:05:00: and you can experience that in 3D space while before,

00:05:05: you only really had your 2D display at your PC.

00:05:13: Now you can look at it in a one-to-one real life scale and explore it

00:05:18: and also talk to other people about it, show them around it,

00:05:21: maybe they don't have the same capability to imagine these 3D products

00:05:28: and the 3D data, and so I think that opens up a lot of opportunities.

00:05:33: However, I think that's one side of the coin,

00:05:36: and the other one is that I think virtual reality lends itself to immersing people

00:05:43: in this new, like in this parallel, separate, subjective reality.

00:05:49: I mean, they're not going to a different dimension, but for people it can't feel like that,

00:05:56: so they're leaving their current context of experience from the real world

00:06:04: and entering a completely different environment, completely different context,

00:06:08: and with that transition, there's also a lot of things happening in the person itself.

00:06:14: Your research also focus on older and unexperienced virtual reality users,

00:06:20: so it probably feels like a completely new world to them.

00:06:23: But why did you choose to focus on this group?

00:06:26: Why not, I don't know, younger people that have more experience?

00:06:31: Well, this kind of goes back to what I was talking about earlier,

00:06:35: about removing people from their context, from their habits,

00:06:40: and maybe from some kind of cognitive barriers like fear or motivational issues.

00:06:48: It all basically started with a little bit of a vision, a little bit of an idea,

00:06:54: seeing as we're living in an aging society,

00:06:57: and seeing that there's a huge deficit in physical activity

00:07:03: in our cooperation project together with Health Psychology at the Trier University.

00:07:09: We wanted to explore virtual reality as a medium for promoting physical activity

00:07:16: in older adults, specifically through exergaming.

00:07:19: And in the beginning, we more narrowly looked specifically at heart disease patients,

00:07:26: who often develop something like fear of physical activity,

00:07:32: because they're misunderstanding the symptoms that are induced by physical activity,

00:07:38: like elevated heart rate, sweating, shortness of breath.

00:07:42: They misinterpret that as symptoms of their disease,

00:07:45: and because of that, they develop fear, often healthcare practitioners,

00:07:51: also are hesitant to recommend too much physical activity,

00:07:56: often telling people to pace themselves.

00:08:00: And this just leads to a situation where people are hesitant, have these barriers.

00:08:05: And that's where we thought our hypothesis was,

00:08:10: that by transitioning these people into a new environment

00:08:19: with some technological safeguards via sensors,

00:08:24: that we would be able to help them, support them,

00:08:29: in overcoming these barriers to physical activity,

00:08:33: either motivational or fear, or whatever.

00:08:37: So the idea was to have them work out in virtual reality in so-called exergames,

00:08:42: which is really a combination of the words "exercise" and "games".

00:08:47: And through these gamified behavior change techniques,

00:08:52: enable them to come to the conclusion that they can be physically active

00:08:58: and still be safe, and then carry this over in their daily lives,

00:09:01: where they can then maybe start a more healthier lifestyle

00:09:05: by working out with or without virtual reality.

00:09:08: You already mentioned your exergames, your exercise games.

00:09:12: And please correct me if I'm wrong,

00:09:14: but those games, they adapt to the heart rate of the user in real time.

00:09:20: Is that right?

00:09:21: That depends.

00:09:22: That's something we experimented with in the virtual reality lab

00:09:27: that we have here at the Triad University of Applied Sciences.

00:09:32: And in the health lab of health psychology at Triad University,

00:09:39: mostly if you're looking at games from the commercial sector,

00:09:45: so what the normal user has available to them

00:09:49: on the different virtual marketplaces of these manufacturers,

00:09:53: they are games that are a bit more limited in that regard.

00:09:58: So often they enable you to connect your devices,

00:10:04: your smartwatches, your sensors in order to count your calories

00:10:10: or determine your exertion,

00:10:13: then calculate your calories that you spend,

00:10:19: the energy that you put into the workout,

00:10:25: or even display it during the gameplay.

00:10:29: Personally, I am not aware of any commercial exergame

00:10:33: that adjusts to your heart rate

00:10:36: by adjusting the challenge or the required movements

00:10:45: for these obstacles that the players encounter on the fly.

00:10:49: Like dynamically during the exercise,

00:10:52: usually you just select before you go into a workout session,

00:10:56: you select the activity level, the exertion level

00:10:59: that you're targeting,

00:11:00: but it's not adjusting during the actual exercise.

00:11:05: And that's something we experimented with,

00:11:09: basically defining algorithms

00:11:11: that would present different kinds of obstacles,

00:11:14: of different exertion or intensity levels,

00:11:18: and hoping to kind of keep the heart rate of the participants

00:11:27: in a given heart rate zone that we wanted to achieve,

00:11:33: which was like 60% of their maximum heart rate.

00:11:37: And this is kind of similar to what you have

00:11:39: when you're walking, when you're running on the treadmill,

00:11:43: or you're on this bike in the gym,

00:11:49: where also according to your heart rate,

00:11:51: they increase or decrease resistance,

00:11:54: or sometimes even incline of the treadmill.

00:12:00: But of course it gets more complicated

00:12:02: when your movements are not as repetitive

00:12:06: as during these kinds of exercises,

00:12:08: but when your movements are pre-free

00:12:12: and kind of limited or determined by the obstacles

00:12:15: that the player is presented with.

00:12:18: If I try to get my parents or my grandparents to exercise,

00:12:24: they would probably call me crazy.

00:12:26: And if I then try to make some kind of gamification

00:12:30: out of these exercises with virtual reality or so on,

00:12:34: They were probably declare me insane.

00:12:37: So how exactly has the VR experience to look in order to engage older people in getting active?

00:12:46: How do you make this work?

00:12:48: So first, before I get into that, I think this aversion against gamification kind of comes from the wrong place in many people.

00:12:58: Because I'm pretty sure that if you would ask your father to throw some balls with you or maybe play a bit of football.

00:13:06: He would love it.

00:13:07: Yeah, I think that's something people are more willing to try or more willing to do, obviously, than just going jogging for five kilometers or something.

00:13:18: And there's also gamified physical activity and can also get pretty strenuous.

00:13:25: Well, to be fair, I think some older people, definitely not all of them, as soon as they hear a gamification, maybe they think of classical video games.

00:13:35: And that's obviously not the point.

00:13:38: Yes, exactly.

00:13:40: And these extra games, they are characterized by requiring a lot of movement and physical activity.

00:13:48: Often they are synced up with music, so they're the problem as you can be when in answering your question.

00:13:55: Problem can be that music tastes are just different.

00:13:58: Even with the appearance of the virtual environment, we did run a survey on that.

00:14:06: I think the results, there was my colleague, so I'm not sure on those results.

00:14:11: However, from interviews that we did with participants after our research studies, it became, well, it's not like it became clear, but it reaffirmed us in our understanding that, of course,

00:14:27: people like different things.

00:14:30: So some people like these virtual worlds, others don't, is not really a matter of age.

00:14:37: Sure, it's more likely that younger people like futuristic, modern or abstract or cartoonish appearances of the virtual worlds.

00:14:49: But the same is true for all the people.

00:14:52: You'll have people who like it and you have people who don't like it.

00:14:55: And that's why one of our goals in our project was to aggregate kind of a catalog of different also commercial extra games.

00:15:07: And based on user preferences and training goals, maybe perform like a matching to recommend specific extra games to specific people.

00:15:20: When exactly did you start the PhD project?

00:15:24: I have to think on that.

00:15:27: It's hard to say.

00:15:29: I've been working with virtual reality for a while, but I think the inception of this research topic was in 2021 or 2020.

00:15:47: Then we applied for funding.

00:15:52: So the actual project started in, I think, November 21.

00:15:59: So now we know when you started and do you have like a set date when you are going to finish it or do you have like an outline already?

00:16:10: Yes, so I am pretty optimistic that I can finish my dissertation this year.

00:16:18: I'm doing a cumulative PhD, so that means that I have to have a certain set of publications, which I then place in a combined frame in my dissertation.

00:16:36: With those publications, it's looking pretty good.

00:16:40: I'm definitely on the right track to accumulate everything that I need in order to write my dissertation this year.

00:16:48: And I'm pretty confident it will work.

00:16:53: On the other hand, it also has to work because deadlines, funding deadlines, everything.

00:16:59: Good luck, I guess.

00:17:01: Thank you.

00:17:02: And if you look back at your project, what has been the biggest challenge of your PhD so far?

00:17:09: It's hard to pinpoint one specific challenge.

00:17:13: I think it's a combination of things.

00:17:16: I always like to say that PhD kind of is a function between resources and that can be money, time, lab equipment and goals that you managed to set for yourself, plans.

00:17:33: Well, and obviously like the time you put into it.

00:17:36: So I think that often you'll see better results if you spend an extended period of time, the actual time on it, as opposed to throwing more and more of your personal time per day on it.

00:17:51: And I think the most challenging part was understanding how different disciplines in research.

00:18:00: So software engineering and, for example, health psychology, how they have very different methodological approaches.

00:18:13: They have different methods.

00:18:15: I had to learn methods of statistical analysis, for example, for my research, which is something that during my studies, though since then it has definitely gained a place in the curriculum, that kind of came short.

00:18:34: So how to design these user tests, collect the data and then analyze it was definitely a challenge than the coordination and communication with the other disciplines who just work very differently.

00:18:48: You're not the first one who is doing a PhD at Trier University of Applied Science and obviously you won't be the last.

00:18:56: Is there anything you can recommend to people that are planning on doing a PhD project?

00:19:03: That's a very loaded question that I would probably answer, well, I would answer differently depending on the audience.

00:19:13: Well, I think it takes a lot of perseverance and commitment to do a PhD.

00:19:19: Don't do it if you don't love your research.

00:19:22: You have to love your research.

00:19:24: You have to like doing it because you'll be doing it nonstop for several years.

00:19:29: You'll be doing it longer than you expect and you're doing it more than you expect.

00:19:35: So I think that's what I want to tell people.

00:19:39: Be sure what you select and don't start it just because it seems convenient.

00:19:48: It's hard to say.

00:19:49: You got to put in the work.

00:19:51: You got to put in the time, you got to put in the work and then it will likely work out like with everything.

00:19:58: So the clear message to the viewers and the listeners, you have to love it.

00:20:03: Let's take one final last step back to your own project.

00:20:08: After you finish it, whenever it will be, how can the industry or in better words, which industry can profit from your project?

00:20:19: So I mostly talked about our research project, which differs a little bit from my PhD project in the way that in the research project,

00:20:31: we have this huge focus on exergaming with older adults.

00:20:35: But my research project really is a bit more narrow and focuses on the measurement of subjective user experience using these virtual reality questionnaires.

00:20:48: So developing that system, designing it, validating it, making sure that it's really usable, comparing it to different solutions.

00:20:58: And we published that software toolkit as open source for people to use.

00:21:07: The intent really is to enable smaller research teams.

00:21:12: And these can be in academia or in industry that lack the resources to set up something like this themselves.

00:21:19: Because by presenting the questionnaires in virtual reality, this enables more economic study designs.

00:21:28: So most importantly, participants are not required to take off the headset in between different study conditions.

00:21:36: So when you're doing an A/B test, like when you have a system that differs a little bit between conditions,

00:21:44: and you want to know which one is better, which interface is better, which process is better, what looks better.

00:21:49: And by enabling people to do their ratings in virtual reality as opposed to out of virtual reality, it's nicer on the participant and it's faster.

00:22:03: So you have higher throughput. And I think this is something that also the industry can really benefit from.

00:22:10: We always hear a lot about user-centered design, user experience design.

00:22:17: And if companies, the industry is working with virtual reality, for example, for prototyping and manufacturing,

00:22:26: I think they can benefit from being able to have users rate their experience.

00:22:32: And it doesn't have to be the experience related to how present do I feel here in this virtual world.

00:22:39: But it can also just be like, how good do I think this prototypical implementation of some machine part fits into the machine

00:22:50: and rating that or maybe rating the design of a car or something that they're viewing in virtual reality.

00:22:59: Doing that there, I think can be a big benefit.

00:23:02: And my work kind of sets the foundation for what is a good way to present these kind of rating scales.

00:23:09: So you basically laid a foundation, you are doing the groundwork and other people can use it and develop it further?

00:23:16: Yeah. I wouldn't say it's foundational science. It's very much still applied.

00:23:24: We're not developing hardware. We're not developing low-layer software.

00:23:30: So it's very much still applied science.

00:23:32: But in the best case, people, other research teams can use it if they don't have the resources to implement it themselves.

00:23:39: And it's a software that can run alongside the major VR engine platform.

00:23:48: So you don't even need to integrate it into your own coding projects in order for it to work.

00:23:53: And it works with commercial software, which was a huge focus of our endeavors.

00:23:59: Well, I'm not expert, but it sounds like it's really helpful to others.

00:24:03: I hope so.

00:24:04: I hope so too. And thank you very much for presenting your project and your research so far.

00:24:10: All the best to it. I hope you will finish it this year.

00:24:14: But please don't feel pressured.

00:24:17: And the last question, how can people contact you?

00:24:21: If they are interested, if they have questions, if they have anything else for you, what are the ways to contact you?

00:24:28: I have a personal website here on the Trier University of Applied Science website.

00:24:36: You can just search my name. I think you'll get there.

00:24:40: Otherwise, maybe you can put my contact information in the video description.

00:24:47: Very good idea.

00:24:49: Then let's do that.

00:24:51: So thank you very much for coming on the podcast. It was very interesting.

00:24:55: And I hope it was interesting for you too.

00:24:58: Thank you very much for listening or for watching.

00:25:02: And until next time at Deep Dive into Applied Science.