Deep Dive #2 - Battling back pain with mixed reality techniques

00:00:00: Hello and welcome to a new episode of Deep Dive Into Applied Sciences.

00:00:11: My name is Martin Zeng, I'm your host and today I have a very special guest and this

00:00:18: guest can introduce himself.

00:00:21: Thank you very much.

00:00:23: My name is Nikola Hebke and I'm a PhD student at the Hochschule Trier for about a bit long

00:00:29: than three years in computer sciences and I've already did my master's and bachelor's degree

00:00:36: here at the university in medical informatics.

00:00:40: Thanks for your introduction and you have a very, well, special topic overall for your

00:00:46: PhD and this topic is back pain.

00:00:50: Well, not only back pain but before we dive deeper please describe your project in one

00:00:57: sense, if possible.

00:00:58: If possible, one sentence that could be hard.

00:01:02: So we are trying to combat back pain using virtual reality techniques or extended reality

00:01:11: techniques.

00:01:12: So we try to develop a low-cost home solution in which you can use a digital physiotherapist

00:01:20: to combat your back pain instead of using a real one or assisting to a real one.

00:01:27: So you got a VR or XR headset and get a digital therapist and he or she will tell you how

00:01:35: to do your exercises so that your back pain gets better.

00:01:40: This was more than one sentence.

00:01:43: No problem, I will allow it.

00:01:45: Well, for someone like me, can you explain the differences between something like VR

00:01:52: and XR because you mentioned both terms and I don't really know what they mean.

00:01:57: So XR is the header term for VR mixed reality, MR augmented reality, AR which so XR can

00:02:08: describe each of them.

00:02:11: And VR is the headset most people know so you take on a headset and you are in a completely

00:02:17: different environment.

00:02:18: Everything you see is digital and AR on the other hand of the spectrum is where you see

00:02:25: so you see your normal reality like an AR glasses.

00:02:30: You can see everything around you like if I had one I could see you but the reality

00:02:36: is augmented by digital objects.

00:02:39: So I see you and maybe like a virtual cup of coffee here for example.

00:02:47: Then there's mixed reality in the middle which means you there are multiple, multiple different

00:02:54: definitions for each of these terms but mixed reality is often referred to augmented reality

00:03:00: which is spatially registered.

00:03:03: So I get a digital coffee mug and I place it digitally on the table.

00:03:08: Then I could move around the table and the object would behave like a normal object would.

00:03:16: So if I move around it I would see the other side if I stood on your part.

00:03:21: And this mixed reality is what we use in our project right now.

00:03:26: So you see everything around you including digital physiotherapists who would stand in

00:03:32: the room and could talk to you.

00:03:36: This actually sounds very complex to me but now that we've established some ground rules

00:03:42: what is what.

00:03:43: The overall topic or the name of your project is development of an immersive XR biofeedback

00:03:51: system for movement therapy in back pain patients.

00:03:56: This very long title.

00:03:57: A better one sentence description than the one I did before.

00:04:02: Well before someone told me it's virtual reality against back pain or XR against back pain.

00:04:10: Maybe that's the headline of the episode.

00:04:13: But you are not working alone.

00:04:16: You're working with various departments.

00:04:18: You're working with psychology, with computer science and also with therapy science.

00:04:24: What are the challenges of your well interdisciplinary approach?

00:04:29: The disciplines work quite differently.

00:04:34: So it starts with the way you publish something.

00:04:37: The way which data is important.

00:04:40: There are some clashes.

00:04:41: If we do a paper together, especially with a PhD student, then we on our informatics

00:04:51: perspective we think you have to include everything like the data, the method, the code you used

00:05:00: to process it.

00:05:02: And then so the people who read the paper can actually recalculate it.

00:05:06: But this is highly unusual in psychology publications.

00:05:11: So we are like, oh, there's missing a big part.

00:05:14: And they're like, oh, what are you talking about?

00:05:16: And then we have to find in the other way around.

00:05:20: We in the informatics department, we do some things that are really weird for the others.

00:05:26: So for each publication we have to kind of work around or keep in mind the way the others

00:05:35: publish and work.

00:05:36: And especially in the work environment, so when we develop the system together, there

00:05:42: are some problems that it's for non IT people usually really hard to estimate the amount

00:05:51: of work any certain task takes.

00:05:54: So if you get, for example, the XR physiotherapist and you tell me there should be an avatar

00:06:04: like a digital human and he should stand on the ground.

00:06:09: And then you probably thinking, oh my god, this is so much work.

00:06:13: He has to get the avatar.

00:06:14: He has to get the registration right and everything so on.

00:06:17: But for me, this could be as simple as downloading an avatar, placing it on the scene, and then

00:06:24: the glasses do the rest for some simple scenes.

00:06:29: But other things like if you want a certain way of grabbing something or you want to include

00:06:36: a change that can be really hard, although you hardly see any difference in the finished

00:06:43: product because in informatics it's really important that you establish what you need

00:06:48: in your project as soon as possible.

00:06:51: So your project needs to be able to do this and this and this and this.

00:06:55: And then you build up on those.

00:06:59: And then if you include another one in the end and you say, hmm, it would be cool if

00:07:04: it has multiplayer, for example.

00:07:07: I get the glasses, I see the digital physiotherapist and then you get one as well and we could

00:07:13: do it together.

00:07:15: This would be really hard because then I'd have to do a network communication for each

00:07:20: and every step along the way and I had to rearrange the entire architecture.

00:07:26: But for you or for the end user who might not work in IT, they would think, okay, I get

00:07:33: one glasses, you get another one, we can place it at the same position.

00:07:37: Why doesn't it work?

00:07:38: Why is it that hard?

00:07:40: So we often get problems where it's hard to estimate it.

00:07:45: And for me, it's hard for me, I really had to learn to communicate.

00:07:53: This is hard.

00:07:54: This is much to do.

00:07:56: And this is maybe easy.

00:07:57: So we started out and had some problems where the colleague was like, oh, I wanted to ask

00:08:03: him for this change, but I think it's too hard.

00:08:07: So I don't ask.

00:08:08: And then later I found, oh, okay, what he wanted was to add some big digital object.

00:08:16: But it really doesn't matter how big this object is.

00:08:19: I mean, I can include it in there.

00:08:21: So it was a change of like 30 seconds.

00:08:24: So we had to find a way to groove in to just say aloud everything that we want to include.

00:08:33: Then I have to really think about how much work would that be.

00:08:37: And then we can together decide what has priority and what could be solved.

00:08:43: Even though you have very different ways of working, you still work together as a team

00:08:48: throughout the whole project.

00:08:50: Yes and no.

00:08:52: Actually I worked together with the physiotherapist as well as the psychotherapist, a psychology

00:09:02: student.

00:09:04: So we worked together with both.

00:09:06: But we really early on saw that what we need from each part is quite distinct.

00:09:16: So there is like a physiological and a psychological component to back pain.

00:09:22: And it's really hard to do a psychotherapy while you're doing squats.

00:09:29: So we kind of split it a little.

00:09:33: And then in the end we bounded back together.

00:09:36: So yeah, I worked together with both.

00:09:38: And then we had some work as a group.

00:09:43: But it's hard to include all three because it could be quite different.

00:09:48: Let us stay with the back pain for a moment because we all know what back pain is.

00:09:53: We all know people who suffer from it.

00:09:55: I know a lot of people that work out a lot.

00:09:58: They still have back pain.

00:09:59: I know people that don't work out at all.

00:10:02: They still have back pain.

00:10:04: So why is back pain such a common phenomenon in our society?

00:10:08: There are basically two different types of back pain we can distinguish between.

00:10:15: So we have like specific, like we're just talking about chronic back pain, not just

00:10:21: back pain.

00:10:22: Everyone can have sleep a little bit.

00:10:24: Then you have back pain for a day.

00:10:26: Then it's fine.

00:10:27: So we don't care about that.

00:10:29: But we are talking about chronic back pain.

00:10:31: And there is specific chronic back pain, which has a specific cause.

00:10:36: Like if you have a cancer in the spine, if you just had a surgeon working on your spine

00:10:44: and you're in operation on a spine, then you have back pain and then you have a reason.

00:10:51: And then you combat the reason and your back pain usually goes away.

00:10:55: What most people suffer from is an unspecific chronic back pain.

00:10:59: This is where you don't know the reason.

00:11:02: And these reasons can be very diverse.

00:11:06: So it could be that you over exert yourself like your friend who does too much sports.

00:11:12: Then your back pain is over exerted and your back is over exerted.

00:11:15: The muscles maybe moved a little wrong and then they start to hurt.

00:11:21: Then you got weak muscles from people who do too little sports.

00:11:28: They have weak muscles.

00:11:29: They can't really hold up the back.

00:11:31: Really that's good.

00:11:32: And this can cause back pain.

00:11:34: Then you have people who have bad posture.

00:11:38: We all know if we're sitting on the table and then we're sitting like this.

00:11:42: It's basically me.

00:11:44: That's me.

00:11:45: Then we do also start to have back pain.

00:11:49: But they are also psychological reasons for back pain.

00:11:53: Like depression, anxiety and stress.

00:11:57: Stress is what people usually have as a reason they cite for back pain.

00:12:05: So how does your project specifically intend to tackle back pain or to combat back pain?

00:12:12: So we have two main pillars.

00:12:15: I hinted on before the work I do with physiotherapists and the psychology work we do.

00:12:22: Like the first pillar, the physiotherapy is we try to explain to the patients which exercises

00:12:33: they should perform.

00:12:35: Like a squat for example.

00:12:38: Then they get the glasses and they see a digital physiotherapist telling them this is the squat.

00:12:43: This is good for these and these muscles.

00:12:47: You should do it like this.

00:12:49: You should take special attention to some of these movements.

00:12:57: And then we track the person while they do the exercise.

00:13:02: So we have little 3D cameras.

00:13:07: We use them to scan.

00:13:10: For example you.

00:13:11: You do the squat.

00:13:13: And then we get a rough estimation of your pose.

00:13:18: All of your joints and how they are set up to one another.

00:13:24: And then we can compare this to a good example of a squat.

00:13:31: And we can tell you, oh you're maybe going to low.

00:13:34: Or maybe you separate your knees too far to the outside.

00:13:39: Or you take your knees too low together.

00:13:42: Or you don't come up as high as you should.

00:13:46: Or your back is maybe round instead of straight.

00:13:50: And then the digital physiotherapist can tell you exactly what's wrong with you.

00:13:56: You get different scales.

00:13:57: Where you see the process you do during the exercises.

00:14:03: And then they could also talk to you and say, okay please keep your back straight.

00:14:09: Like a normal physiotherapist would as well.

00:14:12: If you perform a squat in front of a normal physiotherapist you would or she would tell

00:14:18: you to straighten your back if you do it with a curved back.

00:14:23: And the other pillar is the psychotherapy.

00:14:27: So as I said before back pain can be caused by stress, anxiety, depression.

00:14:34: And for people to know that this can be a cause or a reason or a contributor to the

00:14:42: back pain.

00:14:44: This is usually really helpful.

00:14:46: So if you normally combat lower back pain you usually get something that's called psycho

00:14:53: education where you understand the mechanics involved.

00:14:57: Like the psychological and the physiological and the social that can cause your back pain.

00:15:04: And we have one, we call it SIEM now.

00:15:09: Where you get a digital physiotherapist and they will talk to you about the reasons that

00:15:19: could be caused by back pain.

00:15:21: And you will explain it to you and then you have to do some tasks.

00:15:24: Like understanding where it comes from.

00:15:27: Thinking about the bio-psycho-social model.

00:15:32: Putting it up in 3D so you have to grab something, move it and then fill out tables in that manner.

00:15:40: And that way you can learn about the reasons that cause your back pain more effectively.

00:15:48: And we also have a relaxing exercise.

00:15:52: So it's confusingly also called body scan where we use cameras to scan the body but

00:15:58: this is where you scan your body from within.

00:16:01: So you feel your body, it's like a guided meditation which helps to relieve stress.

00:16:11: And then you get a relaxing environment around you and some things floating around.

00:16:20: Yes.

00:16:21: Let's take us one step back to the digital therapist.

00:16:27: You mentioned the therapist quite a few times.

00:16:31: How does it work?

00:16:33: What I mean exactly if I'm doing like 10 squats and all of them are wrong.

00:16:39: Does the therapist recognize that I'm doing the wrong technique?

00:16:44: Does the therapist recognize patterns?

00:16:45: Is it like an algorithm?

00:16:47: How does it exactly work?

00:16:49: So as I mentioned earlier we have these little cameras that are called Azure Connect.

00:16:56: So maybe some people know the Connect from the game controller before there was the Xbox

00:17:02: Connect which would track your body while you move and then you have to play a game

00:17:06: like this.

00:17:08: And we're using a more advanced version of that.

00:17:12: So we know how you move right now for example.

00:17:18: You can track your entire body and then we can find certain distinguishing points that

00:17:26: are relevant for the exercise.

00:17:29: We can model an exercise as some of different constraints.

00:17:36: For example if you're doing a squat then you need to bend your knees from zero degrees

00:17:42: like standing up to maybe 90 degrees.

00:17:47: Slowly bend down.

00:17:49: And then you can model it for each application.

00:17:52: If you're doing some really tough squats maybe you want more than 90 degrees but usually

00:17:57: the therapist has something in mind.

00:18:00: So the idea is that you shouldn't go over 90 degrees for example.

00:18:06: Then as I mentioned earlier maybe you need your back is curved or straight.

00:18:12: This exercise requires you to keep the back straight but not upright.

00:18:18: Just like not curved.

00:18:22: Then you need to point your knees.

00:18:26: If you're doing a squat if you point your knees too far out wide you could maybe hurt

00:18:30: yourself.

00:18:31: If you do it too far inside you are not stable enough and you could also hurt your knees.

00:18:36: So this is something that we can focus on.

00:18:40: So we have these different constraints and we know because we have the skeleton data

00:18:45: if you break one for example if your back is curved.

00:18:48: some points along your spine.

00:18:51: We can see if they are more or less on a line.

00:18:54: If they deviate from that line, we know it's curved,

00:18:58: so we can say something.

00:18:59: Obviously not if it's just for one frame,

00:19:02: then we see here might be a problem.

00:19:04: And then we have some constraints that are always active,

00:19:11: like your back always has to be upright,

00:19:16: and your knees should point in the same direction,

00:19:20: so you don't move the way your knees point

00:19:25: in any way during the squat.

00:19:27: But then we have active constraints,

00:19:29: like the bending of the knees.

00:19:31: So we can follow how many degrees do you currently have?

00:19:36: Are you at zero degrees?

00:19:38: Are you at 90?

00:19:38: Are you somewhere in between?

00:19:40: And then we can determine where are you within the exercise.

00:19:43: So at the very beginning, at the very end,

00:19:47: maybe you over-exert yourself, you go too deep,

00:19:49: then we can give you a warning.

00:19:52: And then we, or you don't go high enough,

00:19:54: then we can give you a warning.

00:19:56: And if you are anywhere in between and moving,

00:20:00: it's a little more complicated than that.

00:20:02: But then we can say, this is all right,

00:20:06: then we can count how many you do.

00:20:08: And we've quite early found out

00:20:12: that it's really annoying if the person would just talk to you.

00:20:17: Because if you have a real person, it's rather easy.

00:20:21: They can go up to you and say, you have your knees

00:20:28: point into the wrong directions.

00:20:30: Then they go to you and show you where you should do that.

00:20:34: This is really hard in mixed reality.

00:20:38: Therefore, if they just talk to you or do it more outwards,

00:20:44: please complete the exercise currently correctly.

00:20:49: Then this could be really confusing to the users.

00:20:52: So you also have different scales.

00:20:54: So you have a big scale that tells you

00:20:57: where you are in the exercise.

00:20:59: So you know when you're right before the end of the 90 degree.

00:21:02: And then you can go up again.

00:21:04: And that's all right.

00:21:05: And you also have for the direction of the knee

00:21:11: another scale that lies flat on the ground.

00:21:13: So you can actually see, oh, am I in the green?

00:21:16: Like, is it pointing to the right direction?

00:21:19: Or is it much inwards, too much outwards?

00:21:22: And then the therapist would still talk to you

00:21:26: and still say, OK, you're doing this wrong.

00:21:29: But only to point you towards the scales

00:21:32: where you can actually know what you're doing wrong.

00:21:37: So I'm playing devil's advocate here.

00:21:40: I could also go just to a real life physician.

00:21:45: Of course you could.

00:21:46: Yes.

00:21:46: So why should I take the holo glass or the holo lens,

00:21:51: put it on my head, and do the exercise at home?

00:21:54: Am I the wrong target audience, maybe?

00:21:58: No.

00:21:59: You could.

00:22:00: In the perfect world, you wouldn't need that.

00:22:02: Then you had enough physiotherapists

00:22:05: where you go three times a week.

00:22:07: And then you exercise together with them.

00:22:10: And they tell you exactly with all their expertise

00:22:13: what you're doing wrong.

00:22:14: And they would probably--

00:22:17: so I know they would do it better because they are trained

00:22:20: in that way.

00:22:21: And they have no measurement errors.

00:22:25: They know if you're doing something slightly wrong.

00:22:28: They could still improve you.

00:22:30: But we don't live in that world.

00:22:33: We have a shortage of physiotherapists.

00:22:35: So each physiotherapy relies on you

00:22:39: doing your exercises at home.

00:22:42: And so you go once a week there.

00:22:45: And then you will get homework.

00:22:48: And you have to do your homework for the therapy

00:22:53: to work properly.

00:22:54: The deal with that is that many people

00:22:57: are afraid to perform the exercises on their own.

00:23:00: They think maybe not your friend who does too much sports.

00:23:06: They would probably do it fine.

00:23:08: But the one who does too little sports,

00:23:11: they would think, hm, if I do my squat,

00:23:14: the muscles start to hurt.

00:23:16: I don't know if it's the muscles or the joints.

00:23:18: So I'm afraid if I am doing it right now wrong,

00:23:21: I could hurt myself.

00:23:22: And worse in my condition, then if I do nothing.

00:23:25: So they do nothing.

00:23:27: And instead, which is actually worse than doing it slightly wrong.

00:23:32: And for those people, we try to give guidance.

00:23:37: So for one, they are not alone.

00:23:41: They are-- we check if they do it wrong, if they do it right.

00:23:46: Then we show them the exercises again.

00:23:49: So they don't get just a piece of paper where it says, OK,

00:23:53: this is a squat, maybe one picture or two.

00:23:56: They see the entire exercise in 3D.

00:23:59: And then they can do it at home alone

00:24:04: and still get the feeling that they are being watched.

00:24:07: And they can actually show those recordings,

00:24:11: like the recorded positions of their joints

00:24:14: to their physiotherapist and tell them,

00:24:17: am I doing it right if they don't trust the system?

00:24:20: And then they could replay the scene

00:24:22: and say, OK, that's all right.

00:24:24: Or you seem to have problems with these exercises.

00:24:27: You can't really work them out alone,

00:24:29: maybe we give you another exercise.

00:24:31: So it shouldn't replace any real physiotherapist.

00:24:36: And there is really no need to replace them,

00:24:38: because we already get too little.

00:24:41: But we can augment their work with augmented reality.

00:24:47: Like we could help them to do the real hard parts,

00:24:53: like the unamnesia where they talk to you, what is hurting,

00:24:58: where they decide which exercises you should perform

00:25:01: and to which extent how often.

00:25:05: And then the actual performing of the exercise.

00:25:09: This is something that we can automate,

00:25:11: where the valuable time of the physiotherapist

00:25:14: is probably better used somewhere else.

00:25:17: For all these exercises, or for your whole project in general,

00:25:21: you are using the mixed reality glasses HoloLens.

00:25:25: Please correct me if I'm wrong.

00:25:27: HoloLens 2, actually.

00:25:29: HoloLens 2.

00:25:31: And you also use a lot of sensors.

00:25:33: You already mentioned Kinect.

00:25:35: There's a lot of technology and programming involved.

00:25:39: How expensive is it overall?

00:25:42: Because for me, it sounds very expensive,

00:25:45: because of all the technologies and so on.

00:25:48: Yeah, it's quite expensive right now.

00:25:50: So you have HoloLens, it's 3,500 euros.

00:25:55: So a small car that is used.

00:26:00: Then you get the Azure Kinects that are 400 euros apiece.

00:26:06: But each Kinect needs a PC to process the depth image it gets.

00:26:13: And to calculate the movement of the person

00:26:17: that is currently filmed using machine learning techniques.

00:26:21: So we need probably 800 euros as well for each Kinect.

00:26:27: And then we need one PC that can run the rendering of the HoloLens.

00:26:32: This is another 1,000 euros maybe.

00:26:36: So we stack up quite a bill to perform this.

00:26:40: So we are roughly in the range of 8,000 euros.

00:26:45: Yes, maybe.

00:26:48: But the idea is that we are not using this work just for today's usage,

00:26:56: but with the future in mind.

00:26:58: So right now you get VR headsets like the Crest 3

00:27:04: that also has a C through AR mode.

00:27:09: Where you have cameras in the front of your VR headset.

00:27:13: And then you see your environment similar to reality.

00:27:19: And then you can still use an application that was built for the HoloLens

00:27:27: for 3,500 euros, but on this Crest for just 500 euros maybe.

00:27:34: This also has a more powerful processing unit.

00:27:37: So you probably could do without the rendering PC.

00:27:41: So you cut 5,000 of the budget.

00:27:45: And by using different technologies.

00:27:48: Then we currently use three Azure Kinect systems,

00:27:56: which are like I said together maybe 1,200 euros.

00:28:02: But the post estimation technology is improving quite rapidly.

00:28:06: So in a few years you could probably do that with less systems.

00:28:14: And with less powerful systems or less demanding systems.

00:28:19: Modern motion capture, motion estimation systems that work on your smartphone.

00:28:28: So you could use in a hypothetical scenario,

00:28:32: you could use the LiDAR of your iPhone to process your position in 3D,

00:28:39: in real time, send it to a rather cheap VR headset,

00:28:45: which might be more available in homes in the future.

00:28:49: Because we saw in recent years that people bought even more VR headsets.

00:28:54: So they are more available in normal households for gaming or sometimes productivity.

00:29:02: But I don't really believe it.

00:29:05: So if somebody has already a powerful smartphone and a gaming VR headset,

00:29:14: they could do this or run a similar application cheaper.

00:29:20: So maybe for nothing but just the cost of the software.

00:29:24: And for the state we are right now,

00:29:26: these motion tracking systems that run on your smartphones

00:29:30: are not accurate enough for what we demand right now.

00:29:34: So we can't do this at the moment.

00:29:37: But the idea we have in mind is that you could set up the system in a physiotherapist practice.

00:29:47: And then you go on Monday to your real physiotherapist

00:29:52: and on Wednesday and Friday to your digital therapist.

00:29:55: And they could book a session as well.

00:29:58: And then the real physiotherapist could integrate those systems in his practice,

00:30:04: where he says, "Oh, I saw you've been training. I saw you've been training really good.

00:30:08: So I know your progress."

00:30:10: They could show you how much you improved over the time and motivate you like this.

00:30:15: And can say, "Okay, we see this works great for you.

00:30:19: Maybe we dial down the support."

00:30:22: And the digital physiotherapist would not show so many scales.

00:30:26: And would maybe talk less to you up to a point where they don't talk to you at all.

00:30:34: And where they know you can perform the exercises on your own without the supervision.

00:30:39: So this is basically your idea how to transfer your project,

00:30:43: or let's say the data of your project into the industry.

00:30:47: That you work together with different doctors, with different physiotherapists and so on.

00:30:54: I mean, this is an application scenario that we are thinking about,

00:31:02: but the actual implementation is not part of the project.

00:31:07: So your research now basically helps people in the future?

00:31:12: Yes, that is the core idea.

00:31:15: I mean, most research topics are set up that way.

00:31:19: You find a problem, you work on the problem,

00:31:25: and someone else picks up at some point where you are left off,

00:31:29: maybe introduces a new idea we never had,

00:31:32: and then builds it in a different way, builds up on it,

00:31:36: and then somebody else will build up on it,

00:31:38: and then somebody else will use it, refine it,

00:31:43: maybe turn it into a product,

00:31:45: and then introduce to home users or physiotherapists.

00:31:50: Since when do you work on the whole project?

00:31:53: I've been working on this since for three years now,

00:31:58: and yes, with my colleagues.

00:32:01: And when exactly do you plan on finishing?

00:32:03: I don't expect an exact deadline.

00:32:06: We are at a point where we finished most of the software solutions already.

00:32:12: We just did a study about different aspects of the BEO feedback,

00:32:17: where we could define which helps really well,

00:32:22: and which does not work.

00:32:24: So we are currently working on the results of this study,

00:32:29: so I can't talk too much right now,

00:32:32: but the software is in a really good state,

00:32:39: where you could, with some polishing implemented in prototyping practices, for example.

00:32:50: And if you look back at the last three years,

00:32:54: which were the hardest obstacles you faced,

00:32:58: and how did you resolve them?

00:33:00: The hardest obstacle was actually finding out,

00:33:06: like quantifying the accuracy of the Kinect cameras.

00:33:13: The idea is usually you have a system, any system,

00:33:19: which you want to compare for its accuracy.

00:33:21: So you compare it to the gold standard.

00:33:26: For us, we have a home system, the Kinect, which you can buy for 400 euros.

00:33:31: We want to compare it to a vikon system,

00:33:34: like a big motion tracking system they use for movies.

00:33:38: Those are much more expensive, much more effort to set up, but highly accurate.

00:33:44: So there's our gold standard.

00:33:47: The problem was that our gold standard influenced the measurements of the Kinect.

00:33:52: So whenever we simultaneously measure with our system, our Kinect,

00:33:58: and this gold standard, we actually sharply decrease the accuracy of our Kinect measurements.

00:34:05: So the problem was how do we measure without impacting our system?

00:34:11: There we had to build a whole other system.

00:34:16: Maybe I could go into detail with that.

00:34:19: And yeah, to have a really big study,

00:34:25: 20 people each for four hours performing these 10 exercises three times.

00:34:31: And all with back pain.

00:34:33: None with back pain, actually, because if we had anybody with back pain

00:34:37: for four hours doing exercises, I think they would leave.

00:34:43: And the ethics commissions would probably not be happy about that.

00:34:48: If we want to find out the accuracy, we just use people who are athletic,

00:34:55: and then we can work with them and maybe repeat some measurements

00:35:00: over and over without over-exerting them.

00:35:04: So this is actually really important for that.

00:35:07: So to sum it all up, you are combating back pain

00:35:13: with the development of a mixed reality lens,

00:35:17: which directs people with the help of a digital therapist

00:35:22: to do exercises against back pain in the right way.

00:35:28: Yes, for a mixed reality headset.

00:35:31: I don't develop the headset, but yes.

00:35:35: Okay, just to be precise.

00:35:37: So this sounds very interesting.

00:35:41: Maybe I should try it myself with my back pain, but hey, that's my personal issue.

00:35:47: If people are interested in your whole project, and I guess there are some people,

00:35:51: how can they contact you?

00:35:53: And also maybe people from the industry and wherever they come from,

00:35:57: how can they reach you?

00:35:59: So a plan on staying with the Hochschule Trier for a while.

00:36:03: So there's my personal website at the Hochschule Trier with all my contact information.

00:36:09: So just Google "Nicola Hebke", I guess.

00:36:13: And maybe Hochschule Trier, I don't know how many others there are.

00:36:16: And there you'd also find the publications we did.

00:36:21: So our findings and we updated when we publish the paper about the BU feedback.

00:36:31: But there you can also find the paper on the system we developed for combating the interferences

00:36:40: from the vicon to the Kinect.

00:36:42: And maybe some other information about the project.

00:36:47: We also plan on publishing all the code and the introduction on how to build the system

00:36:56: so others can use it and build it.

00:36:58: So Abo can use your code for future projects.

00:37:00: Yes.

00:37:01: So if you have any questions for Nicola Hebke, if you have back pain yourself,

00:37:07: feel free to contact if you're interested.

00:37:10: And yeah, thank you very much for coming, for presenting your PhD project.

00:37:15: Sounds very interesting.

00:37:16: I will look out for it.

00:37:17: And thanks for coming.

00:37:19: And thanks for listening, maybe for watching.

00:37:23: And until next time with Deep Dive into Applied Science.

00:37:35: ?