Download

HoloLens 2 model alignment with QRCode

Hi!
I have 3D model of a room and would like to align to the room in reality so that the model overlays the real room. I thought I would do this with a QRCode. So place a QRCode in the model and then place a printed version in reality in the same location. Then when scanning the QRCode with the HoloLens, the 3D model should snap into place, for example, as shown here: https://www.youtube.com/watch?v=bDB0JLYx3tA

Could anyone point me in the right direction? Are there any finished blueprints or example projects? I only found this small example at Microsoft:
https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-qr-codes?tabs=426

I’ve done exactly this with a QR code and a Hololens 2. The way i did it is by generating a QR code based on a short string of text, sticking it to a spot on the wall of the building, then reading that QR code with the Hololens, extracting the text (I made a ‘QR code scanner’ blueprint actor that looks for QR codes with a timer function), then creating an actor with the building’s mesh at the location of the QR code.

The only trick is matching the building mesh with the correct orientation and location of the QR code. Even Hololens 2 isn’t perfect, however, as you’ll always get a bit of slip and drift in the world tracking. The best it can do is match the location of generated 3D models with the perceived location of the headset, which puts you at the mercy of the Hololens 2’s cameras and tracking software.

Yeah, that’s the direction I was thinking, though I don’t quite get how you match the building mesh with the orientation and location of the QR code?
For example, do you move/create the actor with the building’s mesh at runtime or place it before at the location of the QR code? What kind of text did you store in the QR code? The location/orientation, an id or random text?

The QR code was made with goqr.me, just a super simple QR code generator, and the text was literally ‘building’ I think lol. It doesnt particularly matter what the text is, you can use anything you want as long as you can match it in Unreal and use it to spawn the actor.

For matching location/orientation, I decided ahead of time where to place the QR code in the building, placed a printed QR code in that place (discretely tacked to the wall). Then in Unreal, I made an actor with the building mesh in it and moved it around so that the spot in the building mesh that matches the QR code’s physical location is at the origin point of the actor. It took a few iterations to get the transforms quite right, but it worked fine.

From here I made a function to find the text string associated with the QR code and when the string matched my building QR code, the QR code scanner function stops and instead of tracking the QR code in real-time, the actor is just spawned in the world at the world location of the QR code. This way it remains in the same location when the headset can’t see the QR code.

Thx man, for the detailed description. I’ll try that!

Hey there - i tried to archive the same you all describe - but UE always crashed as soon as i use togglearcaptur (on) - delayed as described in Micrsoft Docs - by 0.5s … any idea what i might have missed here ? Thanks a lot.

Using 4.27 - sorry

Hmm, no, sorry, I have no idea. The example described in the Microsoft Docs works fine for me.

Where do you put the ToggleNode? I tried in LevelBlueprint/Gamemode/Pawn and Playercontroller - same outcome. Someone recommended to use a gameonstance with “on initialized” … i also tried that - samesame …
Can you by chance upload a working project, which i could try on my side? That wpuld be really awesome and helpful …

I activated the ARCapture with capture type QRCode in the level blueprint, but after starting an AR session:

I put the other part in a blueprint actor and added it to the scene:

I’m using version 4.27.1

Thanks for the reply - this is EXACTLY the setup have here … really strange … tried in 4.26 and 4.27… hm … thank you anyway!