Quest Passthrough Camera Access via Camera2 API [UE5 Plugin]

Development Platform & Builds XR Development
,
1

image
image1338×811 77 KB

Quest 2/3/Pro Camera2 API Plugin - Camera Access Implementation Sample for UE5

I’ve created a plugin that accesses Camera2 API via JNI on Quest 2/3/Pro, retrieves camera feed as UE texture, and displays it on a Widget.

Originally developed as a base for QR code recognition using Quest’s camera, I’m releasing this as an implementation example of Camera2 API access.

Environment:

  • Tested with UE 5.3
  • Works on Quest devices only (verified on Quest 3)
  • Does not work in Editor/PCVR (returns null texture by design)

Notes:

  • Fixed resolution at 320x240 (v1.0)
  • Color calibration in progress (slight orange tint)
  • Camera permission required on first launch

Please use this as an implementation sample for accessing Camera2 API on Quest.

Repository: GitHub - tark146/UnrealAndroidCamera2Plugin
License: MIT License (free for commercial and non-commercial use)

4 Likes
2

Very nice, i recently made one for ArUco tracking as well, supports max resolution

1 Like
3

im working on something similar, were you able to do 3d aligment perfectlu?

4

not sure what u mean by 3d alignment, do u mean 3d positioning?

5

yea! I have everything working on my end for texture→solvepnp but just have an issue with the positioning coversion (there is a bit of an offset) and im not sure where its coming from

6

U can get this from android camera characteristics, this is the offset between camera and tracking origin, but i found this not too precise, so u may need to do a hand-eye calibration to figure out the offset as well

7

Hi,
Thank you very much for your plugin. I’m using 5.5 and it’s working on my quest3. I would like to know why it’s not possible to activate the camera in Unreal Editor or with an application .exe (launched from my metaquest link). Thank you again,

8

I haven’t figured out how to access the camera from Unreal Editor or a PC executable yet.
Since PassThrough works via Meta Quest Link, I assume it should be technically possible, but my current scope is focused on enabling QR reading directly on the Quest device. Because of that, I haven’t had the capacity to explore PC or exe support. I’d like to investigate it in the future, but honestly, I just haven’t had enough time. Sorry about that.

By the way, could you tell me what kind of use case you have in mind for accessing the camera image from the PC or within the Editor?