RealityScan - Fully Headless Linux

Development Pipeline & Plugins
1

Hi all,

So I’m trying to get RealityScan running on Linux on an Amazon Ubuntu server (g6.xlarge).
I can get it running when I RDP into the machine, however, when running fully headless (i.e. SSH into the box, no display), RealityScan doesn’t do anything.

Multiple minor issues as well:

  1. -stdConsole does not return anything on Linux (either RDP or SSH)
  2. Linux based paths (e.g. /home/ubuntu/) do not work - I needed to replace /home/ubuntu/ with Y:/

Example command:

/usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit

This command will work when running terminal via RDP, but will NOT work (with no returned error message) when running via SSH.

This is a big blocker for fully automating our pipeline, so any suggestions or support is much appreciated.

2

Hi @NecroticNanite , I’ll forward your feedback to the team.

Have you tried the new Remote Command? With the plugin and samples, it is possible to send a command from one machine to another.

3

Hi @JakubVanko ,

I have not yet tried the Remote Control system - for the time being I’m focusing on a single machine executing (partially to help minimize AWS costs). I believe it likely will have the same issue, as my assumption is that RealityScan does not work unless the computer it is running on has an attached display.

Given that assumption, today I was going to try adding Xvfb to the server, to see if that corrects the issue (though without the -stdConsole command it’s a little trickier to debug) - worth noting I’m running all my commands from Python, so I was going to try:
from xvfbwrapper import Xvfb
with Xvfb():
# Run realityscan here

4

Update: Adding a virtual display does allow RealityScan to actually work!
Decided to use pyvirtualdisplay instead, so updated code:

from pyvirtualdisplay import Display
with Display(visible=False, size=(1400, 900)):
# Execute RealityScan
One issue (minor at this point, though longer term likely to be an issue) is that opening RealityScan in this way appears to add ~45s to the load time of RealityScan

Running RSEmptyProject…
Command: /usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit
Executing command: /usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit
Execution time: 46.12 seconds

Note that I use the same display across multiple commands. (i.e with Display(): is at the root of the project, and I run multiple RS commands within it, to avoid multiple display creation/deletion commands). Totally understand that in many cases opening and closing RealityScan isn’t the intended approach - I currently do each of these as separate commands:

  1. Create Empty Project
  2. Configure (using -set)
  3. Import Folder
  4. Align
  5. Merge
  6. Export list of components (for additional processing)
  7. Export each component as colmap (this one I do as one long command)

So one easy optimization on my end is to unify the first 6 steps into one. (Having them split was super useful for code cleanliness and for debugging as I got the steps set up)

(Still would be useful to have -stdConsole work though!)

5

Hi @NecroticNanite , in short, use realityscan-cli to call RS CLI, it will work with

-stdConsole.

My example command:

DISPLAY=:1 realityscan-cli -stdConsole -addFolder “Y:/RS/Owl_statue_/_geometry” -align -save “Y:/RS/Owl_statue/Owl_statue.rsproj”

In the next post, I’ll elaborate in detail how we resolved the issue with sending commands through SSH.

1 Like
6

In our case, we were testing on a Linux VM running Ubuntu 22.04.5 LTS.

RealityScan Linux has a UI, even in -headless and -silent mode, so it requires a display to create its own Window.

Within Linux, the display can be controlled by X11, Wayland, or other display managers. The UI requests will be forwarded to them, and the window will be shown.

Upon an SSH connection, it will forward the display to the display manager of the SSH host, allowing the user to run the main executable on the remote machine while displaying it in the host system, which is desirable in many cases.

In our case, we do not want to see the window in the SSH host, so we must explicitly keep its display in the remote and not forward it.

To find which DISPLAY is used for the UI, we have a few options:

ps aux | grep Xorg

which may show something like this:


vanko+    2071  0.4  1.8 368448 149732 ?       Sl   20:05   0:28 /usr/lib/xorg/Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log
vanko+   10359  0.0  0.0   9216  2560 pts/0    S+   21:57   0:00 grep --color=auto Xorg

Then 10 in the first line is what we’re looking for, so in the Windows SSH shell, we should enter:

DISPLAY=:10 realityscan-cli

If this one doesn’t give you the correct number (like in our case), another way is to check is:

ls /tmp/.X11-unix/

Which may show you this:

X1

The number after X is your display number, which is 1 in the above case. And now you can run it:

DISPLAY=:1 realityscan-cli
1 Like