RealityScan - Fully Headless Linux

Development Pipeline & Plugins
1

Hi all,

So I’m trying to get RealityScan running on Linux on an Amazon Ubuntu server (g6.xlarge).
I can get it running when I RDP into the machine, however, when running fully headless (i.e. SSH into the box, no display), RealityScan doesn’t do anything.

Multiple minor issues as well:

  1. -stdConsole does not return anything on Linux (either RDP or SSH)
  2. Linux based paths (e.g. /home/ubuntu/) do not work - I needed to replace /home/ubuntu/ with Y:/

Example command:

/usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit

This command will work when running terminal via RDP, but will NOT work (with no returned error message) when running via SSH.

This is a big blocker for fully automating our pipeline, so any suggestions or support is much appreciated.

2

Hi @NecroticNanite , I’ll forward your feedback to the team.

Have you tried the new Remote Command? With the plugin and samples, it is possible to send a command from one machine to another.

3

Hi @JakubVanko ,

I have not yet tried the Remote Control system - for the time being I’m focusing on a single machine executing (partially to help minimize AWS costs). I believe it likely will have the same issue, as my assumption is that RealityScan does not work unless the computer it is running on has an attached display.

Given that assumption, today I was going to try adding Xvfb to the server, to see if that corrects the issue (though without the -stdConsole command it’s a little trickier to debug) - worth noting I’m running all my commands from Python, so I was going to try:
from xvfbwrapper import Xvfb
with Xvfb():
# Run realityscan here

4

Update: Adding a virtual display does allow RealityScan to actually work!
Decided to use pyvirtualdisplay instead, so updated code:

from pyvirtualdisplay import Display
with Display(visible=False, size=(1400, 900)):
# Execute RealityScan
One issue (minor at this point, though longer term likely to be an issue) is that opening RealityScan in this way appears to add ~45s to the load time of RealityScan

Running RSEmptyProject…
Command: /usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit
Executing command: /usr/bin/realityscan -stdConsole -writeProgress Y:/server/work/project/sfm/RealityScan/progress.txt -headless -newScene -save Y:/server/work/project/sfm/RealityScan/Working.rsproj -quit
Execution time: 46.12 seconds

Note that I use the same display across multiple commands. (i.e with Display(): is at the root of the project, and I run multiple RS commands within it, to avoid multiple display creation/deletion commands). Totally understand that in many cases opening and closing RealityScan isn’t the intended approach - I currently do each of these as separate commands:

  1. Create Empty Project
  2. Configure (using -set)
  3. Import Folder
  4. Align
  5. Merge
  6. Export list of components (for additional processing)
  7. Export each component as colmap (this one I do as one long command)

So one easy optimization on my end is to unify the first 6 steps into one. (Having them split was super useful for code cleanliness and for debugging as I got the steps set up)

(Still would be useful to have -stdConsole work though!)