I’m going to be working with some architectural visualization possibly using Unreal and the Oculus in the next month or so. I was hoping to get some opinions on what kind of hardware would be a good fit for Visualization purposes. I am currently looking at the following server system as the base.
Beyond that I would be most likely using a full set of 8 Titan X cards. I honestly don’t know how much of that Unreal will be able to leverage. The machine will also serve a dual purpose of also realtime visualization in Octane, so it’s not just being built for Unreal usage. Any feedback on high end HPC would be greatly appreciated. And before someone asks, I am not sure of what the final budget is, but I know just the base server and the graphics cards are well within the budget.
For UE4 it’s probably overkill. At least for the graphics cards UE4 can only take advantage of one GPU, any more than that will do nothing. The CPU’s will be of most benefit for building lighting, though for fast architectual visualization you should look into downloading the Nvidia VXGI build of UE4, which gives you realtime GI so you don’t have to spend the time building lighting and doing lightmap UV’s. Not as nice of a result as baked lighting, but it depends on how easy you want things and how fast you want to turn out images/animations.
Considering the unique hardware though, I would think you might have some hardware issues–given that there isn’t much development in that type of hardware it might not be too reliable.
That’s also going to be ridiculously expensive.
Thanks for the feedback, I’ll start looking into VXGI. I’m surprised though that Unreal only uses one GPU? What about in SLI configuration, from my understanding the memory would only occupy the space of a single card but the processing itself would be shared among the cards? The end goal is really to hand an investor a game controller and an Oculus and take a virtual tour of the facility, where they are free to explore the entire building before it’s built but to also have all of the lighting and affected by the surrounding buildings.
I realize the hardware configuration is rather unique, typically we would use either quadro cards or tesla cards as they are designed for that sort of usage. As of right now from my understanding of what we’re doing with the hardware is essentially building the first node of a gpu cluster. The plan is to (if the build is successful) expand to more nodes as needed.
SLI requires a specific profile for each game, you have to work with Nvidia to create an SLI profile for your project based on the engine and what features you use. That would be difficult in your situation since you’ll likely be making many different projects that you aren’t selling like game are usually sold.
With SLI, both graphics cards have to load the same stuff, so the memory doesn’t stack. With DirectX 12 it has a new feature that can combine any DX12 graphics card where the memory stacks–however, it’s a similar case as SLI, the game has to be configured specifically how it sends processes to each card so it’s not automatic.
A Titan X will be very good for your situation, gaming cards are faster than the workstation cards.
Given your budget could you split the [workstation + user viewing platform] from the [first node of the UE4 CPU and Octane GPU render] cluster?
This has several advantages.
Firstly, it’s nice to isolate the production user viewing machine from the development machines/cluster farm.
Additionally, especially for Octane, Nvidia GRID etc. you have to establish a client-node set up if you’re going to be using it long term… and with various systems eg. Octane, Nvidia GRID etc. you normally have one client machine that need not be super powerful that just acts as a client to the server render farm for realtime and offline rendering.
Finally, as mentioned above, for UE4 non-realtime lightmapping distribution to Swarm CPU cluster seems like a good idea especially over a fast wired network… This way the UE4 viewing and development is on a production machine (Machine 1 below) that’s stable and not messed with too much, then to scale you just add servers (CPUs in the case of UE4 Swarm) to the render farm. For example, you don’t want to be pushing or updating Octane render farm stuff on the same machine when the investor is using Occulus viewing. Have Machine 1 below be more “consumer” and in line with Occulus viewing and straightforward UE4 development. Have Machine 2, 3, 4 have all the fancy stuff.
Eg:
Machine 1
Purpose: Compiled EXE ArchiViz Occulus Viewer + UE4 development workstation (developer works on it, tests it, client uses it to view)
Spec: Titan X (fastest single GPU, so that for UE4, games, compiled exe, Occulus, you don’t need to mess with SLI stuff) with fastest Core i7 (6 Core Skylake?) available, 16GB RAM, 512GB or more SSD (SSD is a MUST for primary drive), other drives for archiving can be 7200rpm 4TB+ on local machine otherwise SAN etc which is important in the long run and/or required for render farms etc.
Machine 2
Purpose: UE4 CPU offline lightmapping render node and Octane GPU render node (1st node, add accordingly)
Spec: Fast Xeon 8 physical cores or more (for fast UE4 Swarm offline lightmapping), GPUs eg.Tesla, Nvidia GRID as recommended for Octane realtime.
Networking
Super fast connection (wired switch etc) between 1 and 2, SAN storage, etc.
You’ll need this set up anyway if you add more nodes (no. 2 above) whereby more nodes will allow for faster UE4 lightmapping (CPU dependent) and of course faster Octane GPU renders.
I think this will be a very good use of your admirable budget.
Cheers for the info. As per my other thread the realism of rendering is getting really crazy. Check out an Octane realtime of Breaking Bad: Octane Render: realtime ray tracing – fxguide
With having so many different projects being compiled I think you’re right that SLI might be no good in this case. That directx 12 feature sounds interesting though, I’ll look into that, but again like you said it would have to be an individual configuration for each scene.
, you’re idea on splitting the viewing platform from the server is spot on. It got me to thinking about how the Octane demo itself is run on their website where you can essentially control a virtual machine that has octane loaded on it straight from your browser. That might be an interesting avenue to explore but I don’t know how well that would do with the Oculus which needs that 75fps refresh to avoid discomfort. Perhaps the Oculus implementation, while very interesting, is something that we should do in house only when clients come to visit our headquarters. In house, the client server model that you described would be perfect. I do have a question about your model though. In the case of the server you specify either a tesla configuration or a Grid, is there a reason for this? As for the Xeon processors I’ve been looking at those 18 core ones in the E5 family that rolled out I think quarter 3 of last year. I’ve also been looking into perhaps beginning to upgrade our existing network infrastructure to fibre as well so this may be a good excuse to begin rolling out some upgrades. The area that we are based in is supported by Google Fiber, so now may be a good time to switch over.
I’ll see about testing out the Swarm lightmapping, I have a few contacts at SCAD that might be interested in looking into this at the Systems office. Btw your shots from in engine look great! My only critique might be looking into creating scenes with more complex objects or textures/shaders. I’ve started toying with Octane and the VR environment and so far I have to say on my machine the results have been pretty good. I’m excited to see what the speed bost is going to be like using the server though. Right now in my personal machine I have a pair of Quadro K5000s and a GTX 770. Had a second 770 but my riser cable gave out.
Hi, Cheers! Yeah I’m just starting with the Marketplace examples, haven’t created any scenes myself except very rough re-texturing… I’m more from 2D land of design (not much nowadays) and 2D coding (now learning 3D stuff)
OK as for Oculus VR and Octane see no. 4 below.
2a.
For Octane client-server, I think that will depend on the best GPU setup for Octane realtime rendering, eg. Nvidia vGPU GRID etc: Page Not Found | NVIDIA …So they have Supermicro/ others as a certified hardware partner, etc. You can do a test drive of Nvidia GRID here: NVIDIA & VMware Free Test Drive | NVIDIA …These will probably be Telsa cards as they are certified for vGPU cloud-rendering etc for Octane. Although, Nvidia 980Ti and Titan X are very, very impressive cards and much cheaper.
2b.
As of 2 years ago this was Octane realtime on Nvidia GRID vGPU etc: https://www.youtube.com/watch?v=uRdSxZtUpFk …I think this is obviously the future, because firstly you have a scalable render farm accessible from any client workstation, in potentially any part of the world. Secondly, as mentioned in the video, for film, architecture, etc. you reduce previz and iteration time since while it is maybe a few seconds per frame it helps a lot to move the camera and then see the final Octane render for that frame appear in only a few seconds. Fast-foward to today 100 GPUs (Maxwell Teslas) would still be less than 10fps I reckon for path-tracing(?). 200 Maxwell Telsas might get you to 30fps for certain kinds of Octane rendering but I have no idea about Octane just that it looks really sweet.
I think the client machine with a GTX 980 Ti or Titan X with Unreal 4 and Oculus is your safest “client-facing” machine… this would focus on realtime, VR, etc. You’d probably have your Octane GPU server cluster for near-realtime, non-VR, etc. accessed through vGPU for development and for clients that want to come do some quick iterations. Then yeah your Octane GPU cluster could have the Xeons for Swarm for the client or you could have a separate Xeon farm for Swarm. 16, 32 or 64 physical cores of Xeon for Swarm, not sure what the benchmarks are but that would certainly speed up Lightmass massively. Again, I’m no Lightmass or Swarm expert.
Currently Octane VR is more of a 360-panorama type thing: OTOY • OTOY & Oculus present: “Render the Metaverse” Contest …We might be a little while away from fully interactive realtime 75fps to 100fps Oculus Octane VR, even with a massive cloud GPU cluster eg. Octane allowing you to test their GPU cloud: https://orc.otoy.com/
( My test renders for UE4 3K/4K here: