I’m currently writing a custom heighmap generator that will produce .raw/.r16 files as needed. The documentation for generating a simple plane is quite complete but I’ve not been able find any logically coherent information on converting individual point/vertice values into specific heights. Normally I’d assume that this would scale in Unreal Units (uu) of 1cm but other posters on these forums state that the exact default range is -512m to 512m without citing any sources or giving an explanation. Given that the 16-bit value range is 0 to 65535, and the map import code has “Sea Level” as 32768 this would make that range (using uu) to be -327.68m to 327.68m unless additional scaling is going on. I need to translate the my generated height values into the either the range of 0 to 65535 or to the range 32768 to 65535. What is the correct factors for computing this?
Check out L3DT. It writes .raw/.r16 files. L3DT - Large 3D Terrain Generator
I have no idea how UE interprets the 16-bit values. I usually use the brute force method (scaling and translating the Landscape Actor repeatedly) until it fits. Obviously this is not the right way to do it. If you find the correct way of doing it, please let me know.
I had a feeling that I’d get responses like this. I’ve already investigated the generator programs out there; none meet my needs or are within my current budget. I’ve written map editing software for Unreal in the past so writing.raw/.r16 heightmaps is trivial for me. The effort is appreciated but what I need is information; not software.
A good explanation would probably go: “Heights are equal to <some multiplier> * (vertice value - 32768). The documentation for this is located <here>”. But definitely not “I have absolutely no real idea what you are doing with your project, or why (and obviously I have no need to know anything about it to answer a general technical question) but here is a program that I really think is cool that will solve your problem and will also leave you in continued ignorance about the actual matter you needed to know”. This isn’t a personal criticism against but a pattern I’ve observed happening on support forums for years. You want people to try to help if they can and you don’t want them to feel bad for trying. Please note you actually helped by trying to answer as it signals to others that this may be a topic worth paying attention to; so it all works out.
Wow, you must be smart. OBTW, Unreal wants little-endian.
# write a 127x127 bitmap
import struct
h = int(0)
c = h
f = open("Test.r16", "wb")
v = 65535.0/16128.0
for x in xrange(0, 127):
for y in xrange(0, 127):
c = int(h*v)
f.write(struct.pack('<H', c))
h = h+1
f.close()
It does at that. And C# under Windows writes little endian by default.
From a previous project. Normalized map heights across several map tiles in parallel. These tiles were used with World Composition for a large open world. Also add a water level rather simplistically with “level flooding”. shrugs
public class TilePixel
{
public uint Height { get; set; } = 0;
public byte Water { get; set; } = 0;
private double Interpolate(double InputStart, double InputEnd, double OutputStart, double OutputEnd, double Input)
{
if (InputStart == InputEnd)
return (OutputStart + OutputEnd) / 2;
return (Input - InputStart) * (OutputEnd - OutputStart) / (InputEnd - InputStart) + OutputStart;
}
private byte CalculateRed(double offset)
{
if (offset < -1 || offset > 1) return 0;
if(offset<-.25)
{
return 0;
}
if(offset<0)
{
return 0;
}
if(offset<.0625)
{
}
if (offset < .1250)
{
}
return 233;
}
public Color PixelColor
{
get
{
Color color = new Color();
double coloroffset = 3.898162254114119e-9 * Height * Height - 0.000149109892371402 * Height - 0.0302294716459023;
switch (Water)
{
case 0:
color.A = 0;
color.R = 0;
color.G = 0;
color.B = 0;
break;
case 1:
case 2:
color.A = 255;
if (Height == 0)
Height = 0xFFFF;
color.R = 0;
color.G = (byte)(Height & 0xFF);
color.B = (byte)(Height >> 8);
break;
default:
color.A = 0;
color.R = 0;
color.G = 0;
color.B = 0;
break;
}
return color;
}
}
public bool IsHole => Water > 0 && Height == 0;
public bool IsEdge => Water == 0;
public bool IsLand => Water == 2;
public bool IsWater => Water == 1;
public bool IsLandHole => IsLand && IsHole;
public bool IsWaterHole => IsWater && IsHole;
}
public class Tile
{
public const int Width = 1009;
public const int Height = 1009;
public const int Size = Width * Height;
public MapTile Source { get; set; } = null;
public int Holes { get; set; } = 0;
public int Edges { get; set; } = 0;
public int Water { get; set; } = 0;
public int Land { get; set; } = 0;
public uint Lowest { get; set; } = uint.MaxValue;
public uint Highest { get; set; } = uint.MinValue;
public int WaterHoles { get; set; } = 0;
public int LandHoles { get; set; } = 0;
public uint WaterLowest { get; set; } = uint.MaxValue;
public uint WaterHighest { get; set; } = uint.MinValue;
public int WaterLowestCount { get; set; } = 0;
public int WaterHighestCount { get; set; } = 0;
public uint LandLowest { get; set; } = uint.MaxValue;
public uint LandHighest { get; set; } = uint.MinValue;
public int LandLowestCount { get; set; } = 0;
public int LandHighestCount { get; set; } = 0;
public TileCoordinate TopLeftHoleRegion { get; set; } = -1;
public TileCoordinate BottomRightHoleRegion { get; set; } = -1;
public async Task<StorageFile> HeightFile()
{
string FileName = $"ms-appx:///Assets/Map Data/Height Data/Landscape_TitanCityX{Source.Column:00}_Y{Source.Row:00}.r16";
StorageFile file = null;
file = await StorageFile.GetFileFromApplicationUriAsync(new Uri(FileName, UriKind.Absolute));
return file;
}
public async Task<StorageFile> WaterlineFile()
{
string FileName = $"ms-appx:///Assets/Map Data/Waterline Data/WaterTile{Source.Coordinate:000}.r8";
StorageFile file = null;
file = await StorageFile.GetFileFromApplicationUriAsync(new Uri(FileName, UriKind.Absolute));
return file;
}
public async Task<uint]> GetHeights()
{
uint] heights = new uint[Size];
IBuffer fileBytes = await FileIO.ReadBufferAsync(await HeightFile());
DataReader reader = DataReader.FromBuffer(fileBytes);
reader.ByteOrder = ByteOrder.LittleEndian;
for (int Pixel = 0; reader.UnconsumedBufferLength > 0; Pixel++)
{
uint Value = reader.ReadUInt16();
heights[Pixel] = Value;
}
return heights;
}
public async Task<byte]> GetWater()
{
byte] water = new byte[Size];
IBuffer fileBytes = await FileIO.ReadBufferAsync(await WaterlineFile());
DataReader reader = DataReader.FromBuffer(fileBytes);
reader.ReadBytes(water);
return water;
}
public async Task<TilePixel]> GetPixels()
{
TilePixel] pixels = new TilePixel[Size];
IBuffer fileBytes = await FileIO.ReadBufferAsync(await HeightFile());
DataReader reader = DataReader.FromBuffer(fileBytes);
reader.ByteOrder = ByteOrder.LittleEndian;
for (int Pixel = 0; Pixel < Size; Pixel++)
pixels[Pixel] = new TilePixel();
for (int Pixel = 0; reader.UnconsumedBufferLength > 0; Pixel++)
{
uint Value = reader.ReadUInt16();
pixels[Pixel].Height = Value;
if (Value > 0)
{
if (pixels[Pixel].Height != Value)
{
MessageDialog dialog = new MessageDialog("Something ain't Right!!!");
await dialog.ShowAsync();
}
}
}
byte] water = new byte[Size];
fileBytes = await FileIO.ReadBufferAsync(await WaterlineFile());
reader = DataReader.FromBuffer(fileBytes);
reader.ReadBytes(water);
for (int Pixel = 0; Pixel < Size; Pixel++)
{
byte Value = water[Pixel];
pixels[Pixel].Water = Value;
}
return pixels;
}
public async Task GetStatistics()
{
TilePixel] Pixels = await GetPixels();
for (TileCoordinate Pixel = 0; Pixel < Size; Pixel++)
{
TilePixel value = Pixels[Pixel];
if (value.IsEdge)
{
Edges++;
}
if (value.IsHole)
Holes++;
if (value.IsLandHole)
{
Land++;
LandHoles++;
}
if (value.IsWaterHole)
{
Water++;
WaterHoles++;
}
if (value.IsLand)
{
Land++;
if (value.Height < LandLowest)
LandLowest = value.Height;
if (value.Height > LandHighest)
LandHighest = value.Height;
}
if (value.IsWater)
{
Water++;
if (value.Height < WaterLowest)
WaterLowest = value.Height;
if (value.Height > WaterHighest)
WaterHighest = value.Height;
}
if (!value.IsHole)
{
}
}
TopLeftHoleRegion = -1;
BottomRightHoleRegion = -1;
for (TileCoordinate Pixel = 0; Pixel < Size; Pixel++)
{
TilePixel pixel = Pixels[Pixel];
if (pixel.IsEdge)
continue;
int TopLeftX = -1;
int TopLeftY = -1;
int BottomRightX = -1;
int BottomRightY = -1;
if (pixel.IsHole)
{
if (TopLeftHoleRegion < 0)
{
TopLeftHoleRegion = Pixel;
BottomRightHoleRegion = Pixel;
TopLeftX = TopLeftHoleRegion.Column;
TopLeftY = TopLeftHoleRegion.Row;
BottomRightX = TopLeftX;
BottomRightY = TopLeftY;
continue;
}
if (Pixel.Column < TopLeftX)
{
TopLeftX = Pixel.Column;
TopLeftHoleRegion = TopLeftY * Width + TopLeftX;
}
if (Pixel.Row < TopLeftY)
{
TopLeftY = Pixel.Row;
TopLeftHoleRegion = TopLeftY * Width + TopLeftX;
}
if (Pixel.Column > BottomRightX)
{
BottomRightX = Pixel.Column;
BottomRightHoleRegion = BottomRightY * Width + BottomRightX;
}
if (Pixel.Row > BottomRightY)
{
BottomRightY = Pixel.Row;
BottomRightHoleRegion = BottomRightY * Width + BottomRightX;
}
continue;
}
if (pixel.IsWater)
{
if (pixel.Height == WaterLowest)
{
WaterLowestCount++;
continue;
}
if (pixel.Height == WaterHighest)
{
WaterHighestCount++;
}
continue;
}
if (pixel.Height == LandLowest)
{
LandLowestCount++;
continue;
}
if (pixel.Height == LandHighest)
{
LandHighestCount++;
}
}
}
public async Task<byte]> HeightImage()
{
uint] TileData = await GetHeights();
byte] ImageBuffer = new byte[Size * 4];
for (int ReadPixel = 0, WritePixel = 0; ReadPixel < TileData.Length; ReadPixel++, WritePixel += 4)
{
ImageBuffer[WritePixel + 0] = 0;
ImageBuffer[WritePixel + 1] = (byte)(TileData[ReadPixel] >> 8);
ImageBuffer[WritePixel + 2] = (byte)(TileData[ReadPixel] & 0x00FF);
ImageBuffer[WritePixel + 3] = 255;
}
return ImageBuffer;
}
public async Task<byte]> WaterImage()
{
byte] TileData = await GetWater();
byte] ImageBuffer = new byte[Size * 4];
for (int ReadPixel = 0, WritePixel = 0; ReadPixel < TileData.Length; ReadPixel++, WritePixel += 4)
{
ImageBuffer[WritePixel + 0] = (byte)(TileData[ReadPixel] == 1 ? 255 : 0);
ImageBuffer[WritePixel + 1] = (byte)(TileData[ReadPixel] == 2 ? 255 : 0);
ImageBuffer[WritePixel + 2] = (byte)(TileData[ReadPixel] == 0 ? 255 : 0);
ImageBuffer[WritePixel + 3] = 255;
}
return ImageBuffer;
}
public async Task<byte]> IntegratedImage()
{
uint] HeightData = await GetHeights();
byte] WaterData = await GetWater();
byte] ImageBuffer = new byte[Size * 4];
for (int ReadPixel = 0, WritePixel = 0; ReadPixel < HeightData.Length; ReadPixel++, WritePixel += 4)
{
uint Height = HeightData[ReadPixel];
byte Water = WaterData[ReadPixel];
if (Water == 0)
{
ImageBuffer[WritePixel + 0] = 0;
ImageBuffer[WritePixel + 1] = 0;
ImageBuffer[WritePixel + 2] = 0;
ImageBuffer[WritePixel + 3] = 0;
continue;
}
if (Water == 2 && Height == 0)
{
Height = 0xFFFF;
}
ImageBuffer[WritePixel + 0] = (byte)(Height >> 8);
ImageBuffer[WritePixel + 1] = (byte)(Height & 0x00FF);
ImageBuffer[WritePixel + 2] = 0;
ImageBuffer[WritePixel + 3] = 255;
}
return ImageBuffer;
}
public Tile()
{
}
public Tile(MapTile tile)
{
if (tile == null)
return;
Source = tile;
}
}
public class TileCoordinate
{
public int Coordinate { get; set; } = 0;
public int Column => Coordinate % Tile.Width;
public int Row => Coordinate / Tile.Width;
public static implicit operator TileCoordinate(int x)
{
TileCoordinate coordinate = new TileCoordinate();
coordinate.Coordinate = x;
return coordinate;
}
public static implicit operator int(TileCoordinate m)
{
return m.Coordinate;
}
}
public class Map
{
public const int Width = 19;
public const int Height = 23;
public const int Size = Width * Height;
public int Holes { get; set; } = 0;
public int Edges { get; set; } = 0;
public uint WaterLevel { get; set; } = 0;
public int Water { get; set; } = 0;
public int Land { get; set; } = 0;
public int WaterHoles { get; set; } = 0;
public int LandHoles { get; set; } = 0;
public uint WaterLowest { get; set; } = uint.MaxValue;
public uint WaterHighest { get; set; } = uint.MinValue;
public int WaterLowestCount { get; set; } = 0;
public int WaterHighestCount { get; set; } = 0;
public uint LandLowest { get; set; } = uint.MaxValue;
public uint LandHighest { get; set; } = uint.MinValue;
public int LandLowestCount { get; set; } = 0;
public int LandHighestCount { get; set; } = 0;
public Tile] Tiles = new Tile[Size];
public void GetBaseStatistics(MapTile tile)
{
Edges += Tiles[tile].Edges;
Holes += Tiles[tile].Holes;
Water += Tiles[tile].Water;
Land += Tiles[tile].Land;
WaterHoles += Tiles[tile].WaterHoles;
LandHoles += Tiles[tile].LandHoles;
if (Tiles[tile].WaterLowest > 0 && Tiles[tile].WaterLowest < WaterLowest)
WaterLowest = Tiles[tile].WaterLowest;
if (Tiles[tile].WaterHighest > WaterHighest)
WaterHighest = Tiles[tile].WaterHighest;
if (Tiles[tile].WaterLowest > 0 && Tiles[tile].LandLowest < LandLowest)
LandLowest = Tiles[tile].LandLowest;
if (Tiles[tile].LandHighest > LandHighest)
LandHighest = Tiles[tile].LandHighest;
}
public void GetDerivativeStatistics(MapTile tile)
{
if (Tiles[tile].WaterLowest == WaterLowest)
WaterLowestCount += Tiles[tile].WaterLowestCount;
if (Tiles[tile].WaterHighest == WaterHighest)
WaterHighestCount += Tiles[tile].WaterHighestCount;
if (Tiles[tile].LandLowest == LandLowest)
LandLowestCount += Tiles[tile].LandLowestCount;
if (Tiles[tile].LandHighest == LandHighest)
LandHighestCount += Tiles[tile].LandHighestCount;
if (WaterHighest > WaterLevel)
WaterLevel = WaterHighest;
//if (LandLowest <= WaterLevel)
// LandLowest = WaterLevel + 1;
}
public Map()
{
}
}
public class MapTile
{
public int Coordinate { get; set; } = 0;
public int Column => Coordinate % Map.Width;
public int Row => Coordinate / Map.Width;
public static implicit operator MapTile(int x)
{
MapTile coordinate = new MapTile();
coordinate.Coordinate = x;
return coordinate;
}
public static implicit operator int(MapTile m)
{
return m.Coordinate;
}
}
Whatever language/method you choose to generate your heightmap, I don’t see how you are going to get around fiddling with the scale/translation. I think the import function does some smoothing of the terrain whether you like it or not - thus the iterative fooling with the scale/translation. I hope you find an efficient way of doing it, and let us know about it.
If you don’t choose one of those overall sizes listed in the Landscape Technical Guide, UE4 does some screwy stuff to the edges of the Landscape Actor. I know this from bitter experience.
Looking at the Unreal Wiki for Word Machine to Unreal 4 translation I’ve found that the range is indeed -512m to 512m on the Z-axis when the import % is at 100. This means that the multiplier is 1.5625cm. For reference that is around .32 miles. My project has a max height of 7,229m making for each step from “sea level” about .22m. My Z-scale % is 14.12. This should be interesting to see. I think I’ll make my scaling an exact 1400% and my peak, as a consequence 7,168m.
I stick to those sizes for just that reason. And the import does no smoothing. I’ve actually looked that the source. It just reads the values in from the file and shoves them into an array. At least the .raw files. I haven’t looked at the source for .png imports. The docs have a guide on creating a custom importer and I’m thinking of doing just that.
Since so many people use World Machine you would think Epic would have a better pipeline from WM to UE4, than just importing heightmaps.
Good to know that there is no mesh smooth on import.
A little late, I currently use GDAL translate scale
-scale [src_min src_max [dst_min dst_max]]
-scale [img_min img_max[32767 65535]]
So 16 bit png = 0-65536 about half of that is 32767 so when you scale you basically get -32767 to +32767 with sea level being 0. At least this is my understanding and it works.
This sets the landscape imported from the heightmap to sea level. I would love to know how to change the array data of the image manually instead of having to use gdal scale
I’m familiar with GDAL but I’m uncertain how it applies to my original question. For the record Unreal uses a zero value to represent a hole in the map. So the actual height range is 1 to 65535. This allows for 32,787 values below sea level and the same amount above. So sea level then becomes 32,787.
Setting this up as a simple linear slope conversion formula we have
65535m+b=512
m+b=-512
Solving for m:
subtract the bottom equation from the top -
65534m=1024
divide by the coefficient of 65534 -
m=1024/65534 which is approximately 0.0156254768517106
To solve for be we just substitute the value of m back into one of the original equations:
(0.0156254768517106)+b=-512
Subtraction gives us b=-512.156254768517106
If you are trying to reduce rounding errors this formula will work best:
(32,767x+16,777,216)/32,767=y
Simply multiply by 100 to convert from meters to centimeters.
You can take a look at my heightmap generator for real world landscapes. Unreal Map Bridge (justgeektechs.com). You can see that it has a sea level checkbox. I use the GDAL code to adjust the height to sea level. The source code is here delebash/unreal_map_bridge: Import real world locations into Unreal Engine (github.com). You understand the math better than me but I was able to find the solution to make a height map to sea level using GDAL. The relevant code is here unreal_map_bridge/src/javascript/image-utiles.js at master · delebash/unreal_map_bridge (github.com). See the function manipulateImage sealevel=true this is how I use GDAL to change the heightmap to sea level.
You need a free mapbox or maptiler account to use it. Using Mapbox you can generate weightmaps but the free account now requires credit card info which it did not when I originally created the app. Maptilier free accounts don’t require credit card but has reduced features. Both free accounts have very generous download amounts each month but now Mapbox will charge you for the overuse. But unless you are downloading an huge amount you usually won’t go over the free limits, you can check the limits on Mapbox’s site, and they provide an easy way to check how much you have downloaded each month.
OK. This is a very frustrating response to the original question which is about converting scales the scale of one map system to another. You did very complicated reply - with detailed pictures even - about how to generate heightmaps using an app you created.
But I already HAVE a world map!
I currently have a large open world map that is 32,258km x 32,258km in size. To do this from real world data I had to break a larger map into an 8x8 grid of 2017x2017 16-bit pixel tiles accounting for the overlap pixel on the tile join. Can your app even handle this? This is what the tool I created in the original post did. I have already imported this map into the engine. In the engine I had to create an ocean and manually set the height. The world and ocean have a location coordinate consisting of an X, Y, and Z component. I need to translate those Z values into the range of 1 to 65535.
So, tell me why you thought I was looking for a way to import a real-world map.
You went through a lot of trouble to create your last post - including pictures - without any explanation how this connects to converting map height values from one system to another. Look at the title of this thread again.
In the engine sea level is whatever I want it to be. GDAL has absolutely no relationship to that.
So I want to go over every pixel of my world height map file - all 16,129 x 16,129 values and determine from the given world and ocean heights in the engine and determine where the coastline is and what is below the ocean surface. From that I’m generating an image file for other reasons.
My misunderstanding I thought you were trying to make the heightmap, so it imported and created a landscape at sea level, not adjusting it after the heightmap was imported. I did not think you were trying to import a real-world location. The app was just an example of how I modify a heightmap image, so it imports into Unreal at sea level.
Regarding your original post, I thought you wanted to scale your height array to 32768 or sea level. This is exactly what I showed you in the GDAL code example I provided a link to. GDAL transform has an option to scale the image. By using the GDAL scale it transforms each pixel in the image to a scale of 32768 which is what I thought you were asking. I would prefer not to use GDAL to do this and just apply a math formula to each pixel in my height array, but I just did not know what that formula would be. So, I provided an alternate solution of using GDAL to achieve those results. If you create a new blank landscape in Unreal and then export it as a heightmap it will be in the grays range not black and white height values because the exported heightmap is scaled at 32768. By using the GDAL scale set to 32768 it produces a heightmap file in the grays range just like what UE is expecting for a new landscape at world z of 100.
Here is an example of what I was referring to and what I thought you wanted. I generated a heightmap from Gaea using one of their examples without any changes. I exported it as a png and then imported it into Unreal. In the first image you can see that the landscape z position is 100. However, if you look at the sphere placed on the flat part of the landscape it’s z is -22954.542618. This is the actual world position of your landscape and not at the 100 it is supposed to be at. The GDAL code I showed you modifies the heightmap file so that sphere would actually be around 100 z position showing that the landscape did import at the correct world z position of 100. I was just trying to help you did not have to be rude about it.
Yeah, I figured you were confused. You spent a great deal of time trying to understand what was a very old question and give a detailed answer. This effort is appreciated even if it kind of went in the wrong direction. I hope I wasn’t too impolite in my reply.