Need help with a C++ only plugin!

Development Programming & Scripting C++
1

Hey guys and gals! Long time lurker here, just sending a thanks to everyone for all the help over the years. I know I have been able to resolve every problem I have ever had with Google and the search function… until now.

So long story short, I’m playing around with ProceduralMeshComponent and I was using LibNoise, then there’s UnrealNoise (a plugin of LibNoise for Unreal) But LibNoise is LGPLv2, which, in my opinion, is evil, and slow.
So I switched to Noise++, its got a BSD licence (actually free and open source with no strings) is multi-threaded by default and has most of the same modules as LibNoise. Too bad Noise++ has been all but abandoned.
HERE is the link to Noise++ source if anyone wants it.

So I got Noise++ to work (even multi threading) in my project, mostly by brute force, I went through every .cpp file and added my header and fixed up some minor things till the compiler stopped complaining.
But if I wanted to add the same library to another project, I would have to go through and do the same thing again! So I decided to just port it over to a Plugin! Unreal doesn’t have a good Noise plugin (that i’m aware of)(that’s not LGPLv2)
so I ran through the plugin creation wizard, and tossed in the library, fixed up some files and said “I’m only going to use this in C++, so its good for now”

I removed all of my forced in files in my project, commented out the one location where it was used, added the plugin, added it to the “PublicDependencyModuleNames”, remade the visual studio project files, just in case, (yes the plugin compiles free of errors on its own, because I made it in its own empty project) included the plugin header, Visual Studio shows no errors so far, I can get the “noisepp” namespace, I can get all the noise modules, no errors at all! Awesome! But BOOM! as soon as you compile, the worst possible compiler error shows up.



Error	2	error LNK2001: unresolved external symbol "public: virtual void __cdecl noisepp::Module::write(class noisepp::utils::OutStream &)const " (?write@Module@noisepp@@UEBAXAEAVOutStream@utils@2@@Z)	C:\Users\Trojke\Google Drive\Unreal\Projects\ExoPlanet\Intermediate\ProjectFiles\VoxChunkManager.cpp.obj	ExoPlanet
Error	3	error LNK2001: unresolved external symbol "public: virtual void __cdecl noisepp::Module::read(class noisepp::utils::InStream &)" (?read@Module@noisepp@@UEAAXAEAVInStream@utils@2@@Z)	C:\Users\Trojke\Google Drive\Unreal\Projects\ExoPlanet\Intermediate\ProjectFiles\VoxChunkManager.cpp.obj	ExoPlanet
Error	4	error LNK2001: unresolved external symbol "public: virtual void __cdecl noisepp::RidgedMultiModule::write(class noisepp::utils::OutStream &)const " (?write@RidgedMultiModule@noisepp@@UEBAXAEAVOutStream@utils@2@@Z)	C:\Users\Trojke\Google Drive\Unreal\Projects\ExoPlanet\Intermediate\ProjectFiles\VoxChunkManager.cpp.obj	ExoPlanet
Error	5	error LNK2001: unresolved external symbol "public: virtual void __cdecl noisepp::RidgedMultiModule::read(class noisepp::utils::InStream &)" (?read@RidgedMultiModule@noisepp@@UEAAXAEAVInStream@utils@2@@Z)	C:\Users\Trojke\Google Drive\Unreal\Projects\ExoPlanet\Intermediate\ProjectFiles\VoxChunkManager.cpp.obj	ExoPlanet
Error	6	error LNK1120: 4 unresolved externals

I’m not too worried about yall seeing my path to the project on my HD, or the project name, its just fooling around anyway.
A link to the github repo is HERE as I was going to release it to yall eventually.

This happens as soon as you create any of the noise modules. EX: “noisepp::RidgedMultiModule caves;”
It happens in a new project, it happens no matter what, as long as you reference a noise module, and it only happens when you reference a noise module, but the plugin compiles fine on its own, and when you pull the code away from the plugin and “brute force” it into a project, it works just fine.

I have reached the limit of my Google-fu and trying to resolve the LNK2001 error, it’s so stupidly ambiguous that it could be almost anything, and I have gotten it before and been able to fix it. But I need yalls help this time.

Sorry about the rambling, I have tried everything short of re-writing the plugin, which i’m afraid I’ll need to do, as the source has multiple classes in a single header and Unreal wont let you declare multiple UCLASS() like that.

2

Do you understand how C++ compilation process works? I don’t want to needlessly explain that.

I think you’re confusing LGPL with GPL here.

LGPL allows you to dynamically link with the LGPLed project. You’re also allowed to derive classes from LGPLed libraries. Dynamically linking with LGPL does not infect your project with that license.

So, what kind of issue could you possibly have with that? It is same license as the one used in opensource version of qt, which allows creation of proprietary commercial applications.

It is not a compiler error, it is a linker error.

The library you’re trying to compile apparently originally included implementations (i.e. function body) of several methods:
void noisepp::RidgedMultiModule::read(class noisepp::utils::InStream &);
void noisepp::Module::read(class noisepp::utils::InStream &);
void noisepp::RidgedMultiModule::write(class noisepp::utils::OutStream &)const
void noisepp::RidgedMultiModule::read(class noisepp::utils::InStream &)

When you ported the library into unreal, you most likely forgot to include *.cpp files with implementations of those methods, now linker can’t find them. Alternatively, the plugin is not configured to link with the library that contains them, but you have methods that are trying call those methods.

I believe you should learn how LGPL license really work, because I have strong impression that you’re needlessly wasting your time trying to make abandoned project work.

Also, it isn’t that hard to implement N-dimensional Perlin/Simplex noise yourself, so was there some specific feature you needed?

3

Hey, thanks for the help! It may be that i’m wasting my time, but its not like LibNoise is any less abandoned than Noise++, and Noise++ comes pre-packaged with multi-threading. I was thinking of GPL. Sorry about that, I went to TLDRLegal and got straightened out. It’s still not my favorite, but I learned my lesson.

I do have “some” understanding of how compilation works, yes. I have been a programmer for a while, but only recently (within the last year) gotten into C++ with any sort of understanding at all. I guess my level of understanding is student level at best, but when I go to look into it, it seems incredibly complicated and obtuse. I understand that its a linker error and not a compilation error, per se. But the error shows at compile time so its easier for me to think of it that way, again, i’m sorry, its probably my lack of understanding. If you would be so kind as to point me to a reference so that I could learn, I would be grateful, but the links and answers on Stack Overflow are, again convoluted, and difficult to follow.

4

Alright then. I remember writing explanation on that somewhere in the past (on another site), but I can’t find it. So here’s another.

Turning several source code files into program is building process. It is handled by one of the several existing build systems, such as GnuMake, NMake, MSBuild, Scons, cmake (technically, build config generator, etc), jom, etc. The job of build system is reduce amount of work by detecting modified files.

Building consists of two phases: compilation(per file) and linking(per project). Compilation is done by compiler, linking is done by linker.

compilation is compiler turning one *.cpp file into **object ** file. object file is *.obj in case of MS compiler on windows, *.o on unix-likes, etc. The file does not contain truly executable machine code only some sort of “template” or “draft version” of it.

compiler processes program one file at a time. Only *.cpp files are compiled (extension is determined by build system). *.h files are not compiled.

Compilation consists of two phases. Prerprocessor and actual compilation. They’re usually performed by the same tool these day.

Now, this is REALLY important:

C++ compiler treats each *.cpp file as completely separate entity. One cpp file doesn’t know anything about any other cpp files. They’re completely isolated from each other. That is very different from, say, C#/unity, where creating anything anywhere makes it visible in the rest of the project. While compiler proessess one *.cpp file from its point of view other *.cpp files do not exist. They’re in different universes.

Preprocessing phase is done before trying to compile the code or even checking it for syntax errors. Macros is applied during preprocessor phase. include directives are dealt with at preprocessing phase.

important: preprocessor is dumb as a rock and performs almost no sanity checking. It is text insertion and replacement tool. Literally.

important: include directive is a preprocessor directive, not a statement like if/else/do/while, etc. It literally inserts the file it references into program text. You can think of it as of automated Ctrl+C/Ctrl+V. So, it means that if you write simple “int main(){}” program, and include, say <windows.h>, then preprocessor will insert ENTIRE windows.h header and every single header #included by windows.h (and so on, recursively) at the place where include <windows.h> originally existed. And only after that it will try to compile that *.cpp file. That is why C++ compilation is slow. This approach has numerous advantages to it, though. Also, that is the reason for include guards.

Now here’s the thing. Each *.cpp file is completely separate and isolated entity that knows nothing about other *.cpp files. So, how do we actually call any function in the program? That is handled by linking.

During compilation phase, compiler keeps track of declared and defined functions(/methods, etc).
“defined” function is any kind of forward declaration:



void test();

“declared” function has a body:



void test(){
}

.

Important information of functions that are not static (info about non-static functions is not stored) is stored in produced *.obj files. *.obj file basically teslls that “I have body for function with name XYZ” or “I’m using function XXYZYW1 but do not have its body”. Those records act like gateways between different universes (where each universe is *.cpp file).

The whole thing is sewn together by the linker.

Linker looks through every single object and sews all those “links” together. Every occurrence of "At this point function XYZ (whose body we do not have in this *.obj file) is replaced by actual call to the body of a function. Linker also may completely remove functions (that are not used by anyone) from the program. At this stage you get executable.

At the linking stage, two kinds of errors:

  1. unresolved external. That means that there is one *.obj file that wants to call some function, BUT there’s no *.obj file that has the body of the function.
  2. Multiple definition of symbol. That means that there’s more than one function defined with the same name and same argument list and types. So, while linker is stitching object files together into single *.exe, he cannot determine to which function body he should direct actual function call.

Now, consequences of this kind of the system.

*Purpose of .h files. *.h file is pretty much supposed to contain forward definition for anything other *.cpp files might decide they need to know. Forward declarations for methods go there.

*Variables in .h files If you declare global variable in *.h file without marking it as “extern” first, then, because *.h file is inserted into *.cpp code by copy-pasting, you will end up with multiple variables at linking file. Every *.cpp will get one AND that’ll cause “multiple definitions” or similar error at the linking phase.

**Isolated types and functions. **You can create stuff nobody will know about within individual *.cpp file. You can declare a class, create several functions, extend existing namespace, and all that will be unknown in the rest of the program. It is a fairly common approach to declare abstract base class with factory method in *.h file and then in *.cpp file create platform derived class from that abstract AND return pointer to (new instance of) it from factory method.

Caveat for isolated types: However, if in two separate *.cpp files you somehow manage to create two types with identical type names and methods (I saw that happen once), linker may get confused or fail linking those files together.

You do not have to build the whole program. That means that usually while working on the program, instead of trying to build the whole thing, you can just keep compiling currently open *.cpp file when you need to check for compilation errors. It is significantly faster than trying to build everything every time, but last time I checked unreal build system does not support that.

Compiler and Linker errors It is important to distinguish between compiler and linker errors. Compiler errors (error codes start with C on microsoft compiler/visual studio) typically indicate syntax error r similar problem and they occur at compilation phase.
Linker errors (error codes start with L on microsoft compiler/visual studio) indicate errors that occur at linking phase. Typically it is “unresolved external” or “multiple definitions”.

That is C++ program building process.

5

WOW! Thank you so much for this very thoughtfully written explanation of the compilation process. I have a much better understanding of the entire process and its really not all that complicated. I can now see where things might be going wrong. If only forward declarations exist in header files, or header files don’t get compiled, what happens when you write an inline function or crate an entire class in the header? As is the case in the Noise++ library, the entire “utils” sub-namespace is written only in header files.

6

Wow, some pretty awesome info there from NegInfinity.

Specific to the issue you are having, I had a quick look at the GitHub, it looks to me that your issue is probably with DLL exports. To use types and functions from another module, they have to be exported. In UE4, this is done with a macro that is automatically defined for you based on module name. In this case, it will be NOISEPLUSPLUS_API. You need to prefix it to any class that you intend to directly use from outside of the plugin. So for example, in NoiseModule.h:



class NOISEPLUSPLUS_API Module

Do that for any class that is giving you unresolved externals when you try to use it in your project. If you want to read up on this stuff, google for dllexport.

One other thing to bear in mind, those particular functions are wrapped within a ‘#if NOISEPP_ENABLE_UTILS’ statement. Clearly NOISEPP_ENABLE_UTILS is defined for your project. It’s most likely defined somewhere in the headers in which case it’s not an issue, but if for some reason it is not defined for you plugin project, that would likely also lead to unresolved externals.

7

BOOM! That was it! So besides trial and error, or working with the Unreal build system, a lot, how do you figure this out? I’ve dug through the docs, and they are very very wanting. Do you just use trial and error and extensive knowledge of C++? I feel like right as I get the hang of something (I got Multithreading working) I find that there’s a much better system of how to do things (Async<T>) Then I dive into that, and the single forum post is kinda misleading implying that you could use any function when it has to be a TFunction (What!?) So I look into the code on that and subsequently had to look at the code for std::function to figure out how to make one, I still don’t know why one is needed.

But that’s neither here nor there, that was exactly it… As for the ‘#if NOISEPP_ENABLE_UTILS’ that’s a useless statement, if its set to 0 it throws an error. To say I took the library and tossed it into a plugin is just about exactly right. I added #pragma once to all the files and have since cleaned all the #ifndef parts out of all of it as per the coding standard, but haven’t modified any other code.

Thanks a million yall!

8

That’s pretty much it really. UE4 C++ is just C++ with a bit of stuff on top, so you really just need to read up as much as you can on C++, and build up experience. Diving into the UE4 engine code is an excellent way to learn, but it can be scary in there!

9

Contents of *.h file are inserted into *.cpp files at preprocessing phase, then it is compiled.

Inline functions and methods do not really generate “bodies” to be stored in *.obj files, so they don’t cause linker errors.

10

Yeah, i’m sorry, I feel like a moron, I read the entire thing again and my question seems like such a stupid thing to ask, like, you just told me how the system works… then I ask about that. :frowning: But just to clarify one thing, if a class doesn’t have a .cpp file, the whole class is defined, functions, constructor, destructor etc… in a header file, the header file is handled just like it made up a cpp file right?