Currently no, but this feature is planned for the near future .
Hopefully if all goes well I will have some big news for you in the next few weeks.
Awesome and many thanks
Hey Guys!
So 4.8 is almost ready and there are few changes and additions!
- To allow better performance the sensor will be off by default and will require you to explicitly start it using a simple blueprint function.
- You can now turn on and off the sensor on demand!
- Body index frame are now available!
- Cleaned up a lot of garbage under the hood.
Hi Lion
Did you manage to find time to release the 4.8 version ?
[=;340364]
Hey Guys!
So 4.8 is almost ready and there are few changes and additions!
- To allow better performance the sensor will be off by default and will require you to explicitly start it using a simple blueprint function.
- You can now turn on and off the sensor on demand!
- Body index frame are now available!
- Cleaned up a lot of garbage under the hood.
[/]
Hopefully tomorrow 
(Adding few last touches to body tracking to allow very easy avateering)
Hey All!
So I know I am promising you 4.8 for a while but due to time it took to fix various performance issue, better avateering support the up coming 4.9 I skipped 4.8 and ported the plugin to 4.9.
Also there is a new example demo room, it is still wip but I know you waited for a long time for it so there it is:
You will need the latest 4.9 engine to build it.
Enjoy!
so we should use the plugin folder from the demo you have uploaded ? is it a stable one ?
The one that is in the master branch is a stable one, the one in the develop branch is the latest one.
Also I added joint smoothing last night and you can see the usage example in the avateering animblueprint
I could not compile the plugin.
I noticed there are lots of references to non-existing files, but after removing it worked.
plus I still donāt get the purpose of in Editor plugin for kinect, I removed it from the build for myself, but is there any actual reason that you developed that ?
[=plangton;365898]
I could not compile the plugin.
I noticed there are lots of references to non-existing files, but after removing it worked.
plus I still donāt get the purpose of in Editor plugin for kinect, I removed it from the build for myself, but is there any actual reason that you developed that ?
[/]
Thanks for the feedback, I will check it out.
The purpose of the editor module is for future features such as gesture and speech recognition that require importers.
Some other points :
1 - in frameupdate it is not checked if the destination texture is null or is already locked by other threads or sections of codes.
2 - when I run the program in the standalone mode then the plugin does not work
[=plangton;366498]
Some other points :
1 - in frameupdate it is not checked if the destination texture is null or is already locked by other threads or sections of codes.
2 - when I run the program in the standalone mode then the plugin does not work
[/]
Can you please post the line number?
/blob/develop/Plugins/KinectV2/Source/KinectV2/Private/KinectSensor.cpp line number 768 (void FKinectSensor::UpdateTexture(UTexture2DpTexture, const RGBQUADpData, uint32 SizeX, uint32 SizeY))
so the point is that you lock the texture in line 779 this way : uint8 Dest = (uint8*)Texture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);* . without checking if its already locked.
I suggest at least a simple lock check such as Texture->PlatformData->Mips[0].BulkData.IsLocked() can be used and whether check it as a infinite while condition or put the thread in sleep in that while. for sure you have even better suggestion than mine.
this makes it complicated if another thread is trying to manipulated a previously fetched texture from kinect.
PS : is it a big hassle to add CoordinateMapper::GetDepthCameraIntrinsics() function to your plugin since you already have an instance of the coordinateMapper ? I was thinking to add it but wanted to consult with you
[=plangton;368214]
/blob/develop/Plugins/KinectV2/Source/KinectV2/Private/KinectSensor.cpp line number 768 (void FKinectSensor::UpdateTexture(UTexture2DpTexture, const RGBQUADpData, uint32 SizeX, uint32 SizeY))
so the point is that you lock the texture in line 779 this way : uint8 Dest = (uint8*)Texture->PlatformData->Mips[0].BulkData.Lock(LOCK_READ_WRITE);* . without checking if its already locked.
I suggest at least a simple lock check such as Texture->PlatformData->Mips[0].BulkData.IsLocked() can be used and whether check it as a infinite while condition or put the thread in sleep in that while. for sure you have even better suggestion than mine.
this makes it complicated if another thread is trying to manipulated a previously fetched texture from kinect.
PS : is it a big hassle to add CoordinateMapper::GetDepthCameraIntrinsics() function to your plugin since you already have an instance of the coordinateMapper ? I was thinking to add it but wanted to consult with you
[/]
The reason for not checking if the texture is locked comes from the design of the plugin flow, this function is being called only in the game thread by the kinect input device when UpdateTexture is called.
When UpdateTexture is called I use a mutex to guard the memory from being overwritten by the Kinect Thread.
On game thread:
void FKinectSensor::UpdateColorTexture(UTexture2D*pTexture)
{
SCOPE_CYCLE_COUNTER(STAT_KINECT_SENSOR_ColorUpdateTime);
if (m_bNewColorFrame)
{
FScopeLock lock(&mColorCriticalSection); // <--------
UpdateTexture(pTexture, m_pColorFrameRGBX, cColorWidth, cColorHeight);
m_bNewColorFrame = false;
}
return;
}
In the Kinect thread:
void FKinectSensor::ProcessColorFrame(IColorFrameArrivedEventArgs*pArgs)
{
SCOPE_CYCLE_COUNTER(STAT_KINECT_SENSOR_ColorProcessTime);
TComPtr<IColorFrameReference> pColorFrameReferance = nullptr;
HRESULT hr = pArgs->get_FrameReference(&pColorFrameReferance);
if (SUCCEEDED(hr)){
TComPtr<IColorFrame> pColorFrame = nullptr;
if (SUCCEEDED(pColorFrameReferance->AcquireFrame(&pColorFrame))){
RGBQUAD *pColorBuffer = NULL;
pColorBuffer = m_pColorFrameRGBX;
uint32 nColorBufferSize = cColorWidth * cColorHeight * sizeof(RGBQUAD);
{
FScopeLock lock(&mColorCriticalSection); // <------------------------
hr = pColorFrame->CopyConvertedFrameDataToArray(nColorBufferSize, reinterpret_cast<BYTE*>(pColorBuffer), ColorImageFormat_Bgra);
m_bNewColorFrame = true;
}
}
pColorFrame.Reset();
}
pColorFrameReferance.Reset();
}
The textures that are being updated are members of the KinectInputDevice class and so dose the KinectManager instance that triggers the frame events.
When the kinect manager triggers new frame events what actually happens is that this is done by the KinectInputDevice that checks if there is a new frame if there is, it updates the textures using one of the frame update functions of the KinectSensor class than it passes a pointer to the KinectEventManager event dispatcher for that frame.
All of that is done sequentially in the game thread after the frame is updated it wonāt be locked again anywhere else.
P.S.
I am working on coordinateMapper pm me if you have specific feature requests.
Thanks for the explanation.
The issue I face is that I get the depth picture from your plug-in and then I start a thread for processing the depth data.
mainly in the thread I lock the texture with read and fill an array. but your then the process was crashing with error referring to the lines I posted for you.
I think your mutex somehow does not apply to my thread and error happens in the write data to the texture memory.
I solved the issue by some lock and null check, but then I found out that it is speed dependent, meaning that when I decrease the frame rate and make the app slow then the same issue happens
This is strange, AFAIK the texture shouldnāt be be updated till allof the functions that are bound to the depth event finish processing, I will look into it.
It would be grate if you could please send me the problematic code.
I am working on transferring the output of the frame events to render targets, that would decouple the update from the texture and would be more correct with the threading model of the engine.
[=plangton;369107]
Thanks for the explanation.
The issue I face is that I get the depth picture from your plug-in and then I start a thread for processing the depth data.
mainly in the thread I lock the texture with read and fill an array. but your then the process was crashing with error referring to the lines I posted for you.
I think your mutex somehow does not apply to my thread and error happens in the write data to the texture memory.
I solved the issue by some lock and null check, but then I found out that it is speed dependent, meaning that when I decrease the frame rate and make the app slow then the same issue happens
[/]
You gave me some material to think about and I just tested a new method of retrieving camera frames, and from what I see here it boosted the performance by about 20 FPS.
I did so by completely throwing away the update events and therefore the scope locks
Now you retrieve the camera data by getting a pointer to a texture that is being updated directly from the sensor thread by enqueuing the texture update in the rendering thread.
The next step would be to broadcast an event from the rendering after which you would be able to access the data safely (this is similar to the OnHud event).
You can find it here (the body index frame is still using the old model, will update it later today).
So, Iāve grabbed the KinectDemoRoom project from GitHub, attempt to open the uproject with UE 4.9, and it gave me this:
When I press Yes, this appears:
So I opened the VS solution file generated within the same folder, inside it has the UE4 and KinectDemoRoom vc projects. For both, Iāve cleaned and built them using the Develop_Editor configuration.
UE4 didnāt seem to have any compiling problem, but when I try to rebuild KinectDemoRoom, it returned with this errors:
1>------ Rebuild All started: Project: KinectDemoRoom, Configuration: Development_Editor x64 ------
1> Cleaning KinectDemoRoomEditor Binaries...
1> Creating makefile for KinectDemoRoomEditor (no existing makefile)
1> Performing full C++ include scan (no include cache file)
1> Parsing headers for KinectDemoRoomEditor
1> Running UnrealHeaderTool "<root_folder>\KinectDemoRoom\KinectDemoRoom.uproject" "<root_folder>\KinectDemoRoom\Intermediate\Build\Win64\KinectDemoRoomEditor\Development\UnrealHeaderTool.manifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -rocket -installed
1>EXEC : error : Couldn't find parent type for 'K2Node_AsyncAction' named 'UK2Node_BaseAsyncTask' in current module or any other module parsed so far.
1>Error : Failed to generate code for KinectDemoRoomEditor - error code: OtherCompilationError (5)
1> UnrealHeaderTool failed for target 'KinectDemoRoomEditor' (platform: Win64, module info: <root_folder>\KinectDemoRoom\Intermediate\Build\Win64\KinectDemoRoomEditor\Development\UnrealHeaderTool.manifest).
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.MakeFile.Targets(43,5): error MSB3075: The command ""C:\Program Files\Epic Games\4.9\Engine\Build\BatchFiles\Rebuild.bat" KinectDemoRoomEditor Win64 Development "<root_folder>\KinectDemoRoom\KinectDemoRoom.uproject" -rocket" exited with code 5. Please verify that you have sufficient rights to run this command.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========
Iāve just begun using Unreal, and was kind of at lose at where the problem is. Do I have to copy the plugin to under the UE folder and use it as an Engine plugin? Or did I compile the projects incorrectly? How exactly do I use the source code after I cloned it from GitHub?
Thank you for any help given.
First of all welcome to Unreal!
Right click the uproject file -> Change engine version -> select 4.9 -> right click uproject file -> generate visual files -> Open in vs by opening the sln file -> build
[=;370037]
First of all welcome to Unreal!
Right click the uproject file -> Change engine version -> select 4.9 -> right click uproject file -> generate visual files -> Open in vs by opening the sln file -> build
[/]
Thanks for the fast response.
But Iām still getting the same compiling error in VSā¦
Speaking of which, I cloned the develop branch. Perhaps I should use the master branch instead?