Help with accessing Android sensors through JNI - Failed to find method

[HolyDuckTurtle]

Essentially, I am trying to get TYPE_ROTATION_VECTOR values from an android tablet device using JNI and pass them through to a blueprint. Before trying to do this my experience with code was non-existent, so the rest of the project has been built in blueprints. The goal of the project is to use the tablet to view HDRI spheres like google street view or other 360 degree image viewing apps.

I have cobbled a plugin together by attempting to understand the source code of an app called Sensor Fusion Demo (particularly its Rotation Vector Provider) and going by this tutorial: http://isaratech.com/ue4-making-a-an...in-10-minutes/ - The idea is to use an APL.xml file to extend GameActivity.Java

The current error causing the app to crash is triggered on launch once the splash screen exits. It reads:

 

Spoiler

LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error: === Critical error: ===
LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error:
LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error: Assertion failed: Method != 0 [File:\Build\++UE4+Release-4.19+Compile\Sync\Engine\Source\Runtime\Launch\Private\Android\AndroidJNI.cpp] [Line: 189]
LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error: Failed to find Method

Followed by a bunch of Callstack errors:

 

Spoiler

LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!StaticFailDebug(wchar_t const*, char const*, int, wchar_t const*, bool) []
LogPlayLevel: 06-21 02:38:56.274 8181 8201 D UE4 : [2018.06.21-01.38.56:274][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FDebug::LogAssertFailedMessageImpl(char const*, char const*, int, wchar_t const*, ...) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FJavaWrapper::FindMethod(_JNIEnv*, _jclass*, char const*, char const*, bool) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!UAndroidSensors::InitJavaFunctions() []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FModuleManager::LoadModuleWithFailureReason(FName, EModuleLoadResult&) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FModuleDescriptor::LoadModulesForPhase(ELoadingPhase::Type, TArray<FModuleDescriptor, FDefaultAllocator> const&, TMap<FName, EModuleLoadResult, FDefaultSetAllocator, TDefaultMapHashableKeyFuncs<FName, EModuleLoa
dResult, false> >&) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so![Unknown]() []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FPluginManager::LoadModulesForEnabledPlugins(ELoadingPhase::Type) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FEngineLoop::LoadStartupModules() []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FEngineLoop::PreInit(wchar_t const*) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!FEngineLoop::PreInit(int, wchar_t**, wchar_t const*) []
LogPlayLevel: 06-21 02:38:56.275 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!AndroidMain(android_app*) []
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:275][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so!android_main() []
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libUE4.so![Unknown]() []
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libc.so![Unknown]() []
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error: [Callstack] 0x00000000D1067A30 libc.so![Unknown]() []
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error:
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error:
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error:
LogPlayLevel: 06-21 02:38:56.276 8181 8201 D UE4 : [2018.06.21-01.38.56:276][ 0]LogAndroid: Error:

This is the code in AndroidJNI.cpp it's referring to, line 189 being CHECK_JNI_RESULT(Method);

 

Spoiler

jmethodID FJavaWrapper::FindMethod(JNIEnv* Env, jclass Class, const ANSICHAR* MethodName, const ANSICHAR* MethodSignature, bool bIsOptional)

{

jmethodID Method = Class == NULL ? NULL : Env->GetMethodID(Class, MethodName, MethodSignature);

CHECK_JNI_RESULT(Method);

return Method;

}

This is my Function Header:

 

Spoiler

#pragma once

#include "IAndroidSensors.h"
#include "Components/SceneComponent.h"
#include "AndroidSensorsFunctions.generated.h"

UCLASS(ClassGroup = AndroidSensors, BlueprintType, Blueprintable, meta = (BlueprintSpawnableComponent))
class UAndroidSensors : public USceneComponent
{
GENERATED_BODY()

public:

#if PLATFORM_ANDROID
static void InitJavaFunctions();
#endif

//Gets the Android Rotation Vector
UFUNCTION(BlueprintPure, meta = (Keywords = "GetRotVector", DisplayName = "Get Android Rotation Vector"), Category = "Android Sensors")
void AndroidSensors_GetRotVector(FVector& Orientation, float& Angle);

};

This is my .cpp file:

 

Spoiler

#include "CoreMinimal.h"
#include "AndroidSensorsFunctions.h"

#if PLATFORM_ANDROID

#include "Android/AndroidJNI.h"
#include "AndroidApplication.h"

#define INIT_JAVA_METHOD(name, signature) \
if (JNIEnv* Env = FAndroidApplication::GetJavaEnv(true)) { \
name = FJavaWrapper::FindMethod(Env, FJavaWrapper::GameActivityClassID, #name, signature, false); \
check(name != NULL); \
} else { \
check(0); \
}

#define DECLARE_JAVA_METHOD(name) \
static jmethodID name = NULL;

DECLARE_JAVA_METHOD(AndroidThunkJava_AndroidSensors_GetRotVector);

void UAndroidSensors::InitJavaFunctions()
{
INIT_JAVA_METHOD(AndroidThunkJava_AndroidSensors_GetRotVector, "([F)V");
}
#undef DECLARE_JAVA_METHOD
#undef INIT_JAVA_METHOD

#endif



void UAndroidSensors::AndroidSensors_GetRotVector(FVector& Orientation, float& Angle)
{
#if PLATFORM_ANDROID
if (JNIEnv* Env = FAndroidApplication::GetJavaEnv(true))
{
jfloatArray Result = Env->NewFloatArray(1);
FJavaWrapper::CallVoidMethod(Env, FJavaWrapper::GameActivityThis, AndroidThunkJava_AndroidSensors_GetRotVector, Result);

jfloat* ResultArr = Env->GetFloatArrayElements(Result, 0);

FVector tempOrientation(ResultArr[0], ResultArr[1], ResultArr[2]);

Orientation = tempOrientation;
Angle = ResultArr[3];

Env->DeleteLocalRef(Result);

}
else
{
UE_LOG(LogAndroid, Warning, TEXT("ERROR: Could not get Java ENV\n"));
}
#endif

And here is the APL.xml file:

 

Spoiler

<?xml version="1.0" encoding="utf-8"?>
<!-- Android Sensor Additions -->
<root xmlns:android="http://schemas.android.com/apk/res/android">
<!-- init section is always evaluated once per architecture -->
<trace enable="true"/>
<init>
<log text="Android Sensors Init" />
</init>

<androidManifestUpdates>
<addElements tag="application">

<activity android:name="com.Uop.TempleManor.AndroidSensorsActivity"
android:configChanges="keyboard|keyboardHidden|screenLayout|screenSize|orientation"
android:label="@string/app_name" />

</addElements>


</androidManifestUpdates>

<!-- optional additions to proguard -->
<proguardAdditions>
<insert>
<![CDATA[
-keepattributes Signature
-dontskipnonpubliclibraryclassmembers

-keepclassmembers class com.epicgames.ue4.GameActivity
{
public <methods>;
public <fields>;
}
]]>
</insert>
</proguardAdditions>

<resourceCopies>
<!-- Copy the generated resource file to be packaged -->
</resourceCopies>

<AARImports>
</AARImports>

<!-- optional additions to the GameActivity imports in GameActivity.java -->
<gameActivityImportAdditions>
<insert>
import java.util.HashSet;
import java.util.Arrays;
import android.text.TextUtils;
import android.graphics.BitmapFactory;
import android.os.Handler;
</insert>
</gameActivityImportAdditions>

<!-- optional additions to the GameActivity class in GameActivity.java -->
<gameActivityClassAdditions>
<insert>
<![CDATA[
private SensorManager mSensorManager;
private Sensor TypeRotVector;
private SensorEventListener mSensorListener;
final private float[] LastRotVector = new float[4];

public void AndroidThunkJava_AndroidSensors_GetRotVector(float[] RotVector)
{
RotVector = LastRotVector;
}
]]>
</insert>
</gameActivityClassAdditions>

<!-- optional additions to GameActivity onCreate metadata reading in GameActivity.java -->
<gameActivityReadMetadataAdditions>
<insert>

</insert>
</gameActivityReadMetadataAdditions>

<!-- optional additions to GameActivity onCreate in GameActivity.java -->
<gameActivityOnCreateAdditions>
<insert>
<![CDATA[
mSensorManager = (SensorManager)getSystemService(SENSOR_SERVICE);
TypeRotVector = mSensorManager.getDefaultSensor(Sensor.TYPE_ROTATION_VECTOR);
mSensorListener = new SensorEventListener()
{
public void onSensorChanged(SensorEvent event)
{
if (event.sensor == TypeRotVector)
{
SensorManager.getQuaternionFromVector(LastRotVector, event.values);
}
}
};
]]>
</insert>
</gameActivityOnCreateAdditions>

<!-- optional additions to GameActivity onDestroy in GameActivity.java -->
<gameActivityOnDestroyAdditions>
<insert>

</insert>
</gameActivityOnDestroyAdditions>


<!-- optional additions to GameActivity onStart in GameActivity.java -->
<gameActivityOnStartAdditions>
<insert>

</insert>
</gameActivityOnStartAdditions>

<!-- optional additions to GameActivity onStop in GameActivity.java -->
<gameActivityOnStopAdditions>
<insert>

</insert>
</gameActivityOnStopAdditions>


<!-- optional additions to GameActivity onPause in GameActivity.java -->
<gameActivityOnPauseAdditions>
<insert>
<![CDATA[
mSensorManager.registerListener(mSensorListener, TypeRotVector, SensorManager.SENSOR_DELAY_UI);
]]>
</insert>
</gameActivityOnPauseAdditions>


<!-- optional additions to GameActivity onResume in GameActivity.java -->
<gameActivityOnResumeAdditions>
<insert>
<![CDATA[
mSensorManager.registerListener(mSensorListener, TypeRotVector, SensorManager.SENSOR_DELAY_UI);
]]>
</insert>
</gameActivityOnResumeAdditions>


<!-- optional additions to GameActivity onActivityResult in GameActivity.java -->
<gameActivityOnActivityResultAdditions>
<insert>
</insert>
</gameActivityOnActivityResultAdditions>


<!-- optional libraries to load in GameActivity.java before libUE4.so -->
<soLoadLibrary>
<!-- need this if plugin enabled and supported architecture, even if not packaged for GearVR -->
</soLoadLibrary>
</root>

Any assistance here will be massively appreciated. I am a novice at coding in general and have been quite dismayed by my lack of finding solid working examples of this being achieved before.

[IsaraTech.]

Hi, do you have any error message during the packaging of the application?

[HolyDuckTurtle]

Update: I discovered the issue was I had not added the APL as a module dependency in the build.cs file. The app basically didn't know it existed.

Hi, do you have any error message during the packaging of the application?

No, they came up only when it ran. Probably because the code was set to run only on Android. But still, I found the cause, thanks for your help.

[Omar Vector]

HolyDuckTurtle Can I ask you for any resourses that I can learn JNI and warping inside unreal?

I really got tired doing this, bec I want to access microphone data directly and bypassing unreal engine API with that. and i'm unable to get any resources to start with .

[Melksham1977]

Hi,

Ive managed to get this working on Unreal 4.23.1 with the light sensor and also pressure sensore but cant get it working with Sensor.TYPE_ROTATION_VECTOR, X Y and Z of the vector are all 0.0. is ther any kind of permission or somthing else that i need to enable?

thanks

Spriggsy

[Owl_Snowy]

For those who come across this post I suggest using this plugin and calling Java code with a single line of code: https://forums.unrealengine.com/comm...ne-4-jni-obj-c