Download

Face app alternative for android

Hello
i created an android app to track facial landmarks,
is similar to Epic’s face app for ios .

It uses google arcore for the tracking and sets the phone itself as a tcp server so remote clients can
receive the tracked data.

The app is free and available at the play store : https://play.google.com/store/apps/details?id=mx.motion.ar.facemocap

Also im releasing the ue4 project wich connects to the app: http://motion.mx/ue4/AndroidFaceMocap/FaceMocapUe4Project.zip

You can see the app and the example project here:

https://youtube.com/watch?v=RTn3qus_V5I

Note:
the facial tracking of arcore is not as good and somehow limited compared to the ios app,
for example it wont detect blinking,
i hope this app serves as a starting point for anyone who wants to experiment with facial movement,
maybe if the app gets enough users and with ads i can get to implement a better tracking with a comercial tracking sdk.

Enjoy and have fun,
please share if you find the app useful.

3 Likes

Thank you!

Interesting, I tried the android app. Would love to see more development on this.

Here is a new demo using the app, also you can download the source project (link in the video description) https://youtube.com/watch?v=CnATXgWY1iE

App doesn’t support redmi 5 phones or android 8

Hi,
thank you for the feedback,
i created the app using an android 8 so im sure it works with it,

for the other phones those have to support arcore,
so not every phone will run the app,
sometimes it will help installing this https://play.google.com/store/apps/details?id=com.google.ar.core

Wow ! Can you explain how you can use it with unreal ? a video tutorial ? thanks ! First alternative to Iphone solution ! especially great if you don’t like Iphones or don’t have one like me.

For those who are searching “mocap” in your applications search for “AugmentedFace” :wink:

1 Like

Hi,
to use it :

  • install the app in your phone, make sure you are connected to your wifi and run the app (it should show you your ip address)
  • download the example project, open map0 and run the game, it should ask you for the IP then just press the button

How it works:

  • the android app creates a tcp server and sends all the tracked data every frame
  • the ue4 game acts as a client wich connects to the android tcp server

The ue4 game uses a tcp plugin to be able to make the connection, there is one blueprint wich uses the tcp component to
parse all the data coming from the server.

Thats it.

Nice! This is really great.
@paranoio Will you make it open-source and put it up on GitHub or somewhere? I would definitely love to work on the blinking-detection feature! Cheers.

I had not thought about sharing the source , i guess its posible if more users ask for it.
The blink detection is not provided at the moment by arcore ( wich is responsible for the tracking )
and the overall face tracking needs to be improved so
im kind of waiting for google to update the arcore framework , they already added a depth image
so it should be posible for them to improve the tracking at least for lidar phones .

I cannot use BluNet plugin because the binaries is only available for Mac and Windows and I use Linux (Ubuntu)
any solutions?
Thks guys!

Hi , you may use any tcp client you prefer, there are some free tcp clients available on the marketplace , you will have to modify the example project to use the plugin you choosed.

You have a tutorial for use with unreal !

Hi, first love the app its really useful, also i wanted to know is it possible to use the app over a wired connection instead of the provided wireless connection, cause there seems to be a huge delay between what i do on the phone and the actions that happen in the demo project.

We reached 1000 users !!!
Thank you everyone, please send your comments and share your work, i may not answer all the time but lets help each other.

@nywixtv i want to create a tutorial but its been some busy months, cant promise when but i will try to do it
@TheAlienJD i know in some wireless networks it could be slow, as i recall the code in the app will look for the first network connection on the device, i guess you could try connecting a direct ethernet cable to your device and disable the wifi , but again, i dont know if that would work

Hey thanks for the quick reply. Sadly my pc is connected with an ethernet cable, and my phone is connected through wifi and the router is fairly close by but still it lags. I’m not a genius or anything at coding but im fairly sure it should be possible to change the connection type to usb. But if that is not an option currently, is there a method where i can use my pc webcam instead of the phones cam cause that would awesome.

hi just tried the app and it works perfectly , i am definitely going to try this for some cartoon characters i have got , and any idea how i could just export the point data to any 3d software, because i am familiar with the marker facial mocap method in 3d with a pre recorded video , but this live thingy is amazing , i am doing cinematics so this app would be definitely useful if there was any way to export the point animation data as fbx or alembic . This is coool, i am using a ethernet for my pc and wifi for my moto x4 they seem to work pretty good .

Hello Paranoio,
Thank you for this nice code, I’m testing with my Oneplus 5t and it’s working fine.
With the meta human project soon to be released do you know of a way to link the data coming from an android device to those characters?

2 Likes

@Mostah_cg hey , i was thinking about this a few months ago, looking into the code of ue4 seems posible
but there is a catch, i require either an iphone X (or above) or someone to send me a network capture of the udp data transmitted by the iphone to the ue4 computer

i dont want to get an iphone just to for this
so to try it i need that udp network traffic , if anyone reading this post want to help just run wireshark or some udp capture utility
while using the iphone with live link and send me the captured data.

@paranoio Thanks for the app! It’s great! I would like to use it to control a 3D character, but new to UE and no idea how to go forward. Do you have any tips or tutorial you could point to me so I could try to get something done?