9

I am trying to put earings on the face using arcore augmented face. I am editing the 3D model in blender and adjusting the model according to canonical_face_mesh.fbx .

Screenshot of model in blender

But my 3d model is rendering above the head.

I also tried making the pivot point of my model same as that of canonical_face_mesh.fbx, (as mentioned in this SO question) but still when running in app it's rendering earings above head

Twisha Kotecha
  • 1,063
  • 1
  • 2
  • 15
Mehmood Arib
  • 331
  • 1
  • 10

2 Answers2

9

The problem was

  1. I was not placing custom meshes as children of the asset node as mentioned in doc.

    using the hierarchy.

To make the custom mesh or the object a child of the asset node:

i) Load the given canonical face mesh in blender file->import->fbx then browse for your downloaded canonical face mesh.

(Note: Every SDK has its own reference face mesh and they differ; make sure you use correct one).

ii) Then load your custom 3D model.

iii) Scale and align your model according to the canonical face mesh wherever you want it to be.

iv) In object mode press ctrl + shift +p and then click your custom model and then click the bones present in canonical face mesh.A selection menu will appear select object in that menu.

This will make your model, the child of the asset node.

Like this

  1. Now you need to add Vertex Group in your model. To do so:

enter image description here

i) Select all vertex in your model in edit mode.

ii) Then go to context object data menu. Click the + icon and name your Vertex ( NOSE_TIP, FOREHEAD_RIGHT, or FOREHEAD_LEFT ). This you have to assign according to your model. Mine is Nose Pin so i choose NOSE_TIP. Then Click assign.

Now to cross check you can deselect your model and then click your vertex group and click on select button next to assign button you will notice that your model is selected now.

  1. You need to add modifier. To add modifier

enter image description here

i) Click on the modifier option.

ii) Then click add modifier you will see a drop down menu in that select armature.

iii) Name it as asset as shown above select object as asset and select Nose_TIP as shown above.

Now Delete only the facemesh from the asset.

and import your model.

Rishabh Maurya
  • 1,272
  • 3
  • 19
  • 37
Mehmood Arib
  • 331
  • 1
  • 10
  • 1
    i am confused in steps, can you please elaborate more ? – Saurabh Singh Rathore Dec 27 '19 at 07:08
  • sure, in which step you are confused? – Mehmood Arib Jan 06 '20 at 07:02
  • can you share the final 3d model that worked for you? I am facing the same problem. I tried following your steps with the same model that you used but earrings are appearing near my forehead rather than at earlobes. – user7845429 Apr 06 '20 at 08:39
  • 1
    https://drive.google.com/open?id=1BJW6uxXxFfuKCdNgcWWdIdsVVwbGt90u – Mehmood Arib Apr 09 '20 at 03:46
  • 1
    https://drive.google.com/open?id=1CB0M_p3c28X1WTQbwCW70E-X0o6XSXs7 – Mehmood Arib Apr 09 '20 at 03:47
  • 1
    These are in fbx format you can use them directly in your project – Mehmood Arib Apr 09 '20 at 03:48
  • 1
    And the reason that they are appearing at head is that you are unable to lock them take help from 3D modeler or someone who knows Blender – Mehmood Arib Apr 09 '20 at 03:49
  • ok so I tried using your models and it worked (thanx for that btw) but I wasn't able to replicate that so I tried different approach by using [getMeshVertices()](https://developers.google.com/ar/reference/java/arcore/reference/com/google/ar/core/AugmentedFace#public-floatbuffer-getmeshvertices-) and specify the cordinates manually. But now whenever I rotate my head and when the model goes behind the face it turns black. Do you know how to circumvent that so that if earring is occluded by that face it isn't visible or only a part of it is visible? ![example image](https://ibb.co/zX2MjBt) – user7845429 Apr 14 '20 at 07:04
  • Hi, I'm trying to place a 3D Cap on head, following the steps it is appearing over the forehead but it is not getting fit on the head. I'm adding both FOREHEAD_LEFT & FOREHEAD_RIGHT in the Vertex group, but it doesn't seems to work perfectly. Since I'm a beginner in using Blender, I'd appreciate any help. Thx. – Sonu Singh Bhati Jul 10 '20 at 09:14
2

There are three possible problems in your case:

  • A pivot point of 3D Earrings Model doesn't match Face Anchor position.

    • Face Anchor and pivot point of your model must meet each other.

  • A size/proportions of 3D Model doesn't match size/proportions of canonical face mesh.

  • A pivot point in your 3D model is shifted in a wrong direction of Y-axis.

P.S.

Read this post if you need more details about Augmented Faces feature and Face Anchor.

Remember that an orientation of the mesh is different for Sceneform, Unity and Unreal.

Andy Fedoroff
  • 26,838
  • 8
  • 85
  • 144