mediapipe-facelandmark-demo's Introduction

Mediapipe-Facelandmarker-Demo

Animate 3D avatar face using MediaPipe's face-landmark model.

This project aims to test and demonstrate the capabilities of MediaPipe's new face landmark model, which outputs 52 blendshapes. These blendshapes can be interactively tested with avatars from Ready Player Me, with additional features available for customizing these avatars.

Prerequisites

This project requires Node.js to be installed on your local machine.

Installation

Clone the repository:

git clone https://github.com/jays0606/mediapipe-facelandmarker-demo.git

Navigate into the project directory:
```
cd mediapipe-facelandmarker-demo
```
Install the necessary dependencies:
```
npm install
```
Run the following command, Open your browser and visit http://localhost:3000 to view the project.
```
npm run dev
```

Built With

This project was created using:

Contact

Jaeho Shin - [email protected]

mediapipe-facelandmark-demo's People

Contributors

Stargazers

Watchers

mediapipe-facelandmark-demo's Issues

Hand landmark

Hi,
I tried to take this demo forward with detecting the hands and move the avatar's hands
I have created HandLandmarkManager that is doing the detect and drawLandmarks
But i have difficulties to implement "updateHandsTransforms" (on the same convention as "updateFacialTransforms")
i'm missing the part of connecting the HandLandmarkerResult to the hands bones.
and clue on how to implement?

   updateHandsTransforms = (results: HandLandmarkerResult, flipped = true) => {
    if (!results || !this.isModelLoaded) return;
    if(results.handednesses.length > 0){
      let leftIndex,rightIndex;
      for (const categories of results.handednesses) {
        let { index, score, categoryName } = categories[0];
        if(categoryName.includes("Left")){
          leftIndex = index;
        } else if(categoryName.includes("Right")){
          rightIndex = index;
        }
      }
    }
    const LeftHand = this.scene.getObjectByName("LeftHand");
    LeftHand?.traverse((obj) => {
        //implement left hand control
        console.log(obj);
      });
    return
  };

Recommend Projects

jays0606 / mediapipe-facelandmark-demo Goto Github PK

mediapipe-facelandmark-demo's Introduction

Mediapipe-Facelandmarker-Demo

Prerequisites

Installation

Built With

Contact

mediapipe-facelandmark-demo's People

Contributors

Stargazers

Watchers

Forkers

mediapipe-facelandmark-demo's Issues

Hand landmark

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent