could you help me how I could retrieve the eye's ratios correctly? about openseeface HOT 5 CLOSED

Hi! You are correct in that there are issues with the eye open/closed features. I am currently doing some ugly threshold things in C# to actually work with those values for avatar animation. You can see the code here:

I'm not exactly sure where the issues are and how to fix them though.

from openseeface.

Hi.I figured why the ratios were so variant and it's because the landmarks x,y coordinates are based on the whole frame not the detected face frame so when someone is closer to the camera the ratio gets larger and vice versa so I mapped the landmarks to the detected face frame and it got so much better but still it's variant depending on the different individuals understandably.
it's my code:

def lms_on_face (f,frame):
    height, width, channels = frame.shape
    lms = f.lms
    face_bbox = f.bbox
    face_x = face_bbox[0]
    face_y = face_bbox[1]
    face_w = face_bbox[2] + face_x
    face_h = face_bbox[3] + face_y
    face = frame[int(face_y):int(face_h),int(face_x):int(face_w)]
    face_h,face_w,_ = frame[int(face_y):int(face_h),int(face_x):int(face_w)].shape
    face_x = 0
    face_y = 0
    frame_bbox = (0,0,width,height)
    fx = 0
    fy = 0
    fw = width
    fh = height
    points = [36,37,38,39,40,41]

    for i , (x,y,c) in enumerate(lms):
        if i in points:
            x = translate(x,0,fw,0,face_w)
            y = translate(y,0,fh,0,face_h)
            lms[i] = (x,y,c)
    return lms

then I use the ratio formula and get an average based on both eyes.hope it helps someone.

from openseeface.

Interesting, that makes sense! Thank you for posting your solution to this issue!

Thinking about it some more, if this was the cause, you could also try using f.pts_3d for your calculation as it should be somewhat pose invariant.

from openseeface.

from openseeface.

The order of landmarks is the same. There are additional points for the guesstimated eyeball centers in pts_3d.

from openseeface.