Giter VIP home page Giter VIP logo

liveness-detection's Introduction

โš ๏ธ We recommend customers to explore Amazon Rekognition Face Liveness, a new feature to deter fraud in facial verification, that is now generally available. Face Liveness detects spoofs presented to the camera (e.g. printed photos, digital photos or videos, or 3D masks) and spoofs that bypass the camera (e.g. pre-recorded real or deepfake videos). Face Liveness returns a high-quality selfie frame for downstream Amazon Rekognition Face Matching or Age Estimation analysis. Customers can easily add Face Liveness to their React web, native iOS, and native Android applications using open-source AWS Amplify SDKs. Face Liveness automatically scales up or down based on demand and customers pay only for the face liveness checks performed. No infrastructure management, hardware-specific implementation, or machine learning (ML) expertise is required. Face Liveness uses ML models trained on diverse datasets to support high accuracy across user skin tones, ancestries, and devices. Learn more at aws.amazon.com/rekognition/face-liveness . Code samples and developer documentation here: https://docs.aws.amazon.com/rekognition/latest/dg/face-liveness.html

Check the Liveness Detection Framework, a solution that helps you implement liveness detection mechanisms into your applications by means of an extensible architecture. The solution is available in the AWS Solutions Implementations website and its code is available in a GitHub repo.

Liveness Detection sample

Note

  • The solution provided here is not secure against all spoofing. Therefore, before deploying to production, thoroughly test it to validate if it meets your security requirements.
  • The included client (JavaScript frontend web application) is not optimized for small screens and also lacks refinements. You should use it as a starting point or as a sample of how to implement different clients (the backend is client agnostic).
  • The ability to properly monitor a system in production is crucial and the implemented code doesn't provide enough means to do so. Consider using both Amazon CloudWatch Logs and Amazon CloudWatch Metrics to have more visibility of all architecture components before deploying to production.

Overview

This package includes the backend and a sample client (JavaScript frontend) of a liveness detection application.

The application asks the user to perform a challenge: the user must move the nose to a random area shown on the screen. In the end, the following is verified: 1. there was one and only one face, 2. the user moved the nose to the target area, and 3. the user rotated the face.

Following there are some screenshots of a user performing the challenge:

screenshots

Following is the architecture of the application:

architecture

Setup

Prerequisites

  1. Configure the AWS Credentials in your environment. Refer to Configuration and credential file settings .

  2. Download and install AWS CLI. Refer to Installing the AWS CLI.

  3. Download and install AWS SAM CLI. Refer to Installing the AWS SAM CLI .

  4. Download and install Docker. Refer to Docker.

  5. Download and install Node.js. Refer to Node.js Downloads.

Backend

Run the command below to deploy the backend:

sam build --use-container && sam deploy --guided

The command above creates a CloudFormation stack with the following outputs: ApiUrl, StaticWebsiteUrl, and StaticWebsiteBucket. You need those values in the next steps.

Frontend

  1. Enter in the client/ directory.

  2. Download the ML models from face-api.js library:

curl -o public/weights/tiny_face_detector_model-shard1.shard -kL https://github.com/justadudewhohacks/face-api.js/blob/a86f011d72124e5fb93e59d5c4ab98f699dd5c9c/weights/tiny_face_detector_model-shard1?raw=true
echo 'f3020debaf078347b5caaff4bf6dce2f379d20bc *public/weights/tiny_face_detector_model-shard1.shard' | shasum -c

curl -o public/weights/tiny_face_detector_model-weights_manifest.json -kL https://github.com/justadudewhohacks/face-api.js/blob/a86f011d72124e5fb93e59d5c4ab98f699dd5c9c/weights/tiny_face_detector_model-weights_manifest.json?raw=true
echo '1f9da0ddb847fcd512cb0511f6d6c90985d011e6 *public/weights/tiny_face_detector_model-weights_manifest.json' | shasum -c

curl -o public/weights/face_landmark_68_model-shard1.shard -kL https://github.com/justadudewhohacks/face-api.js/blob/a86f011d72124e5fb93e59d5c4ab98f699dd5c9c/weights/face_landmark_68_model-shard1?raw=true
echo 'e8b453a3ce2a66e6fa070d4e30cd4e91c911964b *public/weights/face_landmark_68_model-shard1.shard' | shasum -c

curl -o public/weights/face_landmark_68_model-weights_manifest.json -kL https://github.com/justadudewhohacks/face-api.js/blob/a86f011d72124e5fb93e59d5c4ab98f699dd5c9c/weights/face_landmark_68_model-weights_manifest.json?raw=true
echo 'a981c7adfc6366e7b51b6c83b3bb84961a9a4b15 *public/weights/face_landmark_68_model-weights_manifest.json' | shasum -c
  1. Change the models' paths in the manifests:
perl -i -pe 's/tiny_face_detector_model-shard1/tiny_face_detector_model-shard1.shard/g' public/weights/tiny_face_detector_model-weights_manifest.json

perl -i -pe 's/face_landmark_68_model-shard1/face_landmark_68_model-shard1.shard/g' public/weights/face_landmark_68_model-weights_manifest.json
  1. Open the .env file and replace the value of VUE_APP_API_URL with the API URL (ApiUrl) outputted during the backend deployment.

  2. Run the comamand below to build the frontend:

npm install && npm run build
  1. Copy the static frontend files with the following command (replace {YOUR_BUCKET} with the bucket name
    • StaticWebsiteBucket - outputted during the backend deployment):
aws s3 cp dist s3://{YOUR_BUCKET}/ --recursive

Open your browser and navigate to the CloudFront URL (StaticWebsiteUrl) outputted during the backend deployment.

Clean up (Optional)

If you don't want to continue using the application, take the following steps to clean up its resources and avoid further charges.

Empty the Amazon S3 buckets

Before you can delete a bucket, you must first delete its contents.

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3.

  2. In the Bucket name list, select the option next to the StaticWebsiteBucket and then choose Empty.

  3. On the Empty bucket page, confirm that you want to empty the bucket by entering the bucket name into the text field, and then choose Empty.

  4. In the Bucket name list, select the option next to the FramesBucket and then choose Empty.

  5. On the Empty bucket page, confirm that you want to empty the bucket by entering the bucket name into the text field, and then choose Empty.

Delete the CloudFormation stack

AWS CloudFormation makes it easy to clean up resources that the stack created. When you delete the stack, all resources that the stack created are deleted.

  1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  2. On the Stacks page in the CloudFormation console, select the stack you deployed during the Backend setup.

  3. In the stack details pane, choose Delete.

  4. Select Delete stack when prompted.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

liveness-detection's People

Contributors

amazon-auto avatar dependabot[bot] avatar hsfugita avatar krokoko avatar mherna avatar rafaelwmartins avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

liveness-detection's Issues

Intermittent NetworkError caused by TypeError in verification Lambda

Hi folks,

I've been seeing an intermittent issue which manifests as NetworkError in the front end, but the underlying cause seems to be an unhandled exception in the verification Lambda (because CORS headers don't get set on the error response through API Gateway, so the issue appears as a CORS/network issue on browser side).

Stack trace below but I haven't had chance to dig in to it yet - I wonder if there's something about the number of frames uploaded for the check that can affect the dimensionality of the variable in question?

[ERROR] TypeError: only size-1 arrays can be converted to Python scalars
Traceback (most recent call last):
  File "/var/task/app.py", line 52, in lambda_handler
    status_code, response = execute_if_token_is_valid(body['token'], challenge_id, verify_challenge, challenge_id)
  File "/var/task/app.py", line 68, in execute_if_token_is_valid
    return func(*args)
  File "/var/task/app.py", line 143, in verify_challenge
    state_manager.process(frame)
  File "/var/task/states/manager.py", line 19, in process
    success = self.current_state.process(frame)
  File "/var/task/states/nose.py", line 66, in process
    verified = self.verify_challenge(rek_landmarks, rek_pose, self.challenge_in_the_right)
  File "/var/task/states/nose.py", line 92, in verify_challenge
    trajectory_error = math.sqrt(residuals/len(self.nose_trajectory))

Language implementations

Hi guys, it would be great if you could provide other language reference implementations such as Java and .net.

Frontend npm install command failing - [ ERROR Failed to compile with 1 error ]

Hi Team,

I was checking this up and while replicating at the lab end i notice the npm error while installing the frontend.

Step 5: Run the comamand below to build the frontend:

npm install && npm run build

ERROR  Failed to compile with 1 error                                                                                                                                                           

error  in [internal]

INTERNAL ERROR(undefined,undefined) Cannot destructure property 'script' of 'result.descriptor' as it is undefined.
stack trace:
TypeError: Cannot destructure property 'script' of 'result.descriptor' as it is undefined.
   at Object.parseComponent (/home/ec2-user/liveness-detection/client/node_modules/@vue/cli-plugin-typescript/vue-compiler-sfc-shim.js:6:13)
   at Function.resolveScriptBlock (/home/ec2-user/liveness-detection/client/node_modules/fork-ts-checker-webpack-plugin/lib/VueProgram.js:196:35)
   at Object.host.getSourceFile (/home/ec2-user/liveness-detection/client/node_modules/fork-ts-checker-webpack-plugin/lib/VueProgram.js:106:45)
   at findSourceFile (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101559:29)
   at processImportedModules (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101900:25)
   at findSourceFile (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101607:17)
   at /home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101436:85
   at getSourceFileFromReferenceWorker (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101403:34)
   at processSourceFile (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101436:13)
   at processRootFile (/home/ec2-user/liveness-detection/client/node_modules/typescript/lib/typescript.js:101259:13)

ERROR  Build failed with errors.

To get rid of the error just a quick workaround was to skip the dependency in the packages.json file.

{
  "name": "liveness-client",
  "version": "1.0.0",
  "private": true,
  "scripts": {
    "serve": "vue-cli-service serve",
    "build": "vue-cli-service build",
    "lint": "vue-cli-service lint"
  },
  "dependencies": {
    "axios": "~0.21.1",
    "core-js": "~3.6.5",
    "face-api.js": "~0.22.2",
    "js-logger": "~1.6.0",
    "uuid": "~8.1.0",
    "vue": "~2.6.11",
    "vue-lottie": "~0.2.1"
  },
  "devDependencies": {
    "@types/uuid": "~8.0.0",
    "@typescript-eslint/eslint-plugin": "~2.33.0",
    "@typescript-eslint/parser": "~2.33.0",
    "@vue/cli-plugin-babel": "~4.4.0",
    "@vue/cli-plugin-eslint": "~4.4.0",
    "@vue/cli-plugin-typescript": "~4.4.0",
    "@vue/cli-service": "~4.4.0",
    "@vue/eslint-config-prettier": "~6.0.0",
    "@vue/eslint-config-typescript": "~5.0.2",
    "eslint": "~6.7.2",
    "eslint-plugin-prettier": "~3.1.3",
    "eslint-plugin-vue": "~6.2.2",
    "prettier": "~1.19.1",
    "typescript": "~3.9.3",
    "vue-template-compiler": "~2.6.11"
  }
}

[Feature Request] Centralized failure reason logging

By nature of the way the sample challenge works (split between challenge generation, on-device analytics, and Rekognition-based verification) there are multiple stages within the flow where failure can occur.

We'd like failure events (regardless of on-device vs in-cloud) to be logged in a centralized way independent of the failure reason - both for initial debugging of the app on new platforms, and for management of the challenge flows in production. For example if making changes to the generated challenges, to understand downstream impact on failure rates both for on-device and Rekognition.

CodeUri' is not a valid S3 Uri

hi I tried deploying your solution using the yaml in cloudformation but I get an error and it rollsback - 'CodeUri' is not a valid S3 Uri

Specification document. Number of errors found: 1. Resource with id [ChallengeFunction] is invalid. 'CodeUri' is not a valid S3 Uri of the form 's3://bucket/key' with optional versionId query parameter.. Rollback requested by user.

I would appreciate if you could point me in the right direction to resolve this please thanks

Issue on devices with camera resolution > viewport

Hi team,

From what I can tell, this solution seems to load the media stream for the web client at raw dimensions regardless of the page size. In situations where the camera has high resolution relative to the screen size (e.g. mobile browsers) this causes the page to overflow and seems to somehow break the maths of the challenge in some cases - struggling to get to green box or recognise nose placed in the yellow box.

Observed on Android devices, emulated iPhone, and laptop Firefox and Chrome (by reducing the browser window to minimum width)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.