Giter VIP home page Giter VIP logo

vu-mi's Introduction

vu-mi

vu-mi is a public API to enable view count badges in Markdown. The API returns up-to-date counts, up to {prevComputedCount} + 7999. Any more than that, and it will show a stale count.

It is tremendously over-engineered, using the following technologies:

  • AWS Lambda
  • DynamoDB
  • DynamoDB Streams
  • SQS

I did it to implement concepts I've learned about, but never worked with such as:

  • Modelling one-to-many relationships in a DynamoDB single table design (from The DynamoDB Book, by Alex Debrie)
  • Change Data Capture

Usage

To use this in a Markdown document, simply paste the following, and insert your unique ID:

![visitors](https://img.shields.io/endpoint?url=https://vu-mi.com/api/v1/views?id=<INSERT_UNIQUE_ID_HERE>)

visitors

The ID could be your Github username, Github repo, anything you want really.

API

GET /v1/api/views?id=YOUR-ID-HERE

Response:

200 OK
{
    "schemaVersion": 1,
    "label": "views",
    "message": "12",
    "color": "blue"
}

Design

End-to-End Flow

system-diagram

<get-views-lambda>

  1. GET /v1/api/views?id=jcserv request comes into get-views-lambda

  2. get-views-lambda queries Dynamo for all items with matching PK, limit of 8000

  3. a. Send SQS message with user id & count

    b. Return response with count - 1 (minus one to account for the USER item)

<put-views-lambda>

  1. If the provided count is 0, PUT the USER item

Q: Is there a risk of race conditions here when multiple requests come in for a new user?

A: There is, if the message to the put-views-lambda comes after a batch-update-count-lambda invocation, it can override the count to 0. To circumvent this, we only PUT the user item IFF the user object does not already exist ("attribute_not_exists(PK) AND attribute_not_exists(SK)")

  1. Put VIEW item, with TTL rounded to the next 5 min interval

  2. Items expire; DynamoDB streams sends records with each item

This is essentially the same as having a 5-min cron job to collate View items, but without having to SCAN the entire table.

Streams is also nice here since, rather than one invocation per item expiring, we can do a bulk update. Savings++

<batch-update-count-lambda>

  1. Get the user object using the PK provided from the DynamoDB Streams record

  2. Put the new count onto the user item

Dynamo Model

dynamo-model

Discussion

Q: Was this really necessary?

Not really, it could've been achieved with a simpler design, like so:

system-diagram-v1

But that wouldn't be fun!

Author's Note

This project is named after my cat Yumi, since I implemented the majority of this during an early morning after she woke me up TT

drawing

vu-mi's People

Contributors

jcserv avatar

vu-mi's Issues

perf: tune limit on dynamodb.Query in get-views-lambda

The DynamoDB Query API "... will read up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data".

The current limit of 250 was chosen arbitrarily, but since the item size is pretty small (Avg: 24.5 bytes), this could be increased.

Acceptance Criteria:

  • Do perf testing, determine upper limit of Query
  • Tweak Query to use discovered limit

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.