basic-analytics-dashboard-redis-bitmaps-nodejs

🚀 This repo shows a basic analytics dashboard app that uses Redis Bitmap written in NodeJS (JavaScript) 🚀

Environment variables:

GitHub

git init
git add .
git remote remove origin
git commit -m "first commit"
git branch -M main
git remote add origin git@github.com:coding-to-music/basic-analytics-dashboard-redis-bitmaps-nodejs.git
git push -u origin main

Deploying Docusaurus to Cloudflare Pages

Docusaurus is a commonly used static site generator. It is built on top of react and can be used to make any kind of site (personal website, product, blog, marketing landing pages, etc).

Cloudflare Pages is a JAMstack platform for frontend developers to collaborate and deploy websites.

Creating the App

Open up your shiny terminals and create a docusaurus app How to create: https://docusaurus.io/docs/installation#scaffold-project-website

Creating Docusaurus App

Now, Once you've created change your directory to the project.

Then, Run the start command.

It will open up your browser to the development website: http://localhost:3000/

Now, Your Docusaurus Website is ready, Let's deploy it over cloudflare pages.

Change your docusaurus.config.js url field to https://.pages.dev or your custom domain.

Unlike Vercel and netlify, Cloudflare pages doesn't support cli deployments yet (They might come sooner or later)

So, You'll need to push all the code over github.

git init
git add -A
git commit -m "initial commit"
git branch -M main
git remote add origin https://github.com/apidev234/Docusaurus-Cf-Pages.git
git push -u origin main

After Running these commands, If you check your github, You'll find the code there.

Head to your cloudflare pages dashboard, Signup for pages if you haven't yet.

Create an app and select your github repository.

Choose Create React App as your framework preset.

Create an environment variable NODE_VERSION with the value 16.13.2

Docusaurus requires a node version more than or equal to node v14, 16.13.2 is a LTS(Long time support) Version of Node.js

Then Click Deploy, Within 5mins your app would be ready :)

https://docusaurus-cf-pages.pages.dev/

Add a Custom Domain To Your App https://developers.cloudflare.com/pages/get-started#adding-a-custom-domain

Redirecting www to apex domain https://developers.cloudflare.com/pages/how-to/www-redirect

Redis Analytics Bitmaps demo

Showcases how to implement analytics system using Redis Bitmaps (and other data types) in NodeJS

Overview video

Here's a short video that explains the project and how it uses Redis:

How it works

How the data is stored:

The event data is stored in various keys and various data types.
- For each of time spans:
  - year: like 2021
  - month: like 2021-03 (means March of 2021)
  - day: like 2021-03-03 (means 3rd March of 2021)
  - weekOfMonth: like 2021-03/4 (means 4th week of March 2021)
  - anytime
- and for each of scopes:
  - source
  - action
  - source + action
  - action + page
  - userId + action
  - global
- and for each of data types (types):
  - count (Integer stored as String)
  - bitmap
  - set

Is generated key like: rab:{type}[:custom:{customName}][:user:{userId}][:source:{source}][:action:{action}][:page:{page}]:timeSpan:{timeSpan}, where values in [] are optional.

For each generated key like rab:count:*, data is stored like: INCR {key}
- E.g INCR rab:count:action:addToCart:timeSpan:2015-12/3
For each generated key like: rab:set:*, data is stored like: SADD {key} {userId}
- E.g SADD rab:set:action:addToCart:timeSpan:2015-12/3 8
For each generated key like rab:bitmap:*, data is stored like: SETBIT {key} {userId} 1.
- E.g SETBIT rab:bitmap:action:addToCart:timeSpan:2015-12/3 8 1
Cohort data:
- We store users who register and then bought some products (action order matters).
- For each buy action in December we check if user performed register action before (register counter must be greater than zero).
- If so, we set user bit to 1 like: SETBIT rab:bitmap:custom:cohort-buy:timeSpan:{timeSpan} {userId} 1
- E.g User Id 2 bought 2 products on 2015-12-17. He won't be stored.
- E.g User Id 10 bought 1 product on 2015-12-17 and registered on 2015-12-16. He will be stored like: SETBIT rab:bitmap:custom:cohort-buy:timeSpan:2015-12 10 1.
- We assume that user cannot buy without register.
Retention data:
- Retention means users who bought on two different dates
- For each buy action we check if user bought more products anytime than bought on particular day (current purchase not included).
- If so, we add user id to set like: SADD rab:set:custom:retention-buy:timeSpan:{timeSpan} {userId}
- E.g User Id 5 bought 3 products on 2015-12-15. His retention won't be stored (products bought on particular day: 2, products bought anytime: 0).
- E.g User Id 3 bought 1 product on 2015-12-15 and before - 1 product on 2015-12-13. His retention will be stored (products bought on particular day: 0, products bought anytime: 1) like: SADD rab:set:custom:retention-buy:timeSpan:2015-12 3.

How the data is accessed:

Total Traffic:
- December: BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12
- X week of December: BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12/{X}
  - E.g BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12/3
Traffic per Page ({page} is one of: homepage, product1, product2, product3):
- December: BITCOUNT rab:bitmap:action:visit:page:{page}:timeSpan:2015-12
  - E.g BITCOUNT rab:bitmap:action:visit:page:homepage:timeSpan:2015-12
- X week of December: BITCOUNT rab:bitmap:action:visit:page:{page}:timeSpan:2015-12/{X}
  - E.g BITCOUNT rab:bitmap:action:visit:page:product1:timeSpan:2015-12/2
Traffic per Source ({source} is one of: google, Facebook, email, direct, referral, none):
- December: BITCOUNT rab:bitmap:source:{source}:timeSpan:2015-12
  - E.g BITCOUNT rab:bitmap:source:referral:timeSpan:2015-12
- X week of December: BITCOUNT rab:bitmap:source:{source}:timeSpan:2015-12/{X}
  - E.g BITCOUNT rab:bitmap:source:google:timeSpan:2015-12/1
Trend traffic ({page} is one of: homepage, product1, product2, product3):
- December: from BITCOUNT rab:bitmap:action:visit:{page}:timeSpan:2015-12-01 to BITCOUNT rab:bitmap:action:visit:{page}:timeSpan:2015-12-31
- 1 Week of December: Similar as above, but from 2015-12-01 to 2015-12-07
- 2 Week of December: Similar as above, but from 2015-12-08 to 2015-12-14
- 3 Week of December: Similar as above, but from 2015-12-15 to 2015-12-21
- 4 Week of December: Similar as above, but from 2015-12-22 to 2015-12-28
- 5 Week of December: Similar as above, but from 2015-12-29 to 2015-12-31
  - E.g BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-29 => BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-30 => BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-31
Total products bought:
- December: GET rab:count:action:buy:timeSpan:2015-12
- X week of December: GET rab:count:action:buy:timeSpan:2015-12/{X}
  - E.g GET rab:count:action:buy:timeSpan:2015-12/1
Total products added to cart:
- December: GET rab:count:action:addToCart:timeSpan:2015-12
- X week of December: GET rab:count:action:addToCart:timeSpan:2015-12/{X}
  - E.g GET rab:count:action:addToCart:timeSpan:2015-12/1
Shares of products bought ({productPage} is on of product1, product2, product3):
- December: GET rab:count:action:buy:page:{productPage}:timeSpan:2015-12
  - E.g GET rab:count:action:buy:page:product3:timeSpan:2015-12
- X week of December: GET rab:count:action:buy:page:{productPage}:timeSpan:2015-12/{X}
  - E.g GET rab:count:action:buy:page:product1:timeSpan:2015-12/2
Customer and Cohort Analysis:
- People who registered: BITCOUNT rab:bitmap:action:register:timeSpan:2015-12
- People who register then bought (order matters): BITCOUNT rab:bitmap:custom:cohort-buy:timeSpan:2015-12
- Dropoff: (People who register then bought / People who register) * 100 [%]
Customers who bought only specified product ({productPage} is one of: product1, product2, product3): SMEMBERS rab:set:action:buy:page:{productPage}:timeSpan:2015-12
- E.g SMEMBERS rab:set:action:buy:page:product2:timeSpan:2015-12
Customers who bought Product1 and Product2: SINTER rab:set:action:buy:page:product1:timeSpan:anytime rab:set:action:buy:page:product2:timeSpan:anytime
Customer Retention (customers who bought on the different dates): SMEMBERS rab:set:custom:retention-buy:timeSpan:anytime

Hot to run it locally?

Prerequisites

Node - v12.19.0
NPM - v6.14.8
Docker - v19.03.13 (optional)

Local installation

Go to /server folder (cd ./server) and then:

# copy file and set proper data inside
cp .env.example .env

# install dependencies
npm install

# run docker compose or install redis manually
docker network create global
docker-compose up -d --build

# Run dev server
npm run dev

Go to /client folder (cd ./client) and then:

# copy file and set proper data inside
cp .env.example .env

# install dependencies
npm install

# run development mode
npm run serve

Deployment

To make deploys work, you need to create free account in https://redislabs.com/try-free/

coding-to-music / basic-analytics-dashboard-redis-bitmaps-nodejs Goto Github PK