Giter VIP home page Giter VIP logo

Comments (5)

raunak-r avatar raunak-r commented on September 7, 2024

Esper

esper is the software to handle complex event processing, and EPL is the language (similar to SQL), to achieve this.

from tech-notes.

raunak-r avatar raunak-r commented on September 7, 2024

Redis and Celery with Django

Using Docker-Compose - https://testdriven.io/blog/django-and-celery/
Naked setup in Ubuntu - https://stackabuse.com/asynchronous-tasks-in-django-with-redis-and-celery

Running inside a simple docker container.

Protected Mode is required when connecting from a different server to a different server. Since the container is a different server from localhost, that's why required.

FROM        ubuntu:20.04
RUN         apt-get update && apt-get install -y redis-server
EXPOSE      6379
ENTRYPOINT  ["/usr/bin/redis-server", "--protected-mode no"]

Running Redis

  • sudo docker run -p 6379:6379 --log-opt max-size=10m redis _# Run container
  • sudo docker exec -it CID /bin/bash # access container bash

Deploying Celery | Dockerfile

Dockerfile - https://stackoverflow.com/questions/59651428/runtimewarningyoure-running-the-worker-with-superuser-privilegesthis-is-absol

# --uid and --gid options are required
# else this warning will show up RuntimeWarning:You're running the worker with superuser privileges:this is absolutely not recommended
- celery --app=app worker --loglevel=info --uid=nobody --gid=nogroup &

Testing

  • redis-cli -h 127.0.0.1 -p 6379
  • ping # will output PONG

Steps

  1. Run Redis on a terminal as a docker container
  2. Run Celery Worker on a different terminal
  3. Call API from Postman.

from tech-notes.

raunak-r avatar raunak-r commented on September 7, 2024

ELK elastic - logstash - kibana

from tech-notes.

raunak-r avatar raunak-r commented on September 7, 2024

ElasticSearch

ES Setup

> echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
> sudo apt update
> sudo apt install elasticsearch

Optional - Port Configuration

> sudo nano /etc/elasticsearch/elasticsearch.yml

Run ES

> sudo systemctl start elasticsearch

Create Indexes

> curl -X PUT "localhost:9200/INDEX_NAME?pretty"

Verify Elastic Search

> List of all indexes
    curl -X GET "localhost:9200/_cat/indices?pretty"
> Cluster Health
    curl -X GET "localhost:9200/_cluster/health?pretty"

from tech-notes.

raunak-r avatar raunak-r commented on September 7, 2024

Data at Scale

from tech-notes.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.