Comments (5)
Esper
esper is the software to handle complex event processing, and EPL is the language (similar to SQL), to achieve this.
- getting started with esper, java code, EPL statements, handling events to cep engine - https://coffeeonesugar.wordpress.com/2009/07/21/getting-started-with-esper-in-5-minutes/#:~:text=Esper%20is%20available%20in%20Java,follows%20is%20written%20in%20Java.&text=0%2Fesper%2Flib%20folder.,pipe%20to%20our%20CEP%20engine.
- basics of esper, java code, usage on event handling - https://medium.com/@bruno.felix/complex-event-processing-with-esper-core-concepts-f97394b39c07
from tech-notes.
Redis and Celery with Django
Using Docker-Compose - https://testdriven.io/blog/django-and-celery/
Naked setup in Ubuntu - https://stackabuse.com/asynchronous-tasks-in-django-with-redis-and-celery
Running inside a simple docker container.
Protected Mode is required when connecting from a different server to a different server. Since the container is a different server from localhost, that's why required.
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y redis-server
EXPOSE 6379
ENTRYPOINT ["/usr/bin/redis-server", "--protected-mode no"]
Running Redis
- sudo docker run -p 6379:6379 --log-opt max-size=10m redis _# Run container
- sudo docker exec -it CID /bin/bash # access container bash
Deploying Celery | Dockerfile
Dockerfile - https://stackoverflow.com/questions/59651428/runtimewarningyoure-running-the-worker-with-superuser-privilegesthis-is-absol
# --uid and --gid options are required
# else this warning will show up RuntimeWarning:You're running the worker with superuser privileges:this is absolutely not recommended
- celery --app=app worker --loglevel=info --uid=nobody --gid=nogroup &
Testing
- redis-cli -h 127.0.0.1 -p 6379
- ping # will output PONG
Steps
- Run Redis on a terminal as a docker container
- Run Celery Worker on a different terminal
- Call API from Postman.
from tech-notes.
ELK elastic - logstash - kibana
- When to use elasticsearch - https://qr.ae/pG1xMH
from tech-notes.
ElasticSearch
ES Setup
> echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
> sudo apt update
> sudo apt install elasticsearch
Optional - Port Configuration
> sudo nano /etc/elasticsearch/elasticsearch.yml
Run ES
> sudo systemctl start elasticsearch
Create Indexes
> curl -X PUT "localhost:9200/INDEX_NAME?pretty"
Verify Elastic Search
> List of all indexes
curl -X GET "localhost:9200/_cat/indices?pretty"
> Cluster Health
curl -X GET "localhost:9200/_cluster/health?pretty"
from tech-notes.
Data at Scale
- Nasdaq Case Study - Nasdaq Uses AWS to Pioneer Stock Exchange Data Storage in the Cloud - https://aws.amazon.com/solutions/case-studies/nasdaq-case-study/?pg=ln&sec=c
from tech-notes.
Related Issues (20)
- Django HOT 12
- Web Scraping
- AWS, GCP, Azure | Cloud Security HOT 4
- Ubuntu | WSL HOT 4
- Blog Links - Code | IT | Future HOT 3
- Data Structures | DSA | Interviews HOT 1
- Website | UI/UX | Web Design HOT 5
- Blog Links - Code | IT | Future HOT 5
- IaaS, PaaS, SaaS
- Search and Analytics
- Java | Spring HOT 5
- FileSystem Storages
- Mobile-Dev | Android | iOS
- fastAPI HOT 1
- Microservices Architecture HOT 2
- Scaling a Saas Startup
- Networks HOT 1
- Databases
- Blockchain HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tech-notes.