otacilio-psf Goto Github PK

azure-cloud-handler

Python library to interact with some resources on Azure as AKS (Azure kubernetes service) and Data Lake Storage Gen2

azure-functions-ingestion

Example of how to use Azure Functions to Ingest an API data to datalake

azure-spark-on-kubernetes

Spark on Kubernetes with Azure resources: Azure Kubernetes Service (AKS), Azure Data Lake Storage Gen2 and Azure Synapse

daft

Distributed DataFrame for Python designed for the cloud, powered by Rust

daft-lab

This repository showcases a data engineering project using Daft and Apache Iceberg to transform IMDB datasets

data-academy

project repo to work in Azure enviroment

data-eng-open-source-tools

Explore and test Open-sources tools for Data Engineering in standalone mode or integrated with a ecosystem

data-engineering-roadmap

roadmap de engenharia de dados da jornada 2024

deltalakereader

Read Delta tables without any Spark

desafio-stone-dataengineer

Repositório com a solução proposta para o desafio da Stone de Data Engineer

desafio_bootcamp_data_engineer

dev_api_with_flask

Developing an REST API with Flask MicroFramework

docker-bigdata

Big Data Ecosystem Docker

docker-nosql

Differents NoSql Databases with docker: mongoDB and Redis

dockerswarm-minio

Deploy MinIO storage server in Docker Swarm

elastic-stack-labs

igti-cde-mod3-desafio

Challenge solved using Google Cloud Platform Dataproc and Storage services

igti-cde-mod3-trab-pratico

Trabalho prático modulo 3 Cloud Data Engineer IGTI

igti-cde-practice-one-aws

Practice one from Cloud Data Engineer by IGTI

ingestion-on-postgres

Ingestion of NY Taxy data on Postgres Database with Python or Spark

kafka-samples

Kafka Lab done with docker

lineage-keeper

A lightweight lineage tool based on Spark and Delta Lake

llm-telegram-bot

This project is a sample project of how to create a Telegram Bot leveraging LLM and RAG

llm-zoomcamp-otacilio

My space for LLM Zoomcamp class

otacilio-psf

otacilio-psf.github.io

Portifolio

pyspark-codespace

Template in how to use PySpark + DeltaLake + Azure Storage + test cases in Codespace

spark-dev-env-docker

Spark development environment for kubernetes, spark-submit and jupyter notebook

spark-image

Spark image built-in with connectors to Common data sources and Delta lake

sql-challenge-platform