datacode09 Goto Github PK

hive_table_backup_restore

identify-same-records-in-two-dbs

insert_into_hive_from_csv

learning-spark

linux-and-bash-for-data-engineers

Duke MIDS: Data Engineering and DataOps Course

linux-toolkit

Simple automation & administration tools for Linux

linux_disk_size_queries

long_running_process_detector

merge-in-sql-server-database

mongo_hive_spark_etl

oracle_connection_bash

oracle_permission_checker

oracledb_connection_test

pandas_count_compare

piv-server

powershell_tasks

put_log_file_in_hdfs

pyspark-checker

pyspark-example-project

Example project implementing best practices for PySpark ETL jobs and applications.

python_api_running_a_shell_script

python_query_builder

setl-template

A simple template to start a project with SETL

sfguide-data-engineering-with-snowpark-python

socketsquirrel

Python script made with Paramiko to SSH multiple endpoint to execute commands, upload files and download files. Lazy auditors friend

spark_etl_join_case

split_xlsx_file_in_sheets

sql_creating_minute_level_information

sql_server_partition_view

sqlserver_scripts

You will find the most useful SQL SERVER Scripts, Both for On premisses and Cloud Instances/DBs

the-product-companies-data-warehouse-using-kimball-s-approach

Built a data warehouse for ‘The Product Company’ involving customer, product, supplier, sales, orders and invoice data. Designed a Dimensional Model as per Kimball’s approach with conformed data marts, slowly changing and conformed dimensions. Implemented using SQL, MySQL Workbench, Pentaho