Giter VIP home page Giter VIP logo

databricks_spark_certification's Introduction

Notebook to train for the Databricks Certified Associate Developer for Apache Spark 3.0

Docker configuration for the notebook

Install docker and docker compose for ubuntu

Install curl

sudo apt update && sudo apt upgrade

sudo apt install curl

Install docker

sudo apt-get remove docker docker-engine docker.io containerd runc

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable" 

Install the docker engine

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

Install docker compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

Project Architecture

Databricks
|-- readme.md
|-- .gitignore
|-- docker-compose.yml
|-- docker_entrypoints
|-- |-- jupyter_entrypoint.sh
|-- docker_passwords
|-- |-- jupyter_password.txt
|-- dockerfiles
|-- |-- jupyter_spark_dockerfile

|-- docker_volumes
|-- |-- databricks_certification
|-- |-- |-- requirements.txt
|-- |-- |-- Revision.ipynb
|-- |-- |-- venv/
|-- |-- |-- data/

Launch the notebook

Command to launch container

Change the file Databricks/docker_passwords/jupyter_password.txt.example to jupyter_password.txt, changing the password you want for jupyter token.

Run command with docker compose

docker-compose up -d

Stop and remove the container with

docker-compose down

Access the notebook

Access the notebook through the browser address localhost:8880

And make identification with the password you wrote in Databricks/docker_passwords/jupyter_password.txt

databricks_spark_certification's People

Contributors

davidjulienmillet avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.