adrianriesco / masters-thesis-on-big-data Goto Github PK
View Code? Open in Web Editor NEWMaster's thesis on Big Data
License: Apache License 2.0
Master's thesis on Big Data
License: Apache License 2.0
Configure the work environment where the project will be developed.
Write the python code to gather information from the Twitter API.
Write the python code to gather information from the Spotify API.
Create the front-end of the project with Flask and Bootstrap.
Complete Appendix B "Requirements" of the report.
Write a description of an orchestrator in the project report.
Write the description of the Spotify data in the annex "C. Design".
Write a description of Twitter API in the memory.
Write the project objectives on the memory.
Inspect Twitter API to understand its information and working.
Write a description of NoSQL Databases in the project report.
Create the ETL workflow to load to Cassandra the data extracted from Twitter and Spotify and processed by Spark.
Set up the Spark environment using Docker as the tool to deploy the containers.
Deploy Cassandra environment via Docker.
Add a brief description of the tools selected to the memory of the project.
Write the description of the Twitter data in the annex "C. Design".
Write a description of Flask, Jinja and Bootstrap in the project report.
Write a description of CI/CD in the report.
Write a description of Docker and Docker Compose in the project report.
Integrate Cassandra with Airflow and Spark using Docker Compose.
Configure the DAG in Airflow and ensure it is properly loaded in the Docker container.
Complete Appendix D "Programmer's guide" of the report.
Redesign the project plan for May and June
Include .out, .aux... into gitignore to avoid these files from including in the commits
Complete Appendix E "User manual" of the report.
Learn the basics of Flask in order to build the front-end.
Complete Appendix A "Project plan" of the report.
Configure the project memory template to be aligned with the requirements of the project.
Review the entire project report.
Improve and unify the code written to collect data from both APIs.
Complete Appendix C "Design" of the report.
Write section 5 "Relevant aspects of the project".
Write the API inspection process in the "Programmer guide" section of the memory.
Write section 6 "Related works" of the report.
Write a description of Spotify API in the memory.
Update the main document with a brief explanation of the selected tools and the work methodology used in the development.
Set up the Airflow environment using Docker as the tool to deploy the containers.
Document the objectives and main goals to be achieved with this project.
Write a description of Apache Spark in the project report.
Inspect the API of Spotify to understand the expected input and outputs.
Write section 7 "Conclusions of the project".
Integrate the ETL workflow with the front-end.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.