quantumudit / analyzing-pokemons Goto Github PK

Overview • Prerequisites • Architecture • Demo • Support • License

This project is centered on scraping Pokémon data from the Pokémon Database, followed by conducting exploratory data analysis (EDA) to uncover insights, which are then visualized using Power BI.

Below is a snapshot of the target website:

The workflow involves using Python for web scraping to extract relevant information from the target site, performing necessary data transformations, and then visualizing and reporting the findings in Power BI.

The Power BI report provides a valuable tool for users to analyze statistics of various Pokémon, including their favorites, and compare them with other Pokémon.

Here is a snapshot of the Power BI report:

The project repository exhibits the following structure:

Analyzing-Pokemons/
├── 📁.github
├── 📁conf
├── 📁data/
│   ├── 📁external
│   ├── 📁processed
├── 📁notebooks
├── 📁src/
│   ├── 📁components
│   ├── 📁pipelines
│   ├── 📁utils
│   ├── 🐍constants.py
│   ├── 🐍exception.py
│   └── 🐍logger.py
├── 📁logs
├── 📁reports
├── 📁resources
├── 🐍main.py
├── 🐍template.py
├── 🔒poetry.lock
├── 📇pyproject.toml
├── 🗒️requirements.txt
├── 📜.gitignore
├── 🔑LICENSE
└── 📝README.md

To effectively engage with this project, possessing a robust understanding of the skills listed below is advisable:

• Core comprehension of Python, Web Scraping, and Modular programming
• Acquaintance with data modelling, DAX and Power BI
• Acquaintance with the Python libraries specified in the 🗒️requirements.txt document

These competencies will facilitate a seamless and productive journey throughout the project.

Application selection and setup may vary based on individual preferences and system setups.

The development tools I've employed for this project are:

• Anaconda / Poetry: Utilized for distribution and managing packages
• VS Code: Employed for writing and editing code
• Jupyter Notebook: Used for data analysis and experimentation
• Power BI Desktop: Used for data modeling and visualization
• Notepad++: Served as an auxiliary code editor
• Obsidian: Utilized for documenting project notes
• Figma: Used for crafting application UI/UX designs
• Click Up: Employed for overseeing project tasks

Integrating process automation is entirely elective, as is the choice of the automation tool.

In this project, GitHub Actions has been selected to automate the web scraping and data transformation process as needed.

Should there be a need to adjust data-related settings, simply update the YAML configurations, and the entire development workflow can be executed directly from the repository.

Note: The website may undergo changes in the future, necessitating adjustments to the web scraping script. As a result, the scripts are not completely future-proof and may need to be updated if the website alters its content or presentation.

The architectural design of this project is transparent and can be readily comprehended with the assistance of the accompanying diagram illustrated below:

The project's architectural framework encompasses the following key steps:

This step involves extracting relevant data from a specified website using Python's web scraping modules. These modules helps navigate the website's structure to collect required information efficiently, ensuring that the data is accurately captured and ready for subsequent analysis.

Once the data is scraped, it undergoes a series of transformations to clean and prepare it for analysis. This process involves handling missing values, correcting data types, filtering out irrelevant data, and restructuring the dataset to align with analytical goals. By doing so, the data becomes suitable for accurate analysis and visualization.

The web scraping and data transformation steps are automated using GitHub Actions. This automation allows the process to be executed seamlessly and consistently without manual intervention. The setup ensures that data extraction and preparation can be performed on-demand, enhancing efficiency and scalability.

During this phase, the transformed dataset undergoes analysis to derive meaningful insights and address specific user queries.

A range of analytical techniques is applied to interpret the data, and the findings are showcased through interactive visualizations in Power BI.

The Power BI dashboard is equipped with slicers, cross-filtering, and other advanced features, allowing users to interact with the data, explore specific aspects, and uncover additional insights.

This dashboard offers a user-friendly and engaging interface for exploring data insights, enabling informed decision-making based on the analysis.

The following illustration demonstrates the interactive Power BI report to explore insights from the data:

Access the Power BI report by clicking here: Power BI Report

Should you wish to inquire, offer feedback, or propose ideas, don’t hesitate to contact me via the channels listed below:

Discover and engage with my content on these platforms:

To express your support for my work, consider buying me a coffee or, donate through Paypal

This license allows reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator. If you remix, adapt, or build upon the material, you must license the modified material under identical terms.