The goal of this project is to get insights from a movie list in IMDB website.Link.
The scraped data was from a movie list created by andreea_nastasa based on his/her personal preferences.I have explored a large set of data to learn important things about movies because I'm really interested in the whole movie scene.
Initial scraped data contains 4900 records.Used google sheets and python to clean the messy dataset and handling missing values,after cleaning the data set contains columns such as 'movie_name','Year','Age','Duration','Rating','Gross Profit',and 'Votes' with 1965 rows.
Later we utilized the scraped data to understand the following movie data and correlations using Tableau Dashboard:
- Average Movie Ratings Throughout The Years (Line chart):
Illustrates the trend in average movie ratings over different years using a line chart.
Displays the frequency distribution of movie durations, showing the range and frequency of different durations. 3. Average Movie Duration Throughout The Years (Line chart):
Shows how the average duration of movies has changed over the years using a line chart. 4. Relationship Between Ratings and Gross Profit (Scatter Plot):
Visualizes the correlation between movie ratings and gross profit, indicating any discernible patterns or trends using a scatter plot. 5. Movie Count by Year.(Bar Chart):
Represents the number of movies released each year, offering a bar chart overview of movie count trends over time. 6. Movie Target Age Group (Child < 13 < Teen/Adult) (Text table):
Summarizes movies into target age groups and displays the count of movies for each group using a text table.
- Movie Rating Distribution (Histogram):
Illustrates the distribution of movie ratings, showcasing the frequency of different rating ranges using a histogram.
- Votes Distribution Top 20 Movies (Packed Bubbles):
Visualizes the distribution of votes for the top 20 movies, with each bubble representing a movie and its size indicating the number of votes received.
You can visit the public dashboard here.
Findings and Observations from the Dashboard
- Average Movie Ratings: Over the years, the line graph illustrates a fluctuating trend, starting with a significant decline, followed by intermittent rises, and reaching a peak in 2020, primarily within the range of 5 to 8.
- Movie Duration Distribution: Illustrates that the majority of movies in the dataset have durations ranging from 90 to 110 minutes.
- Average Movie Duration Throughout The Years:Starts going up first, then mostly goes down over the years, with two noticeable increases in between. Movie durations are usually more than 80 minutes and often go beyond 100 minutes.
- Relationship Between Ratings and Gross Profit: Indicates that high ratings don't consistently translate to high profits, with only a few movies surpassing a gross profit of 500 million.
- Movie Count by Year: Highlights that the dataset is concentrated in the years 2000 to 2020, suggesting a prevalence of movies from this time frame.
- Movie Target Age Group: Reveals that the majority of movies in the dataset are categorized as suitable for a Teen/Adult audience.
- Movie Rating Distribution: Displays a concentration of ratings between 6 and 7, suggesting a central tendency in the movie rating distribution.
- Votes Distribution Top 20 Movies: Despite the dataset's size, the visualization distinctly portrays the highest votes distribution among the top 20 movies, offering a clear representation.
- Clone the repo
git clone https://github.com/sadihsn97/IMDB_Movie_Insights.git
- Intialize and activate virtual environment
virtualenv --no-site-packages venv
source venv/bin/activate
- Install dependencies
pip install -r requirements.txt
- Download Chrome WebDrive from https://chromedriver.chromium.org/downloads
- Run the scraper
python web_scraping_IMDB/scraper.py --chromedriver_path <path_to_chromedriver>
- To generate a csv file you can use
df.to_csv("IMDB_Movies_Insights.csv", index=False)