The goal of this project is to get insights from a movie list in IMDB website.Link.
The scraped data was from a movie list created by andreea_nastasa based on his/her personal preferences.I have explored a large set of data to learn important things about movies because I'm really interested in the whole movie scene.

Initial scraped data contains 4900 records.Used google sheets and python to clean the messy dataset and handling missing values,after cleaning the data set contains columns such as 'movie_name','Year','Age','Duration','Rating','Gross Profit',and 'Votes' with 1965 rows.

Later we utilized the scraped data to understand the following movie data and correlations using Tableau Dashboard:

1. Average Movie Ratings Throughout The Years (Line chart):
   Illustrates the trend in average movie ratings over different years using a line chart.

7. Movie Rating Distribution (Histogram):
   Illustrates the distribution of movie ratings, showcasing the frequency of different rating ranges using a histogram.
   
8. Votes Distribution Top 20 Movies (Packed Bubbles):
   Visualizes the distribution of votes for the top 20 movies, with each bubble representing a movie and its size indicating the number of votes received.
   

You can visit the public dashboard here.

1. Average Movie Ratings: Over the years, the line graph illustrates a fluctuating trend, starting with a significant decline, followed by intermittent rises, and reaching a peak in 2020, primarily within the range of 5 to 8.
   
2. Movie Duration Distribution: Illustrates that the majority of movies in the dataset have durations ranging from 90 to 110 minutes.
   
3. Average Movie Duration Throughout The Years:Starts going up first, then mostly goes down over the years, with two noticeable increases in between. Movie durations are usually more than 80 minutes and often go beyond 100 minutes.
   
4. Relationship Between Ratings and Gross Profit: Indicates that high ratings don't consistently translate to high profits, with only a few movies surpassing a gross profit of 500 million.
   
5. Movie Count by Year: Highlights that the dataset is concentrated in the years 2000 to 2020, suggesting a prevalence of movies from this time frame.
   
6. Movie Target Age Group: Reveals that the majority of movies in the dataset are categorized as suitable for a Teen/Adult audience.
   
7. Movie Rating Distribution: Displays a concentration of ratings between 6 and 7, suggesting a central tendency in the movie rating distribution.
   
8. Votes Distribution Top 20 Movies: Despite the dataset's size, the visualization distinctly portrays the highest votes distribution among the top 20 movies, offering a clear representation.
   

1. Clone the repo

git clone https://github.com/sadihsn97/IMDB_Movie_Insights.git

2. Intialize and activate virtual environment

virtualenv --no-site-packages  venv
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Download Chrome WebDrive from https://chromedriver.chromium.org/downloads
5. Run the scraper

python web_scraping_IMDB/scraper.py --chromedriver_path <path_to_chromedriver>

6. To generate a csv file you can use

    df.to_csv("IMDB_Movies_Insights.csv", index=False)