Course materials for General Assembly's Data Science course in Washington, DC (2/21/14 - 5/2/15). View student work in the student repository.
Instructors: Sinan Ozdemir and Josiah Davis.
Office hours: 5-7pm on Tuesday and 5-7pm on Saturday at General Assembly
Saturday | Topic | Project Milestone |
---|---|---|
2/21: | Introduction / Pandas | |
2/28: | Git(hub) / Getting Data | |
3/7: | Advanced Pandas / Machine Learning | One Page Write-up with Data |
ฯ == ฯ/2 day | Model Evaluation / Logistic Regression | |
3/21: | Linear Regression | 2-3 Minute Presentation |
3/28: | Data Problem / Clustering and Visualization | |
4/2 | Naive Bayes / Natural Language Processing | Deadline for Topic Changes |
4/11 | Decision Trees / Ensembles | First Draft Due (Peer Review) |
4/18 | PCA / Databases / MapReduce | |
4/25 | Recommendation Engines | |
5/2 | Project Presentations | Presentation |
- Install the Anaconda distribution of Python 2.7x.
- Install Git and create a GitHub account.
- Once you receive an email invitation from Slack, join our "DAT6 team" and add your photo!
Agenda:
- Introduction to General Assembly
- Course overview: our philosophy and expectations (slides)
- Data science overview (slides)
- Data Analysis in Python (code)
- Tools: check for proper setup of Anaconda, overview of Slack
Homework:
Optional:
- Review your base python (code)
Agenda:
Homework:
- Complete the first Project Milestone (Submit on the Dat6-students repo via a pull request)
Resources:
- Forbes: The Facebook Experiment
- Hacking OkCupid
- Videos on Git and GitHub. Created by one of our very own General Assembly Instructors, Kevin Markham.
- Reference for common Git commands (created by Kevin Markham).
- Solutions to last week's pandas homework assignment (code)
Agenda:
- Advanced pandas (code)
- Iris exploration exercise (exercise, solutions)
- Intro. to Machine Learning (slides, code)
Homework:
- Complete the advanced Pandas homework (Submit on the Dat6-students repo via a pull request)
- Continue to develop your project. If you have a dataset, explore it with pandas. If you don't have a dataset yet, you should prioritize getting the data.(Nothing to turn in for next week).
- Read this excellent article, Understanding the Bias-Variance Tradeoff, and be prepared to discuss it next class. (You can ignore sections 4.2 and 4.3.) Here are some questions to think about while you read:
- In the Party Registration example, what are the features? What is the response? Is this a regression or classification problem?
- In the interactive visualization, try using different values for K across different sets of training data. What value of K do you think is "best"? How do you define "best"?
- In the visualization, what do the lighter colors versus the darker colors mean? How is the darkness calculated?
- How does the choice of K affect model bias? How about variance?
- As you experiment with K and generate new training data, how can you "see" high versus low variance? How can you "see" high versus low bias?
- Why should we care about variance at all? Shouldn't we just minimize bias and ignore variance?
- Does a high value for K cause over-fitting or under-fitting?
Resources:
- For more on Pandas plotting, read the visualization page from the official Pandas documentation.
- To learn how to customize your plots further, browse through this notebook on matplotlib (long!) and check out the matplotlib documentation.
- To explore different types of visualizations and when to use them, Columbia's Data Mining class has an excellent slide deck.
- For a more in-depth look at machine learning, read section 2.1 (14 pages) of Hastie and Tibshirani's excellent book, An Introduction to Statistical Learning. (It's a free PDF download!)
- To learn about NumPy check out this reference code
Agenda: