sk187 / dat6 Goto Github PK

Course materials for General Assembly's Data Science course in Washington, DC (2/21/14 - 5/2/15). View student work in the student repository.

Instructors: Sinan Ozdemir and Josiah Davis.

Office hours: 5-7pm on Tuesday and 5-7pm on Saturday at General Assembly

Course Project information

• Install the Anaconda distribution of Python 2.7x.
• Install Git and create a GitHub account.
• Once you receive an email invitation from Slack, join our "DAT6 team" and add your photo!

Agenda:

• Introduction to General Assembly
• Course overview: our philosophy and expectations (slides)
• Data science overview (slides)
• Data Analysis in Python (code)
• Tools: check for proper setup of Anaconda, overview of Slack

Homework:

• Pandas Homework

Optional:

• Review your base python (code)

Agenda:

• Github: (slides)
• Getting Data (slides)
• Regular Expressions (code)
• Getting Data (code)

Homework:

• Complete the first Project Milestone (Submit on the Dat6-students repo via a pull request)

Resources:

• Forbes: The Facebook Experiment
• Hacking OkCupid
• Videos on Git and GitHub. Created by one of our very own General Assembly Instructors, Kevin Markham.
• Reference for common Git commands (created by Kevin Markham).
• Solutions to last week's pandas homework assignment (code)

Agenda:

• Advanced pandas (code)
• Iris exploration exercise (exercise, solutions)
• Intro. to Machine Learning (slides, code)

Homework:

• Complete the advanced Pandas homework (Submit on the Dat6-students repo via a pull request)
• Continue to develop your project. If you have a dataset, explore it with pandas. If you don't have a dataset yet, you should prioritize getting the data.(Nothing to turn in for next week).
• Read this excellent article, Understanding the Bias-Variance Tradeoff, and be prepared to discuss it next class. (You can ignore sections 4.2 and 4.3.) Here are some questions to think about while you read:
  • In the Party Registration example, what are the features? What is the response? Is this a regression or classification problem?
  • In the interactive visualization, try using different values for K across different sets of training data. What value of K do you think is "best"? How do you define "best"?
  • In the visualization, what do the lighter colors versus the darker colors mean? How is the darkness calculated?
  • How does the choice of K affect model bias? How about variance?
  • As you experiment with K and generate new training data, how can you "see" high versus low variance? How can you "see" high versus low bias?
  • Why should we care about variance at all? Shouldn't we just minimize bias and ignore variance?
  • Does a high value for K cause over-fitting or under-fitting?

Resources:

• For more on Pandas plotting, read the visualization page from the official Pandas documentation.
• To learn how to customize your plots further, browse through this notebook on matplotlib (long!) and check out the matplotlib documentation.
• To explore different types of visualizations and when to use them, Columbia's Data Mining class has an excellent slide deck.
• For a more in-depth look at machine learning, read section 2.1 (14 pages) of Hastie and Tibshirani's excellent book, An Introduction to Statistical Learning. (It's a free PDF download!)
• To learn about NumPy check out this reference code

Agenda:

• The Bias-Variance Tradeoff
• Model Evaluation Procedures (slides, code)
• Logistic Regression (slides, exercise)
• Model Evalutation Metrics (slides, code)