justmarkham / pandas-videos Goto Github PK

View Code? Open in Web Editor NEW

2.1K 2.1K 1.9K 1.88 MB

Jupyter notebook and datasets from the pandas video series

Home Page: https://courses.dataschool.io/pandas-in-30-days

Jupyter Notebook 100.00%

data-analysis data-cleaning data-science jupyter-notebook pandas python tutorial

pandas-videos's Introduction

Welcome to Data School!

My name is Kevin Markham, and I've taught Data Science in Python to over a million students.

My tutorials explain Data Science topics in a clear, thorough, and step-by-step manner.

I'd love to teach you, regardless of your educational background or professional experience.

If you're ready to learn Data Science and start your new career, I'd love for you to join me! 🚀

Here's how to get started (for free):

Sign up for my "Tuesday Tips" newsletter - I'll help you get better at Data Science every week!
Enroll in one of my online courses - I can teach you Python or Machine Learning!
Check out my catalog of resources - I've got tutorials on a dozen Data Science topics!

Let's connect:

YouTube (200,000+)
LinkedIn (50,000+)
Twitter (20,000+)
Facebook (6,000+)

pandas-videos's People

Contributors

Stargazers

Watchers

Forkers

rahulsingh50 rlugojr nanoresearch elinok ln1267 anhmike isaac1989 arkoneogy fernanccervone rayed-therap kuanhoong olinero tsudhahar redwa vinodhkrishnaraju pwaila wanggene memebrain danielcjlee aitatanit mitchshack niteeshkanungo akash-ansari syzdemonhunter marcds littletiger311 atrevas mismz feststelltaste mnalevanko phdkiran amit-dingare tammachari pineapplechiu mtchem ratulghosh pepper1709 leezqcst melakbet aiamateur anhlbt mimirui123 mshekha2 ranjankumarpatel sukanyamandal punyaslokamuduli popsonebz pgnepal drbaguiar githubsniper andy-buv sacridini subhraj07 joshuacourse fajk zhenjunz sandianyang jzhang45 matkal93 oscarrondoni atulsingh0 autred magenti fulkay true-developer mayanksingh09 pengyan-sophia willkriski pandeyaah samirpoonawala chenguanghn tonio145 paisely65 pawanvirsingh sgrim5235 mudragada roosbeh-nowrouzian shyamsy brandeddavid jibinjohney ist256 codetradr imanerezzag madhan1m udayoo7 iamamar venky14 wdillon2 zakved ridzuan05 terratenney phpmind daeil-kwon maitreyeepalghosh andyuan1618 zafrinminoti jessewei sluo6 christfan868 mboge

pandas-videos's Issues

The use of DataFrame.Ix

Thanks for your videos! In the 19 video,the use of DataFrame.Ix has been deprecated now.

Hi, I have the same problem as you ,need some help

Hi,I had the same problem as you, then followed the steps in this comment and modified it, but it still doesn't work, may I ask how did you finally modify it to solve the problem?

Converting pandas.Timestamp into date only by removing timestamp

You've made a good video but you missed this one thing to explain. Actually the situation is in below code.

class NumpyEncoder(DjangoJSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.ndarray):
            return obj.tolist()
        elif isinstance(obj, pd.Timestamp):
            return str(obj)

    return super().default(obj)

with pd.ExcelFile(default_storage.open(path)) as xls:
    sheet_names = xls.sheet_names
    sheets_df = [pd.read_excel(xls, sheet_name, header=None)
                 for sheet_name in sheet_names]
    worksheets_columns = [list(sheet_df.columns) for sheet_df in sheets_df]
    worksheets_data = [json.loads(json.dumps(
        sheet_df.fillna('').applymap(str).to_numpy(), cls=NumpyEncoder))
                       for sheet_df in sheets_df]

I need to convert pandas.Timestamp in the situation above to date only. Currently shows timestamp also.

Thanks in advance.

Drop duplicates from multiple columns

How do I find and remove duplicate rows in multiple columns in pandas?

df.col_1.drop_duplicates()

I have 5 columns how can I do that?

Notebook in text format

Hi,

I just created my account in Github, pardon the simple question.

Is it possible to download the notebooks that you have provided in text format?

Here's the link of the one I'm looking at:
http://nbviewer.jupyter.org/github/justmarkham/pandas-videos/blob/master/pandas.ipynb#25.-How-do-I-work-with-dates-and-times-in-pandas%3F-%28video%29

I'm planning to work with the notebooks without having to type the code one by one.
If it's possible to do this please teach me how! Thank you!

How to add a new time column with new calculated time data?

Hi Just Markham,

I want to add +8 hours to all the existing times and save it to a new column in the DataFrame. I only figured out how to create a new column. But I don't know how to add my delta time to the new time data.

Electronically yours,
Mr. Biggles

from datetime import timedelta
import pandas as pd

ufo = pd.read_csv('http://bit.ly/uforeports')

print("")
ufo['Time'] = pd.to_datetime(ufo.Time)
print(ufo)

print("")
print("_______________________________________________________________________")
print("")

print("ufo.dtypes")
print(ufo.dtypes)

print("")
print("_______________________________________________________________________")
print("")

# Add a new column with new data.

delta = timedelta(hours=-8)
# calc_utc_time = data_cell + delta

ufo['utc_time'] = ufo.Time
print(ufo)

print("")
print("_______________________________________________________________________")
print("")

print("ufo.dtypes")
print(ufo.dtypes)

print("")
print("_______________________________________________________________________")
print("")

How to remove the text Timestamp when saving DataFrame as Dictionary?

Hi Sensei Markham, @justmarkham

I need to save my DataFrame as Dictionary in order to push my data into my database using Flask SQLAlchemy. But even when I change the DataType for column Time from Datetime to Object. My dictionary variable still contains the text Timestamp, how can I remove this text and parenthesis characters from my dictionary?

ufo_dict_2

{0: {'City': 'Ithaca', 'Colors Reported': nan, 'Shape Reported': 'TRIANGLE', 'State': 'NY', 'Time': Timestamp('1930-06-01 22:00:00')}}

x13_change_DataType_for_Series.py

import pandas as pd

#_______________________________________________________________________________

ufo = pd.read_csv('http://bit.ly/uforeports')

print(ufo.head())
print("")
print(type(ufo))
print("")
print("ufo.dtypes")
print(ufo.dtypes)

print("")
print("_______________________________________________________________________")
print("")

#_______________________________________________________________________________

# Convert Time datatype from Object to Datetime.

ufo['Time'] = pd.to_datetime(ufo.Time)

print(ufo.head())
print("")
print(type(ufo))
print("")
print("ufo.dtypes")
print(ufo.dtypes)

print("")
print("_______________________________________________________________________")
print("")

#_______________________________________________________________________________

# Convert DataFrame to Dictionary.

ufo_dict_1 = ufo.to_dict('index')

print("ufo_dict_1")
print("")
print(dict(list(ufo_dict_1.items())[0:3]))
print("")
print(type(ufo_dict_1))

print("")
print("_______________________________________________________________________")
print("")

#_______________________________________________________________________________

# Convert Time datatype from Datetime to Object.

ufo['Time'] = ufo.Time.astype(object)

print(ufo.head())
print("")
print(type(ufo))
print("")
print("ufo.dtypes")
print(ufo.dtypes)

print("")
print("_______________________________________________________________________")
print("")

#_______________________________________________________________________________

# Convert DataFrame to Dictionary.

ufo_dict_2 = ufo.to_dict('index')

print("ufo_dict_2")
print("")
print(dict(list(ufo_dict_2.items())[0:3]))
print("")
print(type(ufo_dict_2))

print("")
print("_______________________________________________________________________")
print("")