A Grammar of Data Manipulation in python
Documentation | Reference Maps | Notebook Examples | API | Blog
datar
is a re-imagining of APIs of data manipulation libraries in python (currently only pandas
supported) so that you can manipulate your data with it like with dplyr
in R
.
datar
is an in-depth port of tidyverse
packages, such as dplyr
, tidyr
, forcats
and tibble
, as well as some functions from base R.
pip install -U datar
# install pdtypes support
pip install -U datar[pdtypes]
# install dependencies for modin as backend
pip install -U datar[modin]
# you may also need to install dependencies for modin engines
# pip install -U modin[ray]
from datar import f
from datar.dplyr import mutate, filter, if_else
from datar.tibble import tibble
# or
# from datar.all import f, mutate, filter, if_else, tibble
df = tibble(
x=range(4), # or f[:4]
y=['zero', 'one', 'two', 'three']
)
df >> mutate(z=f.x)
"""# output
x y z
<int64> <object> <int64>
0 0 zero 0
1 1 one 1
2 2 two 2
3 3 three 3
"""
df >> mutate(z=if_else(f.x>1, 1, 0))
"""# output:
x y z
<int64> <object> <int64>
0 0 zero 0
1 1 one 0
2 2 two 1
3 3 three 1
"""
df >> filter(f.x>1)
"""# output:
x y
<int64> <object>
0 2 two
1 3 three
"""
df >> mutate(z=if_else(f.x>1, 1, 0)) >> filter(f.z==1)
"""# output:
x y z
<int64> <object> <int64>
0 2 two 1
1 3 three 1
"""
# works with plotnine
# example grabbed from https://github.com/has2k1/plydata
import numpy
from datar.base import sin, pi
from plotnine import ggplot, aes, geom_line, theme_classic
df = tibble(x=numpy.linspace(0, 2*pi, 500))
(df >>
mutate(y=sin(f.x), sign=if_else(f.y>=0, "positive", "negative")) >>
ggplot(aes(x='x', y='y')) +
theme_classic() +
geom_line(aes(color='sign'), size=1.2))
# easy to integrate with other libraries
# for example: klib
import klib
from datar.core.factory import verb_factory
from datar.datasets import iris
from datar.dplyr import pull
dist_plot = verb_factory(func=klib.dist_plot)
iris >> pull(f.Sepal_Length) >> dist_plot()
See also some advanced examples from my answers on StackOverflow:
- Compare 2 DataFrames and drop rows that do not contain corresponding ID variables
- count by id with dynamic criteria
- counting the frequency in python size vs count
- Pandas equivalent of R/dplyr group_by summarise concatenation
- ntiles over columns in python using R's "mutate(across(cols = ..."
- Replicate R Solution in Python for Calculating Monthly CRR
- Best/Concise Way to Conditionally Concat two Columns in Pandas DataFrame
- how to transform R dataframe to rows of indicator values
- Left join on multiple columns
- Python: change column of strings with None to 0/1
- Comparing 2 data frames and finding values are not in 2nd data frame
- How to compare two Pandas DataFrames based on specific columns in Python?
- expand.grid equivalent to get pandas data frame for prediction in Python
- Python pandas equivalent to R's group_by, mutate, and ifelse
- How to convert a list of dictionaries to a Pandas Dataframe with one of the values as column name?
- Moving window on a Standard Deviation & Mean calculation
- Python: creating new "interpolated" rows based on a specific field in Pandas
- How would I extend a Pandas DataFrame such as this?
- How to define new variable based on multiple conditions in Pandas - dplyr case_when equivalent
- What is the Pandas equivalent of top_n() in dplyr?
- Equivalent of fct_lump in pandas
- pandas equivalent of fct_reorder
- Is there a way to find out the 2 X 2 contingency table consisting of the count of values by applying a condition from two dataframe
- Count if array in pandas
- How to create a new column for transposed data
- How to create new DataFrame based on conditions from another DataFrame
- Refer to column of a data frame that is being defined
- How to use regex in mutate dplython to add new column
- Multiplying a row by the previous row (with a certain name) in Pandas
- Create dataframe from rows under a row with a certain condition
- pandas data frame, group by multiple cols and put other columns' contents in one
- Pandas custom aggregate function with condition on group, is it possible?
- multiply different values to pandas column with combination of other columns
- Vectorized column-wise regex matching in pandas
- Iterate through and conditionally append string values in a Pandas dataframe
- Groupby mutate equivalent in pandas/python using tidydata principles
- More ...