Comments (6)
Try adding >> ungroup()
before the arrange
.
I regularly use group_by()
, ungroup()
followed by arrange()
, and it works perfectly every time.
In addition, please ensure that the dataframe is sorted by the same key as in the "group_by" before using "group_by". This is a key difference between SQL and Pandas - grouping requires pre-sorting first or it may not work.
from dfply.
@sharpe5's answer is correct, but this behavior unexpectedly diverges from dplyr's.
As far as I can tell, after the summarize
, in dplyr the groups have been eliminated, while in dfply they remain attached to the dataframe.
Is this the intended behavior? If so, should the difference be documented?
Thanks!
In R with dplyr:
r$> library(tidyverse)
r$> diamonds %>%
group_by(cut) %>%
summarize(count_color=n()) %>%
arrange(-count_color)
# A tibble: 5 x 2
cut count_color
<ord> <int>
1 Ideal 21551
2 Premium 13791
3 Very Good 12082
4 Good 4906
5 Fair 1610
r$> diamonds %>%
group_by(cut) %>%
summarize(count_color=n()) %>%
ungroup() %>%
arrange(-count_color)
# A tibble: 5 x 2
cut count_color
<ord> <int>
1 Ideal 21551
2 Premium 13791
3 Very Good 12082
4 Good 4906
5 Fair 1610
In Python with dfply:
In [8]: (dfply.diamonds >>
...: group_by('cut') >>
...: summarize(count_color=dfply.n(X.color)) >>
...: arrange(X.count_color, ascending=False))
Out[8]:
cut count_color
0 Fair 1610
1 Good 4906
2 Ideal 21551
3 Premium 13791
4 Very Good 12082
In [7]: (dfply.diamonds >>
...: group_by('cut') >>
...: summarize(count_color=dfply.n(X.color)) >>
...: ungroup() >>
...: arrange(X.count_color, ascending=False))
Out[7]:
cut count_color
2 Ideal 21551
3 Premium 13791
4 Very Good 12082
1 Good 4906
0 Fair 1610
from dfply.
Yes this is the intended behavior. I am aware that it diverges from dplyr
. I understand the rationale in dplyr
that summarize
would eliminate the groupings, since it is "collapsing" the groups to single rows and therefore the groupings become "meaningless".
I am not opposed to changing it to match the dplyr
behavior if that is what the people want. My rationale for not doing this despite the collapsing into rows is that the ungrouping becomes implicit rather than explicit. My reasoning was that the grouping should be preserved until you explicitly state that groupings should be collapsed into a single dataframe.
Not sure which direction to go. If you think that the dplyr
way is superior I am happy to change it to that.
from dfply.
from dfply.
I'd prefer that summarize()
retain the ungrouping behavior of dplyr, and that if a group-retaining version is desired, that it have a different name. Since I regularly use both dplyr and dfply, it makes me nervous to have the same function name with substantially different behavior. I am worried about confusing them and getting wrong results without noticing.
from dfply.
I'm not wedded to any particular names, but I do think it's important that different functions have different names.
How would you feel about:
- adding a function with a new name,
digest()
, that does ungrouping aggregation, - adding a function with a new name,
review()
, that does aggregation retaining groups, and - removing the
summarize()
function to eliminate that source of confusion with dplyr.
This way there's no confusion and I can just pick the one I want without fear, and I don't have to worry about which version of dfply I'm on.
from dfply.
Related Issues (20)
- A way to outer_join() by index?
- rename(CUT=X.cut, COLOR='color') - counterintuitive IMO HOT 1
- Liveness of this project HOT 1
- group_by ModuleNotFoundError: No module named 'dfply.group'; 'dfply' is not a package
- Performance issue
- NameError: name 'filter_for_jpeg' is not defined
- mutate not support the new variables that created above HOT 2
- Can I use `%in%` operator in dfply framework? HOT 1
- missing cumcount() function in embedded column functions
- unable to aggregate and summarize counts for a categorical variable HOT 1
- When using `spread`, Type error occurs on not callable 'Client' object HOT 1
- Issues to convert to pandas dataframe
- how to use case_when function
- Issue arranging data after summarising with a new variable HOT 1
- Not taking into account None values in group_by HOT 1
- missing sum() from summary function
- inner_join can not use by this way
- #group_by #mode #iter()
- Filtering two or more elements from a column
- how to summarise multiple variables? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dfply.