arrange() not working about dfply HOT 6 OPEN

Try adding >> ungroup() before the arrange.

I regularly use group_by(), ungroup() followed by arrange(), and it works perfectly every time.

In addition, please ensure that the dataframe is sorted by the same key as in the "group_by" before using "group_by". This is a key difference between SQL and Pandas - grouping requires pre-sorting first or it may not work.

from dfply.

@sharpe5's answer is correct, but this behavior unexpectedly diverges from dplyr's.

As far as I can tell, after the summarize, in dplyr the groups have been eliminated, while in dfply they remain attached to the dataframe.

Is this the intended behavior? If so, should the difference be documented?

Thanks!

In R with dplyr:

r$> library(tidyverse)   

r$> diamonds %>%  
        group_by(cut) %>%  
        summarize(count_color=n()) %>% 
        arrange(-count_color)                                                                                                                                                                                        
# A tibble: 5 x 2
  cut       count_color
  <ord>           <int>
1 Ideal           21551
2 Premium         13791
3 Very Good       12082
4 Good             4906
5 Fair             1610


r$> diamonds %>%  
        group_by(cut) %>%  
        summarize(count_color=n()) %>% 
        ungroup() %>% 
        arrange(-count_color)                                                                                                                                                                                        
# A tibble: 5 x 2
  cut       count_color
  <ord>           <int>
1 Ideal           21551
2 Premium         13791
3 Very Good       12082
4 Good             4906
5 Fair             1610

In Python with dfply:

In [8]: (dfply.diamonds >>  
   ...:     group_by('cut') >>  
   ...:     summarize(count_color=dfply.n(X.color)) >>  
   ...:     arrange(X.count_color, ascending=False))                                                                                                                                                                 
Out[8]: 
         cut  count_color
0       Fair         1610
1       Good         4906
2      Ideal        21551
3    Premium        13791
4  Very Good        12082


In [7]: (dfply.diamonds >>  
   ...:     group_by('cut') >>  
   ...:     summarize(count_color=dfply.n(X.color)) >>  
   ...:     ungroup() >>  
   ...:     arrange(X.count_color, ascending=False))                                                                                                                                                                 
Out[7]: 
         cut  count_color
2      Ideal        21551
3    Premium        13791
4  Very Good        12082
1       Good         4906
0       Fair         1610

from dfply.

Yes this is the intended behavior. I am aware that it diverges from dplyr. I understand the rationale in dplyr that summarize would eliminate the groupings, since it is "collapsing" the groups to single rows and therefore the groupings become "meaningless".

I am not opposed to changing it to match the dplyr behavior if that is what the people want. My rationale for not doing this despite the collapsing into rows is that the ungrouping becomes implicit rather than explicit. My reasoning was that the grouping should be preserved until you explicitly state that groupings should be collapsed into a single dataframe.

Not sure which direction to go. If you think that the dplyr way is superior I am happy to change it to that.

from dfply.

from dfply.

I'd prefer that summarize() retain the ungrouping behavior of dplyr, and that if a group-retaining version is desired, that it have a different name. Since I regularly use both dplyr and dfply, it makes me nervous to have the same function name with substantially different behavior. I am worried about confusing them and getting wrong results without noticing.

from dfply.

I'm not wedded to any particular names, but I do think it's important that different functions have different names.

How would you feel about:

• adding a function with a new name, digest(), that does ungrouping aggregation,
• adding a function with a new name, review(), that does aggregation retaining groups, and
• removing the summarize() function to eliminate that source of confusion with dplyr.

This way there's no confusion and I can just pick the one I want without fear, and I don't have to worry about which version of dfply I'm on.

from dfply.