rladies / meetupr Goto Github PK

Add fields argument to find_groups() to retrieve optional fields from the response. I am using this package to work on a project this summer and found out that I needed a count of past events for each R-Ladies group. Thankfully, past_event_count is an optional field within the response of find/groups API method.

For now, I suggest just adding fields to find_groups() and the call to .fetch_results() as shown below:

find_groups <- function(text = NULL, topic_id = NULL, radius = "global", 
                                            fields = NULL, api_key = NULL) {
  api_method <- "find/groups"
  res <- .fetch_results(api_method = api_method,
                        api_key = api_key,
                        text = text,
                        topic_id = topic_id,
                        fields = fields,
                        radius = radius)

An example usage could be:

# add 'past_event_count' and 'upcoming_event_count' to the query parameters using 'fields' argument; 
#these two are among several optional fields provided by the API: https://www.meetup.com/meetup_api/docs/find/groups/
all_rladies_groups <- find_groups(text = "r-ladies",
                       fields="past_event_count, upcoming_event_count", api_key = "key here")

# Cleanup
rladies_groups <- all_rladies_groups[grep(pattern = "rladies|r-ladies|r ladies", 
                                          x = all_rladies_groups$name,
                                          ignore.case = TRUE), ]

#for now, we could easily retrieve the values for these two fields with:
pastevent_count <- purrr::map_dbl(rladies_groups$resource, "past_event_count", .default = 0)
upcomingevent_count <-  purrr::map_dbl(rladies_groups$resource, "upcoming_event_count", .default = 0)

I have been working a little with getting some dat for a presentation, and I realised that the "joined" date, as retrieved by the get_members() function, is the date a person joined meetup, not the date they joined the group.

We might not have the date of joining group (I'm looking into the resource we get and see if I can parse it out), but i'm writing it down here so checking this is not forgotten.

And perhaps the "joined" column should be renamed to "joined_meetup", and if we find group joining, it could be named "joined_group"?

Per comment in Issue #36 as of resolution of issue #31 urlname isn't necessary to run unit tests.

Add a find_topics() function so you can look up the topic ids that match a search term. You can find topic ids by querying here: https://secure.meetup.com/meetup_api/console/?path=/find/topics

This is related to: #13

Via https://www.meetup.com/meetup_api/#limits

I'm not sure if it's the best solution though, but it'd be nice to have this info in the output somehow. :-)

cc @Athanasiamo -- does it sound like a good idea?

Sometimes we don’t list all the options for an argument with a fixed value set, so let's find all the cases of this and add the correct set of values.

Here's an example:

get_events(urlname, event_status = "upcoming")

The default value for event_status, as written in the function definition should be restricted to the set of allowable values (so that R can check if an invalid string is passed and throw an error):

get_events(urlname, event_status = c("upcoming", "cancelled", "draft", 
                                     "past", "proposed", "suggested"))

The actual default value for event_status is the first element of this character vector.

switching via usethis::use_roxygen_md()

https://www.meetup.com/meetup_api/clients/

The build has been failing for 6 months: https://travis-ci.org/github/rladies/meetupr/branches

I don't see any issues with the DESCRIPTION file though..

Building with: R CMD build 
0.26s$ R CMD build  .
* checking for file ‘./DESCRIPTION’ ... OK
* preparing ‘meetupr’:
* checking DESCRIPTION meta-information ... ERROR

Malformed Depends or Suggests or Imports or Enhances field.

Offending entries:
  tibble
knitr

Entries must be names of packages optionally followed by '<=' or '>=',
white space, and a valid version number in parentheses.
See section 'The DESCRIPTION file' in the 'Writing R Extensions'
manual.

The command "R CMD build  ." failed and exited with 1 during .

Your build has been stopped.

To find R-Ladies meetups that aren't using the topic, I do a text search and limit to a category and then apply a regex to the result. We should support category as a parameter.

(TODO: make a reprex)

groups_url <- "https://api.meetup.com/find/groups"

  groups_query_params <- list(
    key=params$meetup_api_key, 
    sign=TRUE,
    page=200,
    radius="global")
  
  # by topic
  meetup_groups_topic <- get_meetups(groups_url, append(groups_query_params, c(topic_id=1513883, order="members")))
  
  # by text + category
  meetup_groups_text <- get_meetups(groups_url, append(groups_query_params, c(text="r-ladies", category=34)))
  
  meetup_groups_aggr <- bind_rows(meetup_groups_topic, meetup_groups_text %>% anti_join(meetup_groups_topic, by="id"))
  meetup_groups <- meetup_groups_aggr %>% filter(str_detect(name, "[Rr]([ -]?)[Ll]adies"))`

Design question - do we want to explicitly support parameters or do we want to consider dynamic arguments?

Hi,

Something might have changed (at some point)

This file
https://github.com/rladies/meetupr/blob/master/R/get_event_attendees.R
Is looking for the word "attendance", but now the word should be "attendees"

You can check https://www.meetup.com/CHAPTERNAME/events/EVENTID/attendees/

We need to make a PR to add it here: https://github.com/meetup/api/blob/master/clients.md

Cf #90 (comment)

CC @Athanasiamo @ledell

There is a blog post by Rick Pack about trying to use the meetupr::get_event_attendees() function and he pointed out that this currently requires the user to be an admin of that group.

It looks like he was able to get around that constraint in his code so we should figure out if we can update the get_event_attendees() to not require group admin rights.

Here's an example:

> past_events <- get_events(urlname = "rladies-nashville", event_status = "past")
> past_events
# A tibble: 9 x 21
         id                                            name
      <chr>                                           <chr>
1 234968855                     R-Ladies Nashville Kickoff!
2 235652800             December Meetup: data.table package
3 235757913              January Meetup: R Resources & Help
4 237055230 February Meetup: RStudio Conference Highlights!
5 237055481                    March Meetup: Intro to Shiny
6 239935025                           An R + Github Journey
7 242098493           Working with Pipes %>% + GIS Tutorial
8 243331077                           Introduction to purrr
9 243331084      Play in the snow, then clean up your mess!
# ... with 19 more variables: created <dttm>, status <chr>,
#   time <dttm>, local_date <date>, local_time <chr>,
#   waitlist_count <int>, yes_rsvp_count <int>, venue_id <int>,
#   venue_name <chr>, venue_lat <dbl>, venue_lon <dbl>,
#   venue_address_1 <chr>, venue_city <chr>, venue_state <chr>,
#   venue_zip <chr>, venue_country <chr>, description <chr>,
#   link <chr>, resource <list>
> get_event_attendees(urlname, event_id = 234968855)
Error in .quick_fetch(api_url = api_url, api_key = api_key, event_status = event_status,  : 
  Bad Request (HTTP 400).

We also need to add some tests to check for this type of thing.

One of our checks should make sure people aren't committing their API keys into the code.

I made a comment on Issue #36 regarding this!

The documentation of meetupr suggests that, in a situation where the user has not provided a token, the packages uses a default token that is shipped with it here: https://github.com/rladies/meetupr/blob/master/R/zzz.R

However, I tried to run a simple call using the package without providing a token, but I am redirected to a browser to Log in, and that seems to be the only way to obtain a token. But once I have logged in once and the authentication is complete, I can now make as many API calls without needing to re-authenticate for several months due to the cache set to true.

To reproduce this scenario, I created a brand new project on RStudio and tried to run the following code:

groups <- find_groups(text = "r-ladies", api_key = "")

Result: I was redirected to a browser for authentication.
Expected behaviour: I expected this to work automatically with the default token shipped with the package.

I hope to explore this more carefully but get_events returns:
Error: Result 1 is not a length 1 atomic vector

when one submits:
grp_events <- get_events("tangoutopia", event_status = "past")

I do not yet know why. There are plenty of past Meetups in https://www.meetup.com/tangoutopia/events/past/ and indeed the function prints:
Downloading 1021 record(s)...

EDIT Can someone see if you also experience this error? I wonder if this is due to me hitting the API too frequently in a given expanse of time.

Tests are mocked in Issue #31

We should have some integration tests that check our client against the Meetup API.

• These should not be run as part of testthat by default
• They should only be run by Travis
• They should only be run on PR, not with every update
• They should not depend on fragile aspects of the API response (eg, expecting a certain number of columns or responses)
• They're function is to check essential compatibility with the Meetup API (authentication, parameters being honored, etc)

I set my API key using Sys.setenv(MEETUP_KEY="mykey"). I can then use all functions without specifying the api_key argument because all functions set it to NULL as default and .fetch_results will then check for NULL and draw it from the Sys env.

The get_boards function does not specify NULL as default for api_key, hence resulting in a missing argument error.

library(meetupr)
get_boards(urlname = "rladies-san-francisco")
#> Error in .fetch_results(api_method, api_key): argument "api_key" is missing, with no default

So that one could use it à la gh::gh() to call any endpoint that's not supported by a high-level function.

Opinions @ledell @Athanasiamo? (and anyone else)

Via Athanasia

Hi folks! I’m playing around with the meetupr package we have on github.
I was wondering whether theget_events function should throw an error when no events are found for a query? Is it really an error if there are no upcoming events? Should it not rather be a warning?

I am looking to see if our code is doing this or if its something from the Meetup API.

https://www.meetup.com/meetup_api/docs/status/

I think we're probably ready to do our first CRAN submission!

These files are essential when other people want to contribute to the package.

To generate this, we can use. the usethis package.

library(usethis)
use_code_of_conduct()
use_tidy_contributing()
use_tidy_issue_template()

The unit tests are currently making "IRL" api calls which can result in failures in the event of no network access. Additionally, these tests may result in errors beyond the scope of what is actually being tested. Ultimately, at least for these tests, we just want to know if our library code is working, we don't care about the api key or httr.

That said, other end-to-end continuous integration tests should be developed that do call the API from TravisCi but that's beyond the scope of this issue. I'll file it as a second one and mention this one to link it.

The README need to be updated to use the new authentication scheme. Related issues (and code examples should be in those issues and PRs): #51 #59

My work using this package to explore R User groups based on a modified version of find_groups() (provided within this pull request #48 ) that retrieves additional fields has revealed an unexpected behavior.

Consider the code below:

all_ruser_groups <- find_groups(text = "r-project-for-statistical-computing", fields = "past_event_count, upcoming_event_count, last_event, topics", api_key = meetup_api_key)

There are 481 records returned for this query. For records, between 0 - 199, everything is fine. But for records between 200 - 481, there are no entries for upcoming_event_count, last_event, topics. Only the first additional field past_event_count is returned.

I guessed that the problem could be from the following portion of .fetch_results() function within internals.R :

# If you have not yet retrieved all records, calculate the # of remaining calls required
  extra_calls <- ifelse(
    (length(records) < total_records) & !is.null(res$headers$link),
    floor(total_records/length(records)),
    0)
  if (extra_calls > 0) {
    all_records <- list(records)
    for (i in seq(extra_calls)) {
      # Keep making API requests with an increasing offset value until you get all the records
      # TO DO: clean this strsplit up or replace with regex
      next_url <- strsplit(strsplit(res$headers$link, split = "<")[[1]][2], split = ">")[[1]][1]
      res <- .quick_fetch(next_url, api_key, event_status)
      all_records[[i + 1]] <- res$result
    }
    records <- unlist(all_records, recursive = FALSE)
  }

I re-wrote this portion and got a working solution for my work. Possibly could send a pull request soon.

@ledell @LucyMcGowan Any thoughts?

Please add any demos or references to the meetupr package that you see on the internet as a comment below. Once there is a short list, we can make this into a markdown file called demos.md (or similar).

Now that we added OAuth support and the API key authentication scheme is no longer functional, we can deprecate the API key methods.

Current plan is to remove the api_key argument and add ... to swallow any api_key args that appear in old/outdated scripts.

For more robustness

Please briefly describe your problem and what output you expect. If you have a question, please don't use this form. Instead, ask on https://stackoverflow.com/ or https://community.rstudio.com/.

Please include a minimal reproducible example (AKA a reprex). If you've never heard of a reprex before, start by reading https://www.tidyverse.org/help/#reprex.

Brief description of the problem

# insert reprex here

It would be nice to have a simple pkgdown website. We could host it at: https://rladies.github.io/meetupr

#84 (comment)

@hfrick and I wanted to pull all R-Ladies chapters from Meetup with "have R-Ladies as topic" as a definition. https://www.meetup.com/topics/r-ladies/all/

Would you consider adding the function for getting meetups for a topic? 🙏

Thanks a lot! 💜

https://github.com/rladies/meetupr/blob/master/R/zzz.R#L6

This stuff doesn't go into a public repo, major security risk. Someone could spoof you for data mining & it may violate Meetup's TOS.

People need to add their own api keys to their .Renviron files. If this isn't in the README or other instructions then in addition to removing that file, add an example to show ppl how.

@GregSutcliffe is working on this in #43

I guess that retrieving the full country name (localized_country_name in the meetup API) could be more useful than just the country code (country in the meetup API). You might want to retrieve both.

Using the following code retrieves this for me in the find_group() function.

country = purrr::map_chr(res, "localized_country_name"),

I received an email this morning from Meetup about changing their API from key-based to OAuth-based on August 15, 2019. It looks like a substantial change that may affect the package: https://www.meetup.com/meetup_api/auth/?utm_medium=email&utm_source=transactional&utm_campaign=api_update&utm_content=en_06142019#oauth2

It also appears that OAuth accounts are free until August 15 but will require a pro account after that :/

Poke @kierisi

ratelimitr

The docs say "You can know your current rate limit status by reading X-RateLimit HTTP headers included in responses. The following table indicates their name and meaning. " so I guess this issue is related to #9.

I imagine the code would need to make a call at the start of each session to determine what to put in the ratelimitr call 🤔

I think it would be more user-friendly for the get_events() function to return an empty (zero rows) tibble instead of an error when there's no results. We could convert the error message into a warning.

I ran into this error when writing a function to get and rbind all R-Ladies events into a single tibble, and my function died in the middle because of this error, rather than just rbinding an empty frame.

Note: This repro will only work if there are still no upcoming events for the Sao Paulo chapter (check here).

events <- get_events(urlname = "rladies-sao-paulo", event_status = "upcoming")

When there are no results, it will give the following error:

Error: Zero records match your filter. Nothing to return.

Right now, we have to wrap in a tryCatch() to be able to use get_events() inside another function w/o potential errors:

events <- tryCatch(get_events(urlname = "rladies-sao-paulo",
                              event_status = "upcoming"),
                   error = function(e) {
                     message("No events found.")
                   })

TO DO: Check if this is the case in other functions and if so, change this ticket to cover all the functions and make the changes all at once.

We need to add add a newline here after the ....

Because of Travis new pricing

The PR which was just merged which adds the fields parameter to find_groups() will add the extra fields to the list (raw data), but the user still has to extract those fields and add them as extra columns to the returned tibble. Should we just add those fields automatically as columns, since it's almost 100% likely that the user will want to do that anyway, since they are explicitly requesting that data?

The other thing I was considering, if we do this, is to make the fields parameter more generic (possibly rename to extra_fields) and then we could allow the user to specify which extra fields they want to add to the returned tibble as extra columns. In this case, those fields could just be extra ones that we already have in the default resource raw list object but are not exposed as columns by default, or they could be the extra ones that must be passed via the API "fields" parameter to the /find/groups endpoint.

Lastly, since it seems like a lot of the endpoints support the "fields" parameter, we could consider adding this functionality to all of the functions. e.g. get_members()

Here's an announcement from meetup.com. We need to add support for OAuth, deprecate api_key (I guess we should not remove it from the functions because it will break people's code but we should tell people that it's not active anymore), update the docs and re-test everything. :-(

Changes to our API access

• API Keys will be replaced by OAuth: We will be removing API keys on August 15, 2019 and requiring you to authenticate with OAuth.
• Move to OAuth soon for continued free API access: Until August 15, members will be able to apply for OAuth access free of charge. After August 15, anyone who wants to apply for API access through OAuth will need to have a Meetup Pro account in order to do so.
• We’re removing API version 2: Before August 15, you will need to switch to API version 3. API version 2 will no longer be available. You can find more information on our API updates and answers to common questions on our Help Center.

So far we only have 2 tests: test-get_events.R test-find_groups.R.

We need to add tests to other functions (https://github.com/rladies/meetupr/tree/master/R)

The README need to be updated to use the new authentication scheme. Related issues (and code examples should be in those issues and PRs): #51 #59

find_groups('R-ladies Paris')
returns
id name urlname created members status organizer lat lon city
1 2.04e⁷ R-Ladi… rladies-… 2016-09-19 17:39:51 298 active Gabriela… 48.9 2.34 Paris

however
get_members('rladies-paris')
returns members with join dates prior to group create date!
id name bio status joined city country state lat lon
1 8.37e⁶ Sarah… I miss skii… active 2008-11-15 22:13:36 Paris fr <NA> 48.9 2.34
2 1.45e⁷ Gabri… <NA> active 2011-04-20 00:57:39 San … us CA 37.8 -122

I get the following message when I run the code:

devtools::install_github("rladies/meetupr")
Downloading GitHub repo rladies/meetupr@master
Error in process_initialize(self, private, command, args, stdin, stdout, :
Command not found