adam-p / danforth-east Goto Github PK

My first implementation of CSRF prevention code used the "Double Submit Cookies" pattern. That works fine for the management site, but not for the self-serve page.

The problem is that Safari prevents iframes from setting cookies except in directly response to user action. There's some info here (or google for something like "safari iframe post cookie"). In practice, I have seen CSRF validation errors from mobile and desktop Safari browsers.

For now I have disabled the CSRF check for the self-serve page (11216ec). This is very distasteful, but... it's a publicly accessible, unauthenticated page, so any CSRF attack doesn't gain privilege escalation, so there's no reason to do one. But it's still distasteful.

Possible alternatives:

• It might be possible to use the form submit as a "user action" that goes to the server, gets a response with a cookie, and then uses that cookie in the AJAX POST. See the description here. It's ugly, though.
• Other mitigations are described in the OWASP cheat sheet. Maybe the "Encrypted Token Pattern" -- although I'm not sure what use in place of a "UserId". Maybe IP address (or other request metadata?).

Email sent via Sendgrid is ending up in spam filters (approximately all the time). This is unacceptable.

This might be improved by sending with Sendgrid via a verified domain rather than a Gmail address.

But it's easier to send via SMTP. We'll use a Google Workspace account with an app password.

I have seen a lot of errors in the demo instance when people are trying to load a page or when the /tasks/members-sheet-cull cron job is running.

BaseHandler exception
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/tasks.py", line 137, in get
    gapps.cull_members_sheet()
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/gapps.py", line 364, in cull_members_sheet
    config.MEMBERS_WORKSHEET_KEY)
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/lib/gdata/spreadsheets/client.py", line 450, in get_list_feed
    **kwargs)
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/lib/gdata/client.py", line 640, in get_feed
    **kwargs)
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/lib/gdata/client.py", line 278, in request
    version=get_xml_version(self.api_version))
  File "/base/data/home/apps/s~mmbrmgmt/1.377551078564221320/lib/atom/core.py", line 520, in parse
    tree = ElementTree.fromstring(xml_string)
  File "<string>", line 125, in XML
ParseError: no element found: line 1, column 0

It appears to be inconsistent, however. This suggests that spreadsheet operations need to be moved to a task that can cope with periodic failure (including really long requests).

So, spreadsheet caching not just for performance but reliability.

We used to basically have this with the "pay later" option. Dig up that code.

It takes about 5 seconds for the embedded membership registration form to load. This is pushing the limits of how long a visitor will wait before deciding that there's something wrong with the page.

The slowness is (almost certainly) due to the GAE instance warming up. Subsequent loads of the page (when the instance is warm) are fast.

A simple-and-sufficient fix would be to put a wait spinner on the page that gets replaced when the form is loaded. That should keep a user waiting long enough.

A better fix would be to figure out how to keep the GAE instance warm, so there's no start-up time. But it seems that isn't easy, and probably costs money.

Right now there's no way for members to check on when their year of membership is expiring, and so know if it's time to renew (besides the reminder email). There's also no way to see if someone is a member at all (it's easy to forget if you've joined or not).

It would be nice to enable checking of membership status and renewal date.

Ideas:

• Field that takes an associated email address and sends an email to that member if they exist. But: This doesn't help if there is no such member (and we probably don't want to send email to every arbitrary email that's entered).

As can be seen in https://github.com/adam-p/danforth-east/blob/b07592ffabccc57418e72eac494e616c974051ed/app.yaml#L1, our runtime is python38. According to the runtime support schedule, python38 stops being supported in October 2024. We should update before then.

(On other projects I have recently had trouble with updates to Python 3.12, so I think we should go to python311.)

The post-self-serve-join modal is near the top, but the user is looking at the bottom of the form when they submit. Either:

• Scroll up
• Show the modal in the current viewport
• Collapse the iframe contents

Spec

In the words of the primary stakeholder:

This was a new position for DECA and I was new to try to manage this sort of thing. So nothing was working. I would get an email from someone, I would determine what they were interested in and direct them to that committee. I don't know if the committee even followed up with them.
I like that they can tick off what they are interested in when they pay or join and if each year they need to go into the membership email and pay their membership fee they could re-evaluate what volunteer positions they would like.
I am not sure then what would happen to this information. It would be useful for each committee to get an update of new volunteers, but also the Volunteer co-ordinator (me), to be able to email each group of volunteers when something is needed or to do a mass call out for volunteers.
I would like a easy, simple (because I am simple minded!) to be able to get in touch and follow up with all the volunteers. DECA Diaries and our Facebook page does a call out when something is needed, but I would like to be able to make it a bit more personable and for the co-ordinators of each group to be able to see who is on the list and then either email them directly or come to me to do it.

(Note that currently functionality includes sending an email to each "Volunteer Interest Area Rep" when a new member joins telling them that the member has expressed a desire to volunteer in their interest area. But that's insufficient.)

Outstanding questions

• Is it okay if mass emails come from an individual's email account/address? Or do they need to come from the organization email address?
• Does mass-mail mean we need unsubscribe? (Would need another field in user record and a GAE email receiver. Or support from MailChimp or whatever.)
• "Skills" probably needs to be included in this. Like, filtering/emailing. But maybe that's feature creep. Should check with the Volunteer Coordinator.

Implementation ideas/plan

New volunteer notifications

It would be useful for each committee to get an update of new volunteers

The current volunteer-rep-email-on-new-user functionality might be sufficient. Or it might need to be expanded to also get triggered in renewals (or just renewals where the indicated volunteer interest areas change).

Another enhancement to this might be to include in the email a list of pre-existing potential-volunteers in the interest area, to refresh the manager's memory. Like:

You are currently listed as a DECA representative for the volunteer interest area of "{{ interest }}". A new member has joined who has expressed interest in volunteering in that area:

Jane Smith
[email protected]

The full list of members who have expressed an interest in volunteering in that area is:

Aaaa Aaaa <[email protected]>
Bbbb Bbbb <[email protected]>
...

...Or maybe that's overkill. Should check with the Volunteer Coordinator.

Volunteer filtering and emailing

to be able to email each group of volunteers when something is needed or to do a mass call out for volunteers.

It seems to me the simplest way to do this is:

• Create a page in the management site where the user can filter members by volunteer interest area to get a list of names and email addresses (or even just email address).
• Provide the list in such a form that it's easy to copy and paste that into the BCC field of an email.
• The user then writes the email by hand and sends it to the volunteer-members.

If the direct mail approach isn't sufficient, then we might have to get into MailChimp, etc. And unsubscribe stuff (tracked by MailChimp? tracked by us?). And maybe using MailChimp's API directly or having users work with it manually.

The “Paid?” status in the sheet doesn’t seem to be updated correctly. All the recent members have “N” in that column, and I just renewed and it says “N”.

The annual archive creation failed with this error:

googleapiclient.errors.HttpError

<HttpError 403 when requesting 
https://www.googleapis.com/drive/v3/files/[redacted]/permissions/[redacted]?transferOwnership=true&alt=json 
returned "Consent is required to transfer ownership of a file to another user.">

Traceback (most recent call last): 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", 
line 2447, in wsgi_app response = self.full_dispatch_request() 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", 
line 1952, in full_dispatch_request rv = self.handle_user_exception(e) 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", 
line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/_compat.py", 
line 39, in reraise raise value 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", 
line 1950, in full_dispatch_request rv = self.dispatch_request() 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", 
line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) 

File "/srv/tasks.py", 
line 250, in member_sheet_archive new_year = gapps.archive_members_sheet(settings.member_sheet_year) 

File "/srv/gapps.py", 
line 422, in archive_members_sheet sheetdata.copy_drive_file( 
  
File "/srv/sheetdata.py", 
line 258, in copy_drive_file drive.permissions().update( 
  
File "/layers/google.python.pip/pip/lib/python3.8/site-packages/googleapiclient/_helpers.py", 
line 134, in positional_wrapper return wrapped(*args, **kwargs) 

File "/layers/google.python.pip/pip/lib/python3.8/site-packages/googleapiclient/http.py", 
line 907, in execute raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/drive/v3/files/[redacted]/permissions/[redacted]?transferOwnership=true&alt=json returned "Consent is required to transfer ownership of a file to another user.">

I haven't yet investigated it.

Handle 500 errors better. Some possibilities:

• Retry XHR on 500
• Have a check server-side to ensure multiple welcome emails aren't sent if the retry causes two successful saves (which happens, for some baffling reason)
• ... maybe 500s happen less often with the increased instance size?