The members' data API is a Play app that manages and retrieves supporter attributes associated with a user.
It runs on https://members-data-api.theguardian.com/.
theguardian.com website is the biggest single consumer of members-data-api
, specifically the /user-attributes/me
endpoint, which it uses to determine both ad-free (when user has a digital subscription or supporter plus or a newspaper product) and if we should hide 'support messaging/asks' (banner, epic, header/footer support buttons etc).
It would be unnecessary to hit members-data-api
on every single page view, so instead it uses cookies to regulate how often calls are made. The gu_user_features_expiry
contains a timestamp for the 'earliest' point it would be allowed to call members-data-api
again, and is updated whenever it does call members-data-api
to 'now + 24hours'.
Various things from the /user-attributes/me
response are stored in cookies, to be used on each render...
GU_AF1
the 'ad-free' cookie which is set to a timestamp for 'now + 48hours' if thecontentAccess.digitalPack
=true
gu_hide_support_messaging
is set totrue
ifshowSupportMessaging
=false
in the responsegu_action_required_for
is set to the value ofalertAvailableFor
in the response, and is used to control the display of the 'payment failure' bannergu_paying_member
=contentAccess.paidMember
in the responsegu_digital_subscriber
=contentAccess.digitalPack
in the responsegu_recurring_contributor
=contentAccess.recurringContributor
in the responsegu_one_off_contribution_date
=oneOffContributionDate
in the response
User attributes (which products a user currently holds) are served from three sources
- The
SupporterProductData-[STAGE]
DynamoDB table - This table is populated by a scheduled extract from Stripe (Guardian Patrons only) and Zuora (digital and print subscriptions & recurring contributions) - The
contributions-store-[STAGE]
Postgres database (one off contributions only) - The mobile purchases api (in-app purchases only)
-
You will need to have dev-nginx installed.
-
Follow the nginx steps for identity-platform.
-
Follow the identity-frontend configuration steps.
-
Then run
./setup.sh
innginx/
. -
Add the following entries to your hosts file:
127.0.0.1 profile.thegulocal.com
127.0.0.1 members-data-api.thegulocal.com
-
Get Janus credentials for membership.
-
Download the config
(you may need tobrew install awscli
to get the command.)
aws s3 cp s3://gu-reader-revenue-private/membership/members-data-api/DEV/members-data-api.private.conf /etc/gu/ --profile membership
-
Get Janus credentials for membership.
-
(optional) Create an ssh tunnel to the CODE one-off contributions database:
- Clone https://github.com/guardian/contributions-platform
- From the contributions-platform project, Run
./contributions-store/contributions-store-bastion/scripts/open_ssh_tunnel.sh -s CODE
(requires marauder) - If you need to close your tunnel and didn't make a note of the process number you can run
ps aux | grep amazonaws.com
to find the process number. The output will look something like thisusername 1693 0.0 0.0 34153208 952 ?? Ss 4:20pm 0:00.00 ssh -i /private/var/folders/yv/dbtm9psd5ddbjm_lvcjm9zf1vdng_j/T/security_ssm-scala_temporary-rsa-private-key.tmp -f machine.eu-west-1.compute.amazonaws.com -L 5432:contributions-store-code.address.rds.amazonaws.com:5432 -N -o IdentitiesOnly yes -o ExitOnForwardFailure yes
where1693
is the process number. You can close the tunnel withkill 1693
.
-
Ensure an
nginx
service is running locally. You can rundev-nginx restart-nginx
to do this.
The /me endpoints can either use the GU_U and SC_GU_U from the Cookie request header or Bearer authorisation header. To get either of these you will need to sign in to the identity frontend.
To authenticate and get a bearer token from identity (recommended)
- Follow this identity thread to set up PostMan https://chat.google.com/room/AAAAFdv9gK8/kxmrtiVjtRs/
To run identity locally for browser based access (not recommended)
- Start up a local Identity service by running script
make dev
in the gateway repo. - Go to https://profile.thegulocal.com/signin.
-
To start the Members' data API service run
./start-api.sh
.
The service will be running on 9400. -
go to https://members-data-api.thegulocal.com/user-attributes/me. If you get a 401 response, it probably means your Identity credentials have expired.
Renew them by following the steps in Identity frontend local sign in -
As of 22/04/2022 the https://members-data-api.thegulocal.com/user-attributes/me endpoint should work correctly if your set up is correct. Other endpoints (
/healthcheck
,/user-attributes/me/mma-membership
) may not work correctly due to upstream dependencies.
run sbt and then test
Identity frontend is split between new (profile-origin) and old (profile), which is the identity project in frontend. Only profile uses the membership-attribute-service. Make sure that it's pointing at your local instance.
devOverrides{
guardian.page.userAttributesApiUrl="https://members-data-api.thegulocal.com/user-attributes"
id.members-data-api.url="https://members-data-api.thegulocal.com/"
}
{
"userId": "xxxx",
"tier": "Supporter",
"membershipNumber": "1234",
"membershipJoinDate": "2017-06-26",
"contentAccess": {
"member": true,
"paidMember": true,
"recurringContributor": false,
"digitalPack": false
}
}
{
"userId":"xxxx",
"recurringContributionPaymentPlan":"Monthly Contribution",
"contentAccess": {
"member":false,
"paidMember":false,
"recurringContributor":true,
"digitalPack": false
}
}
{
"message":"Not found",
"details":"Could not find user in the database",
"statusCode":404
}
{
"userId": "xxxx",
"tier": "Supporter",
"membershipNumber": "324154",
"recurringContributionPaymentPlan": "Monthly Contribution",
"membershipJoinDate": "2017-06-26",
"contentAccess": {
"member": true,
"paidMember": true,
"recurringContributor": true,
"digitalPack": false
}
}
{
"userId": "30000549",
"digitalSubscriptionExpiryDate": "2018-11-29",
"contentAccess": {
"member": false,
"paidMember": false,
"recurringContributor": false,
"digitalPack": true
}
}
{
"userId": "xxxx",
"tier": "Supporter",
"membershipNumber": "1234",
"contentAccess": {
"member": true,
"paidMember": true
}
}
{
"message":"Not found",
"details":"User was found but they are not a member",
"statusCode":404
}
{
"userId": "xxxx",
"tier": "Supporter",
"membershipNumber": "1234",
"contentAccess": {
"member": true,
"paidMember": true
}
}
{
"message":"Not found",
"details":"Could not find user in the database",
"statusCode":404
}
Responses:
{
"adFree": true,
"adblockMessage": false,
"userId": "123",
"membershipJoinDate": "2017-04-04"
}
membership-attribute-service
is installed as systemd service via .deb
package.
Application instances allow SSH via SSM tunnel, not via port 22,
- Get fresh Janus credentials
- Find instance
prism -f instanceName attribute code
- 1ssm ssh -x -i i-123456 --profile membership`
Description | Command |
---|---|
application directory | /usr/share/membership-attribute-service |
service config | /lib/systemd/system/membership-attribute-service.service |
application logs | /var/log/membership-attribute-service/membership-attribute-service.log |
stdout/stderr logs | journalctl -u membership-attribute-service |
service status | systemctl status membership-attribute-service |
healthcheck | curl http://127.0.0.1:9000/healthcheck |
To enable fine metrics for all http calls to external services (Zuora Rest and SOAP, Salesforce, etc), set this in config (either application.conf or s3://gu-reader-revenue-private/membership/members-data-api/DEV/members-data-api.private.conf):
use-fine-metrics=true
When set, calls to all methods / endpoints will be logged as metrics.
You can see it on these dashboards: