originally comes from here datopian/datahub-qa#126

Sometimes pushing the files result in an error from auth service. Usually, re-pushing helps.
Besides this, if you try and push multiple datasets in parallel you might get the same error.

seems like it throws that exception when you hit authz api several times within some short period of time. E.g., I've experienced it while doing multiple "push"s in parallel in the tests - https://travis-ci.org/datahq/data-cli/jobs/344397536#L675

Acceptance Criteria

• we know exact reason why this is happening and fix if possible

Tasks

• reporduce locally
• do analysis

Put back Google - while Github is mainly used by coders, almost everyone has a google account.

Acceptance criteria

• /auth/check responses with option for google

Tasks

• TODO

We currently have 5 repos (+ deploy) (in Python) with several different ways of running tests.

Acceptance criteria

• auth is tested with tox

Bonus:

• able to install with pip
• we have Makefile for test and install
• versioned

Tasks

• refactor tests for auth
• update README

We need another service (module) for communicating with mailing service, like MailChimp. Besides just allowing emails service, to avoid changes in the core of Auth module, we need to create a new external module that will handle permissions and make API requests for populating mailing list.

Acceptance Criteria

• The least modification to auth module
• we have a new API for emails
• new users are added to the mailing list on MailChimp

Tasks

• Create emails module
  • get_permissions() function
  • handler to make requests
• Update list of modified services (allowing emails to get permissions)
• Use emails API to update mailing lists on Mailchimp from Auth

Analysis

Create new module datahub/emails with very simple API right now

add_user(user_info):
    mailchimp_user = os.environ.get('MAILCHIMP_USER')
    mailchimp_pass = os.environ.get('MAILCHIMP_PASS')
    requests.post('https://....', user={mailchimp_user:mailchimp_pass}, data=user_info) 

Inf Auth

# in controllers

import emails

def _get_token_from_profile(provider, profile, private_key):
   ...
   user = create_or_get_user(userid, name, username, email, avatar_url)
   if user.get('new'):
       emails.add_user(user)
   ...

# in models

def create_or_get_user(provider_id, name, username, email, avatar_url):
    id_ = hash_email(email)
    user = get_user(id_)
    if user is None:
        document = {
            'id': id_,
            'provider_id': provider_id,
            'username': username,
            'name': name,
            'email': email,
            'avatar_url': avatar_url,
            'join_date': datetime.datetime.now()
        }
        save_user(document)
        document['new'] = True  # tell controllers it's new
        return document
    return user

Have to set username when user account created

• change username to have unique=True, nullable=False
  • run relevant alter table statements on the DB (or do via alembic)
• github: easy - #8
• google: use user part of email address
• ensure uniqueness of username - check if name is not unique and if not keep adding integer to name until unique
  • tests

When running make test I get various errors.

In the first test there's an error when calling jwt.decode, apparently algorithms is now a mandatory parameter.

    def test___check___on_new_user_is_called(self):
        profile = dict(id='test', name='name', email='[email protected]')
        token = module._get_token_from_profile('test_provider', profile, self.private_key)
>       user_profile = jwt.decode(token, self.private_key)
...
jwt.exceptions.DecodeError: It is required that you pass in a value for the "algorithms" argument when calling decode().

I'm using PyJWT 2.0.1. Installed running make install.

Should I downgrade to a specific PyJWT version?

Attached the make test output.

Thank you,

João
make_test.log

Endpoint: /auth/resolve?username=<username>

Acceptance criteria

• I can get userid for given username

Tasks

• resolve-username(username)
  • Query DB for username
  • Return user-id (if exists) or null (in case it doesn't)

Wondering if there is anything available now (that I might have missed) or if it's possible to add the ability to send description to dataset columns?

We have a case when a user had X primary email when registered for Datahub.io and after a while changed the primary email on GitHub to Y. This cause for him in another user on Datahub. Ideally, we should check all emails from GitHub and if match found to identify it.

Acceptance Criteria

• If the user has multiple Email on GitHub and matches found we identify him as 1

Tasks

• Check all emails and if match not found create user in that case

Currently, we are getting nulls from GitHub API if the user has a submitted email to be private.

Reason and solution for this is in http://stackoverflow.com/questions/35373995/github-user-email-is-null-despite-useremail-scope

To get private emails we need to call: GET https://api.github.com/user/emails

Acceptance criteria

• We have emails even if they are private on GitHub

Tasks

• write tests
• add the check to script and make separate API call

As a developer I want docker image for auth service be updated whenever I mage a change to this repo, so that I do not have to manually build and push datahub-auth image that uses auth image created from this repo.

Acceptance Criteria

• Auth service is redeployed whenever I make changes here

Tasks

• Come up with solution

Analysis

History of the auth service docker image:

• At first we user datahub/auth as is
• Then we used the Dockerfile in https://github.com/datahq/plans/tree/master/auth-docker, which embedded the plans logic with the auth service
• Then we added the email hooks
  • How was that Docker image built?
  • We have a dependency for datahub-emails in auth that needs to be removed:
    • https://github.com/datahq/auth/blob/master/requirements.txt

What we're doing:

• Create repo datahub-auth
  • With Dockerfile and .travis.yaml
  • Adding credentials to the .travis.yaml file:
    • travis env set --private K8S_OPS_GITHUB_REPO_TOKEN "*****"
    • travis env set --private DOCKER_USERNAME "***"
    • travis env set --private DOCKER_PASSWORD "***"
  • Follow instructions from here: https://github.com/datahq/deploy#continuous-deployment

In the frontend, we want to show user profile information such as joined_date (maybe some more in the future). To do so we need to query auth db so we need to have API for it: /auth/get_profile?username=core.

Acceptance criteria

• I can hit /auth/get_profile?username=core and get profile information about the user

Tasks

• add a blueprint in auth/blueprints.py
• add get_user_profile_by_username method in controllers
• reuse get_user_by_username method for querying db

While ago, was registered and able to push files to datahub. Apr ~15 hours later tried to run data login and getting

invalid json response body at http://api-testing.datahub.io/auth/check?jwt=HereGoesToken&next=http://localhost:3000 reason: Unexpected token < in JSON at position 0

Checking the Auth logs we have problem with connecting to database - SQLalchemy invalidRequestError

sqlalchemy.exc.StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back

Not sure, but this might be related to reusing session - according to this https://stackoverflow.com/a/42396447 it happens when using session while SQLachemy error is raised

This is public API method that is not currently documented.