sw-tools / checkin-service Goto Github PK

View Code? Open in Web Editor NEW

9.0 9.0 3.0 750 KB

Checks the user in to a flight

License: MIT License

JavaScript 1.45% TypeScript 98.55%

aws-lambda checkin chrome-aws-lambda flight puppeteer serverless sw

checkin-service's People

Contributors

Stargazers

Watchers

Forkers

mmoore99 aaron-pham rezer

checkin-service's Issues

Logging changes

Currently, the logging looks like this for checkin attempts:

2022-02-23T16:55:34.525Z	c6590d54-2112-5a54-9749-57c448237b84	INFO	Failed to fetch checkin data on attempt 0 of 80 at February 23, 2022, 4:55:34 PM UTC with error: HTTPError: Response code 400 (Bad Request)
    at Request.<anonymous> (/var/task/node_modules/got/dist/source/as-promise/index.js:117:42)
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  code: 'ERR_NON_2XX_3XX_RESPONSE',
  timings: {
    start: 1645635333614,
    socket: 1645635333615,
    lookup: 1645635333615,
    connect: 1645635333622,
    secureConnect: 1645635333630,
    upload: 1645635333630,
    response: 1645635334518,
    end: 1645635334521,
    error: undefined,
    abort: undefined,
    phases: {
      wait: 1,
      dns: 0,
      tcp: 7,
      tls: 8,
      request: 0,
      firstByte: 888,
      download: 3,
      total: 907
    }
  }
}

Before, it looked like this:

2022-02-12T20:04:56.074Z	5b70e155-806f-4200-8a0c-d1c96e8c8f76	INFO	Failed on attempt 1 of 79 at February 12, 2022, 8:04:56 PM UTC with error: 
{
    "code": 400308191,
    "message": "Sorry! This reservation is not eligible for check in.",
    "messageKey": "ERROR__AIR_TRAVEL__BEFORE_CHECKIN_WINDOW",
    "header": null,
    "httpStatusCode": "BAD_REQUEST",
    "requestId": "6FDDCA70-8C3E-11EC-BB37-CB7B12CD8FBA:D8PqHfq2RICJcR-cVZ65IQ:mweb",
    "infoList": []
}

I think the old error message is much more useful and gives us southwest's human readable message. This could help signal to us if our headers are expired or something, or if the checkin is just not ready yet.

There was a bit of changes to how the project actually did the request to the checkin api, so we'd likely need to figure out how to log the entire message based on the "new" way.

Don't allow a user to schedule a checkin that they've already scheduled

Support for round-trip flights

This service has been working well for me recently, thank you again! I found one issue however - it looks like the current logic will always select the earliest leg of a reservation. For a round-trip flight, this can be problematic, since we need to schedule the check-in for the return flight, but the earliest leg of the reservation is the outbound flight.

Would it be reasonable to limit validLegs in findFirstLegDate to flight legs that are in the future?

Development Environment Issues

Hey,

Since I'd like to get everything nice and cleaned up as I'm working on this project, I want my development environment to be as "error" free as possible.

When running npm run compile, I get some TS errors:

npm run compile

> @sw-tools/[email protected] compile /Users/aaronpham/repos/checkin-service
> rm -rf dist && tsc

src/lib/sw-generate-headers.ts:27:30 - error TS7006: Parameter 'request' implicitly has an 'any' type.

27   page.on('requestfinished', request => {
                                ~~~~~~~
src/lib/sw-generate-headers.ts:51:22 - error TS7006: Parameter 'request' implicitly has an 'any' type.

51   page.on('request', request => {
                        ~~~~~~~
Found 2 errors.

Also, in VSCode, .eslintrc.js complains with this error:

Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser.
The file does not match your project config: .eslintrc.js.
The file must be included in at least one of the projects provided.

I fixed the above error by adding .eslintrc.js into my tsconfig.json:

{
  "extends": "@tsconfig/node16/tsconfig.json",
  "compileOnSave": true,
  "compilerOptions": {
    "lib": ["es2021", "dom"],
    "esModuleInterop": true,
    "checkJs": false,
    "allowJs": false,
    "strictNullChecks": false,
    "noImplicitAny": true,
    "useUnknownInCatchVariables": false,
    "outDir": "dist",
    "removeComments": true
  },
  "include": [
    "src",
    ".eslintrc.js"
  ]
}

Are you seeing any of these issues as well? Or is it related to my environment?

Notifications for successful or failed checkin

Are you interested in adding text or email notifications using Amazon SNS?

I have a few friends using this checkin service with my frontend, and this is a highly requested feature. I may start working on this soon and will create a PR if interested!

makeFetchCheckinDataAttempts is taking more than 250 ms between request

Hey,

One thing that I'd like in an automated checkin is knowing that it is faster than me as a human. I notice that in the logs for successful checkins, we are retrying about every 1200 ms instead of 250 ms.

I assume the method will wait for a response each time, before retrying.

This may cause the checkin to be slightly slower than a human (I'll be testing this soon! Have a flight coming up with friends and I'll see who gets the better seat!)

I don't really understand how this function works, but is there anyway we can improve it to actually retry every 250ms?

Logs below for a checkin, with the timestamps.

2022-02-08T09:54:55.103-08:00	2022-02-08T17:54:55.103Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Starting fetch checkin data attempts at February 8, 2022, 5:54:55 PM UTC

2022-02-08T09:54:56.335-08:00	2022-02-08T17:54:56.335Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 1 of 79 at February 8, 2022, 5:54:56 PM UTC with error: {"code":400308191,"message":"Sorry! This reservation is not eligible for check in.","messageKey":"ERROR__AIR_TRAVEL__BEFORE_CHECKIN_WINDOW","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:HLNvijr2Rd-J0_jN4wYfdQ:mweb","infoList":[]}

2022-02-08T09:54:57.489-08:00	2022-02-08T17:54:57.489Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 2 of 79 at February 8, 2022, 5:54:57 PM UTC with error: {"code":400308191,"message":"Sorry! This reservation is not eligible for check in.","messageKey":"ERROR__AIR_TRAVEL__BEFORE_CHECKIN_WINDOW","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:igJRoWOAR9mhm5IPopOfRw:mweb","infoList":[]}

2022-02-08T09:54:58.986-08:00	2022-02-08T17:54:58.986Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 3 of 79 at February 8, 2022, 5:54:58 PM UTC with error: {"code":400308191,"message":"Sorry! This reservation is not eligible for check in.","messageKey":"ERROR__AIR_TRAVEL__BEFORE_CHECKIN_WINDOW","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:W8DxRxxsR0-aK4rffe16sg:mweb","infoList":[]}

2022-02-08T09:55:00.183-08:00	2022-02-08T17:55:00.183Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 4 of 79 at February 8, 2022, 5:55:00 PM UTC with error: {"code":400308192,"message":"This flight is currently not ready for check-in. Please try again momentarily.","messageKey":"ERROR__AIR_TRAVEL__NOT_OPEN","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:OhvnWYMKSPaG8oYA3_XfpQ:mweb","infoList":[]}

2022-02-08T09:55:01.341-08:00	2022-02-08T17:55:01.341Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 5 of 79 at February 8, 2022, 5:55:01 PM UTC with error: {"code":400308192,"message":"This flight is currently not ready for check-in. Please try again momentarily.","messageKey":"ERROR__AIR_TRAVEL__NOT_OPEN","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:8yIM279MSZeivl6jx-2UdA:mweb","infoList":[]}

2022-02-08T09:55:02.634-08:00	2022-02-08T17:55:02.633Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 6 of 79 at February 8, 2022, 5:55:02 PM UTC with error: {"code":400308192,"message":"This flight is currently not ready for check-in. Please try again momentarily.","messageKey":"ERROR__AIR_TRAVEL__NOT_OPEN","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:r73KbXEeRVqib9ZWKsTj8Q:mweb","infoList":[]}

2022-02-08T09:55:03.877-08:00	2022-02-08T17:55:03.876Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 7 of 79 at February 8, 2022, 5:55:03 PM UTC with error: {"code":400308192,"message":"This flight is currently not ready for check-in. Please try again momentarily.","messageKey":"ERROR__AIR_TRAVEL__NOT_OPEN","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:o0w862NyQ3msfktAcawv9Q:mweb","infoList":[]}

2022-02-08T09:55:05.209-08:00	2022-02-08T17:55:05.209Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Failed on attempt 8 of 79 at February 8, 2022, 5:55:05 PM UTC with error: {"code":400308192,"message":"This flight is currently not ready for check-in. Please try again momentarily.","messageKey":"ERROR__AIR_TRAVEL__NOT_OPEN","header":null,"httpStatusCode":"BAD_REQUEST","requestId":"9FC88DE0-8907-11EC-BE80-6DDA96DCCD60:hHu9X5xoTQ6CeL-wLvnrPg:mweb","infoList":[]}

2022-02-08T09:55:06.904-08:00	2022-02-08T17:55:06.903Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Checking in at February 8, 2022, 5:55:06 PM UTC

2022-02-08T09:55:08.991-08:00	2022-02-08T17:55:08.991Z e7ae96a5-bab7-43ad-9eda-489fa7faee43 INFO Checkin succeeded

After checkin, remove the associated EventBridge Rule and EventBridge Trigger

We shouldn't leave around these old resources after we've already checked into a flight.

We should have the EventBridge rule target a second queue.

The existing queue is consumed by an existing Lambda function which performs the checkin.
The new queue should be consumed by a new Lambda function which removes the associated EventBridge Rule and EventBridge Trigger.

Puppeteer timeout on checkin

I set this up recently and it has worked properly for one flight, but I just got this in my error log today (the flight checkin time is 1:35 UTC, so it woke up at the appropriate time):

START RequestId: a4483461-12d6-5170-b06f-6e900d55117c Version: $LATEST

2022-05-08T01:30:50.748Z	a4483461-12d6-5170-b06f-6e900d55117c	INFO	Received SQS message {
  reservation: {
    confirmation_number: 'XXXXX',
    first_name: 'Removed',
    last_name: 'Name'
  },
  checkin_available_epoch: 1651973700
}

2022-05-08T01:31:23.709Z	a4483461-12d6-5170-b06f-6e900d55117c	ERROR	TimeoutError: Navigation timeout of 30000 ms exceeded
    at /opt/nodejs/node_modules/puppeteer-core/lib/cjs/puppeteer/common/LifecycleWatcher.js:106:111

2022-05-08T01:31:23.749Z	a4483461-12d6-5170-b06f-6e900d55117c	ERROR	Invoke Error 	{
    "errorType": "TimeoutError",
    "errorMessage": "Navigation timeout of 30000 ms exceeded",
    "name": "TimeoutError",
    "stack": [
        "TimeoutError: Navigation timeout of 30000 ms exceeded",
        "    at /opt/nodejs/node_modules/puppeteer-core/lib/cjs/puppeteer/common/LifecycleWatcher.js:106:111"
    ]
}

END RequestId: a4483461-12d6-5170-b06f-6e900d55117c

REPORT RequestId: a4483461-12d6-5170-b06f-6e900d55117c	Duration: 33007.91 ms	Billed Duration: 33008 ms	Memory Size: 1600 MB	Max Memory Used: 529 MB	Init Duration: 562.47 ms

Seems to be just an issue with the initial API token page not loading fast enough through puppeteer. I'm unsure if this means the API token page changed or if there just needs to be a retry configured for this part of the process.

Create some endpoints to query a user's active scheduled checkins

429 Error with SW API

It appears that SW has added a request limiter or something, I had two check-ins fail today.

2022-09-13T18:10:09.748Z	a52a7626-6b2b-5e56-a22f-2dbca386fb0f	INFO	Checking in at September 13, 2022, 6:10:09 PM UTC
2022-09-13T18:10:10.183Z	8152a56e-485c-52dd-9b59-fef480f49c0c	ERROR	Invoke Error 	
{
    "errorType": "HTTPError",
    "errorMessage": "Response code 429 (Too Many Requests)",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "name": "HTTPError",
    "timings": {
        "start": 1663092609742,
        "socket": 1663092609743,
        "lookup": 1663092609743,
        "connect": 1663092609752,
        "secureConnect": 1663092609762,
        "upload": 1663092609762,
        "response": 1663092610135,
        "end": 1663092610135,
        "phases": {
            "wait": 1,
            "dns": 0,
            "tcp": 9,
            "tls": 10,
            "request": 0,
            "firstByte": 373,
            "download": 0,
            "total": 393
        }
    },
    "stack": [
        "HTTPError: Response code 429 (Too Many Requests)",
        "    at Request.<anonymous> (/var/task/node_modules/got/dist/source/as-promise/index.js:117:42)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
    ]
}

2022-09-13T18:10:10.016Z	a52a7626-6b2b-5e56-a22f-2dbca386fb0f	ERROR	Invoke Error 	
{
    "errorType": "HTTPError",
    "errorMessage": "Response code 429 (Too Many Requests)",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "name": "HTTPError",
    "timings": {
        "start": 1663092609751,
        "socket": 1663092609752,
        "lookup": 1663092609752,
        "connect": 1663092609759,
        "secureConnect": 1663092609767,
        "upload": 1663092609767,
        "response": 1663092609962,
        "end": 1663092609962,
        "phases": {
            "wait": 1,
            "dns": 0,
            "tcp": 7,
            "tls": 8,
            "request": 0,
            "firstByte": 195,
            "download": 0,
            "total": 211
        }
    },
    "stack": [
        "HTTPError: Response code 429 (Too Many Requests)",
        "    at Request.<anonymous> (/var/task/node_modules/got/dist/source/as-promise/index.js:117:42)",
        "    at runMicrotasks (<anonymous>)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
    ]
}

I had about 50~ or so request happen before the server responded and I got the error. I think a possible fix may be to not start checking in 5 seconds before and maybe even doing the check-ins in longer intervals. In all my check-ins history, it's never checked-in before the open time.

On average, it's usually about 8 seconds after check-in is suppose to open, but as short as 4 seconds after.

If too many checkins are scheduled PolicyLengthExceededException occurs

I noticed this bug when testing my frontend with my flights. It appears that Lambda functions have a max policy length and if there are too many checkins schedule (therefore, many permissions on the HandleScheduledCheckin), we will get PolicyLengthExceededException and weird behavior occurs.

Not really sure how we'd fix this, as this may be a limitation with the lambda functions. Also, since I assume this checkin service is mainly for personal use, this issue is unlikely to occur.

Here's a stack trace from my lambda logs.

2022-02-06T23:17:34.025Z	a2b7e2c6-c73f-4501-be64-2d7fc4e101a3	ERROR	PolicyLengthExceededException: The final policy size (20719) is bigger than the limit (20480).
    at deserializeAws_restJson1AddPermissionCommandError (/var/task/node_modules/@aws-sdk/client-lambda/dist-cjs/protocols/Aws_restJson1.js:2235:41)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async /var/task/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:6:20
    at async /var/task/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:11:20
    at async StandardRetryStrategy.retry (/var/task/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)
    at async /var/task/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22
    at async handleInternal (/var/task/src/handlers/schedule-checkin.js:107:9)
    at async Runtime.handle [as handler] (/var/task/src/handlers/schedule-checkin.js:47:18) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: '239d16c5-7b35-4abe-8486-44a13d7c64cb',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  Type: 'User'
}

Unable to schedule checking due to failed Openflights.org request

Just noticed that scheduling a checkin fails with a 403 forbidden error in the AWS logs but not much information beyond that. I debugged it enough to determine that it's the openflights.org timezone request that's failing, and it seems to be due to openflights.org implementing some kind of cloudflare bot protection. At least for the time being, opening https://openflights.org/php/apsearch.php in postman or a new private browser window shows the cloudflare page.

I was thinking of just incorporating https://raw.githubusercontent.com/hroptatyr/dateutils/tzmaps/iata.tzmap for a local lookup as a workaround on my end for the time being, but I'm unsure if this openflights issue will be an ongoing issue or if there's possibly another API somewhere that can provide the same information.

Circular JSON structure error

I gave this project a spin a couple of days ago, and it looks like it ran into an issue in makeFetchCheckinDataAttempts. It tried once, received an error, and then it looks like it threw an error and stopped (so I went ahead and checked into my flight manually).

I deployed it to AWS Lambda per the README - happy to share any other information that may be helpful. This is a cool project, thanks for sharing this!

INFO Starting fetch checkin data attempts at December 31, 2021, XX:XX:XX PM UTC
INFO failed on attempt 1 of 80 at December 31, 2021, XX:X:XX PM UTC with error {
    "name": "HTTPError",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "timings": {
        "phases": {
            "wait": 1,
            "dns": 98,
            "tcp": 10,
            "tls": 10,
            "request": 0,
            "firstByte": 740,
            "download": 3,
            "total": 862
        }
    }
}
ERROR TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'TLSSocket'
    |     property '_httpMessage' -> object with constructor 'ClientRequest'
    --- property 'socket' closes the circle
    at JSON.stringify (<anonymous>)
    at handleInternal (/var/task/src/handlers/handle-scheduled-checkin.js:74:66)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Runtime.handle [as handler] (/var/task/src/handlers/handle-scheduled-checkin.js:43:18)
ERROR Invoke Error 	{
    "errorType": "TypeError",
    "errorMessage": "Converting circular structure to JSON\n    --> starting at object with constructor 'TLSSocket'\n    |     property '_httpMessage' -> object with constructor 'ClientRequest'\n    --- property 'socket' closes the circle",
    "stack": [
        "TypeError: Converting circular structure to JSON",
        "    --> starting at object with constructor 'TLSSocket'",
        "    |     property '_httpMessage' -> object with constructor 'ClientRequest'",
        "    --- property 'socket' closes the circle",
        "    at JSON.stringify (<anonymous>)",
        "    at handleInternal (/var/task/src/handlers/handle-scheduled-checkin.js:74:66)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
        "    at async Runtime.handle [as handler] (/var/task/src/handlers/handle-scheduled-checkin.js:43:18)"
    ]
}