Comments (9)
That fixed it! Thanks for your help with debugging this.
from auto-southwest-check-in.
Okay, I fixed this. The weird Google requests that were in there were causing the script to parse the wrong request for account name.
def get_flights(self, flight_retriever: AccountFlightRetriever) -> Dict[str, Any]:
"""
Logs into the flight retriever account to retrieve a list of scheduled flights.
Since valid headers are produced, they are also grabbed and updated in the check-in
scheduler. Last, if the account name is not set, it will set it based on the response
information.
"""
driver = self._get_driver()
logger.debug("Logging into Southwest account to get scheduled flights and valid headers")
# Log in to retrieve the account's trips and needed headers for later requests
WebDriverWait(driver, 30).until(EC.invisibility_of_element((By.CLASS_NAME, "dimmer")))
WebDriverWait(driver, 30).until(
EC.element_to_be_clickable((By.CLASS_NAME, "login-button--box"))
).click()
username_element = WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.NAME, "userNameOrAccountNumber"))
)
username_element.send_keys(flight_retriever.username)
password_element = WebDriverWait(driver, 30).until(
EC.presence_of_element_located((By.NAME, "password"))
)
password_element.send_keys(flight_retriever.password)
password_element.submit()
self._set_headers_from_request(driver)
for request_made in driver.requests:
if request_made.url == "https://mobile.southwest.com/api/security/v4/security/token":
response = request_made.response
if response.status_code != 200:
error = self._handle_login_error(response)
raise error
# If this is the first time logging in, the account name needs to be set
# because that info is needed later
if flight_retriever.first_name is None:
logger.debug("First time logging in. Setting account name")
response_body = json.loads(response.body)
self._set_account_name(flight_retriever, response_body)
print(
f"Successfully logged in to {flight_retriever.first_name} "
f"{flight_retriever.last_name}'s account\n"
) # Don't log as it contains sensitive information
# This page is also loaded when we log in, so we might as well grab it instead of
# requesting again later
for request_made in driver.requests:
if request_made.url == "https://mobile.southwest.com/api/mobile-misc/v1/mobile-misc/page/upcoming-trips":
flights = json.loads(request_made.response.body)["upcomingTripsPage"]
print(flights)
driver.quit()
return [flight for flight in flights if flight["tripType"] == "FLIGHT"]
Rewriting the get_flights
method to ensure the proper request is being parsed by checking the request url in webdriver.py
fixes the issue.
from auto-southwest-check-in.
The _set_headers_from_request
method in webdriver.py
also needs to have the URL validated. If any other URLs are present, it won't load the proper headers. So the above fix fixes the error of retrieving the name for the account, the below fixes the next error which was retrieving flight information.
def _set_headers_from_request(self, driver: Chrome) -> None:
# Retrieving the headers could fail if the form isn't given enough time to submit
time.sleep(10)
logger.debug("Setting valid headers from previous request")
for request_made in driver.requests:
if request_made.url == "https://mobile.southwest.com/api/mobile-misc/v1/mobile-misc/page/upcoming-trips":
break
request_headers = request_made.headers
self.checkin_scheduler.headers = self._get_needed_headers(request_headers)
I'm an amateur programmer so I don't know if this is the best way to do it, but this lets me get the right headers. With the fixes above, I'm able to successfully set my account name and retrieve my flight information.
from auto-southwest-check-in.
Thanks for going into so much detail when reporting this (and even providing a fix)!
It appears that your webdriver is capturing requests that we don’t want to capture. The driver.scopes
variable (line 145 of lib/webdriver.py) is supposed to filter these out (more information on what that variable does here).
Does this issue still arise on the master branch?
from auto-southwest-check-in.
Thanks for going into so much detail when reporting this (and even providing a fix)!
It appears that your webdriver is capturing requests that we don’t want to capture. The
driver.scopes
variable (line 145 of lib/webdriver.py) is supposed to filter these out (more information on what that variable does here).Does this issue still arise on the master branch?
Happy to help!
I can confirm the issue is still present on the master branch as well.
from auto-southwest-check-in.
Can you add the following lines on line 111 of lib/webdriver.py
(before response = driver.requests[0].response
) and paste the output?
for r in driver.requests:
print(r.url)
For me, the only two URLs that show up are
https://mobile.southwest.com/api/security/v4/security/token
https://mobile.southwest.com/api/mobile-misc/v1/mobile-misc/page/upcoming-trips
Can you also verify that the driver.scopes
is set on line 145? This is the line that handles only capturing desired requests.
from auto-southwest-check-in.
https://accounts.google.com/ListAccounts?gpsia=1&source=ChromiumBrowser&json=standard
https://clientservices.googleapis.com/chrome-variations/seed?osname=win&channel=stable&milestone=112
https://clients2.google.com/service/update2/crx?os=win&arch=x86&os_arch=x86_64&nacl_arch=x86-64&prod=chromecrx&prodchannel=&prodversion=112.0.5615.121&lang=en-US&acceptformat=crx3&x=id%3Defaidnbmnnnibpcajpcglclefindmkaj%26v%3D0.0.0.0%26installedby%3Dexternal%26uc%26ping%3Dr%253D-1%2526e%253D1&x=id%3Dghbmnnjooekpmoecnnnilnnbdlolhkhi%26v%3D0.0.0.0%26installedby%3Dinternal%26uc%26ping%3Dr%253D-1%2526e%253D1&x=id%3Dgjgfobnenmnljakmhboildkafdkicala%26v%3D0.0.0.0%26installedby%3Dexternal%26uc%26ping%3Dr%253D-1%2526e%253D1&x=id%3Dljglajjnnkapghbckkcmodicjhacbfhk%26v%3D0.0.0.0%26installedby%3Dexternal%26uc%26ping%3Dr%253D-1%2526e%253D1&x=id%3Dnmmhkkegccagdldgiimedpiccmgmieda%26v%3D0.0.0.0%26installedby%3Dother%26uc%26ping%3Dr%253D-1%2526e%253D1
https://mobile.southwest.com/api/security/v4/security/token
https://mobile.southwest.com/api/mobile-misc/v1/mobile-misc/page/upcoming-trips
These are the URLs being captured.
I can also confirm driver.scopes is set.
def _get_driver(self) -> Chrome:
logger.debug("Starting webdriver for current session")
chromedriver_path = self.checkin_scheduler.flight_retriever.config.chromedriver_path
chrome_version = self.checkin_scheduler.flight_retriever.config.chrome_version
driver = Chrome(
driver_executable_path=chromedriver_path,
options=self.options,
seleniumwire_options=self.seleniumwire_options,
version_main=chrome_version,
)
driver.scopes = [LOGIN_URL, TRIPS_URL, RESERVATION_URL] # Filter out unneeded URLs
logger.debug("Loading Southwest Check-In page")
driver.get(CHECKIN_URL)
return driver
from auto-southwest-check-in.
I appreciate you taking your time to debug this and try my suggestions. It appears that the scopes are working. However, Chrome is possibly requesting to Google when it is initialized. Could you add del driver.requests
before the driver.scopes? This will ensure all requests are removed before setting the URL scopes.
from auto-southwest-check-in.
Awesome. Thanks for testing it out
from auto-southwest-check-in.
Related Issues (20)
- Failed to log in to account with username XXXXX. Reason: Invalid credentials. Status code: 400. Passwords with non-alphanumeric characters? HOT 11
- Element click intercepted: Login Button is not clickable (Docker) HOT 13
- ModuleNotFoundError: No module named 'requests' HOT 19
- Forbidden 403 HOT 58
- Encountered a Too Many Requests error while logging in. Skipping reservation retrieval HOT 3
- AttributeError: 'WebDriver' object has no attribute 'add_cdp_listener' HOT 1
- Hang during setting headers_set HOT 7
- Failed to check in due to local time being ahead HOT 17
- Hangs at "Loading Southwest Check-In page" HOT 24
- ModuleNotFoundError: No module named 'requests' HOT 3
- Add devcontainer.json configuration to run the project on GitHub Codespaces
- Continuously hitting 403 error HOT 1
- Add jitter to requests time(s) HOT 1
- Add account / reservation level notification settings HOT 2
- Introduce a custom POST endpoint for "other" notifications HOT 1
- Headers not present on some requests HOT 12
- Reservation monitor hung with lock HOT 6
- Check-in attempt occurred 3 minutes late (didn't sleep computer) HOT 8
- Occasionally getting NTPExceptions - uncaught HOT 7
- "ERROR__AIR_TRAVEL__BEFORE_CHECKIN_WINDOW" on returning flight in a round trip reservation HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from auto-southwest-check-in.