Using js data

[kftang]

Alright, giving this a second go, this time much more refined.

Using data embedded in the response's script tags, I'm able to get much more data than what is parsed from the HTML. There's a few caveats, like having to parse some protobufs, but overall this is in a working state.

See the Original PR for more details on the structure of the js data.

$ uv run test_jsdata.py                                                                  
Best:
Itinerary(airline_code='NK', airline_names=['Spirit'], flights=[Flight(airline='NK', airline_name='Spirit', flight_number='436', operator=None, codeshares=None, aircraft='Airbus A320neo', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[6], arrival_time=[7, 28], travel_time=88, seat_pitch_short='28 in')], layovers=None, travel_time=88, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[6], arrival_time=[7, 28], itinerary_summary=ItinerarySummary(flights='NK436', price=145.98, currency='USD'))

Itinerary(airline_code='NK', airline_names=['Spirit'], flights=[Flight(airline='NK', airline_name='Spirit', flight_number='1371', operator=None, codeshares=None, aircraft='Airbus A320neo', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[10, 19], arrival_time=[11, 46], travel_time=87, seat_pitch_short='28 in')], layovers=None, travel_time=87, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[10, 19], arrival_time=[11, 46], itinerary_summary=ItinerarySummary(flights='NK1371', price=145.98, currency='USD'))

Itinerary(airline_code='DL', airline_names=['Delta'], flights=[Flight(airline='DL', airline_name='Delta', flight_number='3690', operator='SkyWest DBA Delta Connection', codeshares=None, aircraft='Embraer 175', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[13, 35], arrival_time=[15, 9], travel_time=94, seat_pitch_short='31 in')], layovers=None, travel_time=94, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[13, 35], arrival_time=[15, 9], itinerary_summary=ItinerarySummary(flights='DL3690', price=270.98, currency='USD'))

Itinerary(airline_code='WN', airline_names=['Southwest'], flights=[Flight(airline='WN', airline_name='Southwest', flight_number='2872', operator=None, codeshares=None, aircraft='Boeing 737MAX 8 Passenger', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[10, 35], arrival_time=[12, 10], travel_time=95, seat_pitch_short='32 in')], layovers=None, travel_time=95, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[10, 35], arrival_time=[12, 10], itinerary_summary=ItinerarySummary(flights='WN2872', price=290.96, currency='USD'))

Others:
Itinerary(airline_code='AS', airline_names=['Alaska'], flights=[Flight(airline='AS', airline_name='Alaska', flight_number='3443', operator='SkyWest Airlines as Alaska SkyWest', codeshares=[Codeshare(airline_code='HA', flight_number='7057', airline_name='Hawaiian')], aircraft='Embraer 175', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='San Diego International Airport', arrival_airport_name='SAN', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[18, 48], arrival_time=[20, 10], travel_time=82, seat_pitch_short='31 in'), Flight(airline='AS', airline_name='Alaska', flight_number='3427', operator='SkyWest Airlines as Alaska SkyWest', codeshares=[Codeshare(airline_code='HA', flight_number='6512', airline_name='Hawaiian')], aircraft='Embraer 175', departure_airport='SAN', departure_airport_name='San Diego International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[20, 55], arrival_time=[22, 15], travel_time=80, seat_pitch_short='31 in')], layovers=[Layover(minutes=45, departure_airport='SAN', departure_airport_name='San Diego International Airport', departure_airport_city='San Diego', arrival_airport='SAN', arrival_airport_name='San Diego International Airport', arrival_airport_city='San Diego')], travel_time=207, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[18, 48], arrival_time=[22, 15], itinerary_summary=ItinerarySummary(flights='AS3443|AS3427', price=232.0, currency='USD'))

Itinerary(airline_code='AS', airline_names=['Alaska'], flights=[Flight(airline='AS', airline_name='Alaska', flight_number='3429', operator='SkyWest Airlines as Alaska SkyWest', codeshares=[Codeshare(airline_code='HA', flight_number='7056', airline_name='Hawaiian')], aircraft='Embraer 175', departure_airport='SJC', departure_airport_name='San Jose Mineta International Airport', arrival_airport='San Diego International Airport', arrival_airport_name='SAN', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[14, 36], arrival_time=[15, 59], travel_time=83, seat_pitch_short='31 in'), Flight(airline='AS', airline_name='Alaska', flight_number='3427', operator='SkyWest Airlines as Alaska SkyWest', codeshares=[Codeshare(airline_code='HA', flight_number='6512', airline_name='Hawaiian')], aircraft='Embraer 175', departure_airport='SAN', departure_airport_name='San Diego International Airport', arrival_airport='Harry Reid International Airport', arrival_airport_name='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[20, 55], arrival_time=[22, 15], travel_time=80, seat_pitch_short='31 in')], layovers=[Layover(minutes=296, departure_airport='SAN', departure_airport_name='San Diego International Airport', departure_airport_city='San Diego', arrival_airport='SAN', arrival_airport_name='San Diego International Airport', arrival_airport_city='San Diego')], travel_time=459, departure_airport='SJC', arrival_airport='LAS', departure_date=[2025, 10, 4], arrival_date=[2025, 10, 4], departure_time=[14, 36], arrival_time=[22, 15], itinerary_summary=ItinerarySummary(flights='AS3429|AS3427', price=243.2, currency='USD'))
...

I was able to come up with a much better way to do reverse engineering for paths to keys. The Jupyter notebook should have everything you need to ramp up to what I'm trying to accomplish.

Also, any reason for supporting python >= 3.8? Using >=3.11 is very helpful for some better type hinting support