Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to import flights from flightmemory.com #1398

Closed
jb1991dd opened this issue Jul 24, 2023 · 11 comments
Closed

Unable to import flights from flightmemory.com #1398

jb1991dd opened this issue Jul 24, 2023 · 11 comments
Labels
import Issues related to data import (e.g. CSV, FlightMemory). server Things that likely require server access for deployment

Comments

@jb1991dd
Copy link

Unfortunatelly it is not importing my flights from flightmemory.com.
For years I never had any issues. But this time it says:
Upload successful. Parsing...
Results
Tmpfile phpkgOCAs

Any ideas anyone?

@jb1991dd jb1991dd changed the title Unabke to import flights from flightmemory.com Unable to import flights from flightmemory.com Jul 24, 2023
@reedy reedy added import Issues related to data import (e.g. CSV, FlightMemory). server Things that likely require server access for deployment labels Jul 25, 2023
@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

Do you know offhand when it last worked?

I don't have access to the server to check the tmpfile....

@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

And can you provide a copy of your HTML?

@jb1991dd
Copy link
Author

Around 20th May 2023 it was the last time I imported flights.
Attached 2 HTML (one short with only 3 flights - maybe easier to find the problem, one long with 50 flights)
FlightMemory - FlightData.txt
FlightMemory - FlightData2.txt

@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

I signed up for flightmemory and just put one simple flight on, that worked...

Your small file worked fine

Screenshot 2023-07-26 at 17 29 53

Your larger one is rejected because BER doesn't exist in the default dataset (but will exist on openflights.org)... Will add it manually for my dev install (and file a followup issue), and see where we are...

@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

The short file has some logspam (on initial parse, even if it worked)...

"PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618"

The longer file has much more...


"PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null

"; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/ww" while reading upstream, client: 10.13.37.49, server: 10.13.37.36, request: "POST /php/import.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm.sock:", host: "10.13.37.36", referrer: "https://10.13.37.36/html/import"

"w/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_b

"; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt" while reading upstream, client: 10.13.37.49, server: 10.13.37.36, request: "POST /php/import.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm.so

" to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618" while reading upstream, client: 10.13.37.49, server: 10.13.37.36, request: "POST /php/import.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm.sock:", host: "10.13.37.36", referrer: "https://10.13.37.36/html/import"

"; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/impo

"ble $trid in /var/www/openflights/php/import.php on line 618" while reading upstream, client: 10.13.37.49, server: 10.13.37.36, request: "POST /php/import.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm.sock:", host: "10.13.37.36", referrer: "https://10.13.37.36/html/import"

"; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warni

"; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warni

"; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Attempt to read property "textContent" on null in /var/www/openflights/php/import.php on line 36; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; P

"; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618" while reading upstream, client: 10.13.37.49, server: 10.13.37.36, request: "POST /php/import.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php8.2-fpm.sock:", host: "10.13.37.36", referrer: "https://10.13.37.36/html/import"
"; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 567; PHP message: PHP Warning:  Undefined variable $trip_bgcolor in /var/www/openflights/php/import.php on line 617; PHP message: PHP Warning:  Undefined variable $trid in /var/www/openflights/php/import.php on line 618"

Probably explains at least some of the issues on openflights.org...

@jb1991dd
Copy link
Author

jb1991dd commented Jul 26, 2023

Interesting that it worked for you. I looked in my old backup and tried to upload an old file which I saved 2020, which always worked fine in the past.
But I have the same issue now. So either it is openflights which has a 'new import version' or it is browser based problem (But I already tried different ones on different computers)

@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

I haven't actually imported the large one yet, as I've gone down rabbit holes of fixing other issues.

If it was a browser based problem, it would be in the HTML that was saved. The parsing is done "server side" in PHP...

It's also possible whatever the issue was may be fixed in the repo, but not deployed yet. Or it's related to the config being used on the server, and all those warnings are upsetting things.

They are both broken on openflights.org for me, but locally on my dev environment I definitely can at least get past the first pass... And fully import the first, the second I can't because of BER being missing in my dataset (various tasks spun off).

I'm gonna try and clean up some of those warnings and then we might need to get server logs, and/or wait for #1391 and see if it's fixed on openflights.org post deployment.

reedy added a commit that referenced this issue Jul 26, 2023
@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

Can you eyeball this image for me?

Screenshot 2023-07-26 at 22 05 29

Does it look like it has everything it should have?

reedy added a commit that referenced this issue Jul 26, 2023
* No registration and/or no name/title for the plane
* No aiport name, just the City/Town and Country
* Fall back to returning '' in nth_text() if we can't find what we're looking for

More towards #1398
@reedy
Copy link
Collaborator

reedy commented Jul 26, 2023

I'm suspecting there may be some different handling with 36da5e2 than before, and that maybe wasn't deployed when you last tried... It was as of late June.

That second commit just pushed fixes up the other logspam that I pasted above.

It's basically due to it expecting things like plane registrations, plane "names", airport "names"... All of which can be missing as evident by your dataset.

I've added some extra fallback handling so that if this happens on other fields, it will just return an empty string quietly... ish...

@jb1991dd
Copy link
Author

Your image with all flights looks perfect.
Thank you for your help.

@reedy
Copy link
Collaborator

reedy commented Jul 28, 2023

Great. I'm gonna call this fixed then; it should be deployed in the near future.

Feel free to reopen or file a new issue if there are still issues after updated code is deployed!

@reedy reedy closed this as completed Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
import Issues related to data import (e.g. CSV, FlightMemory). server Things that likely require server access for deployment
Projects
None yet
Development

No branches or pull requests

2 participants