Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix missing coordinate text fields for decimal coordinates #5368

Draft
wants to merge 1 commit into
base: production
Choose a base branch
from

Conversation

grantfitzsimmons
Copy link
Member

@grantfitzsimmons grantfitzsimmons commented Nov 2, 2024

Fixes #5365

This essentially runs the following SQL statement:

# There are cases from Specify 6 where Longitude and Latitude are captured as decimal values but
# the accompanying text fields are not populated. In cases where the text fields are not populated,
# it appears the records do not have coordinates. This script will populate the text fields with the
# decimal values where the text fields are empty but the decimal values are not.

UPDATE locality
SET Lat1text = Latitude1
WHERE Lat1text IS NULL AND Latitude1 IS NOT NULL;

UPDATE locality 
SET Long1text = Longitude1 
WHERE Long1text IS NULL AND Longitude1 IS NOT NULL;

UPDATE locality
SET Lat2text = Latitude2
WHERE Lat2text IS NULL AND Latitude2 IS NOT NULL;

UPDATE locality 
SET Long2text = Longitude2 
WHERE Long2text IS NULL AND Longitude2 IS NOT NULL;

⚠️ Note: This PR affects database migrations. See migration testing instructions.

The migration is also fully reversible!

specify@feed99cd5b4e:/opt/specify7$ ve/bin/python manage.py migrate specify 0010_updateDelete_parentcojo
Operations to perform:
  Target specific migration: 0010_updateDelete_parentcojo, from specify
Running migrations:
  Rendering model states... DONE
  Unapplying specify.0012_coordinate_fields_fix... OK
  Unapplying specify.0011_cascading_tree_nodes... OK
specify@feed99cd5b4e:/opt/specify7$ 

Testing instructions

Some databases are known to have this issue due to migration artifacts and this comes up from time to time.

You can usually identify databases with this issue by running the following query on them:

SELECT *  FROM locality  WHERE Lat1text IS NULL AND Latitude1 IS NOT NULL;

If any records are returned, it's a great candidate for testing this migration.

Examples that I found on the test panel are available in this document.

On production before the migration

  • *Modify several locality records to remove values from the Lat1text and Long1text fields while preserving Latitude1 and Longitude1 values OR use one of the test databases mentioned in the earlier doc and find all records where Latitude1 exists but Lat1text does not (using Query Builder or other methods).
  • Verify that the LatLonUI plugin on Locality doesn't display any values.

On issue-5365 after the migration

  • Verify that all cases have been resolved after the migration has completed via Query Builder or SQL
  • Verify that the LatLonUI plugin on Locality now displays the correct values copied from the decimal coordinate fields

@grantfitzsimmons grantfitzsimmons added the Migration Prs that contain migration label Nov 2, 2024
@grantfitzsimmons grantfitzsimmons added this to the 7.9.x milestone Nov 2, 2024
class Migration(migrations.Migration):

dependencies = [
('specify', '0011_cascading_tree_nodes'),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Update this with the last current migration in the specify app before merge

Copy link
Member

@maxpatiiuk maxpatiiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Migration Prs that contain migration
Projects
Status: 📋Back Log
Development

Successfully merging this pull request may close these issues.

Latitude and Longitude coordinates without matching Lat1text and Long1text coordinate fields appear empty
3 participants