Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MITAB: Disable table fields "laundering" for non-neutral charset and add UTF-8 charset #10295

Merged
merged 7 commits into from
Jun 30, 2024

Conversation

drons
Copy link
Contributor

@drons drons commented Jun 25, 2024

Try to fix Toblerity/Fiona#1399

Unfortunately, I do not have a modern version of MapInfo to check the correctness.

@rouault
Copy link
Member

rouault commented Jun 25, 2024

@jratike80 Do you have access to recent MapInfo? If so it could be useful if you could provde a .mif/.mid and/or .tab using UTF-8 encoding

@rouault
Copy link
Member

rouault commented Jun 25, 2024

If so it could be useful if you could provde a .mif/.mid and/or .tab using UTF-8 encoding

and possibly with non-ASCII field names

@jratike80
Copy link
Collaborator

Do you have access to recent MapInfo?

No, we do not have any licenses any more.

@rouault
Copy link
Member

rouault commented Jun 25, 2024

@drons This doesn't build

@drons
Copy link
Contributor Author

drons commented Jun 25, 2024

NB! utf8 mid/mif was generated using iconv utility, requires verification with MapInfo.

@coveralls
Copy link
Collaborator

Coverage Status

coverage: 69.26% (+0.002%) from 69.258%
when pulling d06db89 on drons:mitabUTF8
into 08cbb83 on OSGeo:master.

@drons drons changed the title Draft: MITAB: Disable table fields "laundering" for non-neutral charset and add UTF-8 charset MITAB: Disable table fields "laundering" for non-neutral charset and add UTF-8 charset Jun 26, 2024
@drons
Copy link
Contributor Author

drons commented Jun 26, 2024

@rouault maybe std::iswalpha is a more suitable function for a "laundering"?

@rouault
Copy link
Member

rouault commented Jun 26, 2024

std::iswalpha is a more suitable function for a "laundering"?

I don't know what constraints MapInfo set, if any...

@drons
Copy link
Contributor Author

drons commented Jun 26, 2024

I don't know what constraints MapInfo set, if any...

There is a comment in GDAL code about oldie MapInfo fields name constraints. But there are weak constraints at resent MapInfo.

@rouault
Copy link
Member

rouault commented Jun 26, 2024

But there are weak constraints at resent MapInfo.

so maybe relax / remove laundering if we generate for a recent MapInfo version?

@drons
Copy link
Contributor Author

drons commented Jun 26, 2024

I'll think about testcase

@rouault
Copy link
Member

rouault commented Jun 26, 2024

I'll think about testcase

thanks, and some documentation for the new option in doc/source/drivers/vector/mitab.rst
By the way, I've just merged #10291 into master. It would be great if you could rebase your PR on top of it, and add the UTF-8 entry in the doc/source/drivers/vector/mapinfo_encodings.csv file

@coveralls
Copy link
Collaborator

Coverage Status

coverage: 68.847% (-0.4%) from 69.258%
when pulling 4782c39 on drons:mitabUTF8
into 08cbb83 on OSGeo:master.

@coveralls
Copy link
Collaborator

Coverage Status

coverage: 69.26% (-0.004%) from 69.264%
when pulling a02dab3 on drons:mitabUTF8
into c456677 on OSGeo:master.

@rouault rouault merged commit 8f5d4de into OSGeo:master Jun 30, 2024
35 checks passed
@rouault rouault added this to the 3.10.0 milestone Jun 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Field Name encode error when writing to Mapinfo File
4 participants