-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
move GeoInterface conversions to a separate lightweight package #91
Comments
Hi @disberd , thanks for raising the issue here. Appreciate your attempts to improve the ecosystem.
Can you please confirm that you are creating CountriesBorders.jl because you don't want to take GeoArtifacts.jl as a dependency (which depends on GeoIO.jl)? As explained in Zulip, the Natural Earth dataset is already available as a
It seems to me that your main concern with the current situation is the heavy GeoIO.jl dependency. I would like to ask you again: are you concerned with the heavy dependency in CountriesBorders.jl or in some other user-facing app that you are building that depends on CountriesBorders.jl? In the former case, I'd argue that CountriesBorders.jl is not needed, especially if you simply use/contribute to GeoArtifacts.jl (please let me know if there are feature differences other than just loading the boundaries). In the latter case, your user-facing app will be compiled anyways, and you will be able to load many other formats in the future, without manual intervention. Issue #23 is something that we need to solve in parallel. Contributions are welcome.
We do share your second concern very much. We've been working really hard over the years to get rid of this GDAL madness that is spread across various programming languages. Our ultimate goal is to completely remove this dependency in GeoIO.jl. However, this removal depends on the existence of more native Julia packages to load file formats that are not supported yet. We should probably make a list of the missing formats to make this road map more explicit. In summary, and answering to your request: we do not envision a promising direction with a separate package for geometry conversions. |
Thanks for the quick reply. Indeed my problem is that CountriesBorders is a dependency (direct or indirect) of a number of private packages we develop inernally for work. The problem with the huge GeoIO dependency is that precompilation does not really help in CI where everything is recompiled again from scratch (we use GitLab at work with a self hosted instance so I do not have access to all the julia actions on github that might help with caching). I am working a new PR to allow a more recent version of Meshes here: For the time being, considering your reply about not being willing to move this outside of GeoIO, it seems like I can just live with adding some internal compat based on the files from GeoIO.jl as done here: I added the full copyright from this repo in the comment at the beginning of the file, let me know if this is sufficient or if I have to handle the copyright differently. EDIT: @juliohm Regarding how this differs from what is available in GeoArtifacts.NaturalEarth, not much in the raw content but I do export functionality in CountriesBorders to allow easy selection of specific countries and downfiltering based on region/name. |
As a side comment, I remember that our precompilation time in GeoIO.jl increased massively with the addition of one single format (maybe it was geoparquet?). The issue #23 needs to be investigated by someone with more time and compiler skills. Perhaps, solving the issue can solve your huge CI times as a side effect.
It is perfectly fine :) We are happy that you can find your way into the source code and adapt it to your needs.
That would be super nice to have as a feature in GeoArtifacts.jl for example. This convenience options can help other users in the future. If you have them isolated, and could easily port in your free time, that would be great. Will close the issue, but feel free to continue the discussion here. |
Thanks for confirming I could reuse the code as proposed. Regarding the filtering functionality I will try to see if it can be added once we stabilize the API a bit more. We also have a core internal package that implemented a lot of things that are now in CoordRefSystem for dealing with visibility between satellites and points on ground. Overall I think you are doing a very commendable effort into trying to harmonize these fields in Julia and I would like to re-use and contribute back to the ecosystem when possible :) |
That would be lovely. CoordRefSystems.jl unlocks various paths of innovation that were previously locked by PROJ. Feedback and improvements are very welcome.
Appreciate your kind words. Despite the stubbornness of very few people, we are seeing more community members sharing our long-term vision. |
Hello,
I am opening this issue to understand whether there will be willingness (or if there is already an alternative) to move the functionality to translate geometries implementing the interface from GeoInterface into a separate much more lightweight package.
From what I see this is fully enclosed within the
conversions.jl
file and would only need to depend on Meshes.jl and GeoInterface.jl as a very bare minimum if removing the 4 added methods depending onAbstractGeoTable
:GeoIO.jl/src/conversion.jl
Lines 40 to 43 in 8c0eb84
Alternatively and probably more meaningful within the JuliaEarth ecosystem would be to include GeoTables as dependency and maybe also provide the contents of src/utils.jl to allow easily creating a GeoTable as output from this hypotetical new basic package.
My use case is wanting to provide some basic functionality relying on Meshes and GeoTables in one small package of mine https://github.com/disberd/CountriesBorders.jl where I just want to transform the data related to countries coordinates into a Meshes compatible format so that I can exploit all of the nice functionalities of JuliaEarth.
I had already opened a discussion on zulip on the matter here, where I explained that for the package I did not want to depend on GeoIO.jl since this directly depends on all the possible packages dealing with the formats GeoIO supports, which especially due to GDAL (at least in my experience) can cause issues on windows and long precompilation times.
I understand that things are getting better with precompilation but I still rather not have this huge list of indirect dependencies for some vary basic functionality I know will only depend on one specific geojson file.
The discussion in the zulip above led to the suggestion of just generating the table and saving it as binary, but this comes at the risk that the loaded binary format will be very coupled to the specific version of Meshes and GeoTable that generated it.
I think a solution where the file is simply read with GeoJSON.jl (much lighter than GeoIO.jl) and then converted to a GeoTable upon loading is much cleaner.
I have checked a bit the internals here and indeed it would seem to be rather easy to "excise" this functionality into a separate package that GeoIO would depend on.
This would also open a much easier path for other developers in my shoes to adopt the Meshes ecosystem, and if this was to be included inside JuliaEarth it would give more confidence on its support and it being kept up-to-date with changes to GeoTables and Meshes.
I also believe this work of compatibility maintenance would not be higher than what it currently is as this code already lives in GeoIO which is part of the ecosystem and the effort would just be moved from GeoIO to this new light GeoInterfaceConversion package.
Would this be something you would be open to consider?
The text was updated successfully, but these errors were encountered: