Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OGC Features API at /catalogs/{catalogId}/collections/{collectionId}/features #265

Merged
merged 34 commits into from
Sep 25, 2024

Conversation

recalcitrantsupplant
Copy link
Collaborator

@recalcitrantsupplant recalcitrantsupplant commented Sep 10, 2024

This PR implements the OGC Features API as a subapi within Prez. It has been implemented at: "/catalogs/{catalogId}/collections/{recordsCollectionId}"

The material changes are:

  • Add profiles/endpoints definitions specific to OGC Features under prez/reference_data/profiles|endpoints
  • Add OGC Features Sub Application (similar to adding a Router)
  • Add time and bbox as query parameters for filtering generally, reusing code from CQL implementation where possible.
  • Add /prefixes endpoint
  • Add dependency on rdf2geojson library to support returning geojson mediatype
  • A breaking change to rename the "per_page" query parameter generally to "limit" for conformance with OGC Features and to prevent having two differently named query parameters for one function.

Most of the other changes are renaming things to prevent conflicts, and the addition of a set of CQL queries for testing.

NB the implementation is done as a mount of a separate FastAPI application NOT a FastAPI router. This is so that the Features API can be a "pure" OGC Features API and automatically include docs at a sensible location i.e. /features/docs, and make catering to the specific requirements of OGC Features easier.

The main "data" endpoints are listed below for clarity. A full list of endpoints including management (prefixes, curies), search etc. has been added to the main Readme.

Prez Core Endpoints:

Endpoint Default Profile Default MT
/ N/A text/anot+turtle
/docs OGC API text/html
/catalogs/{catalogId} OGC API text/anot+turtle
/catalogs/{catalogId}/collections OGC API text/anot+turtle
/catalogs/{catalogId}/collections/{recordsCollectionId} OGC API text/anot+turtle
/catalogs/{catalogId}/collections/{recordsCollectionId}/items OGC API text/anot+turtle
/catalogs/{catalogId}/collections/{recordsCollectionId}/items/{itemId} OGC API text/anot+turtle

OGC Features API Endpoints:

The OGC Features API Endpoints are based at the ROOT /catalogs/{catalogId}/collections/{recordsCollectionId}/

Endpoint Default Profile Default MT
{ROOT}/features N/A application/json
{ROOT}/features/docs N/A text/html
{ROOT}/features/conformance N/A application/json
{ROOT}/features/collections OGC Features Minimal Profile application/json
{ROOT}/features/collections/{collectionId} OGC Features Minimal Profile application/json
{ROOT}/features/collections/{collectionId}/items OGC Features All Properties Profile application/geo+json
{ROOT}/features/collections/{collectionId}/items/{featureId} N/A (Query Specified) application/geo+json

…rate application such that it has its own docs and can function as a standalone OGC Features API
…tional (so it can render Sites, FoIs, and other non-Occurrence geo:Features we might have"
Update profile to make props separately optional
Add link headers to JSON responses
…xigraph his supported. Remove BDR specific query.

Optimise imports. Fix 4*tests
…-subapi

# Conflicts:
#	ogcfeaturesapi.Dockerfile
#	prez/services/query_generation/cql.py
#	pyproject.toml
@ashleysommer
Copy link
Contributor

I'm reviewing this now...

@recalcitrantsupplant recalcitrantsupplant marked this pull request as ready for review September 11, 2024 22:47
@lalewis1
Copy link
Collaborator

Note: need to pin the version in pyproject.toml to 3.12.4

I tried installing dependencies with 3.12.5 and cannot install scalene. tried install with 3.11.9 and scalene installs but spaceprez endpoint tests fail due to conflicts with rdflib version. under 3.12.4 everything works as expected.

Looking at the ogctest results I can see that only the crs tests and apidefinition tests fail.
the crs tests fail simply because the api does not declare that it conforms to the crs conformance class at /conformance. the apidefinition tests are failing because the test suite only supports openapi spec v3.0 and prez is using v3.1. (this is an issue with the ogc test suite and I have added a ticket there to add support for other openapi versions)

Issue

Browsing http://localhost:8000/catalogs/ex:DemoCatalog/collections/ex:GeoDataset/features/collections says that no collections are availalable. even though there should be a collection. This is using the ogc_features.ttl test data.

@lalewis1
Copy link
Collaborator

using the updated test data / profile, the collections are now listed and viewable but there is still an issue accessing the individual features.

i.e.

curl --silent localhost:8000/catalogs/ex:DemoCatalog/collections/ex:GeoDataset/features/collections/ex:FeatureCollection/items/ex:Feature1

yields {} but should show the details of the feature.

in addition, putting an invalid feature id like 1 yields a 500 response, expected a 404.

@recalcitrantsupplant
Copy link
Collaborator Author

Thanks I'll check the tests covers those

@lalewis1
Copy link
Collaborator

using the updated test data / profile, the collections are now listed and viewable but there is still an issue accessing the individual features.

i.e.

curl --silent localhost:8000/catalogs/ex:DemoCatalog/collections/ex:GeoDataset/features/collections/ex:FeatureCollection/items/ex:Feature1

yields {} but should show the details of the feature.

in addition, putting an invalid feature id like 1 yields a 500 response, expected a 404.

To give some direction for this.

When requesting the details of ex:Feature1, In the debugger I can see that the item_graph in the ogc_features_object_function only has one triple declaring that the type of ex:Feature1 is a geo:Feature. I feel like this should have more info, so perhaps there is an issue with the query generation. also, the convert function for converting to geojson reduces this single triple down to an empty {} object so there is something not right there too.

hope that helps.

@recalcitrantsupplant
Copy link
Collaborator Author

recalcitrantsupplant commented Sep 24, 2024

Note: need to pin the version in pyproject.toml to 3.12.4

I tried installing dependencies with 3.12.5 and cannot install scalene. tried install with 3.11.9 and scalene installs but spaceprez endpoint tests fail due to conflicts with rdflib version. under 3.12.4 everything works as expected.

Looking at the ogctest results I can see that only the crs tests and apidefinition tests fail. the crs tests fail simply because the api does not declare that it conforms to the crs conformance class at /conformance. the apidefinition tests are failing because the test suite only supports openapi spec v3.0 and prez is using v3.1. (this is an issue with the ogc test suite and I have added a ticket there to add support for other openapi versions)

Issue

Browsing http://localhost:8000/catalogs/ex:DemoCatalog/collections/ex:GeoDataset/features/collections says that no collections are availalable. even though there should be a collection. This is using the ogc_features.ttl test data.

With the CRS one I'll create a separate issue (#267). In short, and Prez will likely support by proxy whichever CRSs the triplestore GeoSPARQL implementation supports, so we probably just need a way to discover that (might be hard) and then expose this info in the various places in the OGC Features API. If this is too hard, we could just create a config option to allow manual specification of different CRSs. It's currently specifying WGS84 as a temporary solution. I'd prefer not to mark it compliant until we've gone through the separate spec in detail.

Not sure what is going on with the python versions. I'll remove scalene as a dev dependency.

couldn't get pytest to run from the command line
…C Features, add geometries to SpacePrez data so it renders. Add links to display of a single feature.
Copy link
Collaborator

@lalewis1 lalewis1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome looks good to me. nice work.

@recalcitrantsupplant recalcitrantsupplant merged commit 4dc3b78 into main Sep 25, 2024
1 check passed
@recalcitrantsupplant recalcitrantsupplant deleted the david/features-subapi branch September 25, 2024 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants