[Feature] HdfsStorage and LocalStorage's TableLocation should be full URI with scheme #121
Open
2 of 8 tasks
Labels
feat
New feature or request
Willingness to contribute
{"label"=>"Yes. I can contribute a fix for this bug independently.", "value"=>"independent"}
Feature Request Proposal
Currently HDFSStorage and LocalStorage set TableLocation without scheme, ie:
/data/openhouse/db/tbl-UUID
instead of full URIhdfs:///data/openhouse/db/tbl-UUID
.This was done to ensure backward compatibility, replication and migration.
Iceberg spec mandates a full-URI (containing scheme) as part of its metadata. source: https://iceberg.apache.org/spec/?ref=blog.min.io#manifests
As part of this bugFix, HDFSStorage and LocalStorage should start using fully-qualified URI.
Motivation
Making this change will generalize tableLocation behavior for various Storages such as s3, hdfs, local etc.
What component(s) does this feature affect?
Table Service
: This is the RESTful catalog service that stores table metadata.:services:tables
Jobs Service
: This is the job orchestrator that submits data services for table maintenance.:services:jobs
Data Services
: This is the jobs that performs table maintenance.apps:spark
Iceberg internal catalog
: This is the internal Iceberg catalog for OpenHouse Catalog Service.:iceberg:openhouse
Spark Client Integration
: This is the Apache Spark integration for OpenHouse catalog.:integration:spark
Documentation
: This is the documentation for OpenHouse.docs
Local Docker
: This is the local Docker environment for OpenHouse.infra/recipes/docker-compose
Other
: Please specify the component.Details
No response
The text was updated successfully, but these errors were encountered: