Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Implement TableOperations-specific FileIO #240

Open
1 of 8 tasks
raymondlam12 opened this issue Oct 28, 2024 · 0 comments
Open
1 of 8 tasks

[Feature] Implement TableOperations-specific FileIO #240

raymondlam12 opened this issue Oct 28, 2024 · 0 comments
Labels
feat New feature or request

Comments

@raymondlam12
Copy link

Willingness to contribute

{"label"=>"Yes. I can contribute a fix for this bug independently.", "value"=>"independent"}

Feature Request Proposal

In order to implement FileIOs that are TableOperations-specific, creating a FileIO instance on a per TableOperations-basis is required.

This is similar to how other Iceberg-compliant Catalogs are implemented: apache/iceberg#10893 .

Motivation

What is the use case for this feature?

This will be required to make OpenHouse data-access-tokens work.
Different tables will require different data-access-tokens to be used by its FileIO implementation.

What component(s) does this feature affect?

  • Table Service: This is the RESTful catalog service that stores table metadata. :services:tables
  • Jobs Service: This is the job orchestrator that submits data services for table maintenance. :services:jobs
  • Data Services: This is the jobs that performs table maintenance. apps:spark
  • Iceberg internal catalog: This is the internal Iceberg catalog for OpenHouse Catalog Service. :iceberg:openhouse
  • Spark Client Integration: This is the Apache Spark integration for OpenHouse catalog. :integration:spark
  • Documentation: This is the documentation for OpenHouse. docs
  • Local Docker: This is the local Docker environment for OpenHouse. infra/recipes/docker-compose
  • Other: Please specify the component.

Details

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant