Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

💎 Implement an Azure store #1542

Open
MarcusSorealheis opened this issue Dec 13, 2024 · 11 comments · May be fixed by #1554
Open

💎 Implement an Azure store #1542

MarcusSorealheis opened this issue Dec 13, 2024 · 11 comments · May be fixed by #1554
Labels
💎 Bounty

Comments

@MarcusSorealheis
Copy link
Collaborator

MarcusSorealheis commented Dec 13, 2024 •
edited
Loading

We need to implement Azure Blob Storage (Azure Store).
We can refer Amazon S3 Bucket Store code - nativelink-store/src/s3_store.rs as a reference.
@algora-pbc Algora PBC
Copy link

algora-pbc bot commented Dec 13, 2024 •
edited
Loading

💎 $2,000 bounty • TraceMachina

Steps to solve:

  1. Start working: Comment /attempt #1542 with your implementation plan
  2. Submit work: Create a pull request including /claim #1542 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to TraceMachina/nativelink!

Add a bounty • Share on socials

Attempt Started (GMT+0) Solution
🟢 @varshith257 Dec 14, 2024, 3:02:16 AM WIP
🟢 @amankrx Dec 23, 2024, 6:16:28 PM #1554

@aaronmondal aaronmondal pinned this issue Dec 13, 2024
@aaronmondal aaronmondal changed the title Implement an Azure store 💎 Implement an Azure store Dec 13, 2024
@aaronmondal
Copy link
Member

See also the discussion about a generalized BlobStore with different backends at #659.

@varshith257
Copy link

varshith257 commented Dec 14, 2024 •
edited by algora-pbc bot
Loading

/attempt #1542

Algora profile Completed bounties Tech Active attempts Options
@varshith257 24 bounties from 11 projects
Scala, Rust,
TypeScript & more
Cancel attempt

@Devking9099
Copy link

Devking9099 commented Dec 14, 2024 •
edited
Loading

Hello, I am Hunt
will solve this issue

@urbit-pilled
Copy link

@aaronmondal Is the generalized BlobStore part of the scope for this bounty?
It's unclear whether #659 should implement it or #1542 should implement it.

@MarcusSorealheis
Copy link
Collaborator Author

The Google Cloud Storage store gcs_store should be implemented first.

@varshith257
Copy link

varshith257 commented Dec 15, 2024 •
edited
Loading

@MarcusSorealheis Almost done with Azure Store. Is GC Store is blockage to this? I think no

@MarcusSorealheis
Copy link
Collaborator Author

gcs_store should be done be first.

@MarcusSorealheis
Copy link
Collaborator Author

But no, it is not a blocker. It just won't get reviewed as soon.

vishwamartur added a commit to vishwamartur/nativelink that referenced this issue Dec 23, 2024
@vishwamartur
Add Azure Blob Storage store
a045a43 
Related to TraceMachina#1542

Implement Azure Blob Storage (Azure Store) for NativeLink.

* **Add Azure Store Implementation**:
  - Create `nativelink-store/src/azure_store.rs` with Azure Blob Storage store implementation.
  - Use Azure SDK for Rust for interacting with Azure Blob Storage.
  - Implement `StoreDriver` trait for Azure Store.
  - Implement authentication using Azure SDK for Rust.
  - Implement chunked uploads for large files using `put_block` and `put_block_list` methods.

* **Update Store Factory**:
  - Modify `nativelink-store/src/default_store_factory.rs` to include Azure Store in the store factory function.

* **Add Unit Tests**:
  - Create `nativelink-store/tests/azure_store_test.rs` with unit tests for Azure Store.
  - Test basic CRUD operations, error handling, authentication process, and chunked upload functionality.

* **Update Dependencies**:
  - Modify `nativelink-store/Cargo.toml` to include necessary dependencies for Azure Blob Storage.

* **Update Documentation**:
  - Modify `README.md` to include information about the Azure store and how to configure it.
  - Add information about different types of blobs supported by the Azure store.
@vishwamartur vishwamartur mentioned this issue Dec 23, 2024
Draft
@amankrx
Copy link

amankrx commented Dec 23, 2024 •
edited by algora-pbc bot
Loading

/attempt #1542

Options

@amankrx amankrx linked a pull request Dec 26, 2024 that will close this issue
Draft
7 tasks
@algora-pbc Algora PBC
Copy link

algora-pbc bot commented Dec 26, 2024 •
edited
Loading

💡 @amankrx submitted a pull request that claims the bounty. You can visit your bounty board to reward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💎 Bounty
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement Azure Blob Store amankrx/nativelink
6 participants
@MarcusSorealheis @aaronmondal @amankrx @varshith257 @urbit-pilled @Devking9099