Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Implement Instance snapshots #381

Closed
wants to merge 5 commits into from

Conversation

nesitor
Copy link
Member

@nesitor nesitor commented Jul 14, 2023

Feature: Upload VM snapshots using Aleph STORE message.
Feature: Restore existing snapshots as STORE Aleph message.

@nesitor nesitor requested review from hoh and odesenfans July 14, 2023 20:47
@nesitor nesitor self-assigned this Jul 14, 2023
@nesitor nesitor requested a review from odesenfans July 15, 2023 09:30
@nesitor nesitor changed the title Upload snapshots to Aleph Feature: Implement Instance snapshots Jul 27, 2023
Andres D. Molins and others added 5 commits September 19, 2023 17:36
* Feature: Restore existing snapshot from reference.

* Feature: Forget old snapshots also before delete the file.

* Fix: Use python SDK instead the connector to retrieve the snapshot messages by ref.

* Fix: Fixed code quality issues.

* Fix: Save snapshots in the correct path.

* Fix: Retrieve the most recent snapshot and fix un-compression and mounting issues.

* Fix: Code quality issues.

* Fix: Fixed missing aioipfs dependency.

* Fix: Downgraded aioipfs version.

* Fix: Added missing python SDK dependency.

* Fix: Added missing wheel dependency for Debian 12.

* Fix: Fixing dependency error for Debian 12.

* Fix: Fixing dependency error for Debian 12.

* Fix: Build dependency error for Debian 12.

---------

Co-authored-by: Andres D. Molins <[email protected]>
@hoh hoh force-pushed the andres-feature-store-snapshot branch from fa1e8a7 to 46c837d Compare September 19, 2023 15:36
# Use local ipfs node by default. TODO: Add a general setting to allow to customize it
client = aioipfs.AsyncIPFS(host="localhost", port=5001)
ipfs_hash = None
async for added_file in client.add(str(path), recursive=True):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this recursive if the goal is to upload a single file ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it accepts different files, and it return different results.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice I only see it used in one place, snapshot_hash = await ipfs_upload_file(self.path), with a single file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mean that the client.add method call, it accepts different files, and return different results.

async def send_forget_ipfs_message(
item_hash: ItemHash, reason: Optional[str] = ""
) -> None:
pkey = get_fallback_private_key()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should never be used in production: the creation of the key should be explicit and during installation. A key that is missing should cause an error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can get it?

@nesitor nesitor requested a review from hoh September 25, 2023 13:11
@hoh
Copy link
Member

hoh commented Dec 11, 2023

I am closing this PR since we are facing performance issues in this implementation and want to evaluate the use of Qcow2 as a replacement.

@hoh hoh closed this Dec 11, 2023
@Psycojoker Psycojoker deleted the andres-feature-store-snapshot branch July 24, 2024 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants