You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The CRAWLER_FULL_PAGE_ARCHIVE is very useful but with a lots of saved pages can lead to a very large storage usage. I was thinking if some external archiving system could be used for having some sort of archiving feature without using local storage, but instead saving on site like Wayback Machine (or other similar sites) and then referencing the saved pages link in Hoarder.
Describe the benefits this would bring to existing Hoarder users
Saving a copy the resources without using local storage.
Can the goal of this request already be achieved via other means?
No, or I didn't know it.
Have you searched for an existing open/closed issue?
I have searched for existing issues and none cover my fundamental request
Additional context
No response
The text was updated successfully, but these errors were encountered:
archive.fo comes to mind but I guess you'd have to check whether they allow "non-human" calls to archive a website.
I'm using CRAWLER_FULL_PAGE_ARCHIVE too, especially since we cannot count on archive.org to be available all the time (see the current attacks on the site). I'd be helpful if the archived websites wouldn't be so enormous: I have only 98 websites archived atm but they take up 9.05 GB of space. That's kinda mind boggling - the underlying archive system (monolith iirc ) is know to produce (imho unnecessary) large website archives.
Describe the feature you'd like
The CRAWLER_FULL_PAGE_ARCHIVE is very useful but with a lots of saved pages can lead to a very large storage usage. I was thinking if some external archiving system could be used for having some sort of archiving feature without using local storage, but instead saving on site like Wayback Machine (or other similar sites) and then referencing the saved pages link in Hoarder.
Describe the benefits this would bring to existing Hoarder users
Saving a copy the resources without using local storage.
Can the goal of this request already be achieved via other means?
No, or I didn't know it.
Have you searched for an existing open/closed issue?
Additional context
No response
The text was updated successfully, but these errors were encountered: