Skip to content
This repository has been archived by the owner on Nov 16, 2021. It is now read-only.

Download from specific positions of a file across multiple farmers -> improves file transfer speed #33

Open
MeijeSibbel opened this issue Aug 24, 2017 · 3 comments

Comments

@MeijeSibbel
Copy link

This is something that in combination with node benchmarking (selection based on for example bandwidth) would reduce the transfer time and increase transfer speed. Bittorrent does this well.

Reference: storj-archived/core#705

@braydonf
Copy link
Contributor

braydonf commented Aug 25, 2017

This should improve the performance towards the end of a file transfer where it's typical to see a decrease in speed when the concurrency decreases due to there only being a few shards left.

@RGBKey
Copy link

RGBKey commented Aug 25, 2017

Am I incorrect in stating that streaming files from the Storj network would greatly benefit from this?

@braydonf
Copy link
Contributor

braydonf commented Aug 25, 2017

Yes it could allow for better performance and bursts an specific positions. The issue with streaming though is that there can be wildly inconsistent behavior. If the first shard happens to be the shard than has gone missing and will need to be recovered, with reed solomon, it will be necessary to download enough data to be able to recover that shard, which would be a noticeable delay. I believe it's possible to do partial recovery without downloading the entire file, however it will still be more data than the size of the first shard.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants