Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot bulk download many files from remote onedrive folder #881

Open
ChNousias opened this issue Jul 24, 2024 · 0 comments
Open

Cannot bulk download many files from remote onedrive folder #881

ChNousias opened this issue Jul 24, 2024 · 0 comments

Comments

@ChNousias
Copy link

Hi, I have the following issue when attempting to download files from a remote onedrive folder. The folder has about ~1800 items, with each item about the size of 5 MB. When attempting to list all files in the folder and download each file separately as in the code below, I usually get a 503 error around the 100th item or so. Even if I use .execute_query_retry() I seldom get beyond the 200th item.:

drive_items = remote_folder.children.get_all().execute_query()

for drive_item in drive_items:

    if drive_item.file is None:
        continue

    local_path = os.path.join(local_folder, drive_item.name)

    with open(local_path, "wb") as local_file:
        drive_item.download(local_file).execute_query()

If I try to download the whole folder as a zip file, as in:

with open(zip_path, "wb") as f:
    remote_folder.download_folder(f, print_progress).execute_query_retry()

the download finishes after downloading the first 200 folder items (which it seems originates from using the folder.children.get() method which for some reason fetches only the first 200 items).

Is this a bug? I guess I can circumvent this, but is there some reason for not allowing downloading more than 200 items from a remote folder at once?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant