ProgressBar with callback not work

[se7enXF]

• I have searched to see if a similar issue already exists.

Describe the solution you'd like

I use aws s3 to upload file, wan to use progress bar to show progress. s3.upload_file have a Callback function param to indicate upload bytes.
I try the follow code but progress bar not move, file is uploaded to s3 successfully:

def upload_file(ecs: EcsInfo, local_path: str, key: str, progress=gr.Progress()):
    file_size = os.stat(local_path).st_size
    progress(0, total=file_size, desc=f"Start upload {Path(local_path).name} to {ecs.bucket_name}/{key}")

    upload_msg = f"Uploading {Path(local_path).name} to {ecs.bucket_name}/{key}"
    t = progress.tqdm(range(file_size), desc=upload_msg)

    def update_progress(value):
        t.update(value)
        # print(value)  # you can unomment this line to show callback value

    s3 = init_ecs(ecs)
    s3.upload_file(Filename=local_path, Key=key, Bucket=ecs.bucket_name, Callback=update_progress)

    new_string = f"{ecs.endpoint_url}/{ecs.bucket_name}/{key}"
    return new_string

I try the for loop example in gradio docs about progress bar and it works. How to work in callback way?

[se7enXF]

I also try with track_tqdm=True when using tqdm in function, there is no progress bar in web but in terminal.

def upload_file(ecs: EcsInfo, local_path: str, key: str, progress=gr.Progress(track_tqdm=True)):
    file_size = os.stat(local_path).st_size
    with tqdm.tqdm(total=file_size) as bar:
        s3 = init_ecs(ecs)
        s3.upload_file(Filename=local_path, Key=key, Bucket=ecs.bucket_name, Callback=bar.update)
    new_string = f"{ecs.endpoint_url}/{ecs.bucket_name}/{key}"
    return new_string

[abidlabs]

Hi @se7enXF please provide a minimal code example that we can use to reproduce the issue above. See: https://stackoverflow.com/help/minimal-reproducible-example

[se7enXF]

Hi @se7enXF please provide a minimal code example that we can use to reproduce the issue above. See: https://stackoverflow.com/help/minimal-reproducible-example

Thanks for reply. Please change the ECS info at the begining of function upload_file and run the script. The progress bar still not move in gradio website.

import os
import tqdm
import boto3
import gradio as gr
from pathlib import Path
from typing import NamedTuple


class EcsInfo(NamedTuple):
    endpoint_url: str
    bucket_name: str
    access_key: str
    secret_key: str


def init_ecs(ecs: EcsInfo):
    s3 = boto3.client(
        service_name="s3",
        aws_access_key_id=ecs.access_key,
        aws_secret_access_key=ecs.secret_key,
        endpoint_url=ecs.endpoint_url,
        region_name="cn-north-1",
        verify=False,
    )
    return s3


def upload_file(local_path: str, key: str, progress=gr.Progress(track_tqdm=True)):
    e = EcsInfo(
        "endpoint_url",
        "bucket_name",
        "access_key",
        "secret_key",
    )
    file_size = os.stat(local_path).st_size
    print(f"Uploading {local_path} to {key}")

    # method 1
    # progress(0, total=file_size, desc=f"Start upload {Path(local_path).name} to {e.bucket_name}/{key}")
    # t = progress.tqdm(range(file_size), desc=f"Uploading {Path(local_path).name} to {e.bucket_name}/{key}")
    # s3 = init_ecs(e)
    # s3.upload_file(Filename=local_path, Key=key, Bucket=e.bucket_name, Callback=t.update)

    # method 2
    with tqdm.tqdm(total=file_size, unit="B", unit_scale=True, desc=f"Uploading {Path(local_path).name}") as bar:
        s3 = init_ecs(e)
        s3.upload_file(Filename=local_path, Key=key, Bucket=e.bucket_name, Callback=bar.update)

    new_string = f"bucket_name/{key}"
    return new_string

# input text0 is local file path
# input text1 is s3 bucket key
gr.Interface(upload_file,["text", "text"],"text").launch()

Python 3.9 with gradio==4.44.1

[abidlabs]

Hi @se7enXF first of all, could you please upgrade to a more recent Gradio version (we only actively maintain 5.x). This issue may already have been resolved.

If not, can you please post a repro that does not require creating a boto client with an access key, etc. A standalone repro please (see: https://stackoverflow.com/help/minimal-reproducible-example)

[se7enXF]

Hi @abidlabs I make a fake function to simulate s3 upload callback. The follow code works well, but not work with boto3. I also try by python3.12 with gradio==5.9.1, meeting the same issue. I have no idea about solving this problem.

import time
import tqdm
from typing import Callable
import gradio as gr


def fake_ecs_upload(total: int, callback: Callable[[int], None]):
    n = 0
    step = total // 10
    while n < total:
        n += step
        callback(step)
        time.sleep(0.5)


def upload_file1(file_size: str, progress=gr.Progress()):
    file_size = int(file_size)

    # method 1
    t = progress.tqdm(range(file_size), desc=f"Uploading {file_size}")
    fake_ecs_upload(file_size, t.update)

    new_string = f"Uploaded {file_size}"
    return new_string


def upload_file2(file_size: str, progress=gr.Progress(track_tqdm=True)):
    file_size = int(file_size)

    # method 2
    with tqdm.tqdm(total=file_size, unit="B", unit_scale=True, desc=f"Uploading {file_size}") as bar:
        fake_ecs_upload(file_size, bar.update)

    new_string = f"Uploaded {file_size}"
    return new_string


with gr.Blocks() as demo:
    with gr.Row():
        in_0 = gr.Text(value="100")
        ot_0 = gr.Textbox()
        bt_0 = gr.Button(value="Progress Bar")
    with gr.Row():
        in_1 = gr.Text(value="100")
        ot_1 = gr.Textbox()
        bt_1 = gr.Button(value="Track tqdm")
    bt_0.click(upload_file1, [in_0], [ot_0])
    bt_1.click(upload_file2, [in_1], [ot_1])
demo.launch()

[abidlabs]

That's strange! I tried the repro you provided, as you mentioned, it works fine for me locally. Perhaps the boto3 callback is not working as expected? We'll need a standalone repro if we are to proceed with this issue