Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout is enforced in completion #8

Open
mrepetto-certx opened this issue Jul 2, 2024 · 1 comment
Open

Timeout is enforced in completion #8

mrepetto-certx opened this issue Jul 2, 2024 · 1 comment

Comments

@mrepetto-certx
Copy link
Contributor

Client timeout is not passed in prompt_completion

From client.py

if include_sources is not OMIT:
            _request["include_sources"] = include_sources
        _response = self._client_wrapper.httpx_client.request(
            "POST",
            urllib.parse.urljoin(f"{self._client_wrapper.get_base_url()}/", "v1/completions"),
            json=jsonable_encoder(_request),
            headers=self._client_wrapper.get_headers(),
            timeout=60,
        )
@lrapp-adan
Copy link

lrapp-adan commented Sep 9, 2024

A fixed timeout of 60s is not only set for prompt_completion, but unfortunately also for multiple other functions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants