Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert_model command not found #173

Open
pint1022 opened this issue May 23, 2023 · 3 comments
Open

convert_model command not found #173

pint1022 opened this issue May 23, 2023 · 3 comments

Comments

@pint1022
Copy link

Hello
get docker image 0.6.0. Just tried to run the two demo command:

  1. docker run -it --rm --gpus all
    -v $PWD:/project ghcr.io/els-rd/transformer-deploy:0.6.0
    bash -c "cd /project &&
    convert_model -m "philschmid/MiniLM-L6-H384-uncased-sst2"
    --backend tensorrt onnx
    --seq-len 16 128 128"
    got the error: convert_model not found

  2. tried the 2nd command to use triton inference server. the service is started fine; the query command got the error:
    curl -X POST http://localhost:8000/v2/models/transformer_onnx_inference/versions/1/infer
    --data-binary "@demo/infinity/query_body.bin"
    --header "Inference-Header-Content-Length: 161"
    {"error":"Request for unknown model: 'transformer_tensorrt_inference' is not found"}

Does infinity download huggingface models and convert to Triton format?

thanks

@sc0eur
Copy link

sc0eur commented Jul 31, 2023

I also got the convert_model not found.

I think pip3 install ".[GPU]" ... was lost somewhere in the latest Dockerfile update: link

I was able to run convert_model after manually running pip install ".[GPU]" inside the container

@jingzhaoou
Copy link

I ran into the same error and fixed it as suggested. That is, manually run pip install ".[GPU]" inside the container.

@aidanrussell
Copy link

aidanrussell commented May 2, 2024

This issue is still there? Can't someone fix it?
EDIT: it seems this repository is not being maintained. It is a shame!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants