Repositories list

• vllm-fork

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  Python

  •

  Apache License 2.0

  •4.5k•0•0•0•Updated Nov 6, 2024Nov 6, 2024

• vllm-hpu-extension

  Public
  Python

  •

  Apache License 2.0

  •7•0•0•0•Updated Oct 16, 2024Oct 16, 2024

• owlite-examples

  Public
  OwLite Examples repository offers illustrative example codes to help users seamlessly compress PyTorch deep learning models and transform them into TensorRT engines.

  Python

  •1•9•0•0•Updated Sep 27, 2024Sep 27, 2024

• owlite

  Public
  OwLite is a low-code AI model compression toolkit for AI models.

  Python

  •

  GNU Affero General Public License v3.0

  •3•38•0•0•Updated Sep 27, 2024Sep 27, 2024

• mlperf_inference_results_v4.0

  Public
  C++

  •

  Apache License 2.0

  •1•0•0•1•Updated Jul 23, 2024Jul 23, 2024

• .github

  Public
  0•0•0•0•Updated Jul 22, 2024Jul 22, 2024

• vllm-quick

  Public
  Python

  •

  Apache License 2.0

  •0•1•0•0•Updated Mar 13, 2024Mar 13, 2024

• QUICK

  Public
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

  Python

  •

  MIT License

  •5•112•5•0•Updated Mar 6, 2024Mar 6, 2024