Open PPL Model Exchange (OPMX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. OPMX provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.
Currently OPMX focus on the capabilities and hardware friendliness needed for Large Language Model(LLM) inferencing.
- Support weight only quant model at llama at 16/07/2024.
- PMX has changed to OPMX at 25/04/2024.
- And the domain of operators are also changed to
opmx(refers to TOC). - You can find the old code at llm_v1
Table of Contents: Link
About add new operator: Link
About update an operator's version: Link
You can install opmx like:
pip install -e .
Then you can just use OPMX like a python api.
OPMX provides functional API based on torch.autograd.Function.
Clone the OPMX repo, and import torch_function like this:
import pmx_llm.torch_function as OPMXAnd then use it as Pytorch's functional API:
norm, skip_out = OPMX.skip_rms_norm(x, weight, skip_in, -1, eps)We can use these API in pytorch to custom your own model.
All OPMX function could be exported as custom operators by torch.onnx.export.
Some opensource model are provided in our model zoo.
Currently models: