ppl.pmx is(but not only) a part of PPL.LLM system.

We recommend users who are new to this project to read the Overview of system.

PPL Model Exchange (PMX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. PMX provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.

Currently PMX focus on the capabilities and hardware friendliness needed for Large Language Model(LLM) inferencing.

Table of Contents: Link

About add new operator: Link

About update an operator's version: Link

PMX provides functional API based on torch.autograd.Function.

Clone the PMX repo, and import torch_function like this:

import pmx_llm.torch_function as PMX

And then use it as Pytorch's functional API:

norm, skip_out = PMX.skip_rms_norm(x, weight, skip_in, -1, eps)

We can use these API in pytorch to custom your own model.

All PMX function could be exported as custom operators by torch.onnx.export.

Some opensource model are provided in our model zoo.

Currently models:

• LLaMA 1/2
  • Exporting Facebook(Meta)'s LLaMA
  • Exporting Huggingfaces's LLaMA