mirror of
https://github.com/bentoml/OpenLLM.git
synced 2026-02-01 19:32:55 -05:00
4cad367ab5
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
68 lines
2.6 KiB
Markdown
68 lines
2.6 KiB
Markdown
# Adding a New Model
|
|
|
|
OpenLLM encourages contributions by welcoming users to incorporate their custom
|
|
Large Language Models (LLMs) into the ecosystem. You can set up your development
|
|
environment by referring to our
|
|
[Developer Guide](https://github.com/bentoml/OpenLLM/blob/main/DEVELOPMENT.md).
|
|
|
|
## Procedure
|
|
|
|
All the relevant code for incorporating a new model resides within
|
|
[`src/openllm/models`](./src/openllm/models/) `model_name` in snake_case.
|
|
Here's your roadmap:
|
|
|
|
- [ ] Generate model configuration file:
|
|
`src/openllm/models/{model_name}/configuration_{model_name}.py`
|
|
- [ ] Establish model implementation files:
|
|
`src/openllm/models/{model_name}/modeling_{runtime}_{model_name}.py`
|
|
- [ ] Create module's `__init__.py`:
|
|
`src/openllm/models/{model_name}/__init__.py`
|
|
- [ ] Adjust the entrypoints for files at `src/openllm/models/auto/*` If it is a
|
|
new runtime, then add it a `src/openllm/models/auto/modeling_{runtime}_auto.py`.
|
|
See the other auto runtime for example.
|
|
- [ ] Modify the main `__init__.py`: `src/openllm/models/__init__.py`
|
|
- [ ] Run the following to update stubs: `hatch run check-stubs`
|
|
|
|
For a working example, check out any existing model.
|
|
|
|
### Model Configuration
|
|
|
|
File Name: `configuration_{model_name}.py`
|
|
|
|
This file is dedicated to specifying docstrings, default prompt templates,
|
|
default parameters, as well as additional fields for the models.
|
|
|
|
### Model Implementation
|
|
|
|
File Name: `modeling_{runtime}_{model_name}.py`
|
|
|
|
For each runtime, i.e., torch (default with no prefix), TensorFlow -`tf`, Flax -
|
|
`flax`, it is necessary to implement a class that adheres to the `openllm.LLM`
|
|
interface. The conventional class name follows the `RuntimeModelName` pattern,
|
|
e.g., `FlaxFlanT5`.
|
|
|
|
### Initialization Files
|
|
|
|
The `__init__.py` files facilitate intelligent imports, type checking, and
|
|
auto-completions for the OpenLLM codebase and CLIs.
|
|
|
|
### Entrypoint
|
|
|
|
After establishing the model config and implementation class, register them in
|
|
the `auto` folder files. There are four entrypoint files:
|
|
|
|
- `configuration_auto.py`: Registers `ModelConfig` classes
|
|
- `modeling_auto.py`: Registers a model's PyTorch implementation
|
|
- `modeling_tf_auto.py`: Registers a model's TensorFlow implementation
|
|
- `modeling_flax_auto.py`: Registers a model's Flax implementation
|
|
|
|
### Updating README.md
|
|
|
|
Run `./tools/update-readme.py` to update the README.md file with the new model.
|
|
|
|
## Raise a Pull Request
|
|
|
|
Once you have completed the checklist above, raise a PR and the OpenLLMs
|
|
maintainer will review it ASAP. Once the PR is merged, you should be able to see
|
|
your model in the next release! 🎉 🎊
|