mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-06 13:26:11 -04:00
Compare commits
1 Commits
fix/libbac
...
bot/issue-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0bac2c3b3b |
@@ -333,49 +333,6 @@ _activateVenv() {
|
||||
unset PYTHONHOME
|
||||
}
|
||||
|
||||
# ensureUv makes sure the `uv` package manager is on PATH before we try to build
|
||||
# or populate a venv with it.
|
||||
#
|
||||
# Minimal base images (notably the nvidia-l4t variants) ship neither uv nor pip.
|
||||
# A backend installed onto a persistent /backends volume there can therefore
|
||||
# never build its venv: `uv venv` aborts with "uv: command not found" and
|
||||
# install.sh leaves a half-installed backend, with no way to recover from inside
|
||||
# the container. See mudler/LocalAI#10720.
|
||||
#
|
||||
# When uv is missing (and we are not in USE_PIP mode) bootstrap a private copy
|
||||
# into the backend directory itself, which lives on the same persistent volume,
|
||||
# so it survives image upgrades and is only downloaded once.
|
||||
function ensureUv() {
|
||||
# pip mode never invokes uv.
|
||||
if [ "x${USE_PIP}" == "xtrue" ]; then
|
||||
return 0
|
||||
fi
|
||||
if command -v uv >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
local uv_dir="${EDIR}/uv-bin"
|
||||
if [ -x "${uv_dir}/uv" ]; then
|
||||
export PATH="${uv_dir}:${PATH}"
|
||||
return 0
|
||||
fi
|
||||
echo "uv not found; bootstrapping a private copy into ${uv_dir}"
|
||||
mkdir -p "${uv_dir}"
|
||||
# The official installer picks the right binary for the current platform and
|
||||
# honors UV_INSTALL_DIR. This is the same mechanism the base images use to
|
||||
# install uv (see backend/Dockerfile.python). INSTALLER_NO_MODIFY_PATH keeps
|
||||
# it from editing shell profiles we don't control.
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="${uv_dir}" INSTALLER_NO_MODIFY_PATH=1 sh
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -qO- https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="${uv_dir}" INSTALLER_NO_MODIFY_PATH=1 sh
|
||||
else
|
||||
echo "ERROR: uv is missing and neither curl nor wget is available to bootstrap it" >&2
|
||||
return 1
|
||||
fi
|
||||
export PATH="${uv_dir}:${PATH}"
|
||||
command -v uv >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# ensureVenv makes sure that the venv for the backend both exists, and is activated.
|
||||
#
|
||||
# This function is idempotent, so you can call it as many times as you want and it will
|
||||
@@ -383,9 +340,6 @@ function ensureUv() {
|
||||
function ensureVenv() {
|
||||
local interpreter=""
|
||||
|
||||
# Make sure uv is available before any uv-based venv work below.
|
||||
ensureUv
|
||||
|
||||
if [ "x${PORTABLE_PYTHON}" == "xtrue" ] || [ -e "$(_portable_python)" ]; then
|
||||
echo "Using portable Python"
|
||||
ensurePortablePython
|
||||
|
||||
@@ -7,6 +7,20 @@ url = '/advanced/model-configuration'
|
||||
|
||||
LocalAI uses YAML configuration files to define model parameters, templates, and behavior. This page provides a complete reference for all available configuration options.
|
||||
|
||||
## Configuration scopes and precedence
|
||||
|
||||
[CLI flags and environment variables]({{% relref "reference/cli-reference" %}})
|
||||
configure the LocalAI server process. Model YAML files configure one model,
|
||||
while supported fields in an API request can override that model's defaults
|
||||
for that request. For example, a request containing `temperature` overrides
|
||||
the model YAML `parameters.temperature` only for that request.
|
||||
|
||||
Precedence is setting-specific rather than one universal ordering. For the
|
||||
overlapping `threads` setting, an explicit nonzero server `--threads` value is
|
||||
applied after model YAML and therefore wins over the YAML `threads` value.
|
||||
Most server flags have no model YAML equivalent, so consult the relevant
|
||||
reference for the scope of each setting.
|
||||
|
||||
## Overview
|
||||
|
||||
Model configuration files allow you to:
|
||||
|
||||
@@ -7,6 +7,9 @@ url = '/reference/cli-reference'
|
||||
|
||||
Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.
|
||||
|
||||
These options configure the LocalAI server process. To configure an individual
|
||||
model, see [Model Configuration]({{% relref "advanced/model-configuration" %}}).
|
||||
|
||||
> **Note:** All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See [.env files]({{%relref "advanced/advanced-usage#env-files" %}}) for configuration file support.
|
||||
|
||||
## Global Flags
|
||||
|
||||
Reference in New Issue
Block a user