mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 17:18:49 -04:00
mudler-agentandEttore Di Giacinto
c60a2e47d6
* feat(grpc): add a Forecast RPC for time-series models Add Forecast to the backend service with columnar request and response messages: flat row-major arrays, NaN for missing values, and one message per series so a caller can send series of different shapes in one call. The response keeps the request order. Wire the RPC through the server, client, embedded client, connection evicting client, in-flight tracking client and the base backend, which answers unimplemented. core/backend gains a Forecast helper that stamps the model identity and records a trace, like the other single-purpose RPCs. Assisted-by: Claude:claude-sonnet-5-5 go-vet * feat(config): add the forecast usecase FLAG_FORECAST marks a model as a time-series forecaster. The timestral-cpp backend declares it as its default usecase, so a config with that backend needs no extra lines. The backend is also listed as a non-text backend, so a forecast model is never offered as a chat model. It shows up in the gallery filter and in the capability list. Assisted-by: Claude:claude-sonnet-5-5 go-vet * feat(api): add POST /v1/forecast Take one or more numeric series and return quantile trajectories for the next prediction_length steps. Targets are [variates][steps] or a flat array for one variate; JSON null marks a missing value. Past and known covariates and categorical codes are checked for length before the backend sees them. Quantiles default to 0.1, 0.5 and 0.9 in the backend. The route is gated by a new default-on forecast API feature. The endpoint is listed in the swagger spec, the /api/instructions index, the UI model filter and the capability symbols. Assisted-by: Claude:claude-sonnet-5-5 go-vet * feat(backends): add the timestral-cpp forecast backend Wrap the timestral.cpp C ABI with purego, the way the other ggml backends do: libtimestral is built from a pinned commit with ggml embedded, and the Go binary loads it at start. Forecast validates the counts against the array lengths, groups the series by shape, makes one library call per group and puts the results back in the order the caller sent them. Quantiles default to 0.1, 0.5 and 0.9, and a level the model does not have is refused with the list of valid ones. A mutex guards the handle because the library is not reentrant. Request size is bounded by two model options. The unit specs use a fake library. The integration specs and a gRPC spec run against the 5M F32 model that test.sh downloads and checks against a fixed SHA-256. Assisted-by: Claude:claude-sonnet-5-5 go-vet * build(backends): register timestral-cpp in the build and the library Add the CPU image to the backend matrix for amd64 and arm64, plus an Apple silicon entry that is not built or tested yet. The image has no GPU keys because the engine runs on the CPU only. Wire the make targets, the path filter and the dependency bump bot. The library entries were generated with backendcatalog. The images are not published yet, so the release entries are unpinned until a maintainer runs --repin. The gallery reader test now counts the names in the library instead of a fixed number, so adding a backend does not break it. Assisted-by: Claude:claude-sonnet-5-5 go-vet * feat(gallery): add the Timestral forecast models Add Timestral 5M, 28M, 123M and 209M as F32 entries, each with its F16 build as a variant. Q8_0 and the K-quants are left out until the quality question is settled. Each entry runs on the timestral-cpp backend, on the CPU only. The weights use Mistral's Modified MIT licence, which bars use by companies above a monthly revenue threshold. The first paragraph after the summary says so, the licence file is installed next to the weights, and the revenue-restricted-license tag makes the entries filterable. The memory each build needs has not been measured, so no size is set. Assisted-by: Claude:claude-sonnet-5-5 * docs(forecast): document /v1/forecast and the Timestral models Add a feature page with the request and response shapes, the error cases, the model options and the licence note. List the engine in the README, the compatibility table and the backend categories. Assisted-by: Claude:claude-sonnet-5-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
81 lines
3.1 KiB
Makefile
81 lines
3.1 KiB
Makefile
# timestral-cpp backend Makefile.
|
|
#
|
|
# The upstream pin is TIMESTRAL_VERSION below, in this file so the dependency
|
|
# bump bot can find and update it.
|
|
#
|
|
# Local dev shortcut: if you already have an out-of-tree timestral.cpp build,
|
|
# symlink the .so + header into this directory and skip the clone/cmake steps:
|
|
#
|
|
# ln -sf /path/to/timestral.cpp/build-shared/libtimestral.so .
|
|
# ln -sf /path/to/timestral.cpp/include/timestral_capi.h .
|
|
# go build -o timestral-cpp-grpc .
|
|
#
|
|
# The default target does the clone-at-pin and cmake build, so CI needs no
|
|
# side checkout.
|
|
|
|
TIMESTRAL_VERSION?=6c0ee070de54e9b2611ea90696dee3a41178700c
|
|
TIMESTRAL_REPO?=https://github.com/mudler/timestral.cpp
|
|
|
|
GOCMD?=go
|
|
GO_TAGS?=
|
|
JOBS?=$(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
|
|
|
|
NATIVE?=false
|
|
|
|
# Build ggml statically into libtimestral (PIC) so the shared library is
|
|
# self-contained: dlopen needs no libggml*.so next to it, only system libs
|
|
# (libstdc++, libgomp, libc) that the runtime image already provides.
|
|
#
|
|
# The engine runs on the CPU only. BUILD_TYPE is therefore not forwarded: a
|
|
# metal, cublas or vulkan build type still produces the same CPU library.
|
|
CMAKE_ARGS?=-DCMAKE_BUILD_TYPE=Release -DTIMESTRAL_SHARED=ON -DTIMESTRAL_BUILD_CLI=OFF -DTIMESTRAL_BUILD_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON
|
|
|
|
ifeq ($(NATIVE),false)
|
|
CMAKE_ARGS+=-DGGML_NATIVE=OFF
|
|
endif
|
|
|
|
.PHONY: timestral-cpp-grpc package build clean purge test all
|
|
|
|
all: timestral-cpp-grpc
|
|
|
|
# Clone the upstream source at the pinned commit, with its ggml submodule. The
|
|
# directory is the target, so make clones only when it is missing. After a
|
|
# TIMESTRAL_VERSION bump, run 'make purge && make' to fetch again.
|
|
sources/timestral.cpp:
|
|
mkdir -p sources/timestral.cpp
|
|
cd sources/timestral.cpp && \
|
|
git init -q && \
|
|
git remote add origin $(TIMESTRAL_REPO) && \
|
|
git fetch --depth 1 origin $(TIMESTRAL_VERSION) && \
|
|
git checkout FETCH_HEAD && \
|
|
git submodule update --init --recursive --depth 1 --single-branch
|
|
|
|
# Build the shared library out of tree, then stage it and the header next to
|
|
# the Go sources, so purego.Dlopen("libtimestral.so") finds it.
|
|
libtimestral.so: sources/timestral.cpp
|
|
cmake -B sources/timestral.cpp/build-shared -S sources/timestral.cpp $(CMAKE_ARGS)
|
|
cmake --build sources/timestral.cpp/build-shared --config Release -j$(JOBS) --target timestral
|
|
cp -fv sources/timestral.cpp/build-shared/libtimestral.so* ./ 2>/dev/null || true
|
|
cp -fv sources/timestral.cpp/build-shared/libtimestral.dylib ./ 2>/dev/null || true
|
|
cp -fv sources/timestral.cpp/include/timestral_capi.h ./
|
|
|
|
timestral-cpp-grpc: libtimestral.so main.go abi.go gotimestralcpp.go options.go plan.go
|
|
CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o timestral-cpp-grpc .
|
|
|
|
package: timestral-cpp-grpc
|
|
bash package.sh
|
|
|
|
build: package
|
|
|
|
# The pure-Go specs always run. The integration specs need
|
|
# TIMESTRAL_BACKEND_TEST_MODEL and a built library; test.sh fetches the 5M F32
|
|
# model and sets both. Without them the integration specs skip.
|
|
test:
|
|
bash test.sh
|
|
|
|
clean: purge
|
|
rm -rf libtimestral.so* libtimestral.dylib timestral_capi.h package timestral-cpp-grpc
|
|
|
|
purge:
|
|
rm -rf sources/timestral.cpp
|