diff --git a/docs/content/features/model-gallery.md b/docs/content/features/model-gallery.md index 3bdfc6d41..75b18a7fc 100644 --- a/docs/content/features/model-gallery.md +++ b/docs/content/features/model-gallery.md @@ -39,6 +39,28 @@ Both views use the same model selection and store the view, search, filter, and selection in the URL. Installing from Explore does not move you away from the catalog; the entry updates in place when the operation finishes. +## RuntimeLabs Kayak screenshot grounding + +[RuntimeLabs Kayak](https://huggingface.co/ayushadarsh7/RuntimeLabs-Kayak-Qwen3-VL-4B) +is a Qwen3-VL-4B fine-tune for locating controls in Kayak flight-search screenshots. +The gallery offers Q4_K_M and Q8_0 GGUF variants for llama.cpp, each with the +matching F16 vision projector and an 8,192-token context. + +Install the Q4_K_M build explicitly: + +```bash +local-ai models install runtimelabs-kayak-qwen3-vl-4b-q4 --variant runtimelabs-kayak-qwen3-vl-4b-q4 +``` + +Send a screenshot and one instruction through `/v1/chat/completions`. Use the +prompt format in the publisher's [inference script](https://huggingface.co/ayushadarsh7/RuntimeLabs-Kayak-Qwen3-VL-4B/blob/main/infer.py), +with `temperature: 0` and `max_tokens: 128`. Fill instructions can include the +text to enter. The response contains an action and, when applicable, a bounding +box in `[x, y, width, height]` format on a 0–1000 scale; x and y identify its +top-left corner. The model predicts coordinates; your application executes the +action. Its training covers Kayak interfaces, so performance on other sites or +changed layouts is unverified. + ## Cyber-Ornith 1.5 9B Cyber-Ornith 1.5 is a Qwen3.5 fine-tune for security auditing, terminal tasks, and tool use. diff --git a/gallery/index.yaml b/gallery/index.yaml index 9e52e70eb..fac0b2da1 100644 --- a/gallery/index.yaml +++ b/gallery/index.yaml @@ -22646,6 +22646,94 @@ - filename: mmproj/mmproj-F16.gguf sha256: 7c5d39a9dc4645fc49a39a1c5a96157825af4d1c6e0961bed5d667a65b4b9572 uri: huggingface://unsloth/Qwen3-VL-30B-A3B-Thinking-GGUF/mmproj-F16.gguf +- name: runtimelabs-kayak-qwen3-vl-4b-q4 + url: github:mudler/LocalAI/gallery/virtual.yaml@master + urls: + - https://huggingface.co/ayushadarsh7/RuntimeLabs-Kayak-Qwen3-VL-4B + - https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF + license: apache-2.0 + description: | + RuntimeLabs Kayak is a 4B Qwen3-VL fine-tune for screenshot grounding on Kayak flight-search interfaces. + It predicts a single action and a normalized bounding box from an image and instruction. + This Q4_K_M GGUF build includes the F16 vision projector and uses an 8,192-token context with llama.cpp. + tags: + - llm + - gguf + - cpu + - gpu + - qwen3-vl + - multimodal + - vision + - screenshot-grounding + variants: + - model: runtimelabs-kayak-qwen3-vl-4b-q8 + overrides: + backend: llama-cpp + context_size: 8192 + known_usecases: + - chat + known_input_modalities: + - text + - image + mmproj: mmproj/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf + options: + - use_jinja:true + template: + use_tokenizer_template: true + parameters: + model: RuntimeLabs-Kayak-Qwen3-VL-4B.Q4_K_M.gguf + temperature: 0 + mmap: true + files: + - filename: RuntimeLabs-Kayak-Qwen3-VL-4B.Q4_K_M.gguf + sha256: 0286db34a9417e41361da1650364708000d437c782355fa15ae9395f1782abdc + uri: https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF/resolve/4228a5ebf59c63b1f809fd553fdde19ede7ac104/RuntimeLabs-Kayak-Qwen3-VL-4B.Q4_K_M.gguf + - filename: mmproj/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf + sha256: 519eeb6616f5d014af407f28bfaad6ce5930468fadadaaf0c3cbfff270b763ad + uri: https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF/resolve/4228a5ebf59c63b1f809fd553fdde19ede7ac104/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf +- name: runtimelabs-kayak-qwen3-vl-4b-q8 + url: github:mudler/LocalAI/gallery/virtual.yaml@master + urls: + - https://huggingface.co/ayushadarsh7/RuntimeLabs-Kayak-Qwen3-VL-4B + - https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF + license: apache-2.0 + description: | + RuntimeLabs Kayak is a 4B Qwen3-VL fine-tune for screenshot grounding on Kayak flight-search interfaces. + It predicts a single action and a normalized bounding box from an image and instruction. + This Q8_0 GGUF build includes the F16 vision projector and uses an 8,192-token context with llama.cpp. + tags: + - llm + - gguf + - cpu + - gpu + - qwen3-vl + - multimodal + - vision + - screenshot-grounding + overrides: + backend: llama-cpp + context_size: 8192 + known_usecases: + - chat + known_input_modalities: + - text + - image + mmproj: mmproj/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf + options: + - use_jinja:true + template: + use_tokenizer_template: true + parameters: + model: RuntimeLabs-Kayak-Qwen3-VL-4B.Q8_0.gguf + temperature: 0 + mmap: true + files: + - filename: RuntimeLabs-Kayak-Qwen3-VL-4B.Q8_0.gguf + sha256: 1e0109e2a5b92f3e369937f35e354652b5c7b4fb0dac040a459556e6af07b036 + uri: https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF/resolve/4228a5ebf59c63b1f809fd553fdde19ede7ac104/RuntimeLabs-Kayak-Qwen3-VL-4B.Q8_0.gguf + - filename: mmproj/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf + sha256: 519eeb6616f5d014af407f28bfaad6ce5930468fadadaaf0c3cbfff270b763ad + uri: https://huggingface.co/mradermacher/RuntimeLabs-Kayak-Qwen3-VL-4B-GGUF/resolve/4228a5ebf59c63b1f809fd553fdde19ede7ac104/RuntimeLabs-Kayak-Qwen3-VL-4B.mmproj-f16.gguf - name: qwen3-vl-4b-instruct url: github:mudler/LocalAI/gallery/qwen3.yaml@master urls: