## What's Changed * Persist server logs so crash reports carry evidence by @tobocop2 in https://github.com/tobocop2/lilbee/pull/339 **Full Changelog**: https://github.com/tobocop2/lilbee/compare/v0.6.66b493...v0.6.66b494

## What's Changed * Emit a proper SSE error event when a chat stream fails by @tobocop2 in https://github.com/tobocop2/lilbee/pull/324 **Full Changelog**: https://github.com/tobocop2/lilbee/compare/v0.6.66b491...v0.6.66b492

The biggest beta in a while, and the headline is speed. The desktop UI used to freeze for tens of seconds whenever a model ran; now it stays responsive no matter what's happening in the background, and its resting CPU is down roughly 80%. There's also a redesigned model catalog, a slash-command palette, hosted-model support, and standalone binaries you can run without a Python install. ## The UI is fast now Inference used to share one process with the event loop and the renderer, so adding a PDF, asking a question, or running a reranked search would lock up the screen — arrow keys took seconds to register, cancel didn't work, and there was no sign anything was happening (#205). That's gone: - **Models run in their own processes.** The UI never does inference work, so scrolling, typing, and switching views stay smooth regardless of model load. Cancel (Esc / Ctrl-C) actually interrupts a running chat now. Models load once and stay warm; idle ones get reaped to free RAM. - **Resting cost cut ~80%.** A profiling pass found the UI thread paying for a 10 Hz animation timer that ran even when nothing was happening, plus a futures handoff on every streaming chat token. Both fixed — idle CPU dropped from ~1.1% to ~0.2%, and streaming no longer interrupts the loop on each token (#214). - **Screens stop rebuilding themselves.** Catalog, Settings, and Sources install once and reuse instead of tearing down and rebuilding on every visit; lists skip the relayout when nothing changed; settings tabs only mount when you open them (#202). ## Imports that don't look frozen Importing scanned PDFs used to sit on one file for minutes with a CPU core pinned and the progress bar refusing to move — three slowdowns stacking: chunk-at-a-time embedding, PDF parsing blocking the whole import, and a fresh vision-model load per file (#210). Now embedding and reranking process chunks in batches, parsing runs on a background thread pool, the vision model loads once and stays resident, and you get per-chunk progress so a big document doesn't look stuck. ## Redesigned model catalog Six tabs — Discover, Chat, Embed, Vision, Rerank, Library — with model families collapsed into single rows. Each row carries a hardware-fit chip (`fits` / `tight` / `won't run`) and its available size variants, computed server-side so the screen just renders them (#218). Pagination is per-tab, so flipping between tabs doesn't refetch (#221). `v` toggles grid/list inside any tab, and each tab has Local and Frontier sub-views. ## Slash-command palette Type `/` and a command palette opens in place, with argument hints inline as you type. Ctrl+N/P and the arrow keys move through it, Enter picks, Esc backs out. The footer shows the help and catalog shortcuts so you don't have to memorize them (#216). ## Hosted models With the `[litellm]` extra installed, hosted providers show up alongside your local models. An API-Keys settings tab lets you paste keys without touching config files, and the catalog's Frontier sub-tabs list what each provider offers (#219). ## Other improvements - **Reasoning cap**: `LILBEE_REASONING_TOKEN_CAP` bounds how long reasoning models think — on cap, the model is told to answer instead of spinning (#224). - **Search/Chat toggle**: F3 flips between retrieval-augmented search and plain chat without leaving the chat screen. - **Standalone binaries** for Linux, macOS (arm64), and Windows attached to every release — no `pip install` needed (#199). - **Defaults retuned** for the 4B–30B local models most people run: top-K, context budget, and sampling now favor coherent multi-turn answers over single-shot precision. ## Release safety Every release now runs through an end-to-end QA matrix — the actual published wheel and standalone binary, exercised on Ubuntu, macOS, Windows, Arch, and Fedora — before anything reaches PyPI (#193). PyPI publishing is a manual step gated on that matrix passing, so a broken build can't ship itself. ## Notable fixes - Model pickers no longer offer half-downloaded models while a pull is in flight. - Empty config values stopped silently overriding defaults. - Vision OCR no longer comes back empty on PDFs where the layout extractor finds no rows. - The Windows standalone binary no longer eats the screen output (native errors go to `tui.log`).

### Always-visible model context The page indicator and active model name now sit at the top of every screen, so you always know where you are and what model you're using. The chat-screen model selectors are a flat single row instead of three boxed dropdowns. ### Wiki off by default The Wiki view tab and the chat scope picker only appear when you enable wiki (Settings or `LILBEE_WIKI=1`). First-run users get a cleaner navigation. ### Dynamic LLM settings Changing the context size or other model-load settings now applies on the next request instead of waiting for the model's keep-alive to expire. When `num_ctx` is unset, chat caps at a safe default so modern 128K-context GGUFs don't try to allocate the full training context and OOM. ### Setup wizard exit + theme polish The wizard's Esc-to-exit hint is finally visible. Ctrl+T cycles themes (now shown in the footer) and the active theme persists across sessions. ### Smaller fixes A CSS quirk that left the wizard's helper text invisible. The highlighted page tab no longer lags the actual screen by one navigation step. Open Select dropdowns get cleaned up before screen tear-down so their borders don't bleed into the next screen.

Runtime CPU dispatch via GGML_CPU_ALL_VARIANTS=ON. One x86_64 wheel works on every CPU from Sandy Bridge (2011) forward — fixes the 'illegal hardware instruction' SIGILL on older Xeons / pre-AVX2 systems.

Wave 2 — full GPU backend matrix on llama-cpp-python 0.3.20. Default PyPI wheels: Vulkan (Linux/Windows), Metal (macOS). Per-backend extra-index publishes for CUDA 12.1/12.4/12.5, ROCm, SYCL, pure CPU.

Fix Memory-is-not-initialized on first chat: bump vendored llama-cpp-python to 0.3.20, build from source on every platform. self-check now exercises the embedding leg too.

### Fixes - **Settings now persist properly.** Fields like `temperature`, `top_p`, `seed`, and `system_prompt` are now saved to and loaded from `config.toml` correctly. Previously only `chat_model` and `vision_model` were loaded on startup — everything else was silently ignored after a restart. - **Empty values are rejected.** `/set chat_model ""` and similar no longer silently accept empty strings. The config validates at the boundary now. - **Quoted values no longer break config.toml.** Setting a value containing `"` or `\` (e.g. `/set system_prompt You are a "pirate"`) previously wrote invalid TOML that failed to parse on next load. ### Improvements - **System prompt is visible in `/settings`.** Rendered as a panel instead of just showing the character count. - **Default system prompt updated.** Now works well for both chat and coding contexts — precise, grounded, and concise. - **Tests run faster.** The conftest was loading the full Ollama model just to check availability. Now it just verifies the model exists.