ITADN

[Feature] Add --query Flag to unsloth chat for Quick, Non-Interactive Questions

#6501ClosedShagon94 创建于 2026-06-20
feature request
S
Shagon94commented
## Overview Currently, `unsloth chat` requires an interactive REPL or manual input to start a session. Users often want to quickly test prompts, benchmark models, or get one-off answers without entering a full chat loop. This feature request proposes adding a `--query` parameter that enables non-interactive, single-turn questioning with built-in shorthand flags and temporary context handling. ## Key Requirements 1. **`--query` / `-q` parameter**: Accepts a string/question to send directly to the model. 2. **Local Server Detection**: - If `unsloth chat` (local API server) is already running, route the query there. - If not running, exit with a clear error message and non-zero exit code. 3. **Temporary Session**: The chat context is ephemeral. No history is saved, and the process exits immediately after returning the response. 4. **Shorthand Flags**: Support compact flags for common options: - `--disable-thinking` / `--no-thinking` - `--preserve-thinking` / `--keep-thinking` - `--stream` / `-s` - `--web-search` / `-w` - `--python` / `-p` (code execution) *(Flags map directly to existing `unsloth chat` parameters for consistency)* ## Usage Examples ```bash # Basic quick question with explicit model unsloth chat --disable-thinking --model unsloth/Qwen3.6-35B-A3B-MTP-GGUF --query="What is the capital of France?" # Using shorthand flags + streaming + web search unsloth chat -q "Explain quantum entanglement simply" -s -w # Disable thinking, enable python execution unsloth chat --no-thinking --python -q "Calculate the 10th Fibonacci number in Python" ``` ## Technical Implementation Notes - **Server Check**: Before sending the query, check for an active local Unsloth server (e.g., via HTTP health endpoint or bound socket). If unavailable, print: ``` [Error] Local Unsloth chat server not running. Start it with 'unsloth studio' first. ``` Exit with code `1`. - **Flag Mapping**: Shorthand flags should be aliases to existing `unsloth chat` CLI parameters. No new backend logic needed; just CLI parsing and session routing. - **Temporary Context**: - Do not write to history files. - Clear conversation buffer after the response is returned. - Non-blocking exit: process terminates immediately after streaming/printing the final output. - **Shell Compatibility**: Properly handle quoted strings, escape sequences, and special characters in `--query` values across bash/zsh/Windows CMD. ## Benefits & Use Cases - **Faster Iteration**: Test prompts or model behavior without manual REPL interaction. - **Scripting & Automation**: Easy integration into CI pipelines, benchmarking scripts, or automated evaluation workflows. - **Consistency**: Leverages existing `unsloth chat` feature set (thinking modes, web search, code execution) in a lightweight wrapper. - **Lower Cognitive Load**: Reduces boilerplate for one-off questions while keeping the full power of the chat interface available when needed. ## Backwards Compatibility - Fully additive: existing `unsloth chat` behavior remains unchanged. - Flags are opt-in; no changes to default interactive mode. - Error handling ensures graceful failure when local server is absent.
关闭于 2026-06-20 1 条评论