[Discussion] xlings as a userspace OS: query, package ownership, resolution, and runtime boundaries

## Status Architecture discussion only. This issue is intentionally separate from the immediate stabilization work for command latency, runtime/graphics regressions, #506, #513, #514, and `subos use` discoverability. The current bugs should be fixed against the existing architecture first. This discussion must not become a reason to delay regression recovery or force an all-at-once migration. ## Motivation Several independently reported failures point at the same long-term boundaries: - local read-only commands rebuilt global state or performed deep payload audits, making `list`, `info`, and the default `self doctor` scale with unrelated payloads; - package identity, XVM targets, workspace state, payloads, delegation, and removal ownership can answer different questions, exposed by #506; - an install host can resolve a dependency correctly but fail to carry the exact result into the hook executor, exposed by #514; - hook failure is still partly represented as a boolean plus side-channel logs, exposed by #513; - runtime selection, process launch policy, and access to host graphics/services are not one explicit contract, exposed by mcpp-community/mcpp#392. These are architecture concerns, but the immediate incidents have bounded bug fixes. This issue asks what the stable long-term contracts should be after those fixes land. ## Design goals 1. Local queries are immediate and depend only on relevant metadata. 2. A successful package release remains identifiable even when it registers zero XVM targets. 3. Resolution happens once per transaction and every hook consumes that exact answer. 4. Install and remove are symmetric over provider-owned effects. 5. Each process uses one coherent core runtime. 6. Host capabilities cross the boundary explicitly and diagnostically. 7. Linux, macOS, and Windows share top-level semantics without pretending their runtime models are identical. 8. Old homes migrate incrementally, observably, and reversibly. ## Proposed architecture boundaries ```text indexes / recipes | v resolver + planner ---------> exact ResolutionContext | | v v install transaction -----------> hook executor | v package records + receipts | +------> query API ------> CLI / TUI / NDJSON / agents | +------> XVM / workspace / shim / sysroot projections SubOS runtime binding | v Runtime selection ------> per-process LaunchPolicy | +------> explicit HostCapability bindings ``` ### 1. Query and control-plane boundary Routine queries should read bounded metadata, not rediscover installed state by walking payload contents. Proposed invariants: - `info <package>` is targeted; it never constructs the inventory for other packages. - `list` is proportional to installed metadata records, not index size or payload file count. - default `doctor` checks control-plane consistency only. - full ELF/Mach-O/PE and payload verification is an explicit deep audit. - no routine query performs network refresh, recipe hooks, child `xlings` calls, or heavy migration. - XVM/workspace/shim/sysroot are queryable projections, not independent authorities that silently override one another. The immediate regression fix can implement targeted readers over today's JSON. A future store is not required to restore these invariants. ### 2. Package lifecycle and resolution boundary A package release and an XVM target are different identities. Proposed minimum package identity: ```text PackageInstanceId { canonical_name version platform arch store_id content_digest } ``` A package instance owns effects such as targets, programs, libraries, files, environment mutations, dependency references, delegation edges, and uninstall hooks. A valid instance may own zero XVM targets. Resolution should produce a total, exact context: ```text ResolvedDep { requested_spec canonical_name exact_version install_dir libdirs store_id source } ``` Hooks consume this record. Directory scanning is an explicitly marked legacy/offline fallback, not another resolver. Modern install hosts that claim the contract should fail closed when a required record is missing or points at a missing payload. Removal starts from the package/provider that is being removed and reverses only its effects. It must not infer package identity from an arbitrary target or remove another provider's bindings. ### 3. Runtime execution and host-integration boundary A SubOS is a package/configuration/policy namespace. It does not imply that every process in its shell must run under one private libc. The long-term runtime contract has three separate responsibilities: 1. **Runtime description** — selected OS/arch/ABI, loader, sysroot, core libraries, package provenance, and generation. 2. **Launch policy** — per-process selection of `host`, `xlings-hermetic`, or `portable-static`, including sanitized child environment. 3. **Host capabilities** — explicit access to Wayland/X11, audio, DRI/GPU, WSLg, open-uri, notifications, and host processes. Core invariant: > One process uses one loader/libc/core-runtime family. A host loader with a private libc, or a private loader with another runtime's libc, is invalid. For an `xlings-hermetic` process, the target is X-complete. Host services normally enter through protocols, sockets, devices, or brokers. Proprietary driver userspace that cannot be reproduced may use an explicitly declared, version-checked library bridge such as today's `*-host-link` packages. Loader/libc can never be such a bridge. This is not a request to make every application hermetic. Host and portable-static domains remain explicit supported choices. ## Proposed decisions ### D1. Query contract Except for explicit download, refresh, migration, or deep audit, xlings queries are local, bounded, perform no payload recursion, and launch no child `xlings` process. ### D2. Migration strategy Recover current regressions first. Introduce contracts incrementally, with old-home comparison and rollback. Do not perform a state/runtime big-bang rewrite. ### D3. Installation anchor The package instance/provider release is the lifecycle anchor. XVM targets are effects, including the valid case of zero effects of that kind. ### D4. Resolution context During an install transaction, every dependency has one exact resolved record. Modern hosts fail closed if that record is absent or invalid. Scanning remains an explicit legacy compatibility path only. ### D5. Runtime policy Runtime selection, per-process launching, and host capability bridging are separate contracts. Hermetic processes aim for X-complete; host/static processes remain explicit; mixed core runtimes are always invalid. ### D6. Storage direction Do **not** introduce SQLite at this stage. Keep business logic behind a storage-independent `state::Store` boundary if/when that work starts, but begin with versioned JSON snapshots, atomic replacement, a bounded transaction journal, and immutable payload receipts. Do not introduce a custom binary database. Do not make a new store a prerequisite for the current bug fixes. A future SQLite ADR should require measured scale/build/maintenance evidence and native Linux/macOS/Windows prototypes before reconsideration. ## Incremental path 1. Land the bounded regression and correctness fixes using current state. 2. Extract targeted query APIs and exact resolution conformance tests. 3. Make provider ownership explicit at lifecycle boundaries. 4. Add readable receipts and versioned JSON transaction/recovery semantics only where current writes need them. 5. Expose a versioned runtime description and per-process launch contract. 6. Convert existing graphics/host-link mechanisms into explicit capability records. 7. Validate old-home migration and native platform behavior before changing any authority. Each stage should be independently useful and reversible. ## Non-goals - replacing the host kernel, GPU kernel driver, compositor, or desktop; - forcing every program into the hermetic domain; - packaging arbitrary host `/usr/lib` trees; - teaching xlings the private directory layout of every consumer; - solving current command hangs by waiting for a database migration; - claiming macOS/Windows/GPU behavior from Linux source review; - silently rewriting existing SubOS runtime bindings. ## Open questions for discussion 1. Is `PackageInstanceId` the right stable lifecycle anchor, or is a smaller provider-release identity sufficient? 2. Which effect classes must be recorded first to eliminate target-to-package inference? 3. What is the smallest versioned RuntimeDescription that mcpp and other consumers can use without directory probing? 4. Should library bridges be a distinct capability class from protocol/device bridges? 5. What old-home compatibility window and rollback promise should precede any authority switch? 6. What measured threshold, if any, would justify reconsidering SQLite over the JSON backend? ## Related incidents - #506 — delegated/zero-target package removal - #513 — hook output and empty failure - #514 — dependency resolved in a shared store but absent from hook context - mcpp-community/mcpp#392 — private glibc, host libraries, shell environment, and graphics/runtime failure The immediate fixes for these issues should remain separately reviewable and should not depend on consensus for this architecture.