Architecture / proposed generalization
Normalized objects and versioned collections
How exact identities, ordered membership, materialized Views, and injected adapters could extend Kam beyond its initial UTC-day History workload without changing existing contracts.
Store one exact object; reference it from many scoped collections; reuse unchanged materializations.
In short
Where byIds and allIds go next: one exact object referenced from many scoped, versioned collections, with materialized Views reused when nothing they read has changed. This is the design direction for generalizing today's History adapter, not a shipped service.
Available today: Planned. Today allIds() and byIds() serve UTC-day History only.
Architecture direction, not a public generalized collection SDK. Existing allIds()/byIds() remain UTC-day History operations; the published State Selection candidate is restricted to one Active State evidence-reference collection. No managed/customer-owned S3 snapshot service is qualified.
One Rust State engine; many typed collections
The foundational distinction is identity versus membership. Exact identity answers which retained object revision is requested. Collection membership answers which exact references belong to a particular ordered, pinned generation. A consumer can enumerate references and hydrate only selected records.
The existing History adapter uses a specific tenant/subject/type, UTC day and generation. That is one supported vertical, not a universal requirement that every future collection be date-based. General collection adapters, including project builds, documents, and tasks, are architectural direction only.
Knowledge is information represented as typed State, with provenance and review status where applicable. Context is a consumer-specific View selecting authorized references. Neither needs a second State engine.
Exact object
Stable identity plus exact revision and declared availability. A stored build record is not proof that the artifact is safe to reuse.
Versioned membership
An ordered collection of exact references with a pinned generation. Membership can change without duplicating objects.
Materialized View
A derived result with exact source/definition references, applicability, and its own publication boundary.
Consumer Context View
A task-scoped selection of eligible State/evidence references; permission is rechecked when serving.
What the current History API actually guarantees
allIds() pages one bounded UTC-day generation. Generation-bound byIds() verifies exact membership; direct byIds() resolves known exact revisions without a membership claim. Missing, unresolved and expired evidence stays explicit; neither API silently follows the latest revision.
The existing Rust State engine already supports versioned Views, dependency membership, change detection and bounded reconciliation. The generalized collection contract described here must reuse those mechanisms rather than create parallel reducers or dirty-propagation loops.
Adapter boundaries, not a different engine per vertical
History adapter — implemented scope
UTC-day ordered outcomes, specific revisions, retention and correction rules.
Project collection — proposed
Enumerate exact build, test, and task references by project instead of date; no build-specific inference or compiler required.
Temporal collection — proposed
Use explicit time and versioned rules as View inputs, so durable source facts may drive new bounded projections without being rewritten each year.
Context collection — proposed
Select exact evidence references for one consumer and budget; representation is separate from the original payload.
Keep the existing dependency DAG
A changed source makes its affected dependent Views dirty. The existing reconciliation lifecycle evaluates bounded affected work and conditionally publishes. Within one traversal, a visited set avoids duplicate work. Across turns, an unchanged applicable materialization can be reused rather than recalculated.
A future nested collection may reference exact child generations, but the parent must pin each child version; old parents must not silently retrieve newer children. Traversal requires total depth, node, edge, byte, and read limits. A trie or Merkle DAG is an optional measured indexing optimization, not a prerequisite.
Hot State and durable snapshots have different jobs
The target storage architecture uses DynamoDB for operational pointers, indexed membership and durable work. Immutable evidence and snapshots may live in an approved S3 store. Kam-managed and customer-owned S3 are proposed storage-ownership modes, not existing product offerings.
Retention, snapshot interval, application validity and hot working-set policy are distinct. Customers might select multi-year retention while only occasionally changing State. The current 365-UTC-date working History window is a real existing contract; generalizing future lifecycles must preserve it until separately migrated and qualified.
Snapshot publication must verify the exact durable object before advancing a conditional hot-State pointer. DynamoDB and S3 do not share a transaction. An absent or inaccessible archived object must not silently resolve to latest.
Illustrative: three agents share one build record
Agent A submits a build result under a stable application-supplied ID. Agents B and C can retrieve the same authorized record or see it in different project collections without copying its payload. A new build gets a distinct exact identity or revision according to the declared producer contract.
Reusing the compiled artifact still depends on an external build cache and its compatibility checks. Kam does not compile Rust, certify arbitrary toolchains, infer build equivalence, or grant access because an agent knows an ID.
Implementation boundary and next experiment
This document describes the intended generalization of existing primitives, not a shipped universal byIds/allIds service. Start by preserving the existing History adapter and qualifying one non-date, bounded collection through the real Rust engine, authorized reads, conditional publication, and recovery.
Only after that source-backed happy path should the website describe generic collections, customer-owned S3, durable Context Views or multiagent artifact reuse as available.