§32 Loams Flow, Event Fabric and House

Status: Proposed · 2026-10-01. Source: the owner's draft "Loams: Final Plan" (Loams Flow, Loams Event Fabric, Loams House), kept in chatdump.md lines 325–549, and two owner inputs of 2026-10-01: first, "Apache Iggy handles raw protocols well … the owner sees Iggy as the streaming house, the ingest and streaming layer"; then, in the owner's words, "our loams wal is optimized for postgres, mysql, text search, vector search, graph workloads, iggy and fluss for event ingestion so separate is good, and we see more contributions to iggy and fluss". This document turns that direction into decisions D330–D351 and open questions Q330–Q347. The connector registry and catalog of Loams Flow are in §33 (D352–D359, Q348–Q359). Everything here is a proposal until the owner confirms it; where it reverses an approved decision (D45, and D74's Loams-side Kafka gateway), §14 says so and the owner decides. No code is written by this document.

Names (owner rulings, 2026-10-01). Published packages use the namespace loams (crates.io loams-*, PyPI loams-*, npm @loams/*), replacing loamdb (D33); Go modules use loams.dev/...; the domain is loams.dev; CloudEvents types Loams defines use the prefix io.loams.dev.<domain>.<name>.v1. The crate and binary names below (loams-fabric, loams-chdb, …) follow the same namespace as the engine's loams-* crates. Java is deferred: Loams writes no Java SDK or Java plugin in this track; the Camel layer runs unmodified Camel with generated YAML routes (§33 D354).

Markers:

• (verify) means not checked against a primary source, or checked only by reading docs or code that were not run. The plan task that depends on it checks it first.
• (estimate) means computed or reasoned, not measured.
• (read 2026-10-01) means read in the upstream repository or website on 2026-10-01 at the release named in §16.
• "The draft" is the owner's "Loams: Final Plan". Its status tags ([verified], [assumed], [build]) are re-checked here; §3 lists what changed.

1. SummaryCopy Anchor Link

2. Goals and non-goalsCopy Anchor Link

2.1 GoalsCopy Anchor Link

1. Managed streaming and analytics without managed Spark or Flink as the default path (the draft's goal 1). Flow compiles to Loams's own primitives, Iggy, Fluss and chDB.
2. Separate logs for separate workloads (owner direction): the Loams WAL keeps its database and retrieval optimizations (§28 Arm A, collections, graphs); event ingestion uses Iggy and Fluss, where Loams also contributes upstream.
3. Buy, not build (user memory): Iggy, Fluss, chDB, Debezium and Camel run unmodified; Loams writes envelopes, bridges, the compiler, the House shim and connectors where no good one exists, and upstreams what is general.
4. ClickHouse compatibility you can test: a declared, versioned surface with a published pass rate and an allowlist (the draft's §1).
5. One envelope everywhere: CloudEvents 1.0 on Loams streams (D270), Iggy messages and Fluss rows (D334).

2.2 Non-goalsCopy Anchor Link

• No new storage engine. Fluss tables, Iceberg on RustFS and Loams's own WAL are the storage; chDB never keeps state between queries.
• No ClickHouse server inside Loams. chDB executes queries; the House never runs ClickHouse's MergeTree storage, Keeper or replication.
• No second graph or vector store. §07 and §06 stay; Grafeo is not embedded.
• No Flink or Spark as Flow targets, and no stateful stream processing in Flow (D341).
• No Loams fork of Iggy or Fluss. Loams's changes go upstream as PRs; Loams pins released versions and carries at most a short-lived patch branch for a merged-but-unreleased fix.

3. What changed since the draftCopy Anchor Link

4. ArchitectureCopy Anchor Link

              Kafka clients  MQTT / AMQP  Webhooks  OTLP  Debezium Server  SDKs (QUIC/TCP/HTTP/WS)
                    │            │           │        │          │              │
                    ▼            ▼           ▼        ▼          ▼              ▼
            Iggy Kafka gw    loams-connect  loams-fabric ingest (CloudEvents HTTP/gRPC)   (direct)
            (contributed)    (Camel → Iggy)          │                                   │
                    └────────────┴───────────────────┴───────────────┬───────────────────┘
                                                                     ▼
┌──────────────────────────── LOAMS EVENT FABRIC (event ingestion) ────────────────────────────┐
│  Apache Iggy (ingest log, VSR, local NVMe, bounded retention)  ── connectors runtime ──┐     │
│        │ fluss_sink (Rust plugin, upstreamed)                     sinks: ES, HTTP, S3,  │     │
│        ▼                                                          Postgres, ClickHouse… │     │
│  Apache Fluss (Log + PK tables, merge engines, remote storage on RustFS)               │     │
│        │ tiering (Flink job under the Flink operator, D212)                             │     │
│        ▼                                                                                │     │
│  Iceberg on RustFS through Lakekeeper ◄──────────────── shared catalog ───────────────┼──┐  │
└────────┬───────────────────────────────────────────────────────────────────────────────┘  │  │
         │ loams_sink (Iggy plugin → ProduceCloudEvents / collection writes)                 │  │
         ▼                                                                                   │  │
┌──────────────────── LOAMS ENGINE (loams binary; Loams WAL, §02/§28) ──────────────────────┐│  │
│ collections (Lance + Tantivy + qdrant-edge), graphs (§07), Loams tables (M4), Live, PG/MySQL││  │
└──────────────────────────────────────────────────────────────────────────────────────────┘│  │
                                                                                            │  │
┌─────────────────── loams-fabric (fabric/ workspace; separate binary) ─────────────────────┘  │
│ flow:   route API (connect-rust), compiler, registry (§33), supervisors (Iggy runtime,       │
│         loams-connect, Debezium Server)                                                      │
│ house:  ClickHouse HTTP 8123 → classifier → DDL/INSERT → Fluss;  SELECT → chDB session:      │
│         Iceberg snapshot (icebergS3) ∪ Fluss tail (ArrowStream) → output format             │
│ coordinator (FL5): scatter-gather, two-stage aggregation (-State / -Merge), native 9000      │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

5. Loams Event FabricCopy Anchor Link

5.1 Two logs, one rule (D331)Copy Anchor Link

The Loams WAL is tuned for what it already carries: Postgres WAL through Arm A (§28 §7.2, D263–D272), MySQL binlog (§29, D276), collections and their implicit streams with consistency tokens (D76), graph sidecars, Loams Live's journal (§20). Event ingestion has other needs: many producers on raw transports, consumer groups, connectors to everything, and keyed real-time tables with merge semantics. The owner's direction is to give those to Iggy and Fluss and contribute there.

The rule: every record has exactly one home log. Writes through Loams's APIs (native, Flight, Qdrant, Elasticsearch, Postgres, MySQL, Live, /events at trigger rates) land in the Loams WAL. Writes through the Fabric's APIs (Iggy transports and SDKs, the Fabric ingest route, connectors, ClickHouse INSERT) land in Iggy or Fluss. A bridge (D336) copies records across when a route asks for it, with the target's own exactly-once mechanism, so no client ever writes both.

What stays in §02: the native stream API core (D72), OTLP logs (D73), CloudEvents on streams (D270, PRs #170 and #171) and changelog streams (D21) are unchanged; they serve the Loams objects and trigger-rate events. What moves: general-purpose, high-rate event streaming, which §02 §7.2 meant to reach through Loams's own Kafka gateway (D74). Q347 asks whether D72's explicit streams should then narrow.

5.2 Apache Iggy: evaluation and fit (D332)Copy Anchor Link

What Iggy is (server 0.9.0, 2026-09-18; README and docs read 2026-10-01): a persistent append-only log in Rust, thread-per-core and shared-nothing on io_uring and compio; transports QUIC, TCP (its own binary protocol), HTTP (REST) and WebSocket, all with TLS; streams → topics → partitions; server-side consumer offsets and consumer groups; retention by message_expiry and max_topic_size; users, permissions and personal access tokens; optional AES-256-GCM encryption; per-topic durability (replicated: VSR quorum commit; persisted: stable storage on the quorum); clustering by Viewstamped Replication with one consensus group for metadata and one per partition, at least three replicas, documented as pre-production; SDKs for Rust, C#, Java, Python, Node.js and Go; a CLI, a web UI and a Helm chart; a connectors runtime that loads Rust plugins implementing Source and Sink (sources: Elasticsearch, HTTP, InfluxDB, PostgreSQL with polling and logical-replication CDC modes, random; sinks: ClickHouse, Delta Lake, Doris, Elasticsearch, HTTP, Iceberg, InfluxDB, Meilisearch, MongoDB, PostgreSQL, Quickwit, RabbitMQ, Redshift, S3, SurrealDB, stdout) with per-stream transforms and Avro support; an MCP server; and an in-development Kafka gateway. Apache-2.0; an Apache top-level project since 2026-08-19 (graduated from the Incubator; announced on the project blog 2026-08-24). Apache Camel 4.22 ships a camel-iggy component (producer and consumer, "Preview" since 4.17).

Gaps that matter to Loams:

How Iggy and the Loams WAL can fit together:

Decision (D332): option A. Iggy runs as the Fabric's ingest log in its own pods (StatefulSet with local NVMe, three replicas, persisted durability for event topics that feed Fluss), deployed by its Helm chart under Loams's umbrella chart (D186). Trade-offs accepted: a second durable system with disks; cross-log visibility through bridges rather than tokens; Iggy's protocol changes absorbed by pinning. Revisit triggers: Iggy 1.0 freezing the protocol; tiered storage with object storage as the source of truth (then the 72 h bound lifts and option A gains D1's property); the Kafka gateway reaching consumer-group offsets (then Q331 can close D74).

5.3 Apache Fluss: evaluation and fit (D333)Copy Anchor Link

What Fluss is (1.0.0, 2026-09-22; read 2026-10-01): streaming storage for real-time analytics. Log tables are append-only, columnar (Arrow) logs; Primary Key tables hold upserts and deletes and emit a changelog, with merge engines LastRow (default), FirstRow, Versioned (a version column) and Aggregation (per-column functions, including rbm32/rbm64 bitmaps), primary-key and prefix lookups, and KV snapshots. Coordinator and tablet servers (Java) coordinate through ZooKeeper; 1.0 adds coordinator HA with epoch fencing. Recent data is on tablet-server local disks; older log segments and KV snapshots move to remote storage (S3 and others); lake tiering writes Paimon, Iceberg, Hudi or Lance through the Flink tiering job, and 1.0 uses Arrow batches for tiering and supports Iceberg REST catalogs. Clients: Java, Flink and Spark connectors; fluss-rs (Rust, 1.0.0, arrow 59), Python and C++; the Fluss Gateway (Rust, REST, preview). A fluss-kafka module exists in the tree (status (verify)).

Fit. Fluss is the table half of the Fabric: what ClickHouse's merge-tree engines and the draft's "real-time tables" need (§7.4). Loams's own keyed and aggregating tables (§08 §1, M4) stay for data written through Loams; the two meet in Iceberg, through the one Lakekeeper catalog.

5.4 The envelope on Iggy and Fluss (D334)Copy Anchor Link

The draft's envelope (tenant, schema id, event time, idempotency key, payload) maps onto CloudEvents 1.0 exactly as Loams streams store it (§02 §7.4):

Deduplication. loams-fabric ingest (the HTTP and gRPC CloudEvents route of the Fabric, FL1) uses D270's claim, append, complete protocol with its ledger in a Fluss PK table _fabric.ce_dedup (LastRow merge engine, so a claim is updated in place when it completes; key (topic_id, dedup_key), value (partition, offset, state, claimed_until)), with the same 1-hour default window and 24-hour maximum. A retried request inside the window gets the first write's partition and offset. Events that reach Iggy directly through its SDKs are not deduplicated by Loams (Iggy's own per-session retry handling applies); downstream PK tables keyed by (_ce_source, _ce_id) dedupe by construction.

5.5 Adapters: protocols into the FabricCopy Anchor Link

5.6 Schemas and dead letters (D335)Copy Anchor Link

• A Fluss table's schema is authoritative for rows in it; ALTER TABLE ADD COLUMN follows Fluss's schema evolution (verify the operations Fluss 1.0 supports).
• An Iggy topic is schemaless; producers name a schema in dataschema. loams-fabric flow stores schema documents in the bucket (ns/<id>/fabric/schemas/<subject>/<version>) and their versions in _fabric.flow_objects (kind = 'schema'), and checks compatibility (backward by default) when a route declares a schema.
• Dead letters: a route that cannot decode, validate or deliver an event writes it to <topic>.dlq with ce_loamserror (code and message) and ce_loamsroute; the DLQ is an ordinary Iggy topic with the route's retention.

5.7 Bridges (D336)Copy Anchor Link

5.8 Tenancy, auth and deploymentCopy Anchor Link

• Namespaces. A Loams namespace gets an Iggy stream ns-<id>, a Fluss database ns_<id>, a Lakekeeper namespace ns_<id> in the cluster's warehouse and a prefix ns/<id>/fabric/ in the bucket. loams-fabric provisions all four idempotently.
• Credentials. In FL1–FL2 everything binds loopback (D111). With the unified auth plan (Q30), a Loams API key maps to an Iggy user (or personal access token) per namespace and to Fluss ACLs (1.0 changed ACLs, verify), and loams-fabric checks OpenFGA (D66) for route and table operations. Q343 asks whether Iggy users are minted per key or per namespace.
• Secrets for connectors go through Dapr's secrets building block via loams-dapr (D189).
• Deployment. Dev: deploy/fabric/compose.yaml (Iggy, Fluss with ZooKeeper, Lakekeeper with Postgres, RustFS, a Flink session cluster with the tiering job, loams-fabric). Kubernetes: subcharts of the umbrella chart (D186): Iggy's chart, a Fluss chart (Fluss ships Helm support (verify)), Lakekeeper's chart, the Flink operator, loams-fabric.
• Usage hooks (§27): loams-fabric exports per-namespace metrics (events in, bytes, House query CPU from chDB's system.query_log equivalents); metering stays in loam-platform (D190, D202).

5.9 Upstream contributions (owner: "we see more contributions to iggy and fluss")Copy Anchor Link

Upstream rules follow §23 §2.2: one concern per PR, nothing posted without the owner's go-ahead per project, Loams carries a fix only between its merge and the next release.

6. Loams FlowCopy Anchor Link

6.1 Routes (D337)Copy Anchor Link

apiVersion: loams.flow/v1
kind: Route
metadata: { name: orders-to-house, namespace: shop }
spec:
  from: { connector: debezium-postgres, instance: orders-db, tables: [public.orders] }
  steps:
    - filter: "type == 'io.debezium.postgresql.datachangeevent' && data.op != 'r'"
    - map:   { sql: "SELECT data.after.id AS id, data.after.total AS total, data.after.updated_at AS ver" }
  to:
    - fabric: { table: orders_current, merge: versioned, version: ver }   # Fluss PK table
    - loams:   { collection: orders_search, key: id }                      # through loams_sink
  delivery: { mode: at_least_once, dlq: true, max_batch: 65536, max_delay: 2s }

Routes are rows of _fabric.flow_objects (kind = 'route'), versioned: an update writes version n+1 and keeps n until no task runs it. A route is valid only if every connector capability it uses is declared (§33 D353); the API answers FailedPrecondition naming the missing capability otherwise.

6.2 Compiler targets (D338)Copy Anchor Link

6.3 DeliveryCopy Anchor Link

Into the Fabric and into Loams objects: exactly-once in the target where the target dedupes (D334's dedup key, PK tables, D270's ledger); into external sinks: at-least-once, with ce_id and an Idempotency-Key header (HTTP sinks) or the sink's own idempotent producer (Kafka) where available. Ordering: per Iggy partition, and per key when the route partitions by partitionkey.

6.4 Where it runs (D339)Copy Anchor Link

loams-fabric flow is a stateless service: routes in the metastore, connector instances supervised as Iggy connectors-runtime processes, loams-connect routes or Debezium Server instances (one per instance, §33 D354), and native connectors as tasks in loams-fabric. Its state is in _fabric.flow_objects (D337); in CN1 one flow process supervises every instance, and FL3 adds task leases through the Loams metastore's network API (the §09 §6 lease model), so the fabric/ workspace never links loams-meta. By open-core.md, all of this is open: a single organisation needs connectors to self-host. loam-platform adds the fleet (per-tenant connector pods, autoscaling, plan limits, billing).

6.5 UI (D340)Copy Anchor Link

A console page lists routes, connector instances, lag and DLQ counts, and edits routes as YAML with a diagram. Embedding Karavan's designer for Camel-backed steps is Q340.

7. Loams HouseCopy Anchor Link

7.1 PatternCopy Anchor Link

The draft's pattern, "WAL → immutable S3 segments → embedded engine → compat shim", holds with the Fabric as the WAL: Fluss's log and KV are the write-ahead state, tiered Iceberg files on RustFS are the immutable segments, chDB is the embedded engine, and the ClickHouse HTTP interface is the shim. House workers are stateless and disposable.

7.2 chDB (D342)Copy Anchor Link

chdb-core packages ClickHouse as libchdb with a C ABI (chdb.h): connections and sessions, buffered and streaming queries, parameterized queries, Arrow output and Arrow input through the C Data Interface, and every ClickHouse input and output format. Versions track ClickHouse (26.9.0, 2026-09-28). Loams's crate loams-chdb-sys (bindgen over the pinned chdb.h) and loams-chdb (a safe wrapper: Engine as a process singleton, Session, Query streams, register_arrow_stream, cancellation, memory limits) are written against that ABI; chdb-rust 2.0 is the reference and may be used directly if FL2 Task 0 finds its API stable enough (Q337). libchdb is fetched by SHA-256 from the chdb-core release in the build script (or taken from LIBCHDB_DIR), never built from source.

Rejected alternatives:

7.3 The binary and the workspace (D343)Copy Anchor Link

fabric/                         # a Cargo workspace of its own; its own Cargo.lock; the shared target dir
  Cargo.toml                    # arrow 59 (fluss-rs), tokio, axum 0.8, connectrpc 0.9
  crates/loams-chdb-sys/         # bindgen over chdb.h; build.rs fetches libchdb by digest
  crates/loams-chdb/             # safe wrapper
  crates/loams-fabric-envelope/  # CloudEvents ↔ Iggy message, ↔ Fluss row (uses loams-cloudevents' core, §FL1 Task 2)
  crates/loams-fabric-ingest/    # CloudEvents HTTP/gRPC → Iggy, dedup ledger on Fluss
  crates/loams-house/            # ClickHouse HTTP, classifier, catalog, DDL/INSERT, read planner
  crates/loams-flow/             # routes, compiler, registry (§33), supervisors
  crates/loams-fabric/           # the binary: loams-fabric {ingest|house|flow|all} --config …
  crates/loams-house-conformance/# the differential harness (D349)

Iggy connector plugins live in Iggy's own workspace (upstream PRs) and are built from a pinned Iggy revision in CI. The engine workspace changes by one small PR: loams-cloudevents gets a default feature log around its record layout, so the Fabric can depend on its codec without loams-log.

7.4 Tables and engines (D344)Copy Anchor Link

The original CREATE TABLE text, the engine and the ClickHouse types go into Fluss table properties loams.ch.ddl, loams.ch.engine, loams.ch.types (custom properties, verify), so the House needs no catalog of its own. Types map ClickHouse → Fluss → Iceberg as in FL2 Task 5's table; types with no Fluss counterpart (Enum8, LowCardinality, Nested, AggregateFunction) are stored as their base type with the ClickHouse type kept in loams.ch.types and restored in query views.

7.5 Reads (D345)Copy Anchor Link

For each query, the House worker:

1. Classifies the statement (§FL2 Task 4) and resolves its tables.
2. Pins, per table, the last tiered Iceberg snapshot and the Fluss log offsets that snapshot covers (Fluss records the lake snapshot's offsets per bucket, verify the fluss-rs API), plus the current Fluss high watermarks — or the offsets in the request's X-Loams-Consistency-Token, waiting up to loams_consistency_wait_ms (default 5 000) until the tail reaches them.
3. Registers in the chDB session, per table, an Arrow stream of the Fluss log between the snapshot offsets and the pinned high watermark, and defines a temporary view db.t = icebergS3(<warehouse>, …, iceberg_metadata_file_path = '<pinned>') UNION ALL the stream, with the ClickHouse types restored. For PK tables the view keeps, per key, the row with the highest (_bucket, _offset) and drops deletes (… ORDER BY _offset DESC LIMIT 1 BY <pk> over the union, where lake rows rank below tail rows).
4. Runs the user's query text unchanged against those views (within the supported surface of §8 and the deny list of §7.8), with FINAL accepted as a no-op, and streams chDB's output in the requested format.

Caching: chDB's own filesystem cache on local NVMe for S3 reads in FL2; sharing Loams's foyer cache is later work.

7.6 Writes and DDL (D346)Copy Anchor Link

• CREATE DATABASE d creates a Fluss database ns_<id>__d (verify allowed characters) and maps d in the session; default always exists.
• CREATE TABLE maps per §7.4 and enables tiering (table.datalake.enabled = true, freshness from SETTINGS loams_tiering_freshness, default 60 s).
• INSERT INTO t [(cols)] VALUES … | FORMAT <fmt> <data> | SELECT …: the worker parses the body with chDB into Arrow batches of the table's schema (FL2 Task 6 fixes the mechanism), writes them with fluss-rs (append for Log tables, upsert or delete for PK tables), and answers once Fluss acknowledges; the summary header X-ClickHouse-Summary carries written_rows and written_bytes, and X-Loams-Consistency-Token the bucket offsets. async_insert = 1 is accepted and served synchronously.
• DROP TABLE, TRUNCATE TABLE (Log tables: drop and recreate, verify Fluss), ALTER TABLE ADD COLUMN (Fluss schema evolution) are in chsurface-1; ALTER … UPDATE/DELETE (mutations), lightweight DELETE FROM, OPTIMIZE (accepted as no-op), RENAME and projections are not.

7.7 Graph (D350)Copy Anchor Link

§07's native mapped graphs (D44) stay the graph surface: they are the GraphRAG workload's shape (seeds → 1–2 hops → rerank, in one planned query), they ship in M3, and they read the same collections that hold the vectors. Grafeo was checked against the draft's claims (§3): its feature list is real and broad, but its repository is eight months old (created 2026-01-26); it would add a second storage engine with local persistence, a second HNSW, arrow 60 beside Loams's 58, and a single-maintainer dependency. If owners of Neo4j-shaped apps ask for Cypher or GQL, the low-risk path is a Flow sink that keeps a grafeo-server (Bolt v5, GQL over gRPC) copy of a mapped graph, run as a separate service (Q339). Graph conformance suites in the draft (openCypher TCK, LDBC SNB, SPARQL, Gremlin) apply only to that companion, not to Loams.

7.8 Security of table functionsCopy Anchor Link

In chsurface-1, these chDB table functions and engines are disabled in House sessions (SUPPORT_IS_DISABLED, code 344): file, url, urlCluster, remote, remoteSecure, cluster, mysql, postgresql, odbc, jdbc, hdfs, s3, s3Cluster, gcs, azureBlobStorage, iceberg*/deltaLake*/hudi with user-supplied URLs, executable, input outside INSERT, dictionary sources outside Loams, system.* writes, and the Kafka, RabbitMQ, NATS, S3Queue engines. The worker itself uses icebergS3 with Loams-generated arguments only. Allowed: numbers, numbers_mt, zeros, values, generateRandom, format, null, view, merge over Loams tables. A per-namespace allowlist for s3 on the namespace's own prefix with vended credentials is later work. chDB's process-wide signal handlers are declined at engine start (chdb-rust's runtime control, verify in the C ABI) so the worker owns its signals.

8. ClickHouse compatibility surface v1 (chsurface-1, D347)Copy Anchor Link

The surface is a file in code (fabric/crates/loams-house/src/surface.rs, one table) from which docs/guides/clickhouse-surface.md is generated, with a test that fails on drift, as D88 does for limits. ClickHouse version: the version chdb-core carries (26.9 at FL2), which is also the reference server's.

8.1 Interface: HTTP (port 8123)Copy Anchor Link

8.2 StatementsCopy Anchor Link

SELECT (ClickHouse's SELECT syntax as chDB parses it, executed over Loams tables, §7.5, with the functions of §8.6 and the table-function restrictions of §7.8), WITH, UNION, INSERT … VALUES | FORMAT | SELECT, CREATE DATABASE [IF NOT EXISTS], CREATE TABLE [IF NOT EXISTS] … ENGINE = <Tier 1> …, CREATE TABLE … AS SELECT (Tier 1 engines), DROP DATABASE|TABLE [IF EXISTS], TRUNCATE TABLE, ALTER TABLE … ADD COLUMN, SHOW DATABASES|TABLES|CREATE TABLE|COLUMNS, DESCRIBE TABLE, EXISTS TABLE, USE, SET, EXPLAIN (AST, SYNTAX, PLAN; passed to chDB), OPTIMIZE TABLE … [FINAL] (no-op), KILL QUERY WHERE query_id = … (own session's queries). Everything else answers code 48 (NOT_IMPLEMENTED) with the statement kind named.

8.3 EnginesCopy Anchor Link

8.4 FormatsCopy Anchor Link

Output and input, through chDB: TabSeparated (and WithNames, WithNamesAndTypes, Raw), CSV (and WithNames, WithNamesAndTypes), JSON, JSONCompact, JSONEachRow, JSONCompactEachRow (and WithNames…), JSONStringsEachRow, RowBinary (and WithNames, WithNamesAndTypes; the Rust and Go HTTP clients use it), Native (clickhouse-connect uses it over HTTP), Parquet, Arrow, ArrowStream, Values, Pretty* (output only), Null (output only). Others answer code 73 (UNKNOWN_FORMAT, verify).

8.5 SettingsCopy Anchor Link

Passed to chDB when in the allowlist: query complexity (max_execution_time, max_result_rows, max_result_bytes, result_overflow_mode, max_rows_to_read, max_bytes_to_read, max_memory_usage capped by the namespace's quota), formats (output_format_*, input_format_*, date_time_input_format, date_time_output_format), join_algorithm, max_threads (capped), session_timezone, allow_experimental_* for analyzer features chDB enables by default, insert_deduplicate (accepted), async_insert (§7.6), wait_for_async_insert. Loams settings: loams_consistency_wait_ms, loams_tiering_freshness, loams_bucket_key. Unknown settings answer code 115 (UNKNOWN_SETTING); known but disallowed ones code 164 (READONLY).

8.6 FunctionsCopy Anchor Link

Supported and allowlisted: chDB's scalar and aggregate functions, except the disabled table functions of §7.8 and functions that read the host (hostName, getMacro, filesystem*, getSetting of server settings; they return Loams values or code 344). version() returns the chDB ClickHouse version; currentDatabase() the mapped database.

8.7 System tablesCopy Anchor Link

Synthesized from the Fluss catalog and the session: system.databases, system.tables (with engine and create_table_query from loams.ch.* properties), system.columns, system.parts (one synthetic part per Iceberg data file, for tools that count rows), system.settings, system.functions, system.formats, system.data_type_families, system.one, system.numbers, system.processes (own namespace), system.query_log (own namespace, recent queries kept by loams-fabric), system.build_options (chDB's), system.contributors (chDB's). Others answer code 60 (UNKNOWN_TABLE).

8.8 ErrorsCopy Anchor Link

Errors carry ClickHouse's text form Code: <n>. DB::Exception: <message>. (<NAME>) (version <v>), the header X-ClickHouse-Exception-Code, and ClickHouse's HTTP status for the code (400 syntax, 401/403 auth, 404 unknown table or database, 500 otherwise; verify the mapping in 26.9). Errors from chDB pass through unchanged. Codes Loams raises itself: 36 BAD_ARGUMENTS, 48 NOT_IMPLEMENTED, 57 TABLE_ALREADY_EXISTS, 60 UNKNOWN_TABLE, 62 SYNTAX_ERROR, 81 UNKNOWN_DATABASE, 115 UNKNOWN_SETTING, 159 TIMEOUT_EXCEEDED, 164 READONLY, 202 TOO_MANY_SIMULTANEOUS_QUERIES, 241 MEMORY_LIMIT_EXCEEDED, 344 SUPPORT_IS_DISABLED, 372 SESSION_NOT_FOUND, 373 SESSION_IS_LOCKED, 394 QUERY_WAS_CANCELLED, 497 ACCESS_DENIED, 516 AUTHENTICATION_FAILED (each checked in src/Common/ErrorCodes.cpp at v26.9.8.3-stable, together with 73 UNKNOWN_FORMAT and 210 NETWORK_ERROR; recheck at the pinned version).

8.9 Out of chsurface-1Copy Anchor Link

The native protocol (FL5), the MySQL and Postgres wire interfaces of ClickHouse, gRPC interface, Distributed fan-out (FL5), Tier 3 engines, mutations, row policies, users/roles/quotas DDL (CREATE USER, GRANT; Loams's auth applies), SYSTEM commands, backups, Keeper, clickhouse-local file semantics.

8.10 The allowlist process (D348)Copy Anchor Link

• Versioning. The surface has a semver id: chsurface-1.0 at FL2's exit. Adding statements, formats or engines is a minor bump; removing anything or changing a result is a major bump and needs the owner.
• Entries. conformance/clickhouse/allowlist.toml:

[[deviation]]
id        = "CH-0007"
tests     = ["corpus/replacing/equal_versions.sql", "stateless/01509_*"]
category  = "merge-timing"     # merge-timing | semantic | error-text | type-display | unsupported | performance
surface   = "1.0"
reason    = "Fluss Versioned keeps the first row on equal versions; ClickHouse keeps the last inserted"
owner     = "house"
issue     = "dina-kar/operon#NNN"
expires   = "chsurface-2.0"    # optional
approved  = "owner 2026-10-15" # required when category = "semantic"

• Strictness. The harness runs every allowlisted test; an allowlisted test that now passes fails CI with "unexpected pass: remove CH-0007" (strict xfail). A test not on the allowlist that fails blocks the PR.
• Who approves. error-text, type-display and performance entries: any reviewer. merge-timing and unsupported: the House owner. semantic (a different answer to a query inside the surface): the owner, recorded in approved.
• Publication. Each build publishes house-compat-<sha>.json and a Markdown summary (pass rate per category and per corpus, the allowlist size per category). The docs page shows the latest release's numbers.
• Release gate. A release is blocked if any declared-surface test regresses against the previous release, or the allowlist grows without an approved entry.

9. Distributed and S3-backed designCopy Anchor Link

10. Compatibility test plan (D349)Copy Anchor Link

11. Phases: track FL (D351)Copy Anchor Link

12. LicencesCopy Anchor Link

Nothing AGPL, BSL, SSPL or ELv2 is linked or required (D11).

13. RisksCopy Anchor Link

14. Conflicts with existing decisions, and how they are resolvedCopy Anchor Link

15. Open questionsCopy Anchor Link

16. SourcesCopy Anchor Link

All read on 2026-10-01 unless stated.

• Apache Iggy: github.com/apache/iggy master (README, core/server/config.toml, core/connectors/{sdk,sinks,sources}, gateways/kafka/README.md), release server-0.9.0 (published 2026-09-18) and its notes; iggy.apache.org/blogs/2026/08/24/apache-iggy-top-level-project-tlp-graduation/ (graduated 2026-08-19); crates iggy 0.11.0 and iggy_binary_protocol 0.11.0 (2026-09-18); iggy.apache.org/docs/clustering/vsr; issue #1419 (closed 2026-05-22) and discussion #3312 "Tiered Storage for Iggy" (2026-05-22, comments to 2026-09-08).
• Apache Fluss: github.com/apache/fluss main (website/docs/maintenance/tiered-storage/{lakehouse-storage,remote-storage}.md, website/docs/install-deploy/deploying-streaming-lakehouse.md, fluss-flink/*, fluss-rust/crates/fluss/Cargo.toml, fluss-kafka/), release 1.0.0 (2026-09-22) notes; incubator.apache.org/projects/fluss.html (graduated 2026-07-16); crate fluss-rs 1.0.0.
• chDB: github.com/chdb-io/chdb v4.4.0 (2026-09-11); github.com/chdb-io/chdb-core v26.9.0 (2026-09-28, release assets); github.com/chdb-io/chdb-rust v2.0.0 (2026-09-20, README).
• ClickHouse: github.com/ClickHouse/ClickHouse v26.9.8.3-stable (2026-10-01), Apache-2.0; HTTP interface and formats documentation (to verify per row in FL2 Task 0).
• Grafeo: github.com/GrafeoDB/grafeo v0.5.43 (2026-09-27; README, Cargo.toml, commit history), created 2026-01-26; github.com/GrafeoDB/grafeo-server v0.5.40 (2026-04-20, README).
• Camel: github.com/apache/camel tag camel-4.22.1 (components/, components/camel-iggy/src/main/docs/iggy-component.adoc, components/camel-clickhouse); github.com/apache/camel-karavan 4.18.1 (2026-04-03).
• Kestra: github.com/kestra-io/kestra v2.0.4 (2026-09-29), README; github.com/orgs/kestra-io plugin repositories.
• Debezium: github.com/debezium/debezium tag v3.7.0.Final; github.com/debezium/debezium-server (debezium-server-http, debezium-server-fluss, …).
• Lakekeeper v0.13.6 (2026-09-22); Flink Kubernetes Operator release-1.16.1; iceberg-rust iceberg 0.10.1 (arrow 58); sqlparser 0.63.0.
• Loams: §00 §7, §02 (§5, §7, §7.4, §8.1), §06 §5, §07, §08, §09, §11, §18 §5, §19, §21, §24, §26 (§10, §16), §27, §28 §7.2, §29 (PR #172), docs/open-core.md; crates/loams-cloudevents (CloudEvent::dedup_key), crates/loams-worker, crates/loams-link; decisions D1, D4–D6, D11, D20–D22, D42, D44, D45, D51, D66, D72–D76, D88, D111, D127, D128, D184, D186, D189, D190, D202, D204, D211, D212, D220, D260, D270, D-SC-13, D-SC-15.
• The draft: chatdump.md lines 325–628 (the owner's "Loams: Final Plan", "Précis (CDMP, Java stack)", "Top 200 connectors", "Rollout").