# Sibyla — Backlog v6.0 **Plan of record:** `docs/V6.md` **Date:** 2026-08-10 · rev. 9 **Supersedes:** `docs/project-todo.md` (v5.0), `docs/project-todo-v4.md` (v4.0) > **S0 is down to one item:** S0-4 — run one real document through the stack and fill in > `docs/STATE.md`. S0-5 is that task's output. Everything else in S0 is closed. ## Legend `[ ]` not started · `[~]` in progress · `[x]` done · `[!]` blocker or standing caution · `[D]` deliberately deferred to Phase 2, with its reinstatement trigger in `V6.md` §7 **Detail is capped at the current phase plus the next** (V6.md §8 rule 3). S0–S2 are itemised. S3–S7 are sketched at the level needed to sequence and staff them, and get itemised when the phase before them closes. This is deliberate — v5.0 itemised fourteen phases in detail and delivered one, because detail written three phases ahead is written from assumptions the current phase is about to invalidate. **Counting.** 68 live items across S0–S7 plus 3 standing platform constraints. Gates, reviews and decisions are not backlog items and never move the numerator; only demonstrated work does. A phase closes on its exit criterion being demonstrated to a person, not on its boxes being ticked. --- ## Decisions (V6.md §2) | # | Decision | Date | Reverses if | |---|---|---|---| | `[x]` V6-D1 | July stack is the spine; FDR tables are the schema | 2026-08-10 | S0-4 finds the July stack unrecoverable | | `[x]` V6-D2 | Cegid izibizi is the first ERP target | 2026-08-10 | izibizi credentials prove unobtainable | | `[x]` V6-D3 | No document data imported; documents arrive one at a time through channels | 2026-08-10 | an accounting or legal requirement to hold history inside Sibyla appears | | `[x]` V6-D4 | Only Layer 2 and Layer 4 data imported | 2026-08-10 | with V6-D3 | | `[x]` V6-D5 | Security hardening deferred while agents and skills are still changing | 2026-08-10 | pilot runs on real company data (O5 stays a production gate regardless) | | `[x]` V6-O1 | **Verification replaces ceremony.** Pipeline green → merge; adversarial pass comments, never gates; fix forward; a review past ~20 min means the change is too large. Stop list = code that governs irreversible behaviour (destructive migrations · ERP write path · permanent code issuance · live-tenant config), **not** runtime events, which are features (`S2-12`, `S3`, `S4-2`). Full text in `V6.md` §2 | 2026-08-10 | a production defect traces to a skipped review — and then by tightening verification first | --- ## S0 — Stop, ground, restart · ~3 days · 8 items The goal is an honest picture, not a plan. Nothing here produces a feature. - `[x]` **S0-1** V6-O1 decided 2026-08-10 — verification replaces ceremony. The v5 per-piece gate structure is withdrawn in full. Recorded in `V6.md` §2 and `AGENTS.md` - `[x]` **S0-2** Close P1-1b. Piece D accepted and merged 2026-08-07. **Scopes 1–8 revoked 2026-08-10** — both go-ahead documents carry a REVOKED banner and moved to `docs/archive/`. Nothing remains - `[x]` **S0-3** The nine P1-1a / Piece A-B-C migration files are on `main`; the GitLab pipeline builds Debug + Release and runs the tests green against them (MR!36). Additive and paid for - `[ ]` **S0-4** Stand up a dev environment and run the July stack end-to-end against the current schema. Upload one real invoice PDF through `DocumentUpload.razor`; follow it through `ExtractDocumentJobHandler`; record what happens at each of the nine steps in V6.md §3 - `[~]` **S0-5** The one-page state note. `docs/STATE.md` exists, replaces `PROJECT-STATE.md` as the live status document, and carries the nine steps honestly marked `unknown`. **S0-4 completes it** — which steps work, which are broken, which are missing - `[x]` **S0-6** Process corpus archived to `docs/archive/` — 62 files, 1.7 MB, `git mv` so history follows. `docs/*.md` went from 86 files to 26. Kept live out of the P1-0 set: `p1-0-schema-mapping.md` and `p1-0-vocabularies.md`. `docs/archive/README.md` states the folder carries no authority - `[x]` **S0-7** izibizi prerequisites defined and owned — **B1 Miguel**, **B2 Luís**; four items reduced to two, both week-1 work. `docs/deployment/izibizi-activation.md` - `[x]` **S0-8** Run restore, Debug/Release builds and tests on every GitLab branch push and merge request to `main`; require a green pipeline before merge **Exit:** the one-page picture of the nine steps exists in `docs/STATE.md`. (V6-O1 was the other half of this criterion and closed 2026-08-10.) --- ## S1 — Reference layer live · ~1 week · 9 items Every table below already exists — `P11aSchemaSql.cs` creates all of them. This phase is a seeder and a screen, not a migration. ### Seed — Layer 2, Entity Item Classification (~4,490 rows) - `[ ]` **S1-1** Idempotent seeder, upsert by natural key, logs what changed, safe to re-run. One command, one transaction per table, no import-provenance machinery - `[ ]` **S1-2** ENTMST `entities.json` 119 · ENTALS `entals.json` 7 · ENTBNK `entbnk.json` 47 - `[ ]` **S1-3** ITMMST `productmaster.json` 391 · ITMALS `itmals.json` 4 · ENTITM `entity_products.json` 327 · ITMCLS `itemclass_taxonomy.json` 51 - `[ ]` **S1-4** MNGACC `itemclass_mngacc.json` 34 · SNCACC `sncacc.json` 3,509 · COCACC `cocacc.json` 1 placeholder ### Seed — Layer 4, Auxiliary Calculation (75 rows) - `[ ]` **S1-5** EXCRAT `exchangerates.json` 75 ### Seed — configuration, not data (~82 rows) - `[!]` **S1-6** DOCTYP `document_type_rules.json` — 19 coded rows, one uncoded row excluded. *Confirmation pending: Miguel named Layers 2 and 4; DOCTYP and DOCEFL are Layer 1 but are rule catalogues rather than document data, and the pipeline is inert without them* - `[!]` **S1-7** DOCEFL `flag_evaluation.json` — 62 rows. Same confirmation as S1-6. Note the set has grown 45 → 52 → 62 since the pin; take the current file, not the pin - `[ ]` **S1-8** CodeLedger high-water marks from `permanent_code_ledger.json`, 7 buckets, so newly issued codes cannot collide with FDR codes if history is ever loaded ### Admin UI - `[ ]` **S1-9** Reference-layer browse, search, edit, create. Extend `Suppliers.razor` rather than building new. Order: entities and aliases → items, item classes and accounts → FX rates **Exit:** an operator searches "CTT" and sees the entity, its aliases, its bank accounts, its usual items and the accounts those items post to. Re-running the seeder changes nothing. --- ## S2 — Single-document intake on the FDR schema · ~3 weeks · 14 items The phase that converts the FDR port from a programme into a feature. **This and S4 are the two that matter.** ### Shape - `[ ]` **S2-1** Adopt the two-record split: `Document` / `ExtractionRevision` / `FileAsset` remain the **intake** record (file, hash, storage, extraction attempts, raw agent output); `DOCLOG` / `FDCHDR` / `FDCDTL` become the **registry** record. One `Document` ↔ one `DOCLOG`, keyed by `LGCode` on `(Filename, EntryCode)`. `DocumentCatalogingService` is the bridge. Upload, storage, hashing, duplicate detection, agent invocation, contract validation and PDF text extraction are untouched — none of it is registry-specific and all of it works ### Persistence and classification - `[ ]` **S2-2** Re-point `DocumentCatalogingService` (237 lines) to write DOCLOG + FDCHDR + FDCDTL transactionally - `[ ]` **S2-3** Classification against DOCTYP rules, replacing the `FinancialDocumentType` enum path. `Treatment` (Include/Exclude) and `ProcessingRoute` (PostFiscal / ArchiveOnly / ReferenceOnly / Review) decide where the document goes - `[ ]` **S2-4** EntryCode issuance from the CodeLedger sequences - `[ ]` **S2-5** `NOT_A_DOCUMENT` routing and byte-identical resubmission auto-discard — both already partly built ### Entity and line resolution - `[ ]` **S2-6** Entity resolution at write time against ENTMST / ENTALS: NIF, then alias, then name. Provisional entities `[ISO2]-SYN-######` when nothing matches - `[!]` **S2-7** The `counterparty_not_internal_company` gate — a supplier must not be one of our own companies. *This is the class of error the Tranquilidade case exposed (`document-cataloging-remediation-plan.md`); its root cause is supplier resolution running only when `Direction != Unknown`. Fix it here, properly* - `[ ]` **S2-8** Line classification against ENTITM → ITMCLS → MNGACC → SNCACC - `[ ]` **S2-9** FX via EXCRAT when the document is not in EUR - `[ ]` **S2-10** VAT-splitting gates and date rules ### Flags - `[ ]` **S2-11** DOCEFL rule evaluation producing DOCFLG instances - `[ ]` **S2-12** `blocking_open_instances()` wired in from the start, not applied by discipline. No grandfathering — with no historical instances, rules simply apply from go-live ### Quality - `[ ]` **S2-13** Build the golden set: ~20–30 real PDFs across purchase, sale, credit note, receipt, foreign-currency, multi-page and unreadable-scan cases. **Build it early in the phase, not at the end** — it is how skill regressions become visible in hours instead of weeks - `[ ]` **S2-14** Golden-set harness re-runs on every skill or model change, emitting one pass rate. **That number replaces the five data-quality baselines and the 55.9% figure as the project's quality metric** **Exit:** drop one real invoice PDF into the web upload; a DOCLOG row, an FDCHDR row and FDCDTL lines appear with the supplier resolved, lines mapped to accounts, and flags raised on whatever was uncertain. Then run all 30 golden documents and record the pass rate. --- ## S3 — Review, correct, learn · ~2 weeks · 6 items Sketch. Itemise when S2 closes. - `[ ]` **S3-1** DOCRQE as the persisted review queue over real DOCFLG instances, severity-ordered - `[ ]` **S3-2** Review UI — extend `DocumentReviewQueue.razor` and `DocumentReviewDetail.razor` to the per-field view - `[ ]` **S3-3** The five decisions: Accept · Reject · Accept with Changes · Need More Information · Defer - `[ ]` **S3-4** **The learning path — the point of this phase.** "Accept with Changes" on an entity or item writes back to Layer 2: a new ENTALS alias, a new ENTITM mapping, a corrected account. The next document from that supplier resolves without a human - `[ ]` **S3-5** Comment log on the document — already built, keep and wire - `[ ]` **S3-6** Discard and purge with tombstones **Exit:** an operator corrects a mis-resolved supplier once; the next document from that supplier resolves automatically, with the alias visible in the admin UI. Measure the golden-set pass rate before and after ten corrections. --- ## S4 — Cegid izibizi push · ~2 weeks · 5 items + 2 blockers Sketch. **The operational blockers are the highest-probability schedule risk in the plan and have zero engineering dependency — start them in week 1, not week 8.** - `[ ]` **S4-1** Re-point `IntegrateDocumentJobHandler` and the Cegid purchase writer from `Document` to FDCHDR / FDCDTL. API client, OAuth, product-code derivation, `external_reference` idempotency, PDF upload and `WriteUnknown` handling stay as built - `[ ]` **S4-2** Approval gate — only a document with no open blocking flags and an explicit human approval is eligible to push - `[ ]` **S4-3** Sales pull poller re-pointed to the registry tables - `[ ]` **S4-4** Decide draft vs closed on `supplierInvoices/insert`. Deferred since July, now on the critical path - `[ ]` **S4-5** Correct `docs/cegid-integration.md` — it still describes the pre-refactor Cegid-specific ledger and a "Moloni not implemented" placeholder, both superseded **Blockers — defined 2026-08-10 in `docs/deployment/izibizi-activation.md`.** Pilot scope is **one** company; the izibizi account is active with API access; the certificate is self-signed. Both are week-1 work with no engineering dependency: - `[!]` **S4-B1** Certificate + key ring **on the host**. The secret-protection code shipped in the July spine (`5f679a5`, 2026-07-17) — nothing to implement, only to provision. **Blocks B2** — no company connection can be saved until it exists. **Owner: Miguel.** Verify before provisioning: `docs/deployment/izibizi-b1-verification.md` (read-only) - `[!]` **S4-B2** Endpoint, client id and client secret for the pilot company. **Owner: Luís.** Depends on B1 — the secret has nowhere to go until the key ring exists - `[x]` ~~**S4-B3** Fiscal-year subentity per company~~ — **not a dependency.** `to_subentity_id` is discovered by *Discover and test fiscal years* via `GET /fiscal_years_list`. An operator task once B1 and B2 land, not something to chase - `[D]` ~~**S4-B4** DPAPI token cache~~ — **not an izibizi prerequisite.** It serves the Microsoft Graph workbook projection; with `ExcelCommit__MicrosoftGraph__Enabled=false` a document reaches `Integrated` on the Cegid commit alone. Moves to Phase 2 with Excel **Exit:** an invoice uploaded Monday morning is in Cegid izibizi Monday afternoon, PDF attached, `external_reference` recorded, and re-pushing it creates nothing. --- ## S5 — Channels · ~2 weeks · 7 items Sketch. The loop works; now let documents arrive the way they actually arrive. - `[ ]` **S5-1** Configure Hermes `orquestrador` (Apolo) to call `sibyla.channel-intake.v1` - `[ ]` **S5-2** Email intake, both forms - `[ ]` **S5-3** Safe link downloader with a domain allowlist - `[ ]` **S5-4** WhatsApp via Baileys, including LID and E.164 dual-alias authorisation - `[ ]` **S5-5** Mattermost - `[ ]` **S5-6** Admin UI for users, companies, departments, memberships, channels, allowlists - `[ ]` **S5-7** `HERMES_HOME` backup including the Baileys session **Exit:** forward an invoice from a phone by WhatsApp; it appears in the review queue within a minute, attributed to the right company and the right sender. --- ## S6 — Archive and non-fiscal documents · ~2 weeks · 5 items Sketch. Absorbs `general-document-archive-scope-plan.md` entirely. - `[ ]` **S6-1** Nextcloud/WebDAV archive wired to the pipeline — client foundations exist - `[ ]` **S6-2** DOCARC for non-fiscal documents: bank statements, leases, insurance policies, employment and supplier contracts, licences, financing. `ProcessingRoute = ArchiveOnly` — filed and findable, **never reaching FDCHDR** - `[ ]` **S6-3** Legal path `{Company}/Legal/{Subtype}/{CounterpartyCodeName}/` - `[ ]` **S6-4** Periodic PostgreSQL ↔ Nextcloud reconciliation - `[ ]` **S6-5** Archive naming without a ledger. `ArchiveNameLedger` with per-context non-reusable sequences is deferred until filename collisions actually occur **Exit:** a lease PDF arrives by email, is archived under Legal, is findable by counterparty, and produced no fiscal entry. --- ## S7 — Pilot · 4 weeks running · 8 items Sketch. - `[!]` **S7-0** **O5 — restricted-token / service-identity validation on the target host.** Owner Miguel, carried since v3.2. A **production gate**, not a development gate: nothing in S0–S6 waits on it, and it must close before S7 runs on real company data. Preparation scripts exist and have been reviewed four times; they need one session on the target host - `[ ]` **S7-1** One company, real documents, every day, through real channels, by the people who will use it - `[ ]` **S7-2** Expired-lease recovery - `[ ]` **S7-3** Dead-letter alerting and replay - `[ ]` **S7-4** Off-host backup with one tested restore - `[ ]` **S7-5** One worker-kill drill - `[ ]` **S7-6** A small metrics set — documents in, auto-resolved, human-touched, pushed, failed - `[ ]` **S7-7** Weekly spot-check of a sample against the ERP Keep one list throughout: what broke, what a human had to fix, what the system should have known. **Exit:** four weeks of daily use. **That list — not the v5 backlog — determines Phase 2.** --- ## Phase 2 — deferred, with reinstatement triggers Not scheduled, not estimated, not designed. Each carries the condition that would bring it back, so it can be revisited on evidence rather than re-argued. Full reasoning in `V6.md` §7. - `[D]` Bank reconciliation, Layer 5 — BNKMOV / BNKCHK / BNKMAT / BNKREC / RECREV, 11 governed match types, statement parsers, Via Verde, Stripe tolerance. Statements are still archived from S6. Reinstate once S7 has produced a few months of real posted documents to reconcile against - `[D]` Layer 6 ground truth — PAYCTR / PAYDTL / RCVCTR / RCVDTL, `ControlGoverningPeriod`, `UpsertControlSnapshot`, the SQLSTATE 23514 period-regression trigger. With Layer 5 - `[D]` The matcher governance triad — `MatcherProposal`, `RejectedMatcherConstraint`, `RejectPreviouslyRejectedProposal`. With Layer 5 - `[D]` `SourceKeyHash` generated column and the `pgcrypto` fail-closed preflight. With Layer 5 - `[D]` Historical document import and its grandfathering machinery — `EnforcementStartsAt` seeding, `IsGrandfathered`, the immutable reject trigger, TC5, `ImportBatch`, `ImportEvidenceRow`, the pinned-blob roster preflight - `[D]` Multi-provider-per-company and Excel-as-provider — 0..N connections, `WorkbookDefinition`, per-flow toggles, `SalesOut` (no consumer), RF6 divergence detection - `[D]` Moloni activation — adapter built and parked. A business blocker, not an engineering one: the client's API subscription and developer credentials - `[!D]` Primavera / PHC — **do a two-day mechanism spike before promising anything to a customer.** Both are commonly integrated over SQL Server, ODBC or file exchange rather than REST, so the `Integration*` HTTP-adapter core may not fit at all. Largest unpriced risk in the roadmap - `[D]` Excel export parity — byte-comparable output against the 31-sheet prototype workbook. Reinstate only if someone's actual work depends on that exact workbook - `[D]` Deep Legal/Procurement metadata and alerting — `EffectiveDate`, `ExpiryDate`, `RenewalDate`, `NoticePeriodDays`, `AutoRenews`, expiry jobs, full-text search. Likely wanted after S7, and cheap once archiving works - `[D]` `document-cataloging-remediation-plan.md` Phase 3 — `SupplierExtractionProfile`, hint transport, `OperatorHintJson`, reprocessing. Its root cause is fixed in S2-7; reinstate only if the golden set shows per-supplier hints are the remaining gap - `[D]` Security hardening (V6-D5) — agent SOUL/SKILL SHA validation, full security battery re-execution per change, API-key secret store, egress policy, cost caps and usage alerting **Not deferred, because they are architecture rather than hardening and are expensive to retrofit:** the AI-proposes / code-decides boundary; per-job agent input and output directories; append-only audit; company scoping on every document. --- ## Platform — three rules that cost nothing now and save a rewrite later Sibyla is the first module of a platform that will also carry Medusa (CRM agents) and Tantalus (marketing agents). No work on either in Phase 1, and no abstraction built in anticipation of them — but these three constraints apply to everything built in S0–S7. - `[ ]` **P-1** The entity registry is platform-level, not Sibyla-level. ENTMST is the single record of who a counterparty is; Medusa's contacts point at it rather than copying it - `[ ]` **P-2** The job engine takes a payload it does not understand. No document-shaped fields on `Job` - `[ ]` **P-3** Channel intake produces a generic "something arrived from someone on some channel" record. Routing to a module is a separate decision — a WhatsApp message may be an invoice (Sibyla) or a client request (Medusa) --- ## Working rules (V6.md §8) 1. One status document, one page, updated in place. Over two pages means it is being used as a diary and needs cutting 2. A phase is done when its exit criterion is **demonstrated to a person**, not when its checklist is ticked 3. Plan detail is capped at the current phase plus a sketch of the next 4. Reviews are scoped, not universal (V6-O1). Fix forward 5. The golden set is the quality metric — one number, honest, trending 6. Decisions live in one table, one line each: decision, date, reason, what would reverse it --- ## Item count | Phase | Items | Detail level | |---|---|---| | S0 Stop, ground, restart | 8 | itemised | | S1 Reference layer | 9 | itemised | | S2 Single-document intake | 14 | itemised | | S3 Review, correct, learn | 6 | sketch | | S4 Cegid izibizi push | 5 + 2 blockers | sketch | | S5 Channels | 7 | sketch | | S6 Archive | 5 | sketch | | S7 Pilot | 8 | sketch | | Platform constraints | 3 | standing | | **Total live** | **69** | | | Phase 2 deferred | 12 groups | not scheduled | For comparison: v5.0 carried roughly 152 open items across fourteen phases, with no estimates on any of them, and ERP integration in none of them.