Memini Docs

Developer guide

Memini runs one Python service and a PostgreSQL database with pgvector. HTTP and MCP share the same query and decision services. The CLI can call HTTP remotely or run operator commands next to the database. Start at the boundary where you observe the problem, then follow the relevant path below.

Starting point Where to look
A CLI option, exit code or HTTP request CLI entry point and command registration, HTTP client, CLI tests with a mock service
An HTTP route or MCP tool FastAPI application factory, MCP server, tool contracts and examples
A scheduled sync or background job Handler wiring, scheduler, queue, worker, failure and recovery tests
A schema change or stored value SQL migrations, migration runner, schema constraints and scope tests

Trace a push from the CLI to the index

  1. memini push reads records or batches and sends them to the source's records route. A batch carries whole records. A second version of a thread replaces its messages as a unit. The route validates the wire shape and checks the token's ingest scope. CLI · HTTP route · sample inputs
  2. Batch preparation validates payloads and redacts secrets. The record ID derives from source ID, kind and external ID, so a record keeps its identity when another stream of the same source encounters it. The batch writer resolves source access, applies filters, upserts records and reconciles eligible snapshots. A complete window advances its checkpoint in the data transaction. An incomplete window can store partial records while retaining the previous checkpoint. Validation · ID contract · batch writer · HTTP retry and scope tests
  3. With embeddings enabled, the payload indexer calls the chunker and index writer in the batch transaction. The writer reuses vectors for identical text under the same embedding key, preserves anchors and refuses a stale workspace fingerprint. A push without an embedding provider can store raw data for a later rebuild. Payload indexer · index writer · fingerprint · rebuild tests

If a late reply disappears, first inspect the submitted whole thread, then its ID and window, then chunk anchors. If raw text exists but search cannot find it, check whether the batch indexed it, whether the fingerprint matches and whether the query token can see the workspace.

Trace a Git or MCP sync

A Git sync fetches into a bare mirror, compares the new commit with the stream's pinned commit and indexes selected files in batches. A failure after a committed batch leaves the previous pin in place and marks the sync partial. A retry reconciles the ref. A search locator identifies the commit and line range to read. Sync and transaction boundaries · chunker · partial sync, retry and reindex tests

MCP intake uses a recipe to plan missing calls. The service executor or agent session collects raw responses. Parsing and completeness checks run on the service, allowing a parser fix to replay stored responses. Look at a window's missing calls before changing its checkpoint logic. Executor · agent session · parser · replay · executor parity tests

Trace a search or journal change

Change storage and authorization together

Check the operation's scope at its route or tool. Authorized reads carry a ReadFilter into the SQL query. Writes carry WriteAccess when locating mutable rows. Workspace-scoped tokens must see the same permitted rows through HTTP, MCP and searches across sources. Test the allowed and denied workspace with actual rows. Authorization predicates · HTTP dependencies · access tests

Add the next consecutive NNNN_name.sql file for a schema change. The runner serializes migrations with an advisory lock and commits each migration with its checksum. Editing an applied file makes the runner refuse the database. Exercise the schema against an empty database and, for changes to existing rows, an upgrade with earlier data. Migration tests · release upgrade check

Verify a change

Run from the checkout after development setup. A quick path through the contracts above is:

uv run pytest -q tests/test_ingest.py tests/test_index_fingerprint.py tests/test_migration_files.py
uv run pytest -q tests/test_api_ingest_db.py::test_a_push_source_takes_a_batch_and_the_same_batch_again_changes_nothing
uv run pytest -q tests/test_query_db.py::test_http_search_routes_mirror_the_tools
uv run pytest -q tests/test_supersession_db.py::test_thread_b_supersedes_thread_a_and_the_restatement_is_a_duplicate

The last three commands need PostgreSQL with pgvector. Start the development database with make db or set MEMINI_TEST_DATABASE_URL to a test server whose role can create disposable databases. Tests create and drop their own databases. For a merge, run make check and make test-integration. For retrieval or prompt changes, also compare the relevant retrieval evaluation or decision evaluation. Use synthetic fixtures. Contribution checks

Implemented and planned

The current tree includes Git indexing, push and recipe-based intake, search, identity resolution, manual and extracted decisions, and one principal with workspace-scoped tokens. The bundled recipe directory contains no server-specific TOML in this version. Hearth is an executable example with a test parser. Multi-user membership and per-user permissions, native mail connectors and a compiled wiki are roadmap items. Use the requirements for scope, the milestone plan for acceptance scenarios and Beads for current work.