Engineering
A result from the literature is reproduced on the actual problem, built as a system, and deployed into existing software or as a standalone product.
01 Research
Current literature is read, reproduced and evaluated against the actual problem before a system is designed around it.
- The question, setup and measurement are fixed in writing before any code runs: which claim of which paper; the data split, baseline and compute budget; the metric and its threshold.
- The result is reproduced on the paper's own setup first, so that a failure on the actual data is attributable to the data and not to the implementation.
- A method f moves to 02 only if it improves on the baseline fbase on held-out data from the problem, m(f, 𝒟test) > m(fbase, 𝒟test) with m the metric fixed in the setup, at an inference cost per request the deployment can carry.
- Each study is published as a note with its code under Research.
02 Systems
The method carried forward from 01 is built as an agent, a trained model, or both, and evaluated against the measurement fixed there.
Agentic systems
An agent is a policy π(a | s) over tool calls, run as an explicit state graph, with its tools exposed over MCP.
- Graph state is typed. Every transition logs the tool called, its arguments, its result, latency and token cost.
- Retrieval returns d* = argmaxd∈𝒞 sim(φ(q), φ(d)), φ an embedding model, over a corpus 𝒞 held in a vector index that is rebuilt from the source system.
- A change to a prompt, model or tool is merged only if Êτ∼π[R(τ)], the mean task score over trajectories τ on a fixed task set, does not decrease.
- Tools with side effects are allow-listed per graph node and execute behind an approval step.
ML systems and MLOps
Training minimises ℒ(θ) = E(x,y)∼𝒟[ℓ(fθ(x), y)]; the data split that defines 𝒟 is versioned with the run.
- Serving: an HTTP service in Docker on Kubernetes, scaled by the Horizontal Pod Autoscaler, exposing request metrics for scraping.
- Each service carries an SLO as an inequality over request latency t: P(t ≤ tmax) ≥ 1 − ε, with tmax and ε set per service.
- Rollout is canary: the candidate model takes traffic share w, and w grows only while the SLO holds.
- Input and prediction distributions are streamed to an event log and compared with the training distribution. Drift raises an alert; it does not trigger an unattended retrain.
- Where a prediction has to be justified to the person it affects, feature attributions are computed on the served model by integrated gradients or a comparable method, and returned with the prediction.
03 Deployment
The system from 02 ships one of two ways: as a dependency of software that already runs, or as a product of its own.
Into existing software
The model is a dependency of your system, called through interfaces you already operate.
- REST endpoints behind your API gateway, consumers on your event streams, reads and writes against your relational database or warehouse.
- Authentication runs through your identity provider; the service keeps no user store of its own.
- Deployment targets your Azure, AWS or GCP account, or your Kubernetes cluster. Data stays there unless the scope states otherwise.
- Handover is the repository: source, Dockerfiles, Kubernetes manifests, CI pipeline, runbook.
As a new product
The same system, run as a standalone service with its own interface, users and billing. Products shipped this way are listed under Labs.
- Interface: a versioned REST API with an OpenAPI schema, and a web client built on the same endpoints.
- Tenancy: every record and index is keyed by tenant k, and retrieval for k takes the argmax over its own corpus 𝒞k only.
- Auth: OIDC sign-in for users, scoped API keys for programmatic access, roles checked per endpoint.
- Metering: every request emits a usage event (tenant, endpoint, tokens, compute time) to an append-only log, and each billing period is invoiced from those events, so invoices and logs reconcile.
- Observability: the per-transition logs from 02, request traces through OpenTelemetry and request metrics, both labelled by tenant.
- The SLO and rollout from 02 apply per public endpoint: P(t ≤ tmax) ≥ 1 − ε, and a release takes traffic share w that grows only while the inequality holds.
04 Delivery
An engagement is a finite list of deliverables D1, …, Dn, fixed before work starts.
- Each Di has acceptance criteria that pass or fail: a bound E(x,y)∼𝒟test[ℓ(fθ(x), y)] ≤ ℓmax on a test set you hold, the service SLO under an agreed load profile, or the CI pipeline building and deploying the repository from a clean checkout.
- Di is accepted when its criteria pass. Exclusive rights to Di pass to you on acceptance.
- Liability is defined in the contract.