Sync the self-hosting stack

This commit is contained in:
2026-10-21 10:11:48 +02:00
parent 6db65866d2
commit e210b4db11
7 changed files with 82 additions and 25 deletions
+28
View File
@@ -9,6 +9,34 @@ one of them changed. An entry says which artefact and which version, so `web 0.1
Every version published to a channel appears here. Nothing is written from memory afterwards: a build Every version published to a channel appears here. Nothing is written from memory afterwards: a build
cannot be published without its entry. cannot be published without its entry.
## 2026-10-21 — The last TypeScript release `api 0.151.0` `web 0.185.0`
**The last release of the TypeScript api and console, and the one to be on before api 1.0.0** —
Jarvis rewritten on .NET — which starts only on a database api 0.149.0 or later has migrated. It
carries the fixes found while rewriting Jarvis, and no new feature.
- **An api that stayed down after an invitation was accepted starts again.** Since api 0.138.0 the
upgrade check counted a redeemed invitation as a broken one: from the first acceptance on, every
start stopped with `invitation(s) hold no profile … Install api 0.139.1 first`. It no longer does.
- **A fresh compose install starts again**: the secrets directory exists in the image, owned by the
api's user.
- **On Kubernetes, the api writes to the Redis primary.** `02-config.yaml` now sets its two
`REDIS_SENTINEL*` lines, which name what `08-redis-sentinel.yaml` deploys. `REDIS_URL` alone at that
file's `redis` Service, the previous default, could land an api pod on a read-only replica: every
write it made failed with `READONLY` while its readiness probe stayed green. The web's
HorizontalPodAutoscaler moved to the last sync wave, so a first Argo sync succeeds.
- **Security fixes**: revoking a session closes its live connections even when one api pod does not
answer; a tenant's Microsoft Graph token is sent to that tenant's Graph endpoint only, and Graph
risk is judged on the request's own path; PowerShell commands double every quote character; a
diagram exported to SVG keeps only drawing elements; a share link's new expiry applies to the
readers it already admitted; a Teams message and an activity notification wait for approval.
- **The console says when a read was refused or failed**, instead of drawing an empty list or a wrong
answer; dialogs stay open while their write runs; actions are offered only to who may take them;
refusals read in the reader's language.
Update the API and web images together. No agent update is required, and the schema does not
change. On Kubernetes, take the new `02-config.yaml`, or set the two `REDIS_SENTINEL*` lines in yours.
## 2026-10-13 — Organization records take only what they expect `api 0.150.0` `web 0.184.0` ## 2026-10-13 — Organization records take only what they expect `api 0.150.0` `web 0.184.0`
**Each field of an organization's record now checks what it is given.** **Each field of an organization's record now checks what it is given.**
+10 -7
View File
@@ -347,11 +347,14 @@ follow their own upstream tags, `postgres:16-alpine` and `redis:7-alpine`. Schem
themselves when the api starts, and the api and the web are versioned independently — their numbers themselves when the api starts, and the api and the web are versioned independently — their numbers
are not meant to match, because usually only one side changed. are not meant to match, because usually only one side changed.
**A schema change is one-way.** Jarvis has no migration history: each boot force-syncs the database **A schema change is one-way.** Jarvis has no migration history: each boot syncs the database to
to the schema its image carries, adding what a release added and dropping what it removed, without the schema its image carries, adding what a release added. A release that would DROP a table or a
prompting. Pulling an older api image does not undo that — unlike the image itself, the schema stays column is refused instead: the schema is not touched, the api logs which columns or tables it would
where the newer build left it. Restoring a dump is the only way back, which is what makes the one drop and stays down, restarting into the same refusal. If those drops are intended, set
below a prerequisite rather than a precaution. `SCHEMA_ACCEPT_DATA_LOSS=true` in `.env`, `docker compose up -d` once, then remove the line. Pulling
an older api image does not undo a change already applied — unlike the image itself, the schema
stays where the newer build left it. Restoring a dump is the only way back, which is what makes the
one below a prerequisite rather than a precaution.
```sh ```sh
docker compose exec -T postgres pg_dump -U jarvis jarvis | gzip > jarvis-$(date +%F).sql.gz docker compose exec -T postgres pg_dump -U jarvis jarvis | gzip > jarvis-$(date +%F).sql.gz
@@ -559,8 +562,8 @@ These are deliberate and documented rather than surprises waiting to be found.
request. The exception is changing your own password: it revokes every *other* session and keeps request. The exception is changing your own password: it revokes every *other* session and keeps
the one you are changing it from, so a token stolen from that session stays valid until it expires the one you are changing it from, so a token stolen from that session stays valid until it expires
— 15 minutes by default (`JWT_ACCESS_TTL`). An admin-forced reset drops every session. — 15 minutes by default (`JWT_ACCESS_TTL`). An admin-forced reset drops every session.
- **The api container runs as root**, and so does nginx's master process in the web container, - **Neither container runs as root**: the api runs as `node` (uid 1000) and the web's nginx as
though its workers drop privileges. Neither image declares a `USER`. `nginx`, listening on 8080. Secret files mounted into them must be readable by that user.
- **`/api/health` answers 200 with `status: "degraded"`** when the database is unreachable, so the - **`/api/health` answers 200 with `status: "degraded"`** when the database is unreachable, so the
container healthcheck alone is not a liveness signal for the database. container healthcheck alone is not a liveness signal for the database.
- **Outbound mail is Microsoft Graph only.** There is no SMTP and no environment variable for any of - **Outbound mail is Microsoft Graph only.** There is no SMTP and no environment variable for any of
+13 -6
View File
@@ -18,17 +18,24 @@ data:
# #
# A SINGLE REDIS IS A SINGLE POINT OF FAILURE FOR EVERY POD AT ONCE. This line is correct for a # A SINGLE REDIS IS A SINGLE POINT OF FAILURE FOR EVERY POD AT ONCE. This line is correct for a
# managed Redis that fails over behind one address, and correct-but-fragile for one Deployment of # managed Redis that fails over behind one address, and correct-but-fragile for one Deployment of
# one Redis. For Sentinel, comment this out and set the two below instead — see # one Redis. It is IGNORED while the two Sentinel lines below are set — and they are, because
# `08-redis-sentinel.yaml`, which deploys exactly that. # `kustomization.yaml` deploys `08-redis-sentinel.yaml`. With a managed Redis instead: remove that
# file from `kustomization.yaml`, delete the two lines below, and point this one at it.
#
# NEVER THIS LINE ALONE WITH 08 DEPLOYED. Its `redis` Service is headless: the name answers with
# all three Redis pods, and two of them are read-only replicas. Each api connection lands on one
# of the three, so a pod that drew a replica fails every write it makes — `READONLY You can't
# write against a read only replica` — while its readiness probe, which only reads, stays green.
# That was this file's default while the two lines below were commented out.
REDIS_URL: "redis://redis.jarvis.svc.cluster.local:6379" REDIS_URL: "redis://redis.jarvis.svc.cluster.local:6379"
# Set BOTH or neither. Sentinel needs the sentinels and the name of the primary they watch; a # Set BOTH or neither. Sentinel needs the sentinels and the name of the primary they watch; a
# name is refused rather than defaulted, because ioredis's own default (`mymaster`) against a # name is refused rather than defaulted, because ioredis's own default (`mymaster`) against a
# primary called anything else is not an error — it is a pod that never becomes ready, silently. # primary called anything else is not an error — it is a pod that never becomes ready, silently.
# When these are set, REDIS_URL above is ignored. # When these are set, REDIS_URL above is ignored. They name what `08-redis-sentinel.yaml` deploys:
# # its three sentinels, and `jarvis`, the primary they monitor.
#REDIS_SENTINELS: "redis-sentinel-0.redis-sentinel.jarvis.svc.cluster.local:26379,redis-sentinel-1.redis-sentinel.jarvis.svc.cluster.local:26379,redis-sentinel-2.redis-sentinel.jarvis.svc.cluster.local:26379" REDIS_SENTINELS: "redis-sentinel-0.redis-sentinel.jarvis.svc.cluster.local:26379,redis-sentinel-1.redis-sentinel.jarvis.svc.cluster.local:26379,redis-sentinel-2.redis-sentinel.jarvis.svc.cluster.local:26379"
#REDIS_SENTINEL_NAME: "jarvis" REDIS_SENTINEL_NAME: "jarvis"
# ONE, not the two the compose files use, and getting this wrong is a security fault rather than # ONE, not the two the compose files use, and getting this wrong is a security fault rather than
# a cosmetic one. # a cosmetic one.
+9 -2
View File
@@ -147,9 +147,16 @@ spec:
apiVersion: autoscaling/v2 apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler kind: HorizontalPodAutoscaler
metadata: metadata:
# With the Deployment it belongs to. # A WAVE AFTER the Deployment it scales, in the last one, and that is the whole point of the line.
#
# Argo waits for every resource of a wave to report healthy before it starts the next, and reads
# an HPA as Degraded while the HPA reads no metric for its pods. On a FIRST install those pods are
# seconds old and metrics-server has not scraped them yet ("the HPA was unable to compute the
# replica count … no metrics returned"), so in wave 1 the first sync failed on it every time, and
# only an automated policy's retry got past it. The last wave is applied and not waited on: the
# sync succeeds, and the HPA turns healthy by itself within a minute. Nothing reads it before.
annotations: annotations:
argocd.argoproj.io/sync-wave: "1" argocd.argoproj.io/sync-wave: "2"
name: jarvis-web name: jarvis-web
namespace: jarvis namespace: jarvis
spec: spec:
+5 -1
View File
@@ -17,7 +17,11 @@
# during the switch — see "What a failover actually costs" at the bottom. # during the switch — see "What a failover actually costs" at the bottom.
# #
# kubectl apply -f 08-redis-sentinel.yaml # kubectl apply -f 08-redis-sentinel.yaml
# # then in 02-config.yaml: comment out REDIS_URL, uncomment the two REDIS_SENTINEL* lines #
# 02-config.yaml points the api at it already: its two REDIS_SENTINEL* lines name these sentinels
# and the primary they monitor. Skip this file for a managed Redis, and delete those two lines with
# it — while they are set, REDIS_URL is ignored. Never point REDIS_URL at the `redis` Service below
# instead: it is headless, and two of the three pods it answers with are read-only replicas.
# #
# ────────────────────────────────────────────────────────────────────────────────────────────── # ──────────────────────────────────────────────────────────────────────────────────────────────
apiVersion: v1 apiVersion: v1
+14 -7
View File
@@ -50,10 +50,13 @@ and an index can only do that because one server decides. Do not point pods at r
`(replicas + maxSurge + jobs) × connection_limit + reserve`: with 3 api pods at 10 connections `(replicas + maxSurge + jobs) × connection_limit + reserve`: with 3 api pods at 10 connections
each, a migration Job at 5, and headroom, about **150**. The stock Postgres image ships 100, and each, a migration Job at 5, and headroom, about **150**. The stock Postgres image ships 100, and
exhausting it presents as an api that starts and then cannot serve. exhausting it presents as an api that starts and then cannot serve.
- **Redis.** [`08-redis-sentinel.yaml`](08-redis-sentinel.yaml) deploys one, or point `REDIS_URL` at - **Redis.** [`08-redis-sentinel.yaml`](08-redis-sentinel.yaml) deploys one — three Redis, one of
a managed Redis that fails over behind a single address and skip that file. Persistence is not them the primary, and three sentinels — and `02-config.yaml`'s two `REDIS_SENTINEL*` lines point
required: everything Jarvis keeps in Redis is ephemeral by design, which is exactly why the run the api at it. Or use a managed Redis that fails over behind a single address: skip that file,
lock is in Postgres instead. delete those two lines, and set `REDIS_URL` to it. Never `REDIS_URL` alone at the file's `redis`
Service: it is headless, two of the three pods it answers with are read-only replicas, and an api
pod that draws one fails every write. Persistence is not required: everything Jarvis keeps in
Redis is ephemeral by design, which is exactly why the run lock is in Postgres instead.
- **An Ingress controller and a TLS terminator.** Jarvis speaks plain HTTP. - **An Ingress controller and a TLS terminator.** Jarvis speaks plain HTTP.
## Order ## Order
@@ -67,7 +70,7 @@ kubectl apply -f 01-secret-job.yaml # the half that is random bytes (optio
kubectl wait --for=condition=complete job/jarvis-mint-secrets -n jarvis --timeout=2m kubectl wait --for=condition=complete job/jarvis-mint-secrets -n jarvis --timeout=2m
kubectl apply -f 02-config.yaml kubectl apply -f 02-config.yaml
kubectl apply -f 08-redis-sentinel.yaml # only if you are running your own Redis kubectl apply -f 08-redis-sentinel.yaml # unless you use a managed Redis (then edit 02-config.yaml)
# The schema, to completion, before any api pod starts. # The schema, to completion, before any api pod starts.
kubectl apply -f 03-migrate-job.yaml kubectl apply -f 03-migrate-job.yaml
@@ -130,10 +133,14 @@ thing being ordered:
| `-1` | `jarvis-mint-secrets` **(hook)** | the migration reads `DATABASE_URL` out of what it writes | | `-1` | `jarvis-mint-secrets` **(hook)** | the migration reads `DATABASE_URL` out of what it writes |
| `0` | `jarvis-migrate` **(hook)** | an api pod against an unmigrated schema is the failure all of this prevents | | `0` | `jarvis-migrate` **(hook)** | an api pod against an unmigrated schema is the failure all of this prevents |
| `1` | api and web Deployments, Services | the web pod's `/readyz` proxies the api's health endpoint, so it cannot be ready before the api is | | `1` | api and web Deployments, Services | the web pod's `/readyz` proxies the api's health endpoint, so it cannot be ready before the api is |
| `2` | Ingress, PodDisruptionBudgets | an Ingress applied early is a route to a 503 | | `2` | Ingress, PodDisruptionBudgets, the web's HPA | an Ingress applied early is a route to a 503; the HPA, below |
Argo does not begin a wave until the previous one reports healthy, which is what the `kubectl wait` Argo does not begin a wave until the previous one reports healthy, which is what the `kubectl wait`
lines are doing by hand. lines are doing by hand. The last wave is applied and not waited on, which is why the web's
HorizontalPodAutoscaler is there rather than beside its Deployment: on a first install it has no
metric for pods seconds old, Argo reads that as Degraded, and in wave 1 it failed the first sync
every time. The application can read Degraded for the minute after a first install, until
metrics-server has scraped the new pods; the sync itself has succeeded.
### Why the two Jobs are hooks ### Why the two Jobs are hooks
+3 -2
View File
@@ -42,8 +42,9 @@ resources:
# host along with the main one. # host along with the main one.
- 06b-ingress-uploads.yaml - 06b-ingress-uploads.yaml
- 07-disruption.yaml - 07-disruption.yaml
# Remove this line if you point REDIS_URL at a managed Redis. It deploys three Redis processes and # Three Redis processes and three sentinels; see the README on what that costs on three nodes.
# three sentinels; see the README on what that costs on three nodes. # 02-config.yaml's two REDIS_SENTINEL* lines point the api at them. For a managed Redis instead,
# remove this line AND those two lines, and set REDIS_URL there to your Redis.
- 08-redis-sentinel.yaml - 08-redis-sentinel.yaml
# The headless browser and the NetworkPolicy that isolates it. COMMENTED OUT: web access is a # The headless browser and the NetworkPolicy that isolates it. COMMENTED OUT: web access is a
# decision rather than a default, and an assistant that can read the internet is a new class of # decision rather than a default, and an assistant that can read the internet is a new class of