diff --git a/CHANGELOG.md b/CHANGELOG.md index db37fb3..a0c23bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,88 @@ one of them changed. An entry says which artefact and which version, so `web 0.1 Every version published to a channel appears here. Nothing is written from memory afterwards: a build cannot be published without its entry. +## 2026-09-26 — Attach files to a message, and upload files to a conversation's workspace `api 0.115.0` `web 0.140.0` + +**Drop, paste or pick files into the composer.** Pictures, text files and PDFs, up to 10 per +message, upload as you add them and travel with the message; they are not listed in the +conversation's workspace. A text file or a PDF can be up to 10 MB. A picture is made smaller in your +browser before it leaves — scaled to 2,048 pixels on its long side and stripped of camera metadata — +so one of up to 40 MB can be dropped, and a full-page screenshot is scaled to fit rather than +refused for its height, up to 100 megapixels. The server checks what each file actually contains, +whatever its name says, and strips metadata again. SVG and HTML, Office documents, archives, +executables, and key or secret files are refused, and so is a text file that holds a private key; +scripts and public certificates are accepted as text. + +**Upload files to the workspace** from the Documents panel, by button or by drop, up to 10 MB each. +They become documents everyone in the conversation sees, marked with who uploaded them. A picture +is kept as it was dropped, and the server strips its metadata. Your browser changes it only when it +has to: it scales a picture larger than 8,000 pixels on a side or 40 megapixels in all, turns a +sideways phone photo upright, converts a format Jarvis does not store, such as HEIC, when the +browser can read it, and re-encodes a picture over 10 MB at its full size until it fits — the +upload says so when it does. A download gives back the picture that was stored. A large picture +also travels with a smaller copy made for the model, so the assistant can look at it; people are +always served the picture itself. + +**The assistant can use them.** It reads text files and the text of PDFs — extracted once, when the +PDF arrives; a scan with no text layer has none — and looks at pictures. **What the model +receives:** an attached picture is sent as the picture itself when the assistant answers the +message it came with, and again only if the assistant asks to look at it; a file's text is sent as +the passages the assistant reads. If the model refuses pictures, the turn is answered without them +and the picture says the model could not see it. This is the same on your own provider and on the +managed model. **If you use the managed model,** what it receives now includes the files and +pictures people attach, when the assistant uses them. + +**Who can:** exactly who can send a message in the conversation. **Public links never include +attachments or uploaded files**; a guest sees that a message had attachments, nothing more. The +assistant's replies are still included, and a reply that describes a file describes it to the guest +too. + +**A sent file can be removed** by whoever attached it, or by the conversation's owner. It is deleted +for everyone, the model stops receiving it from the next request on, and the message keeps a note +saying who removed it. That cannot take back what the assistant already read: whatever it quoted in +its replies or in tool output stays in the conversation. + +**Pictures from other sites are no longer loaded until somebody clicks them**, in replies, in +workspace documents and on public links — in conversations written before this version too. A reply +can point a picture at any address, and fetching it tells that site the conversation is being read; +its address can carry text out of the conversation. The address is shown instead, and a click loads +it. Pictures written into the text itself (`data:` images) now display, where they used to draw as +an empty frame. + +**On the audit trail:** an upload to a workspace is `document.upload` and a removal is +`attachment.remove`. An administrator who attaches or removes a file in a conversation they reach +only through their role is also recorded as `conversation.admin_attachment`, and a message they +send records how many files it carried. These entries hold ids, kinds and sizes, never a file's +name. + +**Self-hosters:** api and web; publish the api first. The schema gains two tables, three types and +new columns beside a captured file's bytes, applied automatically when the api starts; it only +adds, so the schema guard has nothing to refuse. No new environment variable. **Your reverse proxy +must accept request bodies of 13,376 KiB on `/api/uploads/`** ([README](README.md#your-reverse-proxy-has-three-requirements)); +below that, files between its limit and 10 MB fail with the proxy's own error while smaller ones +keep working. The web image's own nginx already does. **The api needs more memory:** an upload of +10 MB with its model copy was measured at about 310 MB while it is checked, so each api process now +takes three uploads at a time, and `04-api.yaml` raises the api's memory limit from 2Gi to 3Gi. +Give a compose host the same headroom. **Kubernetes:** apply the new `06b-ingress-uploads.yaml`, +an Ingress for uploads alone, and patch its host and TLS block like the main Ingress's: an overlay +that patches only the first leaves every file between 4 and 10 MB refused by the controller. +`06-ingress.yaml` also had its two `annotations:` blocks merged into one; `kubectl apply -f` kept +only the second, and so dropped the sync-wave. + +**Rolling back past this version is not a plain image change.** An older build's schema sync would +drop the two attachment tables, the new columns and the three types. The schema guard refuses that +as soon as one file document exists — a single file copied off a machine, even before this version, +is enough, since the new columns are filled for every row — or anything has been attached, and the +api does not start. **On compose, set `JARVIS_SKIP_MIGRATIONS=true` in `.env` for the rollback**: +the older build then starts without syncing the schema, runs on the newer one without touching what +it does not know, and nothing is lost; take the variable out again before you upgrade. **On +Kubernetes** the `jarvis-migrate` Job has no such switch; it fails before the api is touched, so +this version keeps serving. Restore the dump you took before upgrading, or set +`SCHEMA_ACCEPT_DATA_LOSS=true` on the Job for that one run and put it back. Forcing it deletes every +attached file for good, and on the next upgrade files uploaded to a workspace come back as captured +files with no source, which the assistant can no longer read or preview — they still download. Take +a dump before upgrading. + ## 2026-09-25 — A fresh Kubernetes install no longer asks for its own vault key `api 0.114.1` **On the published Kubernetes stack, creating the first administrator asked for the last characters diff --git a/README.md b/README.md index 186bb90..a9b707f 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ is listed in full below. - [How it works](#how-it-works) - [What you need](#what-you-need) - [Install](#install) -- [Your reverse proxy has two requirements](#your-reverse-proxy-has-two-requirements) +- [Your reverse proxy has three requirements](#your-reverse-proxy-has-three-requirements) - [Upgrading](#upgrading) - [The agent](#the-agent) - [Things worth knowing before you trust it with production](#things-worth-knowing-before-you-trust-it-with-production) @@ -286,9 +286,9 @@ adopted into the volume rather than replaced, your `WEB_ORIGIN` becomes the stor your model settings are adopted the same way, and an instance that already has an administrator is marked installed by a backfill — so you will never see the install screen. Nothing to do. -## Your reverse proxy has two requirements +## Your reverse proxy has three requirements -Both are the kind that produce confusing symptoms rather than clean errors. +All three are the kind that produce confusing symptoms rather than clean errors. - **Forward the WebSocket upgrade.** Two separate sockets ride `JARVIS_PORT`: the chat, on `/socket.io/`, and enrolled agents, on `/api/agents/ws`. Neither falls back to plain HTTP. Without @@ -298,6 +298,13 @@ Both are the kind that produce confusing symptoms rather than clean errors. - **Give it a long read timeout** — 300s or so. A reasoning model can go 90+ seconds without emitting a byte, and a 60s default cuts the response mid-stream. The client sees a connection reset rather than a timeout, which reads like a bug in Jarvis. +- **Let uploads through.** Attaching or uploading a file posts it to `/api/uploads/`, up to 10 MB + each, and an image uploaded to a workspace travels with a copy of up to 3 MB prepared for the + model. Your proxy must accept request bodies of at least **13,376 KiB** on that path (nginx: + `client_max_body_size 13376k;` — its default is 1 MB). Below that, larger files fail with your + proxy's own error rather than Jarvis's, and smaller ones keep working, so nothing looks wrong + until somebody attaches a big screenshot or PDF. No other request body Jarvis accepts is larger + than 2 MB, so the higher ceiling can be given to that path alone. Forward `X-Forwarded-For` too. How many proxies rewrite it is a setting, and the install screen is where you answer it — **it shows you the chain your own request actually carried and the address the @@ -483,6 +490,38 @@ These are deliberate and documented rather than surprises waiting to be found. conversation's autonomy level, and at *Full access* nothing does. See [What decides whether a tool call runs](#what-decides-whether-a-tool-call-runs). +### Files people attach or upload + +Anyone who can send a message in a conversation can attach files to it, and upload files to its +workspace from the Documents panel. + +- **What is accepted.** Pictures (PNG, JPEG, WebP, GIF), plain text — logs, CSV, JSON, YAML, + configuration files, scripts, public certificates — and PDF, whose text is extracted when it + arrives. Up to 10 MB each and 10 per message; 200 files and 256 MB per conversation, 2 GB per + organization. A picture attached to a message is made smaller in the browser before it leaves, so + one of up to 40 MB can be dropped; one uploaded to the workspace is kept as it was dropped, and + scaled only past 8,000 pixels on a side or 40 megapixels. Jarvis checks what a file actually + contains rather than trusting its name, strips camera metadata from pictures, and refuses the + rest with a reason: SVG and HTML, Office documents, archives, executables, and key or secret + files — including a text file that holds a private key. +- **They are stored in Postgres, unencrypted, like documents.** Unlike the vault and terminal + recordings, nothing seals them. A secret belongs in the vault, not in a screenshot or a pasted + log, and a database dump carries every attached file in the clear. +- **Every participant in the conversation can see them**, now and later, including people added + after the file was sent — until whoever attached a file, or the conversation's owner, removes + it. Removing it deletes the file for everyone; it does not take back what the assistant already + quoted from it. +- **They go to the model when the assistant uses them.** A picture is sent as the picture itself + when the assistant answers the message it came with, and again only if the assistant asks to look + at it; a text file or a PDF is sent as the passages the assistant reads. That is the same on your + own provider and on the managed model. +- **Public links never include them.** A guest sees that a message had attachments and nothing + more. The assistant's replies are still included, though, and a reply that describes a file + describes it to the guest as well. +- **A CSV opened in Excel is Excel's business.** Jarvis serves text files as downloads and does + not rewrite them, so a cell beginning with `=` is a formula when Excel opens it, and the warning + Excel shows is the only one there is. + ## When it does not come up ```sh @@ -590,11 +629,11 @@ then the agents. ## Backups, and restoring one -Postgres holds everything except the vault key — avatars, documents, exports and terminal recordings -are all rows, not files on disk. **Two volumes are worth backing up, and they must go to different -places:** `postgres_data` and `jarvis_secrets`. `redis_data` carries nonces, rate-limit counters and the -socket fan-out, all of which rebuild themselves; `agent_releases` is refilled by the next -`docker compose pull`. +Postgres holds everything except the vault key — avatars, documents, exports, terminal recordings +and attached files are all rows, not files on disk. **Two volumes are worth backing up, and they +must go to different places:** `postgres_data` and `jarvis_secrets`. `redis_data` carries nonces, +rate-limit counters and the socket fan-out, all of which rebuild themselves; `agent_releases` is +refilled by the next `docker compose pull`. They are separate volumes precisely so that they can be, and must be, backed up separately. A dump that travelled with the key that opens it is a dump that opens itself. @@ -824,7 +863,9 @@ configured.** Every request the assistant then makes to the model carries: - each message, and the conversation so far; - the instructions and the list of tools Jarvis gives the model; - the output of every tool the assistant ran — command output, file contents and API responses from - the systems you administer. + the systems you administer; +- the files and images participants attach to a message or upload to a workspace, when the + assistant uses them — an image as the picture itself, a text file or PDF as the text read from it. That includes runs nobody is watching: scheduled wake-ups, and runs resumed after a restart. Your own provider receives exactly the same content on every request — that is how the assistant works on diff --git a/kubernetes/04-api.yaml b/kubernetes/04-api.yaml index 180a822..ee40a19 100644 --- a/kubernetes/04-api.yaml +++ b/kubernetes/04-api.yaml @@ -141,9 +141,49 @@ spec: cpu: 500m # Not generous — honest. A terminal recording holds up to 8 MiB of Buffers in heap # for the life of the session, and argon2 takes 64 MiB per share verification. + # + # Attached and uploaded files add four holders, every one capped, so that their sum is + # a number rather than a hope. Their figures are MEASURED on this image, in a cgroup + # like this pod's, not estimated — the estimates were five to seven times too low. + # + # Uploads being checked and stored: 3 at a time (`files/upload-slots.ts`). Prisma is + # what costs: its JSON protocol turns the bytes into base64, then into a request + # string, and its engine copies that again. A 10 MiB file adds ~200 MiB of RSS while + # it is written; a workspace image with the 3 MiB copy of it prepared for the model, + # ~310 MiB, and ~330 to 360 MiB each with three at once, three bursts in a row. So + # ~1.1 GiB. Six at a time, as this once allowed, took a 2 GiB container to 1.8-1.95 + # GiB with nothing else in it, and three such bursts were OOM-killed. + # + # Files read back — their bytes over HTTP, a window of their text for the viewer or + # for the assistant's `read_attachment`: 8 at a time, up to ~40 MiB each (a 10 MiB log + # is decoded whole, 25 to 39 MiB measured, until its window is cut). So 320 MiB. + # + # Images for the model: 64 MiB of `data:` URLs, reserved before a byte is read, so the + # same 64 MiB also bounds what is being read for them at any instant — at most ~48 MiB + # raw, which Prisma's transfer copies a few times more for the milliseconds a request + # takes to build. So ~200 MiB at the worst instant. + # + # A PDF worker is the one without a flat cap. `resourceLimits` bounds its V8 heap + # (192 MB, plus 32 MB of young generation) and NOT its ArrayBuffers, where a + # decompressed stream lives. Those are read from outside every 50 ms and the worker is + # killed past 128 MB, which a buffer doubling between two reads can overshoot to ~256 + # MB. So one hostile file can hold up to ~500 MB before it dies, two ~1 GB. Measured: + # two decompression bombs at once added ~310 MB to the process, both refused in under + # two seconds. + # + # Every holder full at once is therefore ~2.5 GiB (1.1 GiB of uploads, 320 MiB of + # files read back, ~200 MiB of images, ~1 GB of PDF workers), on top of the API itself + # (~200 MiB once booted): ~2.75 GiB. That cannot fit 2Gi at any upload concurrency + # worth having — at 2Gi, one upload at a time beside two hostile PDFs is all there is + # room for — so the limit is 3Gi, with ~250 MiB left for the terminal recordings and + # share verifications above. It runs well past the room above this request, and that + # is deliberate: the request is what the scheduler sets aside for an ordinary day, not + # for the worst minute. If more margin is ever wanted, the levers are the PDF worker's + # own caps (`files/pdf-text.ts`) and keeping an upload's bytes out of Prisma's JSON + # protocol — not fewer upload slots. memory: 1Gi limits: - memory: 2Gi + memory: 3Gi securityContext: runAsNonRoot: true runAsUser: 1000 diff --git a/kubernetes/06-ingress.yaml b/kubernetes/06-ingress.yaml index bec1abe..586ffe2 100644 --- a/kubernetes/06-ingress.yaml +++ b/kubernetes/06-ingress.yaml @@ -15,13 +15,16 @@ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: - # Last, so that the first traffic admitted from outside meets Services with endpoints behind - # them. An Ingress applied early is a route to a 503. - annotations: - argocd.argoproj.io/sync-wave: "2" name: jarvis namespace: jarvis + # ONE annotations map. This file used to carry two `annotations:` keys under metadata — the + # sync-wave above the name, the proxy settings below it — and tools disagree about a duplicated + # key: kustomize merged the two, while `kubectl apply -f` kept only the second and dropped the + # sync-wave without a warning. What a cluster received depended on which tool rendered the file. annotations: + # Last, so that the first traffic admitted from outside meets Services with endpoints behind + # them. An Ingress applied early is a route to a 503. + argocd.argoproj.io/sync-wave: "2" # An agent is idle between operations and the server pings every 30s, so this only has to # outlast a quiet period comfortably. Too short and a healthy fleet reconnects all day. nginx.ingress.kubernetes.io/proxy-read-timeout: "3600" @@ -29,6 +32,10 @@ metadata: # Matches BODY_LIMIT in apps/api/src/main.ts. A document may hold 400,000 characters, which is # over a megabyte once accented text is UTF-8 encoded, and a diagram export posts back the SVG # the browser rendered. Below this they are rejected as a bare 413 with no message. + # + # File uploads need more and are NOT raised here: they post under /api/uploads/, which has its + # own Ingress in 06b-ingress-uploads.yaml, because ingress-nginx applies these annotations to + # every path of the object that carries them — 13376k here would be 13376k on every API route. nginx.ingress.kubernetes.io/proxy-body-size: "4m" # # NO affinity annotation, and none should be added. The reflex is understandable — there are diff --git a/kubernetes/06b-ingress-uploads.yaml b/kubernetes/06b-ingress-uploads.yaml new file mode 100644 index 0000000..5173fb7 --- /dev/null +++ b/kubernetes/06b-ingress-uploads.yaml @@ -0,0 +1,60 @@ +# File uploads: a second Ingress on the same host, for one path. +# +# A SEPARATE OBJECT, NOT A FOURTH PATH IN 06-ingress.yaml. ingress-nginx applies an Ingress's +# annotations to every path that object declares, so the only way to give /api/uploads/ a larger +# body ceiling without handing the same ceiling to every API route is an object of its own. The +# controller merges both Ingresses into one server for the host and picks the longest matching +# prefix per request, so /api/uploads/ lands here and everything else under /api/ keeps the 4m of +# the main Ingress. +# +# PATCH ITS HOST LIKE THE MAIN ONE. An overlay that rewrites `jarvis.example.com` in 06-ingress.yaml +# must rewrite it here too, TLS block included. Left unpatched, this rule serves a host nobody +# visits and uploads quietly fall back to the /api/ rule — where every upload between 4 and 13 MB +# meets the controller's HTML 413 instead of Jarvis's own answer, and small files keep working, so +# nothing looks broken until somebody attaches a large one. +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: jarvis-uploads + namespace: jarvis + annotations: + # With the main Ingress, for the same reason: last, once the Services have endpoints. + argocd.argoproj.io/sync-wave: "2" + # 13376k is 13 MiB and 64 KiB: exactly the most the API itself reads on this path — a workspace + # image of up to 10 MiB, the copy of it prepared for the model (at most 3 MiB), and 64 KiB for + # the multipart framing and the name (`BODY_MAX_BYTES.documents` + `MULTIPART_OVERHEAD`, in + # apps/api/src/files/upload-gate.interceptor.ts; both refuse strictly above). Not less: a body + # the API would take would meet this ceiling first, and the controller's answer is an HTML page + # the client cannot translate. Not more: the API refuses anything larger unread. A composer + # attachment over 10 MiB still gets through here, and meets the API's own JSON 413. + nginx.ingress.kubernetes.io/proxy-body-size: "13376k" + # Everything the API does with a file happens before it answers: a wait for an upload slot, a + # PDF's text extraction (capped at 20s), a 13 MB row write. Two minutes is headroom over that, + # and far below the hour the main Ingress keeps open for idle websockets, which have no business + # on this path. + nginx.ingress.kubernetes.io/proxy-read-timeout: "120" + nginx.ingress.kubernetes.io/proxy-send-timeout: "120" + # Request buffering is left at the controller's default, ON: the controller reads the whole body + # before it opens a connection to the api, so a slow client costs the controller's disk rather + # than one of the api pod's few upload slots and the memory behind it. + # + # And no `custom-http-errors` override. A controller that sets it replaces the body of the + # statuses it lists, and every refusal the api makes on this path is a 4xx — busy included, + # which is a 429, never a 503. The one 4xx such lists usually name, 404, is a status whose body + # the client never reads. +spec: + ingressClassName: nginx + tls: + - hosts: ["jarvis.example.com"] + secretName: jarvis-tls + rules: + - host: jarvis.example.com + http: + paths: + - path: /api/uploads/ + pathType: Prefix + backend: + service: + name: jarvis-api + port: + name: http diff --git a/kubernetes/README.md b/kubernetes/README.md index 268395b..4483298 100644 --- a/kubernetes/README.md +++ b/kubernetes/README.md @@ -76,6 +76,7 @@ kubectl wait --for=condition=complete job/jarvis-migrate -n jarvis --timeout=15m kubectl apply -f 04-api.yaml kubectl apply -f 05-web.yaml kubectl apply -f 06-ingress.yaml +kubectl apply -f 06b-ingress-uploads.yaml # uploads alone get a ~13 MiB body ceiling kubectl apply -f 07-disruption.yaml ``` @@ -175,12 +176,30 @@ images: - name: git.luxit.be/luxit/jarvis-api digest: sha256:... patches: - - path: replicas.yaml # your node count, your ingress host, your storage class + - path: replicas.yaml # your node count, the host of BOTH Ingresses, your storage class ``` Then the digests you run are in a commit you control, upgrading is a diff you can review and revert, and `?ref=` pins the base to a tag rather than a branch that moves under you. Pin it. +**There are two Ingresses on the one host, and the host patch must reach both**: `jarvis`, and +`jarvis-uploads` in `06b-ingress-uploads.yaml`, TLS block included. Uploads have an Ingress of their +own because annotations apply to every path of the object that carries them, and they are the one +path that needs bodies over 13 MiB. An overlay that rewrites only the first leaves uploads on the +`/api/` rule, where small files still work and anything between 4 and 13 MB meets the controller's +own 413 — so it looks fine until somebody attaches a large PDF. + +To check it once deployed, send an upload of more than 4 MB without signing in, on your own host: + +```sh +head -c 5000000 /dev/zero > big.bin +curl -s -o /dev/null -w '%{http_code} %{content_type}\n' -F file=@big.bin \ + https://jarvis.example.com/api/uploads/orgs/x/conversations/y/attachments +``` + +`401 application/json` is Jarvis turning away an anonymous request, so the body got through. A +`413` with an HTML page is the controller, and means the host patch did not reach `jarvis-uploads`. + ## The three things most likely to bite **Never mint secrets per pod.** On Compose a one-shot generates the vault key and the JWT secrets @@ -263,8 +282,11 @@ outcome. Run it again. `emptyDir` filled by an initContainer is right; a shared RWO volume is not — either every pod is pinned to one node, or the second sits in `Multi-Attach error`. Leaving it unset is supported: everything works except the agent installer, which answers 503 saying no build is published. -- **No application volume, and it should stay that way.** Avatars, documents and terminal recordings - are all in Postgres; exports are rendered in memory. +- **No application volume, and it should stay that way.** Avatars, documents, terminal recordings + and attached files are all in Postgres; exports are rendered in memory. +- **The Ingresses are written for ingress-nginx.** Under another controller their `nginx.ingress…` + annotations do nothing, so set that controller's own equivalents: an hour's read timeout on the + two websocket paths, a 4 MB body ceiling on `/api/`, and 13,376 KiB on `/api/uploads/`. - **Autoscale `jarvis-web`, not `jarvis-api`.** Every api scale-in kills a pod holding agent websockets, live runs and open shells, and CPU is a poor proxy for a load made of long-lived connections. Raise api replicas deliberately. diff --git a/kubernetes/argocd-application.example.yaml b/kubernetes/argocd-application.example.yaml index 29f03dc..af5c03d 100644 --- a/kubernetes/argocd-application.example.yaml +++ b/kubernetes/argocd-application.example.yaml @@ -25,7 +25,8 @@ # - name: git.luxit.be/luxit/jarvis-web # digest: sha256: # patches: -# - path: replicas.yaml # your node count, your ingress host, your storage class +# - path: replicas.yaml # your node count, the host of BOTH Ingresses, your storage class +# # (jarvis and jarvis-uploads, TLS blocks included) # # Then the Application points at YOUR repository, your digests are in a commit you control, and # upgrading is a one-line diff you can review and revert. `?ref=` pins the base to a tag or a commit diff --git a/kubernetes/kustomization.yaml b/kubernetes/kustomization.yaml index ceeadaa..fc5cf72 100644 --- a/kubernetes/kustomization.yaml +++ b/kubernetes/kustomization.yaml @@ -38,6 +38,9 @@ resources: - 04-api.yaml - 05-web.yaml - 06-ingress.yaml + # Uploads get their own Ingress so their larger body ceiling reaches no other path. Patch its + # host along with the main one. + - 06b-ingress-uploads.yaml - 07-disruption.yaml # Remove this line if you point REDIS_URL at a managed Redis. It deploys three Redis processes and # three sentinels; see the README on what that costs on three nodes.