Sync the self-hosting stack
This commit is contained in:
+34
-50
@@ -14,14 +14,21 @@ spec:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: jarvis
|
||||
app.kubernetes.io/component: api
|
||||
# RECREATED, FOR api 1.0.0 ONLY — the release that crosses from the TypeScript images. A rolling
|
||||
# update would keep a TypeScript pod and a .NET pod serving side by side behind one Service, and
|
||||
# their realtime protocols differ under /socket.io/ (Socket.IO against SignalR): every old pod stops
|
||||
# before the first new one starts, a gap of about thirty seconds, once. The next release puts the
|
||||
# rolling update back:
|
||||
#
|
||||
# type: RollingUpdate
|
||||
# rollingUpdate:
|
||||
# maxUnavailable: 0 # never fewer pods than are being served now
|
||||
# maxSurge: 1
|
||||
#
|
||||
# With `kubectl apply` or Argo's default client-side apply this switches both ways. Under
|
||||
# server-side apply, a field manager that owns `rollingUpdate` makes the API server refuse it.
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
# Never fewer pods than are being served now. A run holds its conversation through a unique
|
||||
# index, so a surge pod cannot double-answer anything — the old constraint that made an
|
||||
# overlap dangerous is gone.
|
||||
maxUnavailable: 0
|
||||
maxSurge: 1
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
@@ -31,7 +38,7 @@ spec:
|
||||
# 90 seconds, and every part of it is accounted for:
|
||||
# 10s preStop, so this pod leaves the Service's endpoints BEFORE it starts draining
|
||||
# 25s RUN_SHUTDOWN_GRACE_SEC — in-flight assistant runs stop and close their transcripts
|
||||
# ~15s agent sockets closed and their rows written, httpServer.close(), Prisma, Redis
|
||||
# ~15s agent sockets closed and their rows written, Kestrel, the database pool, Redis
|
||||
# the rest is margin
|
||||
# The default of 30 would SIGKILL the pod in the middle of the run drain, which is the one
|
||||
# thing the drain exists to prevent.
|
||||
@@ -75,14 +82,14 @@ spec:
|
||||
preStop:
|
||||
exec:
|
||||
# Endpoint removal and SIGTERM race each other, and losing that race is visible:
|
||||
# a prompt that arrives after Nest sets `draining` is persisted and broadcast as
|
||||
# a prompt that arrives after the api starts draining is persisted and broadcast as
|
||||
# queued, and this pod will never answer it. Ten seconds is comfortably more than
|
||||
# kube-proxy needs to stop sending new connections here.
|
||||
command: ["/bin/sh", "-c", "sleep 10"]
|
||||
env:
|
||||
- name: JARVIS_SKIP_MIGRATIONS
|
||||
# The schema is the migration Job's business, once, before this rolls. N pods each
|
||||
# running `prisma db push` is N writers racing the same DDL while the previous
|
||||
# migrating is N writers racing the same DDL while the previous
|
||||
# generation is still selecting the columns being altered.
|
||||
value: "true"
|
||||
envFrom:
|
||||
@@ -139,48 +146,25 @@ spec:
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
# Not generous — honest. A terminal recording holds up to 8 MiB of Buffers in heap
|
||||
# for the life of the session, and argon2 takes 64 MiB per share verification.
|
||||
# Not generous — honest. A terminal recording holds up to 8 MiB in memory for the life
|
||||
# of the session, and argon2 takes 64 MiB per share verification.
|
||||
#
|
||||
# Attached and uploaded files add four holders, every one capped, so that their sum is
|
||||
# a number rather than a hope. Their figures are MEASURED on this image, in a cgroup
|
||||
# like this pod's, not estimated — the estimates were five to seven times too low.
|
||||
# Attached and uploaded files are the holders that grow, and every one is capped:
|
||||
# uploads being checked and stored, 3 at a time per pod (`Files/UploadSlots.cs`); files
|
||||
# read back, their bytes or a window of their text; images prepared for the model; and
|
||||
# a PDF's text, read in a worker PROCESS of its own (`Files/PdfText.cs`,
|
||||
# `Files/PdfTextWorker.cs`) that is killed past its own limits, so a hostile file costs
|
||||
# that process and not this one.
|
||||
#
|
||||
# Uploads being checked and stored: 3 at a time (`files/upload-slots.ts`). Prisma is
|
||||
# what costs: its JSON protocol turns the bytes into base64, then into a request
|
||||
# string, and its engine copies that again. A 10 MiB file adds ~200 MiB of RSS while
|
||||
# it is written; a workspace image with the 3 MiB copy of it prepared for the model,
|
||||
# ~310 MiB, and ~330 to 360 MiB each with three at once, three bursts in a row. So
|
||||
# ~1.1 GiB. Six at a time, as this once allowed, took a 2 GiB container to 1.8-1.95
|
||||
# GiB with nothing else in it, and three such bursts were OOM-killed.
|
||||
#
|
||||
# Files read back — their bytes over HTTP, a window of their text for the viewer or
|
||||
# for the assistant's `read_attachment`: 8 at a time, up to ~40 MiB each (a 10 MiB log
|
||||
# is decoded whole, 25 to 39 MiB measured, until its window is cut). So 320 MiB.
|
||||
#
|
||||
# Images for the model: 64 MiB of `data:` URLs, reserved before a byte is read, so the
|
||||
# same 64 MiB also bounds what is being read for them at any instant — at most ~48 MiB
|
||||
# raw, which Prisma's transfer copies a few times more for the milliseconds a request
|
||||
# takes to build. So ~200 MiB at the worst instant.
|
||||
#
|
||||
# A PDF worker is the one without a flat cap. `resourceLimits` bounds its V8 heap
|
||||
# (192 MB, plus 32 MB of young generation) and NOT its ArrayBuffers, where a
|
||||
# decompressed stream lives. Those are read from outside every 50 ms and the worker is
|
||||
# killed past 128 MB, which a buffer doubling between two reads can overshoot to ~256
|
||||
# MB. So one hostile file can hold up to ~500 MB before it dies, two ~1 GB. Measured:
|
||||
# two decompression bombs at once added ~310 MB to the process, both refused in under
|
||||
# two seconds.
|
||||
#
|
||||
# Every holder full at once is therefore ~2.5 GiB (1.1 GiB of uploads, 320 MiB of
|
||||
# files read back, ~200 MiB of images, ~1 GB of PDF workers), on top of the API itself
|
||||
# (~200 MiB once booted): ~2.75 GiB. That cannot fit 2Gi at any upload concurrency
|
||||
# worth having — at 2Gi, one upload at a time beside two hostile PDFs is all there is
|
||||
# room for — so the limit is 3Gi, with ~250 MiB left for the terminal recordings and
|
||||
# share verifications above. It runs well past the room above this request, and that
|
||||
# is deliberate: the request is what the scheduler sets aside for an ordinary day, not
|
||||
# for the worst minute. If more margin is ever wanted, the levers are the PDF worker's
|
||||
# own caps (`files/pdf-text.ts`) and keeping an upload's bytes out of Prisma's JSON
|
||||
# protocol — not fewer upload slots.
|
||||
# MEASURED on this image, in a cgroup like this pod's (lot 11 P, proof 2j): every upload
|
||||
# slot of two pods busy with a 10 MB, 500-page PDF — the most pages the worker reads —
|
||||
# peaked at 619 MiB and 740 MiB of `memory.current`, the API and its workers together,
|
||||
# from 446 and 496 MiB before. The TypeScript image sized this limit: Prisma's JSON
|
||||
# protocol copied an upload's bytes several times over (~1.1 GiB for three at once)
|
||||
# and a PDF worker's buffers could reach ~500 MB, ~2.75 GiB with every holder full.
|
||||
# None of that is in this process; the 3Gi limit stands until it is lowered on purpose
|
||||
# (docs/dotnet/PLAN.md, "Left to Antoine after lot 11"). The request is what the
|
||||
# scheduler sets aside for an ordinary day, not for the worst minute.
|
||||
memory: 1Gi
|
||||
limits:
|
||||
memory: 3Gi
|
||||
|
||||
Reference in New Issue
Block a user