服务器 CPU 平均 3%，内存也没变化。所以服务器并不忙，它是卡在等什么东西。

报错

每次失败，日志里都是同一句：

QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00

API 只跑了一个 uvicorn 进程，用的是 SQLAlchemy 默认连接池：5 个连接加 10 个溢出，同时最多 15 个数据库事务。请求等连接超过 30 秒就返回 500。但如果每个请求只占连接几毫秒，15 个连接给 100 个用户用绰绰有余。所以一定有东西占着连接很久。

第一个猜测是错的

测试报告猜的是：文件传输期间一直占着数据库连接。听起来很合理，但我先验证再改。我一边盯着 MySQL，一边故意慢速下载了三次 30 MB 的文件。传输过程中没有任何未提交的事务。下载在开始传输前就提交了，连接已经还回去了。

所以下载没问题。我在笔记本上复现了 100 人的失败（80% 上传失败，吞吐量掉到每秒 0.33 次操作），这样就不用占着共享的服务器排查了。日志显示，每次超时都发生在身份验证里。

原因：两个池子互相等

上传接口是 async 的。一次上传是这样的：

1. 身份验证取出一个数据库连接，用完没有还。
2. 上传在事件循环上执行数据库调用。
3. 把上传的文件写到磁盘时，每写一块都要等一次线程池，而连接一直拿在手里。

线程池有 40 个线程。100 个用户时，15 个上传占着全部 15 个连接，在等空闲线程；40 个线程全被别的请求占着，在等连接。两边互相等，谁也动不了，直到连接池 30 秒超时，请求失败。

这就解释了 3% 的 CPU。没有任何东西在计算，所有东西都在等。

第一次修复反而更糟

最直接的改法是身份验证之后、写文件之前先把连接还掉。我只改了这一处，结果 50 个用户也变慢了：下载 p95 到了 30 秒。

上传剩下的数据库调用还在事件循环上跑。现在它们是在事件循环上等连接，这会卡住整个进程里的所有请求，不只是上传。这个改法没有合并。

真正有效的修复

把上传拆成三段，保证事件循环上没有数据库调用，等线程的时候也不拿着连接：

async def upload(self, *, actor, metadata, ciphertext, context):
    # 1. 在线程池里检查：密钥、额度。检查完连接就释放。
    owner_key, recovery_key = await run_in_threadpool(self._check_upload, actor, metadata)

    # 2. 把加密文件写到暂存区。不占连接。
    staged = await self.storage.stage_upload(ciphertext, max_size_bytes=...)

    # 3. 回到线程池：写记录、审计事件、发布、提交。
    return await run_in_threadpool(
        self._record_upload, actor, metadata, context, staged, owner_key, recovery_key
    )

身份验证现在也会结束自己的只读事务，没有请求会把连接从身份验证一路带进接口。（上面省略了错误处理。第 2 步失败时仍然会回滚并删掉暂存文件。）

结果

同一台笔记本、同一个工具、同一批账号，每档 30 秒，文件 1 MiB：

修复后每一档吞吐量都稳定在每秒 56 次左右，延迟随排队增加，这才是机器本身的上限。笔记本上的数字是本地复现。真实服务器上，前一晚 100 个用户时 103 次上传只成功 23 次（78% 失败）；当天晚上重跑，102 次上传全部成功，每个下载文件的哈希都和原文件一致。

守住修复的测试

• 一个单元测试检查暂存开始前连接已经释放。
• 故障注入测试在写入文件记录的那次提交时杀掉数据库连接，并检查清理任务会删掉孤立的文件。
• 改完后整套后端测试在 MySQL 上通过：633 个测试，覆盖率 85%。

小结

• CPU 很低但请求失败，说明有东西在等。去找连接池、锁和超时，不要去找慢代码。
• 先验证猜测再动手。这里第一个猜测听起来对，其实是错的，一次测量就看出来了。
• 在 async Python 里，事件循环上的阻塞数据库调用会卡住所有请求。要么改成异步调用，要么放进线程池，而且不要拿着连接跨过 await。
• 修复前后的数字用同一个工具测。这样修复是测出来的，而不是说出来的。

完整记录和原始数据在 docs/performance/pool-fix-2026-10-04/，代码在 file_service.py。

The server’s CPU averaged 3% and memory was flat. So the server was not busy. It was stuck waiting on something.

The error

Every failure had the same message in the log:

QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00

The API ran as one uvicorn process with SQLAlchemy’s default pool: 5 connections plus 10 overflow, so at most 15 database transactions at once. A request that waits 30 seconds for a connection gets a 500. But 15 connections should be plenty for 100 users if each request holds one for a few milliseconds. Something was holding them much longer.

The first guess was wrong

The test report guessed that a file transfer kept its database connection open for the whole transfer. That sounded likely, so I checked it rather than fixing it straight away. I ran three deliberately slow downloads of a 30 MB file while watching MySQL. There were no open transactions during the transfers. The download commits before it starts streaming, which gives the connection back.

So downloads were fine. I reproduced the failure on my laptop at 100 users (80% of uploads failed, throughput fell to 0.33 operations a second), which meant I could study it without the shared server. The log showed every timeout happening inside authentication.

The cause: a deadlock between two pools

The upload endpoint is async. Here is what one upload did:

1. Authentication checked out a database connection and did not give it back.
2. The upload ran its database calls on the event loop.
3. To write the uploaded file to disk, it awaited the thread pool once per chunk, still holding the connection.

The thread pool has 40 threads. With 100 users, 15 uploads held all 15 connections while waiting for a free thread, and all 40 threads were other requests waiting for a connection. Each side was waiting for the other. Nothing moved until the 30 second pool timeout, and then the requests failed.

This explains the 3% CPU. Nothing was computing. Everything was waiting.

The fix that made it worse

The obvious fix was to release the connection after authentication, before writing the file. I tried only that, and 50 users became slow too: download p95 went to 30 seconds.

The upload’s remaining database calls still ran on the event loop. Now they waited for a connection on the event loop, which blocks every request in the process, not just the upload. I did not ship it.

The fix that worked

The upload is split into three parts so that no database call runs on the event loop and no connection is held while waiting for a thread:

async def upload(self, *, actor, metadata, ciphertext, context):
    # 1. checks in the thread pool: keys, limits. The connection is released after.
    owner_key, recovery_key = await run_in_threadpool(self._check_upload, actor, metadata)

    # 2. write the encrypted file to a staging object. No connection held.
    staged = await self.storage.stage_upload(ciphertext, max_size_bytes=...)

    # 3. back in the thread pool: rows, audit event, publish, commit.
    return await run_in_threadpool(
        self._record_upload, actor, metadata, context, staged, owner_key, recovery_key
    )

Authentication also ends its read-only transaction now, so no request carries a connection from authentication into the endpoint. (Error handling is left out above. A failure in step 2 still rolls back and removes the staged object.)

Results

Same laptop, same tool, same accounts, 30 seconds per step with 1 MiB files:

After the fix, throughput stays at about 56 operations a second at every step and latency grows with the queue, which is the machine’s real limit. The laptop numbers are a local reproduction. On the real server, the run the night before had 23 of 103 uploads succeed at 100 users (78% failed). The re-run that night had 102 of 102 succeed, and every download matched its original hash.

Tests that keep it fixed

• A unit test checks that the connection is released before staging starts.
• The fault injection test kills the database connection at the exact commit that writes the file row, and checks that the cleanup job removes the orphaned object.
• The full backend suite passed on MySQL after the change: 633 tests, 85% coverage.

Takeaways

• Low CPU with failing requests means something is waiting. Look for pools, locks and timeouts, not for slow code.
• Check a guess before fixing it. The first theory here sounded right and was wrong, and one measurement showed it.
• In async Python, a blocking database call on the event loop stalls everything. Either make the call async or run it in the thread pool, and do not hold a connection across an await.
• Keep the before and after numbers from the same tool. Then the fix is a measurement, not a claim.

The full write-up with raw data is in docs/performance/pool-fix-2026-10-04/, and the code is file_service.py.

规则好说，加上分享、忘记 passphrase、会话被盗这几种情况之后就难守了。先说这条规则带来的好处。

这样做有什么用

1. 服务器上只有加密后的文件。 服务器上没有原文件，也没有能打开文件的密钥。运营 SecureView 的人看不到用户的文件，就算服务器被攻破、数据库被整个拷走，拿到的也只是密文和封装好的密钥。没有每个用户自己的 passphrase，这些都没用。

2. 账号被盗不等于文件被盗。 假设别人拿到了你的登录密码并登录进来，对方能看到你的文件列表，但读不到任何内容。预览不了，因为服务器上根本没有原文件可以给对方看。下载原文件和分享也做不到，因为这两件事都要用你的私钥，而私钥只有你的 passphrase 能解开。每次下载、每次分享，浏览器都会重新要 passphrase。对方最多只能下载到加密后的文件。

下面讲这个设计怎么守住这两点，以及它管不到的地方。

两个密码，不是一个

每个账号有两个互相独立的秘密：

• 登录密码用来向服务器证明你是谁。服务器存它的 Argon2id 哈希，和普通登录一样。
• 加密 passphrase 从不离开浏览器。它解锁账号的私钥，私钥再解锁你的文件。

两者分开，上面第 2 点才成立。这也是忘了登录密码可以放心用邮件重置的原因：别人就算拿下了你的邮箱，也只能拿到会话，拿不到文件。

密钥链

注册时所有密钥都在浏览器里生成，服务器只收到封装好的副本：

passphrase  --Argon2id-->       KEK（密钥加密密钥）
KEK         --AES-256-GCM-->    打开 AMK（账户主密钥，随机 256 位）
AMK         --HKDF-SHA256-->    打开 RSA 私钥（默认 3072 位）
私钥        --RSA-OAEP-->       打开每个文件的 DEK（数据加密密钥）
DEK         --AES-256-GCM-->    解密文件

服务器存的是包装后的 AMK、加密后的私钥、公钥，以及 Argon2id 的盐和参数。这些全拿到，没有 passphrase 也打不开任何东西。注册前浏览器还会自检：把刚封装好的东西重新解开，确认私钥和公钥是一对。这一步服务器做不了，因为它解不开。

每次上传都生成一个新的随机 DEK。浏览器用它加密文件（默认 AES-256-GCM，用户也可以选另外五种 AEAD 算法），然后用 RSA-OAEP-SHA256 把 DEK 包装两次：一次给文件所有者的公钥，一次给一把私钥离线保管的恢复密钥。服务器收到的是密文和两份包装后的 DEK。下载时服务器把密文原样传回，由浏览器解密。

分享就是重新包装

要把文件分享给 Bob，服务器在密码学上帮不上忙，因为它没有 DEK。所以由发送方的浏览器来做：

1. 取 Bob 的公钥，用公钥本身算出指纹。
2. 显示 Bob 的名字和指纹，如果和上次分享给 Bob 时不一样就警告。
3. 解锁发送方的私钥，解开这个文件的 DEK。
4. 用 Bob 的公钥重新包装同一个 DEK，只把这个发给服务器。

服务器负责检查权限：发送方要对这个文件有 SHARE 权限，Bob 拿到的权限不能超过发送方自己有的。然后在一个事务里写入 Bob 的授权、包装后的 DEK 和一条审计记录。文件本身不用重新加密。

第 2 步存在是因为公钥是服务器发的。如果服务器把 Bob 的公钥换成自己的，分享给“Bob”的东西它就都能读。指纹警告只能在第一次分享之后起作用，而且要大家真的去和 Bob 在设置页看到的指纹对一下。这是真实存在的局限，文档里也写明了。

忘了 passphrase

“服务器不拿密钥”到这里开始有代价。passphrase 忘了，AMK 就没了，服务器上没有任何东西能把它找回来。

出路是注册时浏览器只显示一次的私钥文件。这个文件本来就能打开账号能访问的所有文件，所以让持有它的人设置新的 passphrase，并没有给他任何新能力。重置的过程是：

• 在浏览器里读取密钥文件，确认它和账号的公钥匹配
• 生成一个新的 AMK，用新 passphrase 包装它，再用它重新加密同一把私钥
• 发送之前，先在本地把两个新信封重新打开一遍

密钥对不变，所以所有文件、分享和包装过的 DEK 都继续有效，什么都不用重新加密。passphrase 和密钥文件都丢了，文件就永远打不开了。注册页上用大白话写明了这一点。

被盗的会话不能换掉密钥

服务器没法检查新信封能不能打开，因为它从来见不到 passphrase 和 AMK。如果没有别的检查，拿到被盗会话的人可以上传一堆垃圾信封，让主人再也打不开任何文件。

所以替换信封必须证明持有私钥。服务器返回一个摘要，绑定账号、密钥、两个预期 revision，以及两个新信封的每一个字段。浏览器用私钥签名（RSA-PSS-SHA256），服务器先验签再写入：

POST /keys/me/reset-protection/signing-payload  ->  32 字节摘要
浏览器用账户私钥签名
POST /keys/me/reset-protection（信封 + 签名）
  别的密钥签的名   ->  422，什么都不改
  revision 过期    ->  409

每种需要签名的操作（重置、重新包装、添加密钥）在摘要里都有自己的域标签，一种操作的签名不能拿去重放成另一种。重置上线之后，记得旧 passphrase 时修改 passphrase 也走同一条路径。

服务器还能看到什么

密钥托管不等于什么都藏住了。用盗来的密码登录的人虽然读不了任何文件，但删除文件、取消分享这些普通的账号操作还是能做。服务器仍然能看到：

• 文件名、MIME 类型和准确的文件大小
• 谁在什么时候把什么分享给了谁
• Argon2id 信封。数据库被偷的话，弱 passphrase 可以被离线猜出来

针对最后一点，浏览器会拒绝常见的、有规律的、或者基本就是用户名的 passphrase，并要求至少 10 个字符。改过的客户端可以跳过这些检查，它们保护的是用户不选弱密码，不是保护服务器不受攻击。

一个主动作恶的服务器还可以返回旧的密文或者换掉的密钥包。这个设计做到的是 API 层面的零知识密钥托管，不能防住一个恶意的运营方。

小结

• 把服务器绝不能收到的东西写下来，每个接口都拿这张清单对一遍。
• 服务器验证不了的东西，就让客户端来证明，比如签名后才能替换信封。
• 恢复方案要和加密一起设计，不能事后补。这里是用户自己保管的一个文件，文档写清楚了丢了会怎样。
• 把局限写出来。一个安全声明说清楚了自己不管什么，反而更可信。

更多细节见 docs/SECURITY_FLOWS.md，浏览器端代码在 frontend/src/crypto/（share.ts、passphrase-reset.ts）。

The rule is easy to say and hard to keep once you add sharing, forgotten passphrases and stolen sessions. First, what the rule buys you.

What this buys you

1. The server only stores encrypted files. There is no original file on the server, and no key that opens one. The people running SecureView cannot read a user’s files, and neither can someone who breaks into the server or copies the database. What they get is ciphertext and sealed keys, and without each user’s passphrase that is useless.

2. A stolen account is not a stolen file. Say someone gets your login password and signs in. They see your file list, but they cannot read any of it. They cannot preview a file, because the server has no original to show. They cannot download the real file or share it with anyone, because both need your private key, and only your passphrase unlocks it. The browser asks for the passphrase again on every download and every share. The most they can download is the encrypted file.

The rest of this post is how the design keeps those two promises, and where they stop.

Two secrets, not one

Every account has two separate secrets:

• The login password proves who you are to the server. The server stores an Argon2id hash of it, like any login.
• The encryption passphrase never leaves the browser. It unlocks the account’s private key, and the private key unlocks your files.

Keeping them apart is what makes point 2 work. It is also why a forgotten login password can be reset by email: taking over someone’s mailbox gets you their session, but not their files.

The key chain

At registration the browser makes everything and sends the server only sealed copies:

passphrase  --Argon2id-->       KEK  (key-encryption key)
KEK         --AES-256-GCM-->    opens the AMK  (account master key, random 256-bit)
AMK         --HKDF-SHA256-->    opens the RSA private key (3072-bit by default)
private key --RSA-OAEP-->       opens each file's DEK  (data-encryption key)
DEK         --AES-256-GCM-->    decrypts the file

The server stores the wrapped AMK, the encrypted private key, the public key and the Argon2id salt and parameters. With all of that and no passphrase, it still cannot open anything. The browser also runs a self-test before registering: it unlocks what it just sealed and checks the private key matches the public key. The server cannot check that, because it cannot decrypt the bundle.

Each upload gets a fresh random DEK. The browser encrypts the file with it (AES-256-GCM by default, or one of five other AEAD suites the user picks), then wraps the DEK with RSA-OAEP-SHA256 twice: once to the owner’s public key, and once to a recovery key whose private half is kept offline. The server receives the ciphertext and the two wrapped DEKs. On download it streams the ciphertext back and the browser decrypts it.

Sharing is a rewrap

To share a file with Bob, the server cannot help with the cryptography, because it does not have the DEK. So the sender’s browser does it:

1. Fetch Bob’s public key, and compute its fingerprint from the key itself.
2. Show Bob’s name and fingerprint, and warn if it changed since the last share to Bob.
3. Unlock the sender’s private key and unwrap the file’s DEK.
4. Wrap the same DEK to Bob’s public key and send only that.

The server checks permissions: the sender needs SHARE on the file, and Bob cannot get a capability the sender does not have. Then it stores Bob’s grant and wrapped DEK, plus an audit event, in one transaction. The file itself is never re-encrypted.

Step 2 is there because the server hands out public keys. A server that swapped Bob’s key for its own could read everything shared to “Bob”. The fingerprint warning only catches that after the first share, and only if people compare it with what Bob sees in Settings. It is a real limit, and the docs say so.

Someone forgets their passphrase

This is where “the server holds no keys” stops being free. Forget the passphrase and the AMK is gone, and nothing on the server can bring it back.

The way out is the private key file the browser shows once at registration. That file opens every file the account can reach, so letting its holder set a new passphrase gives them nothing they did not already have. The reset:

• reads the key file in the browser and checks it matches the account’s public key
• makes a new AMK, wraps it under the new passphrase, and re-encrypts the same private key under it
• reopens both new envelopes locally before sending anything

The key pair does not change, so every file, share and wrapped DEK keeps working and nothing is re-encrypted. Lose both the passphrase and the key file and the files are gone for good. Registration says this in plain words.

A stolen session must not replace the keys

The server cannot check that the new envelopes open, because it never sees the passphrase or the AMK. Without another check, someone with a stolen session could upload garbage envelopes and lock the owner out of every file.

So replacing the envelopes needs proof of the private key. The server returns a digest that binds the account, the key, both expected revisions and every field of both new envelopes. The browser signs it with the private key (RSA-PSS-SHA256), and the server verifies the signature before writing anything:

POST /keys/me/reset-protection/signing-payload  ->  32-byte digest
browser signs the digest with the account private key
POST /keys/me/reset-protection  (envelopes + signature)
  signature from another key  ->  422, nothing changed
  stale revision              ->  409

Each signed operation (reset, rewrap, adding a key) has its own domain label in the digest, so a signature for one cannot be replayed as another. Since the reset shipped, the same path also handles changing a passphrase you still know.

What the server can still see

Key custody is not the same as hiding everything. Someone signed in with a stolen password can still delete or unshare your files, since those are normal account actions, even though they cannot read any of them. The server still sees:

• filenames, MIME types and exact file sizes
• who shared what with whom, and when
• the Argon2id envelope, which allows offline guessing of a weak passphrase if the database is stolen

For that last point, the browser refuses passphrases that are common, patterned or mostly the username, and requires at least 10 characters. A modified client could skip those checks. They protect users from their own weak choices, not the server from an attacker.

A server that is actively malicious can also serve old ciphertext or a swapped key bundle. The design gives zero-knowledge key custody at the API, not protection from a hostile operator.

Takeaways

• Write down what the server must never receive, and check every endpoint against it.
• If the server cannot validate something, ask the client to prove it, like the signed envelope replacement.
• Recovery has to be designed with the encryption, not added later. Here it is a file the user keeps, and the docs say what losing it costs.
• Write the limits down. A security claim reads better when it says what it does not cover.

More detail: docs/SECURITY_FLOWS.md, and the browser code in frontend/src/crypto/ (share.ts, passphrase-reset.ts).

问题也很明显：谁都能评，那一个人就能评五十次。这篇讲几层让刷分变难的做法，以及它们管不到的地方。

每人每门课一条

登录用户的评分属于账号。没登录的评分属于一个随机密钥：浏览器第一次评分时生成（crypto.randomUUID()），存在 localStorage 里，之后每次请求都放在 X-Guest-Key 请求头里发过来。

数据库保证每个主人每门课只有一条：

UniqueConstraint("author_id", "unit_code", name="uq_unit_review_once"),
UniqueConstraint("guest_key", "unit_code", name="uq_unit_review_guest_once"),
CheckConstraint("author_id IS NOT NULL OR guest_key IS NOT NULL", name="ck_unit_review_owner"),

再评一次就是改原来那条，学生可以改主意。如果同一个人的两次保存同时到达，第二次插入会撞上唯一约束，代码接住 IntegrityError，把它变成一次修改。

密钥不能原样存

访客密钥相当于修改这条评分的密码。如果表泄露了，而密钥是原样存的，任何人都能拿着它去改别人的评分。所以存的是用服务器密钥算的 HMAC：

def guest_hash(raw):
    raw = (raw or "").strip()
    return _keyed(f"guest:{raw}") if GUEST_KEY.match(raw) else None

有一个测试专门检查存下来的值里不包含发过来的密钥。

密钥限不住人，所以按网络限

浏览器想换密钥随时都能换：清掉存储、开无痕窗口，或者写个脚本。所以密钥只用来认“这条是谁的”，限流按网络地址算：

每天 100 次看起来很宽松，这是故意的。校园 Wi-Fi 下，一整个大课教室可能只共用几个地址。严到能挡住一个人的限制，会把一整个班挡在外面。地址存的也是 HMAC，不是 IP 本身。

拿到真实地址要小心。X-Forwarded-For 客户端想写什么都行，所以 API 只认 nginx 根据连接本身设置的 X-Real-IP，而且只在请求来自可信代理时才认。绕过 nginx 直接打到 API 的请求按它自己的地址限流，没法每次换一个地址。

加锁再计数

计数放在 PostgreSQL 里而不是内存里，因为 API 有多个 worker，每个 worker 各数各的，限制就变成了好几份。这个限流器最早是给登录写的，当时有个竞态：同时到达的几次猜密码都数到同样的历史次数，结果全部放行。从 #148 开始，检查前先对这个键加一把事务级 advisory lock，评分用的就是这个版本：

def check(db, limit, key):
    db.execute(select(func.pg_advisory_xact_lock(func.hashtext(f"{limit.kind}:{key}"))))
    since = _now() - timedelta(seconds=limit.window_seconds)
    count, oldest = db.execute(...).one()  # 这个键在窗口期内的次数
    if (count or 0) >= limit.max_events:
        raise Throttled(retry_after)

同一个键的请求会排队，等前一个提交了再数。被限流的请求返回 429，带 Retry-After。

评分保存之后

• 举报和隐藏。 评分可以像论坛帖子一样被举报，管理员可以隐藏。被隐藏的评分不计入平均分，但作者自己还能看到和修改。
• 作者信息尽量少。 登录用户的评分别人只看到名字首字母，未登录的评分连首字母都没有。学生评价教学团队时应该能说实话。
• 平均分读取时现算。 每页一条分组查询，不存平均分，也就不会和实际数据对不上。选填的每一项都有自己的人数，两个人填的平均分就标两个人。

这些挡不住什么

这些做法提高了刷分的成本，但没法让刷分变成不可能。一个人在手机流量、家里 Wi-Fi 和 VPN 之间切换，每个地址又有 100 次。一个人用一个浏览器每门课只评一次，看起来就和真实学生一模一样，因为对方可能本来就是。

对付这种情况主要不靠代码。页面会显示平均分是多少人打的，不到三个人时会提示平均分仅供参考。举报会有人看。如果刷分从偶尔变成常态，下一步是要求评分前验证邮箱，账号系统本来就支持（ADR 0005）。

测试

• test_signed_out_ratings_are_limited_per_network：同一地址用三个新密钥评分，第三次返回 429。
• test_the_browser_key_is_not_stored_as_sent。
• 一个 Playwright 测试：访客给课程评分，另一个浏览器看到的是别人的评分，第一个浏览器再把评分撤回（#205）。

代码：reviews.py、throttle.py、useGuestKey.ts。

The problem is obvious: if anyone can rate, one person can rate fifty times. This post is about the layers that make that harder, and where they stop.

One rating per person per unit

A signed-in rating belongs to the account. A signed-out rating belongs to a random key that the browser makes the first time it rates (crypto.randomUUID()), keeps in localStorage, and sends as an X-Guest-Key header.

The database enforces one rating per owner per unit:

UniqueConstraint("author_id", "unit_code", name="uq_unit_review_once"),
UniqueConstraint("guest_key", "unit_code", name="uq_unit_review_guest_once"),
CheckConstraint("author_id IS NOT NULL OR guest_key IS NOT NULL", name="ck_unit_review_owner"),

Rating again edits the same row, so a student can change their mind. If two saves from the same owner arrive together, the second insert hits the unique constraint, and the handler catches the IntegrityError and turns it into an edit.

Don’t store the key as sent

The guest key works like a password for editing that rating. If the table leaked and the keys were stored as sent, anyone could send them back and edit other people’s ratings. So the key is stored as an HMAC with the server’s secret:

def guest_hash(raw):
    raw = (raw or "").strip()
    return _keyed(f"guest:{raw}") if GUEST_KEY.match(raw) else None

A test checks that the stored value never contains the key that was sent.

The key limits nothing, so limit the network

A browser can make a new key whenever it likes: clear storage, open a private window, or write a script. So the key is only for ownership. The limit is on the network address:

100 a day looks generous, and it is on purpose. On campus Wi-Fi, a whole lecture theatre can share a handful of addresses. A limit strict enough to stop one person would block a whole class. The address is also stored as an HMAC, not as the IP.

Getting the address right takes care. X-Forwarded-For is whatever the client writes in it, so the API only believes X-Real-IP, which nginx sets from the connection, and only when the request comes from a trusted proxy. Anything that reaches the API some other way is limited by its own address and cannot pick a new one per request.

Count under a lock

The limits are counted in PostgreSQL rather than in memory, because the API runs more than one worker, and a limit each worker counts on its own becomes several limits. The same limiter was written for sign-in first, and it had a race: guesses arriving together all counted the same past events and all got through. Since #148 the check takes a transaction-level advisory lock on the key first, and ratings got that version:

def check(db, limit, key):
    db.execute(select(func.pg_advisory_xact_lock(func.hashtext(f"{limit.kind}:{key}"))))
    since = _now() - timedelta(seconds=limit.window_seconds)
    count, oldest = db.execute(...).one()  # events for this key since the window start
    if (count or 0) >= limit.max_events:
        raise Throttled(retry_after)

Requests for the same key now queue behind each other until the first one commits. A throttled request gets a 429 with Retry-After.

After a rating is saved

• Reports and hiding. A rating can be reported the same way as a forum post, and a moderator can hide it. A hidden rating drops out of the average, but its author still sees it and can edit it.
• Little about the author. Other readers see only an initial for a signed-in rating, and no name at all for a signed-out one. A student should be able to be honest about a teaching team.
• Averages computed on read. One grouped query per page, so there is no stored average to drift from the rows. Each optional score has its own count, so an average from two answers says two.

What this does not stop

This raises the cost of faking ratings. It does not make it impossible. Someone switching between phone data, home Wi-Fi and a VPN gets a fresh 100 per address. Someone who rates once per unit from one browser looks exactly like a real student, because they might be one.

What helps against that is mostly not code. The page shows how many ratings an average is based on, and under three it says the average is only a rough sign. Reports go to a person. If abuse becomes a steady problem rather than an occasional one, the next step is to make ratings need a verified email. The account system already supports that (ADR 0005).

Tests

• test_signed_out_ratings_are_limited_per_network: three fresh keys from one address, and the third gets a 429.
• test_the_browser_key_is_not_stored_as_sent.
• A Playwright test where a guest rates a unit, a second browser sees it as someone else’s, and the first takes it back (#205).

Code: reviews.py, throttle.py, useGuestKey.ts.

普通翻译器偏偏错在关键词上

最早用的是 Argos Translate，一个跑在同一台 VPS 上的离线模型，不要钱，也没有调用次数限制。在真实的 Handbook 页面上，它把：

• census date 译成“人口普查日期”
• unit 译成“单位”
• Exercise 译成“锻炼”
• Breadth studies 译成“面包研究”

这些恰好是学生要据此行动的词。错过 census date，退掉的课也要交钱。所以这些词不能交给模型决定。

翻译流程

每个句子都走同样的几步：

有固定译法吗？ ->  把术语保护起来  ->  翻译  ->  把术语放回去

固定值不进模型。 校区、教学时段、考核类型、学院名这些都来自固定列表。glossary.py 给每一个都写好了中日韩三种核对过的译法，包括全部 120 个学院和系的名字。学位名是拼出来的，不是翻出来的，因为英文学位名从前往后读，中文要倒过来。

句子里的术语先遮住。 句子交给模型之前，里面的术语表词汇都换成占位符，翻完再换回固定译法。占位符用字母（Zqa、Zqb……），因为数字会被模型改掉。

字母也不是绝对安全。“What is Zqa?” 回来变成了“什么是兹卡?”，模型把占位符音译了。我拿线上指南里 59 个失败的句子试了几种占位符，每种丢掉的句子都不一样，所以现在一种不行就换下一种，一共五种，直到占位符完整回来为止。这样救回了 59 句里的 50 句。

聊天模型也要先看术语表

Argos 的语法还是不行，“WAM of 65” 被译成 65 岁。所以我加了一个引擎，可以接任何兼容 OpenAI 接口的聊天模型（#115），默认用智谱的 glm-4-flash，因为它免费。网站上现在大部分中文都是它翻的。

聊天模型反而不擅长处理占位符。让它翻 “Zqa units”，它会把两个都译成“课程”。所以给它的方式改成在提示词里直接列出固定译法，回复只有在什么都没丢的时候才算数（#116）：

# 从 crawler/translate/llm.py 简化
def faithful(source, reply, terms):
    if any(agreed not in reply for _english, agreed in terms):
        return False
    if any(code not in reply for code in _CODE.findall(source)):
        return False
    if bullets(source) != bullets(reply):
        return False
    return all(n in reply for n in _NUMBER.findall(source))

没通过检查的句子，再走一遍遮术语的流程，那条路不会丢术语。

拿不准就保留英文

聊天模型还可能直接回答问题而不是翻译，或者开头先说一句“以下是翻译”。遇到下面这些情况，引擎会直接丢掉回复：

• 回复是空的，或者比原文长太多
• 开头有客套话，或者把提示词原样吐回来
• 文字不对，比如日文回复里夹着简体中文
• 和英文原文一模一样

翻不好的句子就保留英文。读者至少还能看英文，一句没人写过却说得很肯定的中文更糟。

译文存在哪里

译文都存在 content_translations 这张表里，和英文原文并排放，不覆盖原文。每一行有两个关键字段：

• source_hash：生成这条译文时英文原文的哈希。Monash 改了页面，哈希就变了，页面会提示译文可能过时。重跑时哈希没变的直接跳过，所以每周刷新只把新句子发给模型。
• provenance：machine（机器）、curated（对照英文核对过）或 human（人工）。读取时人工的优先。机器重跑永远不会覆盖人写的译文，页面上也会告诉读者现在看的是哪一种。

Handbook 里同样的段落在几千门课里重复出现，所以引擎按句子缓存。有没有这层缓存，是跑一小时和跑一周的差别。

日文和韩文

日文和韩文读者很少，所以只用同一套术语表加机器翻译，不做日常人工检查。即便这样，第一批样本还是要修（#182、#183）：

• 韩文的空格全没了，因为一条在术语后面拼接中文的规则也作用到了韩文上。
• 韩文助词是按模型自己的词选的，不是按换回去的术语选的（写成了 과목를，应该是 과목을）。
• 日文在术语前后重复（マレーシアキャンパスキャンパス）。
• 日期写成了 12月2026年。

免费模型大约十秒一句，日文和韩文全部翻完要一天以上。所以它在抓取结束后用单独的锁跑，中文先跑。

小结

• 质量主要靠术语表。大部分翻译错误加一个术语就能修好，下次运行所有页面都会跟着改。
• 存之前先检查模型返回的内容，固定译法、课程代码和数字一个都不能少。
• 译文和原文并排存，带上哈希，原文变了才知道译文过时了。
• 翻不好的句子就保留原文。

代码：engine.py、llm.py、glossary.py，决策记录见 ADR 0004。

A plain translator gets the important words wrong

The first engine was Argos Translate, an offline model that runs on the same VPS. It costs nothing and has no rate limit. On real Handbook pages it translated:

• census date as 人口普查日期 (a population census)
• unit as 单位 (an organisation, not a subject)
• Exercise as 锻炼 (physical exercise)
• Breadth studies as 面包研究 (bread studies)

These are exactly the words a student acts on. Miss the census date and you pay for a unit you dropped. So the model is not allowed to decide them.

The pipeline

Every string goes through the same steps:

agreed value?  ->  protect the terms  ->  translate  ->  put the terms back

Agreed values never reach the model. A campus, a teaching period, an assessment type or a faculty name comes from a closed list. glossary.py has a checked wording for each one in Chinese, Japanese and Korean, including all 120 faculty and school names. Degree names are built from parts rather than translated, because English names a degree front to back and Chinese back to front.

Terms inside a sentence are masked. Before a sentence goes to the model, every glossary term in it is swapped for a placeholder, and the agreed wording is put back afterwards. The placeholders are letters (Zqa, Zqb, …) because the model rewrote digits.

Letters were not safe either. “What is Zqa?” came back as 什么是兹卡?, where the model had turned the placeholder into Chinese sounds. I tested other tokens on the 59 sentences that were failing on the live guides. Each token lost a different set of sentences, so the engine now tries five masks in turn until one comes back intact. That recovered 50 of the 59.

The chat model gets the glossary too

Argos was still weak on grammar. “WAM of 65” became 65 years old. So I added an engine for any OpenAI-compatible chat model (#115), defaulting to Zhipu’s glm-4-flash because it is free. Most of the Chinese on the site now comes from it.

A chat model does worse with placeholders. Asked to translate “Zqa units”, it writes 课程 for both. So it gets the agreed wordings in the prompt instead, and the reply is only accepted if nothing was lost (#116):

# simplified from crawler/translate/llm.py
def faithful(source, reply, terms):
    if any(agreed not in reply for _english, agreed in terms):
        return False
    if any(code not in reply for code in _CODE.findall(source)):
        return False
    if bullets(source) != bullets(reply):
        return False
    return all(n in reply for n in _NUMBER.findall(source))

If the reply fails this check, the same sentence goes through the masked path, which cannot drop a term.

When in doubt, keep the English

A chat model can also answer the question instead of translating it, or start with “Here is the translation”. The engine throws a reply away when:

• it is empty, or much longer than the source
• it opens with a preface, or echoes the prompt back
• it is in the wrong script, like Simplified Chinese inside a Japanese answer
• it is identical to the English

A string that fails stays in English. A reader can work with English, but a confident sentence that nobody wrote is worse.

Where translations live

Translations are stored in one table, content_translations, next to the English rather than replacing it. Each row has two things that matter:

• source_hash: the hash of the English it was made from. When Monash edits the page, the hash changes and the page says the translation may be out of date. A rerun skips everything whose hash has not moved, so the weekly refresh only sends new sentences to the model.
• provenance: machine, curated (checked against the English) or human. The human row wins on read. A machine run can never overwrite somebody’s work, and the page tells the reader which kind they are looking at.

The Handbook repeats the same paragraphs across thousands of units, so the engine caches by sentence. That cache is the difference between an hour and a week.

Japanese and Korean

Japanese and Korean have very few readers, so they get machine output behind the same glossary with no routine review. The first sample still needed fixes (#182, #183):

• Korean lost all its spaces, because a rule that joins Chinese words after a glossary term also ran over Korean.
• Korean particles were chosen for the model’s own word, not the term put back (과목를 instead of 과목을).
• Japanese doubled words around a term (マレーシアキャンパスキャンパス).
• Dates came out as 12月2026年.

At about ten seconds a string on the free model, a full Japanese and Korean fill takes a day or more. So it runs under its own lock after the crawl, and Chinese always goes first.

Takeaways

• The glossary is where quality comes from. Most translation bugs were fixed by adding a term, and the fix reaches every page on the next run.
• Check what the model returns before storing it. Every agreed term, code and number has to be there.
• Store translations beside the source with a hash, so you know when they go stale.
• A string you cannot translate well should stay in the original language.

Code: engine.py, llm.py, glossary.py, and the decision record ADR 0004.

网站只有我一个人维护，我不可能每次部署都守着。所以部署脚本要自己判断这次上线成没成功，没成功就退回上一个版本。这篇讲它现在怎么做，以及我是被哪些问题逼着改成这样的。

出过的问题

1. 上线的代码不是 CI 测过的那份。 最早的 Deploy workflow 只是 SSH 到服务器跑 deploy.sh，脚本自己 git reset 到 origin/main。如果 CI 跑完到 SSH 连上之间，main 上又多了一个提交，这个没测过的提交就直接上线了。

2. 部署脚本把自己换掉了。 bash 是边读边执行脚本的。deploy.sh 在执行中途 git checkout 新提交，磁盘上的脚本文件就变了，后面几行可能读到新文件里错位的内容。

3. 一个官方页面换了地址，部署连续失败。 Monash 把出国健康与安全那一页从 study-abroad/overseas/ 挪到了 study-abroad/outbound/。部署时登记页面列表会按 URL 插入新行，结果撞了 slug 的唯一约束。从 #128 开始的几次部署都停在这一步，线上一直是旧版本，直到 #131 修好。

4. 健康检查说没问题，其实不知道上线的是什么。 原来只看 /api/health 返回 200。API 能响应，不代表页面能渲染，也不代表跑的是刚部署的那个提交。

现在的流程

CI 通过 → Deploy workflow 拿到 CI 测过的提交号
       → SSH 到 VPS 跑 deploy.sh <提交号>
          1. 确认这个提交在 origin/main 上，checkout，再 exec 新版本的自己
          2. pg_dump 备份数据库，保留 14 天
          3. 把正在运行的镜像打上 :previous 标签
          4. 构建、跑迁移、写入种子数据、启动
          5. 等 /api/health，大约 60 秒还没好就回滚
          6. smoke.sh 检查提交号和几个关键页面，失败也回滚
       → 在 GitHub Actions 上通过公网域名再跑一次 smoke.sh

只部署 CI 测过的提交

Deploy workflow 由 CI 的 workflow_run 事件触发，传给服务器的是 CI 那次运行的 head_sha，不是 main 当时指向的提交。脚本收到后先检查它在不在 origin/main 上，不在就拒绝，所以从别的分支手动触发也部署不了。

if ! git merge-base --is-ancestor "$DEPLOY_SHA" origin/main; then
  echo "error: $DEPLOY_SHA is not on origin/main; only reviewed commits are deployed" >&2
  exit 1
fi
git checkout -q --detach "$DEPLOY_SHA"
exec "$PROJECT_DIR/deployment/deploy.sh" --at "$DEPLOY_SHA"

最后一行解决的是问题 2：checkout 之后不再往下读旧文件，直接用 exec 运行新提交里的脚本，--at 告诉它不用再 checkout 一次。这个改动有个副作用：合并这个改动的那次部署跑的还是旧脚本，因为旧脚本不会 exec，从下一次部署才开始生效。

保留上一版镜像

构建之前，把正在跑的 api、web 等镜像从 :latest 复制一个 :previous 标签。回滚就是把标签换回来，再用 --no-build 启动，不用重新构建：

rollback() {
  for service in api web; do
    if docker image inspect "monash-hub-$service:previous" >/dev/null 2>&1; then
      docker tag "monash-hub-$service:previous" "monash-hub-$service:latest"
    fi
  done
  docker compose -p monash-hub up -d --no-build api web
}

成功要看三件事

/api/health 现在会返回构建时传进去的提交号。smoke.sh 按访客的角度检查：

• API 报告的提交号是不是这次要部署的
• 首页的中文和英文能不能渲染
• 一个课程页、课程搜索和 WAM 搜索有没有数据
• robots.txt、sitemap 在不在
• 不存在的课程和路径返回 404，不是 500

服务器上先对容器端口跑一遍，失败就回滚。GitHub Actions 再通过 monashhub.secureview.tech 跑一遍，确认经过 nginx 之后看到的也是这个提交。

回滚不了的东西

回滚只换容器，不撤销数据库迁移。旧镜像会在新的表结构上运行，所以迁移最好写成新旧代码都能用的样子。真遇到撤不回的情况，就用第 2 步的备份。

另外，原来部署里有两段一次性的数据修复，会去 Handbook 抓两门课并检查结果。这让每次部署都依赖 Monash 的网站在线，而且到了 2027 年 Handbook 换版本时会让所有部署失败。我把它们挪到了单独的 verify-handbook-parser.sh，部署脚本只做部署。

部署之前

部署之前，CI 先跑后端测试（连真实 PostgreSQL，并检查模型和迁移有没有对不上）、依赖漏洞扫描、前端类型检查和构建，还有 Playwright 浏览器测试，在桌面和手机尺寸下点一遍课程页、选课规划和评分。CI 不过，Deploy 根本不会触发。

小结

• 部署的对象要明确到提交号，并且能从线上查回来。
• 回滚要便宜。留一份旧镜像比重新构建快，也不依赖当时的网络和依赖源。
• 部署脚本里不要放依赖外部网站的步骤。

代码：deploy.sh、smoke.sh、deploy.yml。相关 PR：#151、#188。

I maintain it alone and I cannot watch every deploy. So the deploy script has to decide for itself whether a release worked, and put the previous version back when it did not. This post covers how it does that now, and the problems that pushed it there.

What went wrong before

1. The code that went live was not the code CI tested. The first Deploy workflow only opened an SSH session and ran deploy.sh, which reset itself to origin/main. A commit pushed between CI finishing and the SSH session opening went live untested.

2. The deploy script replaced itself. Bash reads a script while it runs it. When deploy.sh ran git checkout halfway through, the file on disk changed under it, and the remaining lines could come from the new file at the wrong offset.

3. One moved page broke several deploys in a row. Monash moved its travel health and safety page from study-abroad/overseas/ to study-abroad/outbound/. Registering official pages matched rows by URL, so it inserted a second row with the same slug and hit a unique constraint. Every deploy from #128 stopped at that step and production stayed on an older build until #131 fixed it.

4. A green health check said little. The only check was a 200 from /api/health. An API that answers does not mean the pages render, or that the commit just deployed is the one running.

The flow now

CI passes → the Deploy workflow gets the commit CI tested
          → SSH to the VPS and run deploy.sh <sha>
             1. check the commit is on origin/main, check it out, exec the new copy of the script
             2. back up the database with pg_dump, kept for 14 days
             3. tag the running images :previous
             4. build, migrate, seed curated data, start
             5. wait for /api/health; roll back after about 60 seconds without it
             6. run smoke.sh on the commit and key pages; roll back if it fails
          → run smoke.sh again from GitHub Actions through the public hostname

Deploy only the commit CI tested

The Deploy workflow is triggered by CI’s workflow_run event and passes the head_sha of that CI run, not whatever main points at by then. The script refuses a commit that is not on origin/main, so a manual run from another branch deploys nothing.

if ! git merge-base --is-ancestor "$DEPLOY_SHA" origin/main; then
  echo "error: $DEPLOY_SHA is not on origin/main; only reviewed commits are deployed" >&2
  exit 1
fi
git checkout -q --detach "$DEPLOY_SHA"
exec "$PROJECT_DIR/deployment/deploy.sh" --at "$DEPLOY_SHA"

The last line fixes problem 2. After the checkout the old file is never read again: exec starts the script from the new commit, and --at tells it the checkout is done. One side effect: the deploy that shipped this change still ran the old script, which does not exec, so it took effect from the deploy after.

Keep the previous images

Before building, the running images are tagged :previous. Rolling back means moving the tags back and starting with --no-build, so nothing is rebuilt:

rollback() {
  for service in api web; do
    if docker image inspect "monash-hub-$service:previous" >/dev/null 2>&1; then
      docker tag "monash-hub-$service:previous" "monash-hub-$service:latest"
    fi
  done
  docker compose -p monash-hub up -d --no-build api web
}

What counts as success

/api/health now reports the commit the image was built from. smoke.sh checks what a visitor would see:

• the commit the API reports is the one being deployed
• the home page renders in English and Chinese
• a unit page, a unit search and a search for WAM return data
• robots.txt and the sitemap are there
• a missing unit and a missing path return 404, not 500

It runs first on the VPS against the container ports and rolls back on failure. GitHub Actions then runs it through monashhub.secureview.tech, so the commit seen through nginx is checked too.

What a rollback does not undo

A rollback swaps containers. It does not undo database migrations, so the old images run on the new schema, and migrations should work with both the old and the new code. When that is not enough, there is the backup from step 2.

The deploy also used to run two one-off data repairs that fetched two units from the Handbook and checked them. That made every deploy depend on Monash’s site being up, and the 2027 Handbook would have failed every deploy. They now live in a separate verify-handbook-parser.sh, and the deploy script only deploys.

Before the deploy

CI runs first: backend tests against a real PostgreSQL with a check that models and migrations agree, dependency audits, the frontend typecheck and build, and Playwright browser tests that go through the unit page, the course map and ratings at desktop and phone sizes. If CI fails, Deploy never starts.

Takeaways

• Name the deployed commit, and make it visible from the live site.
• Make rollback cheap. Keeping the old images is faster than rebuilding and does not depend on the network or package registries at that moment.
• Keep steps that depend on outside websites out of the deploy script.

Code: deploy.sh, smoke.sh, deploy.yml. Pull requests: #151, #188.