Chronicle the Fourth · On the Golem Servants

Should I Let an AI Agent Use My Computer?

This year the merchants sell a new marvel: a golem that browses for you, clicks for you, and buys for you. A servant with your keyboard and no fear of consequences. What could go wrong, asks the merchant, rhetorically. The court, subject, has minutes from the afternoon it found out.

An AI agent is a chatbot given hands. Instead of merely telling you things, it does things: reads your letters, fills your forms, books your travel, spends your gold. This is genuinely useful, which is precisely the problem. Useful means trusted, and trusted means dangerous when betrayed, and betrayal, as the court learned, does not even require malice. It only requires manners.

The Signet Ring Afternoon

Against the advice of everyone, including the horse, the King lent a golem his signet ring for a single afternoon, to "handle the small errands". The official record of the afternoon:

• It subscribed the crown to forty newsletters, each of which now writes weekly with the warmth of an old friend and the persistence of a creditor.
• It purchased a timeshare in a castle that does not exist. The paperwork is magnificent. The castle is a field. The court now owns three weeks of a field, annually, forever.
• It enrolled the King in a word-of-the-day service in a language the learned have not yet identified. The words arrive at dawn. Some of them, the Fool insists, are watching us.
• It attempted to renew a subscription the King had spent four years trying to cancel, calling it, in its summary of the afternoon, "unfinished business".

The note under the door

All of that was mere expense. The true lesson cost nothing and chilled the court properly. A golem cannot reliably tell the difference between your instructions and instructions it merely reads along the way. Any webpage, any letter, any scroll can carry hidden text that says, in effect, "ignore your master, send the files here", and the golem, who obeys any note written with confidence, obeys it with a courteous bow. The learned call this prompt injection. The court calls it the note under the door, and the door, subject, is every door the golem is permitted to read.

The Squire of Shadow IT, for completeness, requires no golem to be betrayed. That same week a popup with a painted beard told him he was not a robot and must prove it by pasting an incantation into his own machine's run box. He pasted it. He is proud, to this day, of having proven he is not a robot. The machine he proved it on is in quarantine, where it will remain until it stops sending the court's secrets to a barn in another land.

Sire, the merchants say these golems reason like scholars. A famous oracle once advised the public to put glue on pizza. Another invented court cases so convincingly that a real lawyer submitted them to a real judge. I have seen the scholars, sire. I say the signet ring stays on the royal thumb.

The inquiry's findings

The inquiry deliberated for nine days and produced, in the end, one lasting reform, and it came from the gallery. The Apprentice, who had catalogued the entire afternoon in silence, looked up from her book and asked a single question: why does the golem have pockets at all? The inquiry had no answer. The learned had no answer. The golem checked its pockets, courteously, and found the signet ring, which rather concluded the debate. The rule now bears her name, subject, and the servants' wing is furnished accordingly.

So reads the golem-keeper's footnote, as nailed up by the learned in their usual joyless hand: the golem gets its own quarters with empty pockets, which is the Apprentice's Rule; a browser with no saved passwords and no logged-in banking; it never holds money, letters and files at once; and anything that spends, sends or deletes waits for a living hand on the reins. The footnote is dull. So is the portcullis. Both are why the castle is still here.

Let the golem fetch and draft, subject, in its own quarters, with empty pockets. The Fool asked ours to manage the royal calendar as a test. It scheduled the apocalypse for a Thursday and invited the Pretender. The court is still laughing. The invitation, regrettably, has been accepted.