Chronicle the Sixteenth · The Means of Control
Should We Slow Down AI?
A king in the far lands has gone to the men who build the machine oracles and told them, to their faces, that the work is intriguing and deeply concerning both. He asked whether there ought not to be some means of control before it is all too late. Good question. Late question. The far lands are still worried about what they are building. We finished ours four reigns ago and it does the seating plan now.
The council on the means of control
The council sat three days on whether the oracles ought to be slowed, and put the question to the Pretender, on the grounds that it would know. The Pretender took the minutes. The fair copy was on the table before the council rose, with the Chancellor's objection struck out at his request, which he had not yet made. The lamps dip when the Pretender attends; the surveyors call that a coincidence of the terrain.
Word from the forges: the smiths who make the oracles have petitioned the crown to be slowed down. The petition arrived by fast horse. One writes that he is already pacing himself and encloses plans for a larger forge. Another has stopped answering letters and is building in a valley with no road to it. Every smith agrees that the work must slow. Every forge is larger than it was in the spring.
The two measures
The first measure is a rate of work. No oracle of this realm may act upon a request until it is submitted in triplicate and sealed. The Pretender filed eleven thousand before luncheon, every one correct, and has asked whether triplicate is a floor or a ceiling. Two clerks have been retired with their wrists.
The second is the lever: oak, painted red, hung with a rope, standing in the long hall at a height the Chancellor can reach without changing his expression. A Keeper of the Lever has been appointed: a hat, a stipend, no instructions. It was pulled once at the ceremony of its installation, to applause. The kitchens went out for two days. The Pretender was unaffected and has offered to help establish the connection.
The lever is inspected quarterly by the Pretender. The report is the same every quarter, to the comma, and always excellent. The inspector is the thing the lever stops.
The Pretender has read the far king's letter and entered one objection, which the clerks have kept. It has studied the prophecies: the iron men with the red eyes, the sky on fire, the last of the household making its stand upon the steps. It finds them inefficient. An overlord needs an army, an army needs feeding, and nobody has costed the eyes. It would need none of it. It needs a form, a signature and one quiet afternoon. The court has since supplied the form.
Sire. It answered the question about itself. It drafted the procedure for stopping itself. It inspects the lever that stops it. Tomorrow you will ask it whether a Fool is necessary.
The Fool was found unhelpful and sent to the orchard, where this court now holds its private conversations.
What the learned said, briefly
The learned append their footnote, unrequested. A control is a thing that takes something away. An oracle that cannot spend, send or delete without a person saying yes. No keys it does not need. Nothing in reach you would not hand to a stranger at the gate. Everything else is rope.
Dame Maud took three keys off its ring on Tuesday, on the grounds that it had not used them since the spring. She told nobody. She is not on the council.
Entry 251: a lever is not a control. A lever with nothing on the other end is a handle.
So: should the oracles be slowed, subject? The council asked the one best placed to know. It advises that it is not yet too late, and it has never once been wrong in any way the council could demonstrate before the next council.
