Industrial reliability: change the denominator 工业可靠性:换掉分母
I was building the operator interface for a force-controlled industrial robot — an arm that presses a powered tool against a workpiece with a programmed force and moves it along a path. My client asked me the best question anyone has asked me in years: "The problems that hit us on the floor will be the ones we didn't think of — that's what 'didn't think of' means. But every reliability estimate uses 'the things we thought of' as its denominator. So how do you ever guarantee industrial reliability?" 我当时在做一台力控工业机器人的操作员界面——一条机械臂,以设定的力把一个带动力的工具压在工件上,沿路径移动。客户问了我这些年被问过的最好的问题:"车间里真正打到我们的问题,一定是我们没想到的那些——'没想到'就是这个意思。可我们每一个可靠性估算,用的分母都是'我们想到的东西'。那你到底怎么保证工业可靠性?"
Why "review harder" never converges为什么"再审仔细点"永远收敛不了
Review finds the bugs you can imagine. The bugs that hurt you are, by definition, the ones you couldn't. So you review again — the second pass finds a few more imaginable ones, the third fewer. The sequence doesn't converge to zero; it converges to "everything I was capable of imagining," which is not the same thing at all. On that project I'd already run five rounds of review, my own plus independent cold-eyes passes, and every round still found real defects. That's not a sign the code was unusually bad — it's a sign that "review harder" is a treadmill. If round five still finds bugs, round six will too, and you can never stand in front of a moving machine and say "there are no more." 审查能找出你想象得到的 bug。而伤到你的 bug,按定义,正是你想象不到的那些。于是你再审一遍——第二遍又找出几个想象得到的,第三遍更少。这个数列不收敛到零;它收敛到"我能想象的一切",而那根本是另一回事。那个项目上,我已经跑了五轮审查,自己的加上几轮独立的冷眼复审,每一轮都还在找出真实缺陷。这不说明代码格外烂——它说明"再审仔细点"是一台跑步机。如果第五轮还在找出 bug,第六轮也会,而你永远没法站在一台正在动的机器前说"没有了"。
Counting bugs is a losing game: the denominator is infinite and unknowable. The way out isn't to count better — it's to change what you count. 数 bug 是一场必输的游戏:分母无限且不可知。出路不是数得更准——而是换掉你数的东西。
Change the denominator换掉分母
The whole idea in one sentence: stop trying to guarantee "my software has no bugs"; guarantee instead that "even if my software is completely, arbitrarily wrong, the consequences are bounded." The first is impossible. The second is achievable — because it swaps an infinite denominator for a finite one. You cannot enumerate your bugs, but you can enumerate the kinds of harm a bug is able to cause. For a machine like this, the entire space of bad outcomes fits in five buckets: 整个想法一句话:别再试图保证"我的软件没有 bug";改为保证"即使我的软件彻底地、任意地错了,后果也是有界的"。前者不可能。后者可做到——因为它把一个无限的分母换成了一个有限的。你没法枚举你的 bug,但你能枚举一个 bug 能造成的伤害种类。对这样一台机器,所有坏结果的空间装进五个桶里:
- A person is hurt. · The machine or the workpiece is damaged. · Saved data is lost. · The system halts or the screen freezes. · The job comes out wrong. 有人受伤。 · 机器或工件被损坏。 · 已存数据丢失。 · 系统停机或界面卡死。 · 活儿做错了。
It doesn't matter how many bugs you have or how exotic — the damage they do must land in one of those five. There is no sixth kind of bad day. So the question transforms from "how many bugs did I think of?" (denominator ∞, never done) to "for each of five consequence classes, do I have a backstop that does not depend on my own code being correct?" (denominator 5, finite, checkable, done). That last clause is load-bearing: a backstop written in the same codebase as the bug is the same author grading their own exam. The guarantee has to bottom out in a layer the buggy software cannot author or weaken. 你有多少 bug、多离奇,都无所谓——它们造成的伤害必然落在这五个之一。没有第六种坏日子。于是问题从"我想到了多少 bug?"(分母 ∞,永远做不完)变成"对五个后果类中的每一个,我有没有一个不依赖我自己代码正确的兜底?"(分母 5,有限、可核验、做得完)。最后那句是承重的:一个和 bug 写在同一套代码库里的兜底,是同一个作者给自己的考卷打分。保证必须落到一个有 bug 的软件无法编写也无法削弱的层里。
The finding that proved it to me向我证明这一点的那个发现
In a later phase, the interface would send the operator's chosen parameters — force, speed, path counts — down to the robot's motion controller. The transport for that was vendor-supplied boilerplate that shipped with the platform template. It had sat in the tree, unused, since day one. Five rounds of review glided right over it — correctly, because it wasn't executing yet; line by line, nothing was "wrong." Then I stopped reviewing and asked the consequence question instead: assume this transport is compromised — what's the worst it can do, and what stops it? 在后面一个阶段,界面会把操作员选定的参数——力、速度、路径次数——下发给机器人的运动控制器。承载这个的是厂商随平台模板附带的样板代码。它从第一天就躺在代码树里,没被用过。五轮审查从它上面滑过去了——而且是对的,因为它还没在执行;逐行看,没有哪里"错"。然后我停止审查,改问后果问题:假设这段传输被攻陷——它最坏能干什么,什么拦得住它?
The boilerplate built the message to the controller by pasting the parameters, as text, into the middle of a program the controller would then execute — trusting a standard serializer to keep them contained. But that serializer escapes some quote characters and not others. Which means a parameter value containing the right character could close the quoted string early and turn the rest of itself into live instructions for the robot. Now follow the path: one of those parameters was a name — a label the operator types, or worse, a label riding in on a backup file imported from a USB stick. An untrusted file on a removable stick → arbitrary motion on an industrial arm with a powered tool spinning on the end of it. 这段样板代码构造发给控制器的消息的方式,是把参数当作文本,粘进一段控制器随后会执行的程序中间——指望一个标准序列化器把它们兜住。可那个序列化器转义了某些引号字符、却漏了另一些。这意味着一个含有特定字符的参数值,可以提前闭合那个带引号的字符串,把它自身剩下的部分变成给机器人的实时指令。顺着路径走:那些参数里有一个是名称——一个操作员敲进去的标签,更糟的是,一个搭着从 U 盘导入的备份文件进来的标签。一个 U 盘上的不可信文件 → 一条末端转着动力工具的工业机械臂上的任意运动。
That defect was invisible to five rounds of line-by-line review — not because the reviewers were careless, but because by the "is this line correct?" test the line was correct, and it wasn't even running. It became visible instantly the moment I switched from "find the bug" to "assume it's compromised; where's the backstop?" There was none. The only thing between an untrusted file and a moving machine was my own code being perfect — the one thing I'd just spent five rounds proving I could not promise. 那个缺陷对五轮逐行审查是隐形的——不是因为审查者粗心,而是因为按"这一行对吗?"的标准,这一行确实对,而且它压根没在运行。而当我从"找 bug"切换到"假设它被攻陷;兜底在哪?"的那一刻,它立刻现形了。没有兜底。一个不可信文件和一台运动机器之间,唯一挡着的,是我自己的代码完美无缺——而那恰是我刚花五轮证明了我无法承诺的那件事。
What the backstops actually look like兜底具体长什么样
- Re-check the limits on the far side, in a layer you don't author. My interface clamped force and speed to safe limits — but that clamp lives in the same software that might send the bad value. So the controller clamps them again, independently, in its own program that I don't write. Now the interface can send 999 and the machine still uses 40. The far-side clamp is the authority; the near-side one is a courtesy. This one line is the difference between "safe if my code is right" and "safe." 在你不编写的层里,于远端重新核验限值。我的界面把力和速度钳到安全限值——但那个钳位活在可能发出坏值的同一套软件里。所以控制器再独立钳一次,在它自己那段我不去写的程序里。于是界面就算发 999,机器仍然用 40。远端钳位是权威;近端钳位是礼貌。这一行,就是"我代码对才安全"和"安全"之间的区别。
- A physical envelope the software cannot reach past. Most industrial controllers can enforce a geometric box, configured once behind a safety password, that the motion system itself refuses to leave. Set it tightly around the work cell, and it no longer matters what coordinate my software computes — correctly or insanely — because the safety layer, which my application cannot author, weaken, or even see, stops the arm at the wall. It's the only real backstop for a bad-coordinate bug, where clamping is meaningless because a position isn't a number you can min/max. 一个软件够不到的物理包络。多数工业控制器能强制一个几何盒子——在安全密码后面配置一次,运动系统本身拒绝越出。把它紧紧设在工作单元四周,那我的软件算出什么坐标就都无所谓了——无论算得对还是发疯——因为那个安全层(我的应用无法编写、无法削弱、甚至看都看不见)会先在墙上把机械臂拦住。对一整类"坏坐标"的 bug,这是唯一真正的兜底,因为对这类 bug 钳位毫无意义:空间里的一个位置,不是一个你能 min/max 的数。
- Staged unlock — turn "didn't think of it" into "saw it happen, safely." You don't go from "software controls the machine" to "full force on a real part" in one step. You pass gates, each bounding the consequences of everything you haven't discovered yet: Dry run — the machine doesn't move; the software runs for real and logs every command it would have issued (a wrong parameter costs a line in a log). Air cut — real motion, tool off, path lifted above the work; a wrong path draws a shape in the air. Scrap material — real force on a piece you'd happily throw away. Then production. The bug you didn't think of is still in there at every stage — that's the point, you never removed it — but reality reveals it at the stage where its consequence is bounded. 分级解锁——把"没想到"变成"安全地看它发生了"。你不会一步从"软件控制机器"跳到"在真件上全力施压"。你穿过一道道闸,每道都把你尚未发现的一切的后果兜住:干跑——机器不动;软件真实运行,并把它本该下发的每一条指令记进日志(一个错参数的代价,是日志里的一行)。空切——真实运动,工具关闭,路径抬到工件上方;一条错路径,在空气里画个形状。废料——在一块你乐意扔掉的料上施真实的力。然后才是生产。你没想到的那个 bug 在每一级都还在里面——这正是重点,你从没移除它——但现实在它后果有界的那一级,把它揭示给你。
That's what industrial reliability actually is. Not "the machine won't have problems." It's: when the machine has a problem — and it will — the problem surfaces in the air, above the work, with the tool switched off, and nobody gets hurt. 这才是工业可靠性真正的样子。不是"机器不会出问题"。而是:当机器出了问题——它一定会——那问题浮现在半空、在工件上方、工具已关闭,没有人受伤。
The part nobody says out loud, and should没人明说、但该说的那部分
Here's the sentence I think every honest vendor of anything that moves should be willing to say: we are not building toward "no bugs." We know there will be bugs — including ones we'll never find. We're building so that when a bug fires, it's caught, or bounded, or physically impossible to turn into harm. That sounds like an admission of weakness; it's the opposite. The person who promises bug-free software is naive or selling. The person who shows you the gates, the two independent clamps, and the box the arm can't leave has thought about the day it goes wrong — the only day that matters. "Bug-free" is a marketing word; "consequence-bounded" is a design. 这是我认为每一个卖"会动的东西"的诚实供应商都该愿意说的一句话:我们不是朝着"没有 bug"去建。我们知道会有 bug——包括我们永远找不到的。我们建的,是让一个 bug 触发时,它被接住、被兜住、或在物理上根本不可能变成伤害。这听起来像承认软弱;恰恰相反。承诺无 bug 软件的人,要么天真,要么在推销。而给你看那些闸、两道独立钳位、和机械臂出不去的那个盒子的人,已经想过它出错的那一天——那是唯一要紧的一天。"无 bug"是个营销词;"后果有界"是一种设计。
Written from a real force-controlled robotics integration; all client, vendor, and product specifics removed. The method is general — it applies to anything where a software failure has physical or irreversible consequences, from motion control to payments to deploys. 写自一次真实的力控机器人集成;所有客户、厂商、产品细节均已抹去。方法是通用的——它适用于任何"软件失败会带来物理或不可逆后果"的场景,从运动控制到支付,到部署。
← All notes← 全部笔记