视频库 / NO.120
--:--
点击播放，这里会跟随视频显示当前句的中英字幕。
The Ethical Algorithm | Michael Kearns & Aaron Roth | Talks at Google
这期还没有生成章节与讲者信息。
0:05
AARON ROTH: The starting point of our book is the observation that although machine learning has been around for a long time, we are now starting to use it for increasingly consequential tasks, as many people here at Google will know. So for example, as was in the news recently, lending decisions, like if you apply for an Apple credit card, will you get a high credit limit or a low credit limit, are now often made without any human intervention at all, just an algorithm. HR departments make hiring and compensation decisions informed by machine learning algorithms.
AARON ROTH：我们这本书的出发点是这样一个观察：虽然机器学习已经存在很长时间了，但我们现在开始把它用在越来越事关重大的任务上，在座的许多 Google 同仁都清楚这一点。比如说，就像最近新闻里报道的，信贷决策——比如你申请一张 Apple 信用卡时，你会拿到高额度还是低额度——现在往往完全没有人工介入，就只是一个算法说了算。人力资源部门做招聘和薪酬决策时，也会参考机器学习算法的结果。
0:39
And in a number of states, including in Pennsylvania, bail and parole decisions are informed by trained models. And so it's sort of natural when you start using machine learning to make important decisions about people that you start worrying that maybe those algorithms will violate some of the social norms that we would expect of human decision makers when they were making those decisions. And indeed, there's lots of evidence that this happens. We've seen these articles every week, and there's a number of very good popular books that have pointed out the problem.
而在包括宾夕法尼亚州在内的一些州，保释和假释的决定也会参考训练出来的模型。所以当你开始用机器学习来做关于人的重要决策时，你自然会开始担心，也许这些算法会违背我们对人类决策者做同样决定时所期待的一些社会规范。事实上，有大量证据表明这种情况确实在发生。我们每周都能看到这类文章，也有不少非常优秀的通俗读物指出了这个问题。
1:11
And here's three books here that we admire, but what these books do is very much point out the problem. But where they say less is about what you can do to fix these problems, what the solutions are. They talk about the need for regulation, for example, which we by and large agree with. But they don't talk about technically what you would do to make the algorithms better behaved, and that's the goal of our book to explain in plain English the emerging science of what we talk about as embedding ethical norms into algorithms.
这里有三本我们很欣赏的书，但这些书基本上都只是在指出问题。而它们谈得较少的是，你能做些什么来解决这些问题，解决方案是什么。比如说，它们会谈到监管的必要性，这一点我们大体上是认同的。但它们没有从技术层面谈，你该怎么做才能让算法表现得更好，这就是我们这本书的目标——用通俗易懂的语言解释一门正在兴起的科学，我们把它称作把伦理规范嵌入算法。
1:46
Then there's now a community of hundreds of people, including us, working on these problems. One comment we often get, in fact, something that one of our early reviewers asked us is, does the very premise of the book, like the title of the book, does it make any sense, the ethical algorithm? Because algorithms are, in the end, tools. They're human artifacts, like hammers. Well, algorithms, like hammers, can be used to do bad things. They can be used as instruments of violence. I could whack you on the hand with a hammer.
现在已经有数百人组成的一个社群，包括我们在内，都在研究这些问题。我们常常收到一种评论，实际上我们早期的一位审稿人就问过：这本书的前提本身，比如这本书的书名，「合乎伦理的算法」，这说法讲得通吗？因为算法归根结底只是工具。它们是人造物，就像锤子一样。没错，算法和锤子一样，可以被用来做坏事。它们可以被当作施暴的工具。我可以用锤子砸你的手。
2:16
But if I did that, we wouldn't think of this as some moral failing of the hammer, right? Like you would attribute that action directly to me, and that's basically how we regulate and write laws about violence induced by hammers, right? We say, if I whack you on the hand with a hammer, then I'm very likely going to have to go to jail, and I'll take that into account when I'm deciding whether I want to do that. But algorithms-- and when we say algorithms, we really mean models. The sort of trained models that at the output of machine learning pipeline are different.
但如果我那么做了，我们不会认为这是锤子的道德缺陷，对吧？你会把这个行为直接归咎于我，而这基本上就是我们针对锤子引发的暴力进行监管和立法的方式，对吧？我们规定，如果我用锤子砸了你的手，那我很可能得去坐牢，而我在决定要不要这么做的时候，就会把这一点考虑进去。但算法——而我们说算法的时候，其实指的是模型。机器学习流水线输出的这类训练好的模型是不一样的。
2:57
They're different in a number of ways, but one that's salient for this discussion is that it's very difficult to predict the outcome in every situation of an algorithm that you've trained using the principles of machine learning. So many of you are aware of what the machine learning pipeline looks like, but let's just briefly recount it. You'll start with some data set. A data set might these days consist of records of millions of people. You might have hundreds of thousands of features for each person.
它们的不同体现在很多方面，但对今天的讨论来说，最突出的一点是：对于一个你用机器学习原理训练出来的算法，你很难预测它在每一种情况下会给出什么结果。对于一个你用机器学习原理训练出来的算法，很难预测它在每种情况下的输出。你们很多人都知道机器学习流水线长什么样，但我们还是简单回顾一下。你会从某个数据集开始。如今，一个数据集可能包含数百万人的记录。每个人可能有几十万个特征。
3:28
And in the best case-- if you're lucky, you're in the best case. You're not always in the best case. In the best case, you understand this data set as the data scientist developing an algorithm in the sense that maybe you know how the data was gathered. Maybe you know what all of the features represent, but it's hard to say that you really understand all of the information contained in such a massive object. Then you use this data set to formulate some usually narrow objective function, some proxy for classification error or maybe profit.
而在最好的情况下——如果你运气好，处于最好的情况下。但你并不总是处于最好的情况。在最好的情况下，作为开发算法的数据科学家，你理解这个数据集，比如说你知道数据是怎么收集的。比如说你知道数据是怎么收集的。也许你知道所有这些特征代表什么，但很难说你真的理解了这样一个庞大对象里包含的所有信息。但很难说你真的理解了这么庞大的对象中所包含的全部信息。然后你用这个数据集来构建某个通常很狭窄的目标函数，某种分类误差的代理指标，或者也许是利润。
4:00
And you use some tool, something like stochastic gradient descent, to search over some enormous class of models to find the one that is best or very good at maximizing your narrow objective function. And then you get out some model, right? If you're training a deep neural network, this might consist of millions of parameters. And it's hard to say anything at all about this model, except that it's probably very good as measured according to the objective function you specified. And so the problem is when and if this model goes on to inflict some harm on some person or some group of people, it's typically not the case that this harm was the result of some mal intent of the software engineer or scientist who is sitting behind the scenes and building this algorithm, right?
然后你使用某种工具，比如随机梯度下降，在某个极其庞大的模型类中进行搜索，去找出那个在最大化你那个狭窄目标函数方面表现最好、或者非常好的模型。然后你就得到了某个模型，对吧？如果你训练的是一个深度神经网络，它可能包含数百万个参数。而关于这个模型，你几乎说不出什么，除了它按照你指定的目标函数来衡量大概表现很好。按照你所指定的目标函数来衡量。所以问题在于，当这个模型最终对某个人或某个群体造成伤害时，这种伤害通常并不是幕后那位构建算法的软件工程师或科学家出于恶意造成的，对吧？以及构建这个算法的人，对吧？
4:52
If that was the case, the situation would be much simpler. Existing regulatory tools could be used to weed out bad actors. But the problem is that the harms that we see from algorithms are typically the unanticipated and unintended side effects of optimization over large classes of models of the very basic premise of machine learning. And so if we're going to prevent this bad behavior by learned algorithms, we need to figure out how to embed our social values, the actions that we want the algorithms not to exhibit into the design process itself.
如果是那样的话，情况反而要简单得多。现有的监管手段就可以用来清除那些不良行为者。但问题在于，我们看到的算法带来的危害，通常是在庞大的模型类上做优化时——也就是机器学习最基本的前提——所产生的意料之外、非本意的副作用。在庞大的模型类上进行优化，而这正是机器学习最基本的前提。所以，如果我们要防止学习出来的算法出现这些不良行为，我们就需要弄清楚如何把我们的社会价值观，也就是我们不希望算法表现出来的那些行为，嵌入到设计过程本身当中。
5:37
OK, and that's hard, right? Because words like privacy, and fairness, and accountability, these are big and vague. They mean many things, but it's important to be precise about definitions when you say privacy or fairness, and in particular, when you say you want an algorithm to be private or fair. It's not enough to speak about these things as a philosopher might, for example. Because just at a very practical level, if you're going to embed these things as constraints into some optimization, you need to be mathematically precise.
好的，这很难，对吧？因为像隐私、公平、问责这样的词，都是宏大而模糊的概念。它们有很多含义，但当你说隐私或公平时，对定义保持精确是很重要的，尤其是当你说你希望某个算法是保护隐私的或公平的时候。比如说，仅仅像哲学家那样谈论这些东西是不够的。因为从非常实际的层面来说，如果你要把这些东西作为约束嵌入到某个优化问题中，你就必须在数学上足够精确。
6:11
It's also an enlightening exercise, even separately from the need to embed mathematical constraints when you're designing algorithms to think about what you really mean. What are different kinds of privacy? What are different kinds of fairness? And the very act of trying to be very precise about these things is illuminating and can reveal new trade offs that maybe weren't immediately evident. So we've written these words in decreasing degree of gray scale starting with privacy, ending with morality, and you know, you can't even see it.
而且这也是一个很有启发的练习，即便抛开设计算法时需要嵌入数学约束这一点不谈，去思考你真正的意思。隐私有哪些不同的种类？公平又有哪些不同的种类？而试图对这些东西给出非常精确定义的这个行为本身就很有启发性，能揭示出一些可能并不是一眼就能看出来的新权衡。所以我们把这些词按灰度递减的方式写了出来，从隐私开始，到道德结束，而且你甚至都看不见它了。
6:47
But Michael assures me he wrote singularity in white at the bottom there. In proportion basically to how much progress we've made trying to understand these things at a sort of mathematically precise level, how much progress we've made in thinking about the consequences of embedding constraints representing these notions into algorithms. So it's not to say that we've solved privacy or that we have precise ways of thinking about all of the many different kinds of privacy. But as we'll talk about in a moment, we've made some progress.
但 Michael 向我保证，他在下面用白色字写了 singularity（奇点）。基本上，这个顺序对应于我们在从数学精确层面理解这些概念方面取得了多少进展，以及在思考把代表这些概念的约束嵌入算法之后会带来什么后果这方面，我们又取得了多少进展。所以这并不是说我们已经解决了隐私问题，或者说我们对各种各样不同类型的隐私都有了精确的思考方式。但正如我们稍后会谈到的，我们确实取得了一些进展。
7:18
Fairness isn't there yet, but it's along a good path. And for these other things, accountability, interpretability, and even more as you go further down the list-- and people are working on these things, and they're important. But we feel like we don't have the right definitions yet that are sort of a necessary prerequisite to making the kind of scientific progress that we talk about in the book. MICHAEL KEARNS: OK, thanks, Aaron. OK, so what we want to do with most of the remaining time is just go through two quick vignettes.
公平性还没到那个程度，但它正走在一条不错的路上。而对于其他这些方面——可问责性、可解释性，以及再往下看还有更多——人们确实在研究这些问题，它们也很重要。但我们觉得，我们还没有找到正确的定义，而这些定义是取得我们在书中所讲的那种科学进展的必要前提。MICHAEL KEARNS：好的，谢谢你，Aaron。好，接下来剩下的时间里，我们主要想做的就是过一遍两个简短的案例。
7:48
One on privacy and one on fairness, which as per Aaron's last slide, are there that we feel are, in relative terms, the most mature for the type of scientific research or algorithmic research that we're discussing. And as Aaron said, sometimes like the very exercise of having to think so precisely about the definitions of these social norms is itself greatly beneficial and might not only reveal trade offs that you weren't aware of, but like flaws in your intuitions about these ideas if you just talk about them at the level that, let's say, a moral philosopher might.
一个是关于隐私的，一个是关于公平性的。正如 Aaron 最后一张幻灯片所说，相对而言，这两个是我们认为最成熟的领域，适合我们正在讨论的这类科学研究或算法研究。而且正如 Aaron 说的，有时候，光是被迫如此精确地去思考这些社会规范的定义，这个过程本身就非常有价值，它不仅可能揭示出你原本没意识到的各种权衡取舍，还可能暴露出你对这些概念的直觉当中的缺陷——如果你只是停留在，比方说，那种层面上去谈论它们的话。就像道德哲学家会做的那样。
8:19
And so privacy is a good case study. We argue in the book that we feel like there's a definition of privacy for at least of the type of privacy that I'm going to talk about here is the right definition, which is differential privacy. But it's preceded by definitions that, I think, we and others feel like are fundamentally flawed. And unfortunately, these fundamentally flawed concepts are the ones almost exclusively in force in practice these days. So if you look at a end user license agreement or a privacy policy of a large company, they will normally refer to policies.
所以隐私是一个很好的案例研究。我们在书中论证，我们觉得对于隐私——至少是我接下来要讲的这类隐私——是存在一个定义的，那就是正确的定义，也就是差分隐私（differential privacy）。但在它之前还有一些定义，我认为我们和其他人都觉得那些定义存在根本性缺陷。不幸的是，这些存在根本性缺陷的概念，恰恰是如今实践中几乎唯一在使用的。所以如果你去看一家大公司的最终用户许可协议或隐私政策，它们通常会提到一些策略。
8:52
If they're precise at all, they'll talk about various forms of anonymization or removing PII, Personally Identifiable Information. And to give you a sense of why we think those definitions are fundamentally flawed, I have here a toy example in which there is two different databases from two different hospitals of medical records. And due to privacy concerns, there's been some anonymization done here, and anonymization largely consists of operations of redaction, like entirely removing certain columns from a database, or coarsening in which you sort of fuzz up the information.
如果它们表述得足够精确的话，会谈到各种形式的匿名化，或者移除 PII，也就是个人可识别信息。为了让你们理解我们为什么认为这些定义有根本性缺陷，我这里有一个玩具示例，其中有两个不同的数据库，分别来自两家不同医院的病历记录。出于隐私方面的考虑，这里做了一些匿名化处理，而匿名化主要包括涂抹（redaction）操作，比如把数据库中的某些列整个删掉，或者做粗化处理，也就是把信息模糊化。
9:29
And then the hope is that somehow when you're done with this, you have some sort of privacy guarantees. So in this top database, somebody has gone in and decided like, well, let's entirely redact a name. Rather than giving precise ages, let's group them into decades. So are you 10 to 20, 20 to 30, et cetera? Let's give some information about zip code but redact the last two digits. And let's keep some of the medical information, like whether you're a smoker or not-- we'll come back to smoking in a minute-- and what the particular diagnosis that you were given in your visit was.
然后大家的期望是，做完这些之后，你就多少能获得某种隐私保障。所以在上面这个数据库里，有人进去后决定：好吧，我们把姓名整个抹掉。与其给出精确年龄，不如把它们按十年分组。比如你是 10 到 20 岁、20 到 30 岁，等等？我们给出一些邮政编码的信息，但把最后两位抹掉。然后保留一部分医疗信息，比如你是不是吸烟者——我们等一下会再回到吸烟这个话题——以及你这次就诊时被给出的具体诊断是什么。
10:04
And of course, in reality, you know, these databases would be much, much larger for a large hospital, like the University of Pennsylvania's. There might be tens of thousands of records, but the conceptual flaw can already be demonstrated in this toy example. Suppose you have some additional information aside from this database, like you have a neighbor named Rebecca, who you happen to know is female. And that she's 57 years old. And you know this because she's your neighbor and you're friends with her, OK?
当然，在现实中，对于像宾夕法尼亚大学医院这样的大医院，这些数据库会大得多得多。可能会有几万条记录，但这个概念上的缺陷在这个玩具示例中就已经能展示出来了。假设除了这个数据库之外，你还掌握一些额外信息，比如你有个邻居叫 Rebecca，你恰好知道她是女性。而且她 57 岁。你知道这些是因为她是你的邻居，你和她是朋友，对吧？
10:32
So if with that side information you also managed to get hold of this allegedly anonymous database, then already in it there are exactly two records which match your knowledge about Rebecca. And they're the two highlighted in red. And notice that already from this side information you can infer that your neighbor either is HIV or has colitis, and she might reasonably already consider that to be a violation of her privacy alone. Now, again, in a real large database and in a real application of these methods, you might go for a criterion like what's called k-anonymity.
所以如果带着这些旁侧信息，你又设法拿到了这个所谓匿名的数据库，那么在里面恰好有两条记录与你对 Rebecca 的了解相匹配。就是标红的那两条。注意，仅凭这些旁侧信息，你就能推断出你的邻居要么是 HIV 感染者，要么患有结肠炎，而她完全有理由认为，光是这一点就已经侵犯了她的隐私。再说一次，在真实的大型数据库和这些方法的实际应用中，你可能会采用一个叫做 k-匿名（k-anonymity）的标准。
11:09
So what's k-anonymity? K-anonymity basically asks that you do enough of this coarsening and redaction so that any row in the allegedly anonymized database matches at least k other-- there at least k matches to that row, k identical records, OK? So then you wouldn't know-- rather than knowing this is like a two anonymous database. But in general, we might hope to get more privacy by asking for 100 anonymity rather than two anonymity. The real problem, of course, comes when you also know that your neighbor Rebecca happened to also have a visit to a second hospital whose database is at the bottom, and this hospital has also, in an effort to provide some kind of privacy, done some redaction.
那么什么是 k-匿名呢？k-匿名基本上要求你做足够多的粗化和涂抹处理，使得所谓已匿名化的数据库中的任何一行都至少能匹配到另外 k 条——也就是说至少有 k 条记录与该行匹配，k 条完全相同的记录，好吧？这样一来你就无法确定——而不是像这样，这只是一个 2-匿名的数据库。但总体而言，我们可以期望通过要求 100-匿名而不是 2-匿名来获得更多隐私保护。当然，真正的问题出现在你同时还知道，你的邻居 Rebecca 恰好也去过第二家医院，它的数据库在下面这里，而这家医院同样为了提供某种隐私保护，也做了一些涂抹处理。
11:53
The same kind of redaction and coarsening in their database, and now three records match Rebecca there. And of course, the real problem here is the join of these two databases, right? Which is sometimes called linkage analysis, or triangulation, or various other names. When I take the intersection of the top red records and the bottom red records, I uniquely now know that Rebecca is HIV. And you might try to wish these problems away with fancier definitions or by appealing to scale. But the real problem with these types of definitions is that they pretend that the data set in front of you is the only data that is ever going to exist now or forever in the world.
在他们的数据库里做了同样的涂抹和粗化，现在那边有三条记录与 Rebecca 匹配。当然，这里真正的问题在于这两个数据库的连接（join），对吧？这有时被称为链接分析（linkage analysis），或者三角定位，或者其他各种名称。当我取上面那些红色记录和下面那些红色记录的交集时，我就能唯一地确定 Rebecca 是 HIV 阳性。你可能会想用更花哨的定义或者诉诸规模来把这些问题搪塞过去。但这类定义真正的问题在于，它们假装摆在你面前的这个数据集是世界上现在或将来唯一会存在的数据。（承上）现在或永远。
12:33
And they don't anticipate attacks on privacy that come from triangulation of multiple databases, other information you might have about people, even publicly declared information that they weren't particularly trying to hide. Many of you might have seen this sort of mainstream news frenzy over articles that, I think, surprised probably very few people in this room. One was about a month ago, and it basically said, here are 18 apparently innocuous attributes that if I know them about you, they serve as a fingerprint for you among all US citizens.
它们没有预料到那些来自多个数据库三角定位的隐私攻击，也没有预料到你可能掌握的关于这些人的其他信息，甚至是他们并没有特意想要隐藏、公开声明过的信息。你们很多人可能都看到过主流新闻对某些文章的疯狂报道，我想，那些文章大概让在座各位中的极少数人感到惊讶。其中一篇大约是一个月前的，它基本上是说，这里有 18 个看似无害的属性，如果我知道了你的这些属性，它们就能作为你在全体美国公民中的指纹。
13:07
So you know, I'm not sure what they were. But you can imagine if you tell me what kind of car you drive, you tell me your zip code, you tell me what color your eyes are, you tell me whether you have dogs or cats, each one of these things, of course, is like exponentially cutting away the remaining possibilities. And it doesn't take long to kind of have that sort of innocuous information, undo the privacy promises of these anonymity methods. OK, so these are bad privacy definitions as we discuss in the book.
你知道，我不太确定具体是哪些属性。但你可以想象，如果你告诉我你开什么车，告诉我你的邮政编码，告诉我你的眼睛是什么颜色，告诉我你养狗还是养猫，当然，这里的每一项都在指数级地削减剩余的可能性。用不了多久，这类看似无害的信息就足以瓦解这些匿名化方法所承诺的隐私保护。好的，正如我们在书中讨论的，这些都是糟糕的隐私定义。
13:37
What would be a good privacy definition? Well, let me start by proposing a definition, which has been, I think, proposed since at least the '70s, which if you could get it would be a nice definition. But we argue in the book that it's basically asking for too much in the sense that if you enforce this kind of privacy, we will never be able to do useful, interesting things with data, including things like medical research studies. OK, so what is the definition I have in mind? So imagine-- and you can make this mathematical, but I won't bother here.
那么什么才算是好的隐私定义呢？那我先提出一个定义吧，这个定义我想至少从上世纪70年代起就有人提出过了，如果你真能做到，它会是个不错的定义。但我们在书中论证说，它本质上要求得太多了，因为如果你强制执行这种隐私，我们就永远没法用数据做任何有用、有趣的事情，包括医学研究之类的事。好，那我心目中的这个定义是什么呢？设想一下——你可以把它写成数学形式，但这里我就不费这个劲了。
14:10
Imagine we basically said the definition is that no harm should ever come to you of any kind as the result of a data analysis in which your data was included, OK? So let's think about that as a privacy definition for a second. So certainly, it's a strong privacy guarantee, right? I'm sort of allowing the notion of harm to be entirely general, and I'm basically saying if your data was used, no harm should come to you of that study. OK, so why is this asking for too much? So imagine that it's 1950, and you are a smoker, OK?
设想我们基本上是说，这个定义要求：只要你的数据被纳入某项数据分析，你就绝不应该因此受到任何形式的伤害，好吧？那我们花一秒钟想想，把它当作一个隐私定义看看。首先，它当然是一个很强的隐私保障，对吧？我在这里允许“伤害”这个概念完全泛化，基本上是在说：如果你的数据被使用了，你就不该因为那项研究受到任何伤害。好，那为什么说这要求太多了呢？设想现在是1950年，而你是个吸烟者，好吧？
14:44
And if it's 1950, you are a smoker. Because in 1950, pretty much everybody smokes. There is no social or medical stigma associated with smoking. In fact, it's seen as a glamorous habit, and so you do it openly in public. Everybody that knows you knows that you're a smoker. Maybe even your health insurer knows that you're a smoker. Who cares? And suppose you're asked to contribute your medical record to the famous series of studies that were done in the 1950s in England that firmly established a correlation or connection between smoking and lung cancer.
如果是1950年，那你就是个吸烟者。因为在1950年，几乎人人都吸烟。吸烟没有任何社会或医学上的污名。事实上，它被看作一种有格调的习惯，所以你会公开地在大庭广众之下抽烟。认识你的人都知道你吸烟。甚至你的健康保险公司可能都知道你吸烟。谁在乎呢？然后假设有人请你贡献自己的医疗记录，用于上世纪50年代在英国做的那一系列著名研究——正是这些研究牢固确立了吸烟与肺癌之间的相关性或联系。
15:18
So your data was included in this analysis. This analysis announced to the world that there is a connection between smoking and lung cancer. And now we can say real harm has come to you as a result of this study, right? Now everybody's posterior beliefs about the likelihood that you have cancer go up in light of this study, and your data was part of this study. And in particular, real harms might come to you of the financial variety. Your health insurer might decide to double your premiums, for example, OK?
于是你的数据被纳入了这项分析。这项分析向全世界宣布：吸烟和肺癌之间存在联系。那么现在我们可以说，这项研究确实给你带来了实实在在的伤害，对吧？现在所有人对于“你患癌可能性”的后验信念，都会因为这项研究而上升，而你的数据正是这项研究的一部分。尤其是，你可能会遭受实实在在的经济上的伤害。比如，你的健康保险公司可能决定把你的保费翻倍，对吧？
15:48
Now the key observation-- so in particular, if we adopt this definition, this study would have been disallowed. This would have been a violation of the privacy of everybody whose data was included in this study. The key observation, though, here is that, of course, it's not the case that your particular medical record was the crucial piece of data that allowed the link between smoking and lung cancer to be established, right? Any sufficiently large collection of medical records would have been enough to establish this fact, because the fact that smoking and lung cancer are connected is not like a fact about you particular or your data.
那么关键的一点是——特别地，如果我们采用这个定义，这项研究就会被禁止。它会构成对所有数据被纳入其中的人的隐私侵犯。不过这里的关键观察在于，当然，事实并不是说你那一份特定的医疗记录就是让吸烟与肺癌之间的联系得以确立的那块关键数据，对吧？任何足够大的医疗记录集合都足以确立这个事实，因为“吸烟和肺癌相关”这个事实，并不是关于你个人或你的数据的事实。
16:27
It is what we might call a fact about the world that can be discovered provided we have enough data. So this brings us to what we claim as the right definition of privacy, which is differential privacy, which slightly refines the definition I give to sort of account for this fact that your data wasn't the crucial missing piece in this analysis. And this is a schematic, but in English, what does differential privacy ask? It basically asks you to consider two alternative worlds. One in which an analysis is done, and your data is included in that analysis.
它是我们可以称之为“关于世界的事实”，只要有足够的数据就能被发现。这就把我们带到了我们所主张的正确的隐私定义，也就是差分隐私。它对我给出的那个定义做了微小的修正，以便照顾到“你的数据并不是这项分析中缺一不可的那块拼图”这一事实。这是一张示意图，但用大白话说，差分隐私要求的是什么呢？它基本上要求你考虑两个平行世界。一个世界里做了某项分析，而你的数据被包含在这项分析中。
17:01
And let's say that there are N medical records total in the analysis, and the other one is the same analysis is done but on N minus medical records, where the missing one is yours. So what we want is the harm that comes to you is basically identical in these two situations. So whatever your definition of harm is, whatever it is you're worried about, the chances that that harm comes to you in the case where your medical record is included compared to the one where it's only your medical record that's excluded is sort of controllably close, OK?
比方说分析里总共有N份医疗记录；另一个世界里做同样的分析，但只有N减一份医疗记录，缺的那一份正是你的。我们想要的是：这两种情形下你所受到的伤害基本上是一样的。所以不管你对“伤害”的定义是什么，不管你担心的是什么，在你的医疗记录被纳入的情况下，你遭受那种伤害的概率，与仅仅排除掉你这份医疗记录的情况相比，是可控地接近的，好吧？
17:38
And as many people in this audience know, the definition of differential privacy involved-- it's a property of an algorithm, first of all, not about a data particular data set. An algorithm either is or is not differentially private. And differential privacy is generally achieved by adding noise to computations, so you move from deterministic to randomized algorithms. And you typically add noise in a way that kind of obscures the contribution of any individual piece of data in the analysis while preserving broad statistics.
在座很多人都知道，差分隐私的定义涉及——首先，它是算法的一种性质，而不是关于某个特定数据集的性质。一个算法要么满足差分隐私，要么不满足。而差分隐私通常是通过向计算中添加噪声来实现的，所以你从确定性算法转向了随机化算法。而且你添加噪声的方式，通常是既能掩盖分析中任何单个数据点的贡献，又能保留整体的统计特征。
18:14
So the first time-- Aaron's been working on differential privacy much longer than I have. And I remember the first time I saw the definition of it. I thought, well, that's a great definition, but I'm still worried that it's too strong, right? It's got many universal quantifiers in it. The algorithm has to provide differential privacy on absolutely any input database. The definition of harm can be anything you want it to be, and still the increase in harm as a result of including your data is controlled.
所以第一次——Aaron研究差分隐私的时间比我长得多。我还记得我第一次看到它的定义时，我想，嗯，这定义真不错，但我还是担心它太强了，对吧？它里面有一大堆全称量词。算法必须在任意的输入数据库上都提供差分隐私。“伤害”的定义可以是你想要的任何东西，而纳入你的数据所带来的伤害增量，依然是受控的。
18:40
And so my first reaction was like, you know, maybe we still won't be able to do anything with this definition either. Luckily, that's turned out to be far from the truth. And in particular, pretty much any technique you know from statistics or modern machine learning has a variant. It is not differentially private in its original form. But it has a variant, which gives differential privacy. So for example, back propagation in neural networks or stochastic gradient descent have differentially private variance.
所以我的第一反应是：你知道吗，也许在这个定义下我们同样什么都做不成。幸运的是，事实证明远非如此。特别是，你所知道的统计学或现代机器学习中的几乎任何一种技术，都有一个变体。它在原始形式下不是差分隐私的。但它有一个能提供差分隐私的变体。比如说，神经网络中的反向传播、或者随机梯度下降，都有差分隐私的变体。
19:11
So differential privacy has kind of just in recent years started to make it out of the lab or maybe kind of more precisely off the whiteboard into practice. And the big moonshot for differential privacy is coming up next year when the US census has decided that every single report or statistic it results based on the raw underlying census data will be released under the constraint of differential privacy. And this is a huge engineering effort, and it'll be interesting to see how it turns out. And I'm going to turn it over to Aaron now to talk about fairness of it.
所以差分隐私最近这些年才刚刚开始走出实验室，或者更准确地说，走下白板，进入实践。而差分隐私的一个大型登月计划就在明年——美国人口普查局已经决定，它基于原始普查数据得出的每一份报告或每一项统计量，都将在差分隐私的约束下发布。这是一项浩大的工程，看看结果如何会很有意思。接下来我把话筒交给Aaron，让他讲讲公平性。
19:45
AARON ROTH: Yeah, so we're not there yet on fairness. So we sort of assert that if you think about differential privacy for a while, read chapter 1 in the book, that many of you will agree that, at least for a particular kind of privacy, statistical privacy and data sets, it's somehow the right definition. It's capturing what you want. There's nothing like that in the fairness literature yet. There's dozens of definitions of what we might mean by fairness, and for each one, I could tell you one reason why it's lacking.
AARON ROTH：是的，在公平性上我们还没走到那一步。我们大致可以断言：如果你花点时间琢磨差分隐私，读一读书里的第一章，你们中的很多人都会同意，至少对于某一类特定的隐私，也就是数据集上的统计隐私来说，它就是那个正确的定义。它抓住了你想要的东西。而在公平性的文献里，还没有这样的东西。关于“公平”可能意味着什么，已经有几十种定义，而对每一种，我都能告诉你一个它不足的理由。
20:19
It's not capturing everything you want. In fact, we even know the study of fairness is going to be more complicated than the study of privacy, because there are already known different reasonable definitions of fairness. That in isolation, you would nod your head and agree, yes, that's something I would like that are known to be incompatible with one another. Nevertheless-- so maybe you think about the study of fairness in machine learning as where the study of privacy was 15 years ago. Nevertheless, it's an extremely important problem.
它没有涵盖你想要的全部。事实上，我们甚至已经知道，公平性的研究会比隐私的研究更复杂，因为已经有若干种不同的、都很合理的公平性定义。孤立地看，你会点头同意：是的，这正是我想要的；但它们已被证明彼此是不相容的。尽管如此——也许你可以把机器学习中的公平性研究，看成15年前隐私研究所处的阶段。但无论如何，这都是一个极其重要的问题。
20:55
Here on the slide are two headlines just from the last week, two applications that have attracted New York state regulatory scrutiny. One, a lending application, the Apple credit card that you might have heard about. There's a number of tweets from prominent people alleging that the algorithm that determines what your credit limit will be exhibits gender bias. The other article was about a widely deployed algorithm targeting health care interventions that seems to exhibit racial bias, so I don't want to talk too much about definitions of unfairness.
幻灯片上是就在上周的两条新闻标题，两个引起纽约州监管部门审查的应用。一个是借贷方面的应用，就是你们可能听说过的Apple信用卡。有不少知名人士发推指控，那个决定你信用额度的算法存在性别偏见。另一篇文章讲的是一个被广泛部署的、用于定向医疗干预的算法，它似乎存在种族偏见。所以我不太想过多讨论“不公平”的定义。
21:31
Because I don't think we've yet hit upon exactly the right ones, but I do want to give some idea for why machine learning might be unfair in the first place. Because I think a lot of people's first reaction is that, well, bias of the sort that we talk about when we talk about like racism or sexism, this is some human property. And we're removing it just by removing human beings from the decision making pipeline and using objective optimization procedures. And it's a little more complicated than that.
因为我不认为我们已经找到了完全正确的那些定义，但我确实想说明一下，机器学习一开始为什么可能是不公平的。因为我想很多人的第一反应是：嗯，我们谈论种族主义或性别歧视时所说的那种偏见，那是人类才有的属性。而我们只要把人类从决策流程中拿掉、改用客观的优化程序，就把偏见去掉了。但事情比这要复杂一点。
22:03
Here's a little cartoon to illustrate why. OK, so suppose that Michael and I volunteer to help out with Penn admissions. And we're going to design a machine learning algorithm to help admit students to Penn. OK, so maybe in this cartoon, we've got two observations about each applicant, their SAT score and their GPA. And there's some concrete thing we're trying to predict. OK, so maybe, for example, we're trying to predict whether students if admitted will graduate in at most five years with at least a 3.5 GPA.
这里有个小漫画来说明原因。好，假设Michael和我自告奋勇去帮忙做宾大的招生工作。我们打算设计一个机器学习算法，来帮助录取宾大的学生。好，在这个漫画里，也许我们对每个申请者有两项观测指标：SAT成绩和GPA。并且我们要预测的是某个具体的东西。好，比如说，我们可能要预测：学生如果被录取，是否会在至多五年内以不低于3.5的GPA毕业。
22:33
Maybe we're trying to predict whether within 30 years of graduating, they'll donate at least $10 million. Whatever it is, some concrete thing such that we're trying to admit the people who we've labeled as plus, and we want to reject the people we've labeled as minus. And there's all sorts of problems you might imagine gathering this data. You might imagine that there's potentially the biases of past admissions officers embedded in this data, but let's wish that all away and imagine for this cartoon example that the data really is what it says it is.
或者我们要预测：他们是否会在毕业后30年内捐赠至少1000万美元。不管是什么，总之是某个具体的东西，使得我们想录取被标为“加号”的人，而拒掉被标为“减号”的人。收集这类数据时你可以想象出各种各样的问题。你可以想象，这些数据里可能嵌入了过去招生官的偏见，但我们先把这些都放到一边，在这个漫画例子里假设数据真的就如它所说的那样。
23:03
Because I want to show you that things can be a little bit more complicated, even in the best case when you've got good data. So there's going to be two populations. You're looking at the green population now, and there's a couple of things I want you to notice about them. So first, slightly fewer than half of the green population is qualified for college, by which I mean, there's slightly more minus signs on this slide than there are plus signs. Second, there's a pretty good although not perfect decision rule.
因为我想让你们看到，即便在数据很好的最佳情形下，事情也可能会更复杂一些。那么会有两个群体。你们现在看到的是绿色群体，关于他们我想让你们注意几点。首先，绿色群体中略少于一半的人是够格上大学的，我的意思是，这张幻灯片上的减号比加号稍微多一点。第二，存在一条相当不错、但并不完美的决策规则。
23:30
There's a line I can draw through space. And by and large, although not exclusively, the positive points lie above the line, and the negative points lie below the line. OK, so that was the green population. Here's the orange population, and again, a couple of things I'd like you to notice about them. Maybe the first one you noticed is that the orange population is a minority, by which I mean, literally just that there are fewer orange points, OK? Like in this context, all it means to be a minority is that there's fewer of them.
我可以在空间中画一条线。总体上——虽然不是绝对的——正例点位于这条线之上，而负例点位于这条线之下。好，这是绿色群体。这是橙色群体，同样，我希望你注意关于他们的几点。你可能首先注意到的是，橙色群体是少数群体，我的意思是，字面上就是橙色的点更少，对吧？在这个语境下，所谓少数群体，仅仅意味着他们的数量更少。
23:58
The second thing you might notice is that the points seem to be drawn from a different distribution. In particular, they're shifted downwards on this plot. They seem to systematically have lower SAT scores. That could be for one of any number of reasons. For example, maybe the green points come from a wealthy population. They take SAT tutoring classes. They take the SAT three times and report only the highest score. The orange points take it once cold. That naturally results in a higher distribution on SAT scores for the green population, but it doesn't necessarily make them more qualified for college.
你可能注意到的第二点是，这些点似乎来自一个不同的分布。具体来说，它们在这张图上整体向下偏移。他们的 SAT 分数似乎系统性地偏低。这可能有很多种原因。比如，也许绿色的点来自富裕人群。他们上 SAT 辅导班。他们考三次 SAT，只报最高的那次分数。而橙色的点只裸考一次。这自然会让绿色群体的 SAT 分数分布更高，但这并不一定意味着他们更适合上大学。
24:28
In fact, when you look at the labels, when you look at the actual thing that we're trying to predict, it's the orange population that's better here. And the orange population is better in two ways. First, on average, they're more qualified for college, right? Half of them are positive examples here compared to fewer than half for the green population. And second, it's even easier to tell who is who. There's now a linear decision rule that I can implement that makes no mistakes at all. OK, we've got two populations.
事实上，当你看标签，看我们真正想预测的那个变量时，反而是橙色群体在这里表现更好。而且橙色群体在两个方面更好。第一，平均而言，他们更有资格上大学，对吧？他们中有一半是正例，而绿色群体中正例还不到一半。第二，甚至更容易分辨出谁是谁。现在我可以实现一条完全不出错的线性决策规则。好，我们有两个群体。
24:56
And in this example, the minority population is the better one. When I say they're better, I mean they're more qualified on average, and it's easier to determine who are the qualified ones. And yet, here are the two populations together. And remember, we're only giving the algorithm SAT score and GPA. So you can see the colors of the points but the algorithm cannot. And suppose what we ask for is the standard objective in machine learning. We would like to find the model, in this case, the linear decision rule, that makes as few mistakes as possible.
在这个例子里，少数群体反而是更好的那个。我说他们更好，意思是他们平均而言更合格，而且更容易判断出谁是合格的那些人。然而，把两个群体放在一起就是这样。记住，我们只给算法提供 SAT 分数和 GPA。所以你能看到这些点的颜色，但算法看不到。假设我们提出的是机器学习中的标准目标。我们想找到一个模型——在这里就是线性决策规则——让错误尽可能少。
25:30
What could be more objective than that? Minimize the number of mistakes, and what you get is just the rule that best fits the green population. You can think about why that is, right? If I were to shift that decision boundary downwards, I would make fewer mistakes on the orange population, but I would make more mistakes on the green population. And that wouldn't be worth it from the point of view of minimizing overall error, because there are more green points. And so mistakes on the green population count more for overall error.
还有什么比这更客观呢？最小化错误数量，而你得到的，就是最贴合绿色群体的那条规则。你可以想想这是为什么，对吧？如果我把决策边界往下移，我在橙色群体上的错误会变少，但在绿色群体上的错误会变多。从最小化整体误差的角度看，这不划算，因为绿色的点更多。所以绿色群体上的错误对整体误差的影响更大。
25:59
OK, so we have an example here, where the orange population was better than the green population, but drawn from a slightly different distribution. And when I asked to find the model that minimized overall error, it ended up rejecting every single member of the orange population despite the fact that they were more qualified and despite the fact that they actually had more signal in their features. Note, by the way, that were I allowed to use group membership color in this case in my model, for example, if I were allowed to build a decision tree that said, well, for green points, use the blue line.
好，我们这里有个例子，橙色群体比绿色群体更优秀，但来自一个略有不同的分布。而当我要求找到最小化整体误差的模型时，它最终拒绝了橙色群体中的每一个人，尽管他们其实更合格，尽管他们的特征里实际上包含更多信号。顺便说一句，注意，如果在这个例子中我被允许在模型里使用群体身份，也就是颜色，比如说，如果我可以构建一棵决策树，说：对绿色的点，用蓝线。
26:34
For orange points, use the purple line. Then I could have improved things, and I could have improved things for everybody, right? I would have had a more accurate model. It wouldn't have changed the decisions for the green population, and all of a sudden, I'd be making the right decisions for the orange population. And so two things I want you to learn from this cartoon. The first is that if you just blindly optimize for error, that will tend to fit the majority population typically at the expense of the minority population.
对橙色的点，用紫线。那我本可以改进结果，而且是对所有人都有改进，对吧？我会得到一个更准确的模型。它不会改变对绿色群体的决策，而与此同时，我对橙色群体的决策就会变得正确。所以我希望你从这个示意图中学到两点。第一，如果你只是盲目地优化误差，那通常会去拟合多数群体，而牺牲少数群体。
27:06
Not for any kind of underlying-- not because there's any kind of underlying like racism baked into the objective function, but simply because larger populations contribute more to overall error. And second, although it's a knee jerk reaction to say, OK, like if I don't want like racial or gender bias in my algorithm, I shouldn't use those features. That's not always the case. This is an example where using those features can actually make things better. Not just for fairness, whatever that is-- we haven't defined it-- but for accuracy as well.
这并不是因为有什么潜在的——不是因为目标函数里内建了什么种族主义，而仅仅是因为更大的群体对整体误差的贡献更多。第二，虽然人们的下意识反应是说：好吧，如果我不想让算法有种族或性别偏见，那我就不该使用这些特征。但情况并非总是如此。这就是一个例子，说明使用这些特征实际上可以让结果更好。不只是对公平性更好——不管公平性到底是什么，我们还没定义它——对准确率也更好。
27:44
This is an example of something that intuitively seems unfair. We have this better population, and we've learned a model that nevertheless rejects all of them simply because there's fewer of them. If we want to design algorithms that correct this, we have to pick a definition. We have to specify what we mean by unfair. I don't want to dwell too much on definitions. But for example, in this application, you might decide that the people who are being harmed by the mistakes made by our algorithm are the qualified applicants, the positive examples, who are mistakenly rejected by our algorithm.
这是一个直觉上就显得不公平的例子。我们有这个表现更好的群体，但我们学到的模型却仍然拒绝了他们所有人，仅仅因为他们人数更少。如果我们想设计算法来纠正这一点，我们就必须先选定一个定义。我们必须明确指出，我们所说的“不公平”到底是什么意思。我不想在定义上过多纠缠。但举个例子，在这个应用场景里，你可能会认为，被我们算法犯的错误所伤害的人是那些合格的申请者，也就是正例，他们被我们的算法错误地拒绝了。
28:18
These are the people who-- like it's really too bad that our algorithm rejected them. They would have done well had they come to our college. And maybe the thing that you object to in this model is that the rate at which the algorithm is doing harm in these two populations, in this case, the rate of false rejection, the false negative rate, is drastically different between the two populations. It's 100% on the orange population. It's close to zero on the green population. And so you could imagine asking-- and this has become a popular thing to ask for-- we should find a model that comes close to equalizing these false rejection rates.
这些人就是——算法把他们拒了，实在是太可惜了。如果他们来了我们学校，本来是会表现得很好的。也许你对这个模型不满的地方在于，算法在这两个群体中造成伤害的比率，在这个例子里就是错误拒绝率、假阴性率，在两个群体之间差异极大。在橙色群体上是 100%。在绿色群体上则接近于零。所以你可以设想去要求——而这已经成了一个很流行的诉求——我们应该找到一个模型，让它尽可能拉平这些错误拒绝率。
28:54
Maybe it exactly equalizes them. Or maybe it equalizes them up to 5%, or 10%, or 50%. So you've got some quantitative notion of unfairness that you can ask for. There's a knob that you can turn, trading off this notion of unfairness with other things you care about, like error, and what you find when you start designing algorithms that achieve these goals. And then you've got this knob that you can tune. And by the way, differential privacy also comes with such a knob, and so you can draw similar pictures when you're thinking about privacy.
也许是完全拉平。也可能是拉平到相差 5%、10% 或者 50% 以内。这样你就得到了一个可以提出要求的、可量化的不公平性概念。你有了一个可以旋转的旋钮，用它在这种不公平性概念和你在意的其他东西之间做权衡，比如误差，以及当你开始设计能达成这些目标的算法时会发现的东西。然后你就有了这么一个可以调节的旋钮。顺便说一句，差分隐私也带有这样一个旋钮，所以当你思考隐私问题时，也可以画出类似的图。
29:29
What you find is that although there are inevitably trade offs that you have to grapple with, you can illuminate what those trade offs are. OK, so these are Pareto frontiers. These are on different data sets for a real machine learning task. The optimal rate of unfairness that you can achieve here measured by the difference between false negative rates between populations, that's what's plotted on the y-axis, with the optimal rate of error you can achieve, that's what's plotted on the x-axis. OK, so for a particular class of models, you can achieve an error unfairness trade off represented as any points on this Pareto frontier.
你会发现，虽然不可避免地要面对一些权衡取舍，但你可以把这些权衡清楚地揭示出来。好，这些是帕累托前沿。这些是在不同的数据集上、针对一个真实的机器学习任务得到的结果。这里能达到的最优不公平程度，用两个群体之间假阴性率的差值来衡量，这就是画在 y 轴上的量；而你能达到的最优误差率，就是画在 x 轴上的量。好，所以对于某一类特定的模型，你可以达到的误差与不公平之间的权衡，就是这条帕累托前沿上的任意一点。
30:08
And it is not possible to get a model that simultaneously improves on both of these metrics. And what you can see, if you're lucky, like in the plot on the left, you can sometimes get a dramatic decrease in this unfairness metric, in this case, the difference between false negative rates at the beginning, and only a very small cost to error. That's what happens when this curve looks very steep. Of course, these trade offs become more severe as you start asking for more and more stringent conditions.
而你不可能得到一个在这两个指标上同时改进的模型。你能看到的是，如果运气好，比如左边这张图，有时候你能让这个不公平指标大幅下降，在这个例子里就是一开始假阴性率之间的差值，而付出的误差代价却非常小。当这条曲线看起来非常陡峭时，就是这种情况。当然，当你开始要求越来越严苛的条件时，这些权衡就会变得越来越严峻。
30:41
And so as we describe in the book, the science can only take you so far. It can elucidate what these trade are, but it can't tell you where on this trade curve you want to live as a society and a particular application. And there's not going to be universal answers. We want to prioritize fairness for privacy more. In certain applications, we'll want to prioritize accuracy. Other things more in other applications, but you know, there's no avoiding that we have to make hard decisions. And what the science can do is it can help us make those decisions with our eyes open.
所以正如我们在书中所写的，科学能带你走的路是有限的。它可以阐明这些权衡是什么，但它无法告诉你，作为一个社会、在某个具体应用中，你想要落在这条权衡曲线的哪个位置。而且这也不会有放之四海而皆准的答案。我们希望更优先考虑公平，或者更优先考虑隐私。在某些应用中，我们会想优先考虑准确性。在另一些应用中则优先考虑别的东西，但你知道，我们无法回避的是，必须做出艰难的抉择。而科学能做的，是帮助我们在做这些决定时看得清清楚楚。
31:16
MICHAEL KEARNS: What we've described so far-- plus with the introduction gets us to about the halfway point of the book. And in midway through the book, we kind of take a wide left turn that we think is interesting and well motivated. And I just want to give you a teaser for what that wide left turn is. So in the different scenarios and applications we've talked about so far, it was fair to a first approximation to think about individual people, consumers, as the vehemence of algorithms. So you might be denied admission to a college you wanted to go to unfairly, or you might have your privacy leaked by a data set or a computation.
MICHAEL KEARNS：我们到目前为止讲的这些——再加上引言部分——差不多带我们走到了这本书的中间位置。而在书的中段，我们来了一个大转弯，我们觉得这个转弯很有意思，也很有理由。我只想先给你们透露一下这个大转弯是什么。在我们目前谈到的各种情境和应用中，粗略地说，把个体的人、消费者当作算法的承受者，是说得通的。所以你可能会被你想去的大学不公平地拒之门外，或者你的隐私可能因为某个数据集或某次计算而被泄露。
31:52
And you might not even know it, right? And you might not also know that your data was being used to build these models that are being applied to decisions made about other people. There are other situations in which there is an algorithm or maybe more precisely an app. And there's a large base of users of that app, and it's not so easy to entirely blame the algorithm alone for the antisocial behavior that it exhibits. Because that antisocial behavior is sort of a function of the algorithm, but also of the incentives of the users who are using the app.
而且你甚至可能都不知道，对吧？你也可能不知道，你的数据正被用来构建这些模型，而这些模型又被用于对别人做出的决策。还有另一些情形，其中存在一个算法，或者更准确地说是一个应用程序。这个 App 有着庞大的用户群体，而它表现出的那些反社会行为，并不那么容易完全归咎于算法本身。因为那种反社会行为在某种程度上是算法的函数，但同时也取决于使用这个 App 的用户们的动机。
32:28
And this takes us into the realm of game theory. And so in particular, there are many, many apps these days that we can really think about as the word that's often used is personalization, but we might think about the game theory term as being computing your best response, right? So one concrete example is commuting using apps, like Waze and Google Maps. Where in response to real time traffic, mainly the activity of all the other drivers on the roads, there's this app that computes your best response.
这就把我们带进了博弈论的领域。具体来说，如今有非常非常多的 App，我们其实可以用一个常被使用的词来理解它们，那就是“个性化”，但用博弈论的术语来说，我们可以把它看作是在计算你的最优反应，对吧？一个具体的例子就是用 App 通勤，比如 Waze 和谷歌地图。它会���据实时路况——主要就是路上所有其他司机的活动——由这个 App 来计算你的最优反应。
32:58
It basically says this is the lowest latency or the shortest driving route for you to take from your point A to your point B. And you might think like, oh, well, what could be better than that? I've got this thing that uses real time traffic information right now and tells me which route to drive. But it is driving us all collectively towards a selfish equilibrium of some very, very large, complicated multiplayer game, like literally the Nash equilibrium of that game. And any of you that have had any basic game theory know that just because something is in equilibrium doesn't mean it's a good thing for you or necessarily for any of the players in that game.
它基本上是在说，这是你从 A 点出发所能选择的延迟最低、或者说最短的行车路线。B。你可能会想，哦，那还有什么能比这更好呢？我现在有这么个东西，它利用实时路况信息，告诉我该走哪条路线。但它正把我们所有人共同推向一个非常非常庞大、复杂的多人博弈的自私均衡，字面意义上就是那个博弈的纳什均衡。学过一点博弈论的人都知道，一件事处于均衡状态，并不意味着它对你有好处，也不一定对博弈中的任何参与者有好处。
33:34
And so in particular, in the case of driving apps, there's well-known both toy examples and evidence that this happens in the real world, even though we're individually optimizing all the times with these apps. We might be collectively driving more, because we're in this competitive equilibrium. And in the book, we kind of take this semi-metaphor and apply it to areas that, I think, are less clearly mathematically formulatable as a game as commuting, including things like product recommendation on services like Amazon, or what you see in your Facebook news feed.
具体到导航应用这个例子，既有众所周知的玩具模型，也有现实世界中确实发生这种情况的证据，尽管我们每个人都在用这些应用时时刻刻做着个体最优化。我们整体上可能反而开了更多的车，因为我们身处这种竞争性均衡之中。在书里，我们把这个半隐喻式的想法拿来，应用到一些在我看来不像通勤那样能清晰地用数学建模成博弈的领域，比如亚马逊这类服务上的商品推荐，或者你在 Facebook 信息流里看到的内容。
34:11
And talk about the tensions between individual optimization and self-interest versus the collective equilibrium that we're at, let's say, in the form of filter bubbles or vulnerability to fake news in the case of Facebook, things like that. And then the final chapter of the book before we get to the catch all chapter that discusses everything from sort of interpretability to every AI alarmists favorite dystopia, the Singularity. We talk about specifically sort of the competitive sport that machine learning has become.
我们讨论了个体最优化、自身利益与集体均衡之间的张力，这种均衡表现出来，比如说就是过滤气泡，或者在 Facebook 的情形下是对假新闻的易感性，诸如此类。然后是书的最后一章之前的那一章——最后那章是个大杂烩，从可解释性一直谈到每个 AI警世论者最爱的反乌托邦，也就是奇点。我们专门谈了机器学习是如何变成一项竞技运动的。
34:46
And in particular, we talk about sort of game theoretic ways of thinking about that and the consequences that it has for things like the reproducibility crisis in the sciences. So in a very quick nutshell, I think many people in this room will be familiar with the fact that machine learning, in some sense, has become a competitive sport, where there are these benchmark data sets. There's selection bias in the reporting of results, because journals won't publish negative findings for the most part.
特别地，我们用博弈论的方式来思考这件事，以及它对诸如科学界可重复性危机之类的问题带来的后果。非常简要地说，我想在座很多人都清楚，机器学习在某种意义上已经变成了一项竞技运动，有那些基准数据集。结果的报告存在选择偏差，因为期刊大多不会发表阴性结果。
35:16
There's so many people in the field right now that we really have no idea how many experiments are actually being run, and how to correct for the complexity a number of those experiments to make sure that we're not going down the road that food science has already gone down, where some significant fraction of the published results are not reproducible under our kind of false discoveries. So that's a teaser for the second half of the book. And we wanted to invite Emily back up and chat with us. EMILY DENTON: So I really want to start with one of the major theses in your work is that the solutions to the ethical concerns that are arising from this prevalence of algorithmic decision making systems should themselves be in large part algorithmic.
现在这个领域里的人太多了，我们其实完全不知道到底有多少实验正在被运行，也不知道该如何对这些实验的数量和复杂度做校正，以确保我们不会走上食品科学已经走过的老路，那里有相当一部分已发表的结果无法重复，属于某种假发现。以上就是本书后半部分的一个预告。我们想请 Emily 回到台上来，和我们聊一聊。EMILY DENTON：我特别想从你工作中的一个核心论点开始：算法决策系统的普及所引发的那些伦理问题，其解决方案本身在很大程度上也应该是算法性的。
36:03
And so I was wondering if you could talk a little bit about how you came to this perspective, if you're thinking on this matter as involved at all in the past few years. MICHAEL KEARNS: Yeah, I mean, I think we came to that perspective through our technical research work, right? So we were relatively early adopters of sort of the whole fate view of machine learning and algorithms, like many people in this room. And so we knew even while we were reading reports of our field, violating basic social norms, that we and others were thinking about, well, you could wait for better laws and regulations.
所以我想请你们谈谈，你们是怎么形成这种观点的，以及在过去几年里，你们对这个问题的思考有没有发生变化。MICHAEL KEARNS：好的，我想我们是通过自己的技术研究工作形成那种观点的，对吧？我们算是比较早接受机器学习和算法领域整套 FATE（公平、问责、透明、伦理）视角的人，就像在座的很多人一样。所以，即便我们在读到有关我们这个领域违反基本社会规范的报道时，我们也很清楚，我们和其他人都在想：好吧，你可以等待更好的法律和监管出台，
36:37
Or you could go fix that problem in the code this way, like right now. I definitely think our view has evolved and even the draft of the book evolved. We talked to many people outside of the computer science machine learning community who care about these issues, like regulators, like policymakers, like people who work in social agencies that see firsthand the damages caused by criminal sentencing models that have gender or racial bias in them, for example. And I think the main evolution it had, at least on the book, is to point out that we don't think that algorithms can solve every problem.
也可以现在就直接在代码里、用这种方式去解决那个问题。我确实觉得我们的观点在演变，连这本书的草稿也在演变。我们和计算机科学、机器学习圈子之外很多关心这些问题的人聊过，比如监管者、政策制定者，还有那些在社会服务机构工作、亲眼看到伤害的人——比如说，看到带有性别或种族偏见的刑事量刑模型造成的伤害。我想，至少在这本书上，最主要的转变是我们明确指出：我们并不认为算法能解决所有问题。
37:19
And that there's still a great deal of room and importance for laws, regulations, and more traditional solutions. And that, also, there are some problems that the really hard problems remain are kind of social. If it's the case that your police on the street are racially biased on who they decide to arrest or stop and frisk, that's going to kind of show up in the data. You may not know it, and the only solution for it is to make police less racist, right? And that's like not an algorithmic problem.
法律、监管以及更传统的解决方案，仍然有很大的空间，也仍然非常重要。而且，还有一些问题——那些真正棘手的问题——本质上是社会性的。如果街上的警察在决定逮捕谁、拦下谁搜身时带有种族偏见，那这一点就会体现在数据里。你可能都不知道，而唯一的解决办法是让警察少一些种族歧视，对吧？这就不是一个算法问题了。
37:54
It's not even an easy regulatory or policy problem. AARON ROTH: The only other thing I would say is that-- so of course, like all of these problems are complicated. And their solutions probably can't be derived from just thinking about some very narrowly scoped algorithm without thinking about the sort of broader social and algorithmic ecosystem in which they live. But many of the issues that have come to light when thinking about, for example, algorithmic fairness, like trade offs between different reasonable notions of fairness.
它甚至也不是一个容易解决的监管或政策问题。AARON ROTH：我唯一想补充的是——当然，所有这些问题都很复杂。它们的解决方案，恐怕不可能只靠思考某个范围很窄的算法就推导出来，而不去考虑这些算法所处的更大的社会与算法生态。但很多在思考诸如算法公平性时浮现出来的问题，比如不同合理的公平性定义之间的权衡——
38:38
Yeah, it's not that they're specific to algorithmic decision making. They've only come to light now. Because there's no avoiding, when you're using algorithms, making quantitative measurements, and specifying precisely what you want. But these issues are-- these trade, for example, are fundamental to any life decision making process. They apply also to human decision makers, and so I think many people think of like algorithm as a scary word. Of course, like as computer scientists and as folks at Google, we probably think of it as less scary.
其实并不是算法决策所特有的。它们只是到现在才被摆到台面上。因为当你使用算法时，你没法回避做定量测量，也没法回避精确地说明你到底想要什么。但这些问题——比如这些权衡——对任何决策过程来说都是根本性的。它们同样适用于人类决策者。所以我觉得很多人一听到“算法”这个词就害怕。当然，作为计算机科学家、作为 Google 的同行，我们大概觉得它没那么可怕。
39:13
But it's not that just as simple tweaks to algorithms can't fix complicated problems. Saying, get rid of algorithms, like also isn't a workable solution. It doesn't fix anything. EMILY DENTON: Something else that you talk about in the book is how a lot of these outcomes are the results of professional scientists and engineers very carefully and rigorously applying principled machine learning methodology just to massive complex data sets. And so you do kind of get at this a little bit about the things that are missing in this standard sort of methodology, and so I'm really thinking that this points to how many different aspects of the kind of rigorous scientific practice that we would strive for actually fall outside the kind of standard machine learning sort of framing on educational training.
但问题在于：简单地调整算法固然解决不了复杂的问题，可要说“把算法都取消掉”，同样不是一个可行的方案。那什么也解决不了。EMILY DENTON：你们在书里还谈到一点：很多这样的后果，其实是专业的科学家和工程师非常小心、非常严谨地把有原则的机器学习方法论应用到海量复杂数据集上所导致的。你们也稍微触及了标准方法论中所缺失的东西。我在想，这其实指向了一点：我们所追求的那种严谨科学实践中，有很多方面其实落在了标准机器学习的教育和训练框架之外。
40:07
And so for example, one of the examples that you gave just now on the screen was this algorithmic health care system that was sort of reproducing racial biases in the health sector. And if I recall correctly, one of the problems with that system was this kind of equating of health care with health care costs. And so this is something that's been discussed a little bit in the algorithmic fairness community, this kind of failure to really precisely articulate and justify the kind of operationalization of abstract sort of social constructs into precise variables that are then predicted by the machine learning system.
比如你们刚才在屏幕上给出的一个例子，就是那个算法化的医疗系统，它某种程度上复制了医疗领域的种族偏见。如果我没记错的话，那个系统的问题之一，就是把“医疗需求”等同于“医疗费用”。这在算法公平性社区里也有一些讨论：也就是没能真正精确地阐明并论证——如何把抽象的社会构念操作化为具体的变量，再由机器学习系统去预测这些变量。
40:49
And so I'm wondering if you could just talk a little bit about the sort of new machine learning education and practices that would kind of get at these things that fall slightly outside the kind of traditional machine learning thinking, but are still kind of in this algorithmic frame. MICHAEL KEARNS: Yeah, I mean, I guess in some ways, it's fair to characterize the fairness and privacy chapters of our book at least as kind of a tutorial on what you can do to make things better without leaving the field of machine learning and going and becoming a social worker, OK?
所以我想请你们谈谈：什么样的新式机器学习教育和实践，能够涵盖这些略微超出传统机器学习思维、但仍然属于算法框架之内的东西？MICHAEL KEARNS：嗯，我想在某种意义上，可以把我们书里关于公平性和隐私的那几章看作一份教程，讲的是你在不离开机器学习领域、不去改行当社工的前提下，能做些什么让情况变好，对吧？
41:26
And by the way, in writing this book, we often would talk to people that would basically say to us like, well, if you really want to help, you should like quit your day job and go become a social worker. And I was like, OK, well, I'm not going to do that. But I mean, I think that maybe one of our points, especially to an audience like this, is that there are things that we can do that are just adjacent to what we're doing already. I mean, the hard part will be things like the Pareto curves that Aaron showed.
顺便说一句，在写这本书的过程中，我们经常遇到有人基本上是这么对我们说的：如果你真想帮忙，你就该辞掉现在的工作，去当社工。我当时就想：好吧，我不会那么做的。但我觉得，我们的一个要点——尤其是对在座这样的听众来说——是我们其实有很多事情可以做，而且就紧挨着我们已经在做的事情。真正难的部分，是像 Aaron 刚才展示的那种帕累托曲线。
41:53
There will be hard trade offs between error and fairness or error and privacy. But it's not like a different kind of beast. I mean, if I had to phrase it very dryly, it's like the difference between solving the optimization problem that you're solving now to solving a constrained optimization problem, where your objective is the same. But now there is like fairness or privacy constraints. And so, I think, in many ways for the machine learning community, this is like low hanging fruit. It's low hanging fruit that will result in perhaps difficult decisions with leaders of your business units when you tell them like, oh, this will make our ad placement more fair.
在误差与公平之间、误差与隐私之间，会有艰难的权衡。但这并不是一头完全不同的怪兽。如果要我用很枯燥的方式来表述：这就像是从你现在求解的优化问题，变成求解一个带约束的优化问题，你的目标函数还是同一个，只不过现在多了公平性或隐私方面的约束。所以我觉得，在很多方面，对机器学习社区来说，这是唾手可得的成果。这是唾手可得的成果，但它可能会让你和业务部门的负责人面临一些艰难的决定——当你告诉他们：哦，这会让我们的广告投放更公平，
42:35
But CTR prediction will be this much worse, which translates into this much less profit every year. But at least you sort of put the discussion on scientific grounds, and I think those parts are appropriate to put on scientific grounds. AARON ROTH: Yeah, I think there's two separate things here. So the discussion we had on this slide was in this idealized world, where the data was clean. It was correct. The labels were right, and even there, there's something to do to learn. But that's really the scenario in which you're talking about constrained optimization problems and having to deal with trade offs.
但点击率（CTR）预测会差这么多，换算下来每年利润要少这么多。但至少你把讨论放到了科学的基础上，而我认为这些部分是适合放到科学基础上讨论的。AARON ROTH：是的，我觉得这里有两件不同的事。我们在这张幻灯片上讨论的，是一个理想化的世界：数据是干净的，是正确的，标签也是对的——即便在那种情况下，学习本身仍然有事可做。那才是你真正需要谈带约束的优化问题、需要处理权衡的场景。
43:10
In the UnitedHealth case, the problem for those who aren't aware is this model was supposed to predict, given a patient with some collection of symptoms, health outcomes so that new interventions could be targeted. But they didn't have outcome data. Instead, they trained on health costs. How much did this patient cost the health care system down the line with the thought that patients who are sicker cost more? And it's thought the reason for the bias that the model exhibited that sort of two similarly sick patients, one of whom was Caucasian, one of whom was black.
至于 UnitedHealth 那个案例，对不了解的人解释一下：这个模型本来是要根据一位有某些症状的病人，去预测健康结果，好据此有针对性地安排新的干预措施。但他们没有健康结果的数据。于是他们改用医疗费用来训练模型：这位病人日后给医疗系统带来了多少花费？背后的想法是，病得越重的人花费越多。而这个模型表现出偏见的原因被认为是：对于两位病情相似的病人，一位是白人，一位是黑人，
43:54
The model would tend to suggest more health care intervention for the Caucasian patient. Well, the reason was because black patients who were similarly ill tended to cost less. Not because their health outcomes were better, but because they had less access to health care. So this is a case where you don't necessarily have to deal with a trade off in the sense that you trained your model on the wrong data. If you were able to go out and get the right data, then it might solve this sort of unfairness problem and simultaneously make your algorithm better at predicting the thing you really wanted it to predict.
模型往往会建议给白人病人更多的医疗干预。原因在于，病情相似的黑人病人往往花费更少——不是因为他们的健康结果更好，而是因为他们获得医疗服务的机会更少。所以在这个案例里，你未必需要处理什么权衡问题，因为问题在于你用错误的数据训练了模型。如果你能去拿到正确的数据，那可能既解决了这种不公平问题，同时也让你的算法在预测你真正想预测的东西上变得更准。
44:29
But you know, again, I think this is something that's sort of made salient. Just because people are thinking about, in this case, fairness. But this is like a part of data science education that would have been important, even if we didn't care about fairness. In the sense that you could have made the model, even if you just cared about like overall accuracy. You could have made the model more accurate by training it on the correct data, and you know, it's only because someone wrote an article in science about the unfairness of the model that it was brought to light.
不过话说回来，我觉得这件事之所以被凸显出来，只是因为在这个案例里，人们正好在关注公平性。但这其实属于数据科学教育中很重要的一部分，即使我们根本不在乎公平性也一样重要。也就是说，哪怕你只关心整体准确率，你也能通过用正确的数据来训练，让模型变得更准。而你知道，正是因为有人在《科学》杂志上写了一篇关于这个模型不公平的文章，这件事才被曝光出来。
44:59
EMILY DENTON: Yeah, yeah, that's kind of what I'm getting at is that there are sort of really rigorous scientific practices in related fields for kind of turning these abstract constructs into measurable variables. And this kind of interdisciplinary kind of work, I think, is not really being adopted as much as it should be, and I think could really-- I don't necessarily view it as to entirely separate from the algorithm design. And I think that really should be integrated, and so yeah, I'm just glad to hear that you think it's also important.
EMILY DENTON：对对，这正是我想说的——在相关领域里，其实有一些非常严谨的科学实践，专门用来把这些抽象的构念转化为可测量的变量。而这种跨学科的工作，我觉得并没有被采纳到应有的程度，我认为它其实可以——我并不认为它和算法设计是完全分开的两件事。我觉得这两者真的应该整合起来。所以，很高兴听到你们也认为这很重要。
45:30
OK, one more high level question. So you've detailed a lot of different sort of troubling practices prevalent within the machine learning community, and how these sorts of practices, like biases in reporting, kind of reliance on a small number of data sets, these types of things. These lead both to the reproducibility crisis, but also to a lot of really ethically questionable design and development of algorithms. And so I'm curious what your thoughts are on how the community as a whole can start to shift its practices, what types of new incentive structures you'd like to see in place.
好，再问一个宏观一点的问题。你们详细描述了机器学习社区里流行的很多令人担忧的做法，以及这些做法——比如报告结果时的偏倚、过度依赖少数几个数据集之类的——既导致了可重复性危机，也导致了大量在伦理上非常可疑的算法设计与开发。所以我很好奇，你们认为整个社区该如何开始转变它的做法？你们希望看到什么样的新激励机制？
46:00
Obviously, this is not a quick fix. This is a very long term thing, but I think a lot of us are members of this academic community. And so I'd love to hear your thoughts on how we as a group can kind of shift in a more socially, and beneficial, and ethically informed direction. AARON ROTH: Yeah, I mean, I guess my quick answer would be we do suggest some technical things in the book, and we talk about the pre-registration movement and things like that, which I think we view as too restrictive of a solution.
显然，这不是一朝一夕能解决的。这是一件非常长期的事，但我想我们很多人都是这个学术共同体的成员。所以我很想听听你们的看法：我们作为一个群体，怎样才能朝着更有社会意识、更有益、更有伦理自觉的方向转变。AARON ROTH：好的，我的简短回答是，我们在书里确实提出了一些技术性的建议，也谈到了预注册（pre-registration）运动之类的做法，不过我们认为那种方案限制性太强了。
46:29
But you know, maybe to answer that question to make a broader social comment. I think it would be good for the field of machine learning to become less like a competitive sport, again, and this a relatively recent phenomenon. It's, I think, a byproduct of the tremendous empirical successes that areas like deep learning had and the need for these massive data sets and kind of concentrated focus by a large number of people undiminished in an intense period of time. And that's all been great. And I don't-- no knock whatsoever on the actual advances in those technologies, which are large, I think, in vision, speech, and NLP.
但也许，为了回答这个问题，我可以做一个更宽泛的社会性评论。我觉得，如果机器学习这个领域能少一点“竞技体育”的味道，那会是件好事——再说一次，这其实是相当晚近才出现的现象。我想，这是深度学习等方向取得巨大经验性成功的副产品：那种成功需要海量数据集，也需要大量的人在一段紧张的时间里高度集中地投入。这一切都很棒。我完全没有要贬低那些技术上的实际进展的意思——那些进展非常大，尤其是在视觉、语音和自然语言处理方面。
47:11
And I'll use my seniority here to point out that the field of machine learning, you know, it used to be that people were considering many, many different types of learning frameworks of different learning models. There wasn't this sort of uniformity of data sets to the extent that there is now. Or at least if there were, they were really kind of toy data sets that nobody considered like a serious benchmark for developing and deploying services. It's things like the UC Irvine data set, which you went to check your results on.
我想倚老卖老地指出一点：机器学习这个领域，以前人们会去考虑许许多多不同类型的学习框架、不同的学习模型。当时并不像现在这样，数据集如此高度统一。或者至少可以说，即便有，那也是些玩具级的数据集，没人会把它当成开发和部署服务的严肃基准。服务。比如 UC Irvine 数据集，你就是拿它来验证一下自己的结果。
47:40
But it wasn't like, OK, on the UC Irvine data set, I've now developed this service that I'm now going to unleash on a billion users. And I think the field, I'm hoping, will organically balance itself back more to an earlier era, where there's not this single minded focus on sort of one framework for learning and a few data sets. And so maybe things like pre-registration or sort of smarter leader boards, which we do discuss in the book, maybe that's a part of the solution. But maybe part of it is just kind of a cyclical move back towards kind of a more diverse research landscape in the field.
但绝不是说，好，我在 UC Irvine 数据集上开发出了这个服务，现在要把它推给十亿用户。我觉得这个领域，我希望，会自发地回摆到更早期的状态，那时候并没有这种只盯着某一套学习框架和少数几个数据集的单一思路。所以也许像预注册，或者更聪明的排行榜机制——我们在书里也讨论过——也许那会是解决方案的一部分。但也许另一部分只是一种周期性的回摆，回到这个领域中更多元的研究格局。
48:16
EMILY DENTON: Cool. Do you have a question? AUDIENCE: Jack Dorsey from Twitter made the announcement that they're not going to do political ads at all. And machine learning algorithms are being used to target users with those ads. So algorithmic accountability and fairness is-- we get questionnaire. And you're chiming in with Aaron that there is some bit of-- not all algorithms are bad. We are trying to make them better. So a player leaving the field kind of creates this added pressure on the other players in the field to be really accurate about it.
EMILY DENTON：很好。你有问题吗？观众：Twitter 的 Jack Dorsey 宣布他们将完全不做政治广告。而机器学习算法正被用来向用户精准投放这些广告。所以算法问责和公平性——我们收到了不少这方面的提问。而你和 Aaron 都表示，并不是所有算法都是坏的。我们是在努力把它们变得更好。那么一个玩家退出这个领域，某种程度上会给场上其他玩家施加额外的压力，逼他们在这件事上做得更严谨。
48:57
MICHAEL KEARNS: Briefly, I don't have deep thoughts on this particular issue. I think the policy to pull those ads entirely is better than having no policy whatsoever. On the other hand, I'm not sort of convinced that pulling things that are "designated" political ads eradicates kind of the penumbra of worries that people have around the politicization of social media. Like, I don't think it directly addresses things like fake news and the like. But I think it's better to have a clear policy than to have no policy at all.
MICHAEL KEARNS：简单说，我对这个具体问题没有太深入的思考。我认为完全撤下这些广告的政策，要好过完全没有任何政策。另一方面，我并不觉得，撤掉那些被“认定”为政治广告的内容，就能消除人们围绕社交媒体政治化所产生的那一整片模糊的担忧。比如说，我不认为这直接解决了假新闻之类的问题。但我认为，有一个明确的政策总比完全没有政策要好。
49:36
And secondly, I do think it's good for the competitive landscape of the tech industry to have actors that take stands on issues and try to create internal pressure in the industry to sort of think harder about these issues and adopt them. To give an example, sort of, Apple has successfully-- you can debate how deserved it is. But Apple has carved out a reputation for greater concern about consumer privacy and was an early adopter of differential privacy. And I think that that does create kind of an environment where there is more internal pressure from the industry rather just then from regulators.
其次，我确实认为，科技行业里有一些参与者愿意在某些议题上表明立场，对整个竞争格局是有好处的，并且努力在行业内部形成压力，促使大家更认真地思考这些问题并采取行动。举个例子，比如说，苹果就很成功地——当然你可以争论它是否名副其实。但苹果确实树立了更重视消费者隐私的声誉，而且是差分隐私的早期采用者。我认为这确实营造出一种环境，让压力更多来自行业内部，而不只是来自监管机构。
50:17
AUDIENCE: Thank you. AUDIENCE: I was wondering how much do you think that your book is sort of a snapshot of the current moment in time? So certainly it wouldn't have made sense to have this book published like 10 years ago. And how much do you think it is really something that's an enduring set of problems? And because of the list of problems, we understand less and less. It's really creating an outline and a framework that's going to have a significant impact over time. AARON ROTH: So as we say at the beginning of the book, like this is an emerging science.
观众：谢谢。观众：我想问，你们觉得这本书在多大程度上算是对当下这个时间点的一个快照？毕竟，这本书如果放在十年前出版肯定是说不通的。那你们又觉得它在多大程度上真正触及了一组会长期存在的问题？而正因为这份问题清单，我们理解得越来越少。它其实是在勾勒一个轮廓和框架，会在很长时间里产生重大影响。AARON ROTH：正如我们在书的开头所说，这是一门正在兴起的科学。
50:44
And you might reasonably think that that means it's sort of too early to write such a book. But we think that it's sort of exactly the right time because it's when the ideas are developing that somehow the intellectual process of thinking about them is most exciting. So I certainly think that, especially as you go down that list-- maybe even already, you know, fairness, which was the second thing in the list-- that if you look at what the technical approaches are going to look like 15 years down the line, they might be quite different from what they look like today.
你可能会合理地认为，这意味着现在写这样一本书还为时过早。但我们觉得现在恰恰是最合适的时机，因为正是在这些想法成型的阶段，思考它们的那种智识过程才是最令人兴奋的。所以我当然认为，尤其是当你顺着那份清单往下看——也许现在就已经是这样了，比如公平性，也就是清单上的第二项——如果你去看十五年后的技术路径会是什么样，它们可能和今天的样子相当不同。
51:17
But I think that the basic premise of what needs to be done-- and that's been sort of successfully carried out from whiteboard, to product, to national scale deployments for privacy-- is enduring. Which is that what you need to do is you need to think very hard, in a rigorous, precise way, about what you mean when you say you want algorithms to be blah, where blah can represent any word you want, where a human being would just know what you meant if you told them you wanted accountability, fairness, transparency.
但我认为，关于需要做什么的基本前提——这一点在隐私领域已经被成功地走通了，从白板，到产品，再到全国规模的部署——是会长期成立的。那就是，你需要做的是非常认真地、以严谨而精确的方式去思考，当你说你希望算法具备某某属性时，你到底指的是什么，这里的“某某”可以换成任何词。如果你告诉一个人你想要问责、公平、透明，他自然就明白你的意思。
51:56
But that is not obvious to an algorithm. And then after you come up with a plausible definition-- and coming up with the definition is the hard part. But when you come up with a plausible definition, you have to think about how to-- you have to think about the scientific problem of how to design models satisfying that definition and think quantitatively about trade-offs because typically these things don't come for free. I think that general methodology is going to have to be enduring. And that even if the specifics of how people are thinking about these things 15 years down the line are going to be different, they will be thinking about these things.
但算法并不会自动明白。然后，在你提出一个说得通的定义之后——而提出定义正是最难的部分。但当你提出一个说得通的定义之后，你就得思考如何——你得去思考那个科学问题：如何设计出满足这个定义的模型，并且定量地思考各种取舍，因为通常这些东西都不是免费的。我认为这套总体方法论是会长期延续下去的。而且即便十五年后人们思考这些问题的具体方式会有所不同，他们仍然会在思考这些问题。
52:30
EMILY DENTON: I'm going to throw in the story question really quickly. What skills are there, other than computer science, that are most needed for work on ethical algorithms? What advice do you have for successful interdisciplinary collaborations? MICHAEL KEARNS: Let's see. I mean, if by skill we mean sort of an academic or specific technical skill, I think it's more-- I think what I would advise most is sort of a willingness, an actual interest, in talking to people in adjacent fields that think about the same issues, but from a non-technical perspective.
EMILY DENTON：我插一个快速的问题。除了计算机科学之外，从事伦理算法的工作还最需要哪些能力？对于成功的跨学科合作，你们有什么建议？MICHAEL KEARNS：我想想。我是说，如果这里的“能力”指的是某种学术或具体的技术技能，我觉得更多是——我最想建议的是一种意愿，一种真实的兴趣，去和相邻领域中同样思考这些问题的人交流，只不过他们是从非技术的视角出发的。
53:04
So I think we've benefited greatly, for instance, in conversations we've had with people at the law school at Penn, who think hard about fairness and privacy, including in technological settings from a legal perspective, and just kind of understanding their views. And also, especially understanding the constraints that come from their world and in talking to regulators. It's quite revealing to talk to tech regulators and realize the handicaps that they face. These are smart people. But these are smart people, kind of with many, many shackles on what they can and can't do, that really kind of force them to lag in many ways the companies that they're regulating.
比如说，我觉得我们从和宾大法学院的人交流中受益良多，他们对公平和隐私有很深入的思考，包括从法律视角看技术场景中的这些问题，而理解他们的观点本身就很有价值。另外，尤其是要理解他们那个世界带来的种种约束，还有就是和监管者交流。和科技监管者聊天，会让你很直观地意识到他们面临的种种掣肘。这些都是聪明人。但这些聪明人身上戴着许许多多镣铐，限制着他们能做什么、不能做什么，这真的在很多方面迫使他们落后于他们所监管的那些公司。
53:45
And so I think it's important to, in working in this area, even if it doesn't like, oh, you talk to some regulator. And then you got a research idea, that you then go work on, to just kind of understand that landscape more than any other particular field outside of CSI or machine learning. AUDIENCE: One thing that I've seemed to notice is that when people notice that, say, an algorithm is maybe not fair, then what algorithm designers and even society at large tend to do is to think of quick solutions on how to fix it, when the solution itself may not inherently be fair.
所以我觉得，在这个领域工作时，这一点很重要——即便不是说，哦，你跟某个监管者聊了天，然后就得到一个研究点子，接着去做研究，而只是去更好地理解这片图景，这比计算机科学或机器学习之外的任何其他具体领域都更重要。观众：我注意到的一件事是，当人们发现，比如说某个算法可能不公平时，算法设计者、甚至整个社会往往会去想一些快速的修补办法，而这些方案本身可能在本质上并不公平。
54:27
To take an example, for instance, the college selection slide that you showed, where you had two populations. And clearly, the initial solution of having of general cutoff was a problem. But like you just suggested, one thing that could have been done is consider the two populations separately, which seems like an OK thing to do. People might even agree with it. But then, when you look at the data, you do see that even the green population, there may be some data points which are positives. But they fall within the cutoff range of maybe population two.
举个例子，比如你展示的那张大学录取的幻灯片，上面有两个群体。很明显，最初那种设定统一分数线的做法是有问题的。但正如你刚才所建议的，一种可行的做法是把两个群体分开考虑，这看起来似乎是个不错的办法。人们甚至可能会认同它。但接着，当你去看数据时，你确实会发现，即便是绿色的那个群体里，也可能有一些数据点其实是正样本。但它们落在了第二个群体的分数线区间之内。
55:04
And I do think that this happens a lot in real life too, that the easy solution is to maybe just say that one very easily observable variable, this population type. But maybe the actual hidden variable that you need to consider is maybe, like you mentioned, maybe the income. Maybe you couldn't test that three times, instead of one. So today, what I see is maybe people from the green population, perhaps as a result. If there is somebody who falls within cutoff, then they're completely screwed as well here.
我确实觉得这种情况在现实生活中也经常发生：简单的解法往往就是抓住某一个非常容易观测的变量，也就是群体类别。但也许你真正需要考虑的那个隐藏变量，正如你提到的，可能是收入。也许你可以让人考三次，而不是只考一次。所以今天我看到的情况是，绿色群体里的人，可能就成了这种做法的结果。如果有人落在了分数线之内，那他在这里也一样彻底没戏了。
55:36
So I'm wondering as one of the designers, what are the things that--
所以我想问，作为设计者之一，有哪些事情是——
55:44
how can this problem be solved? Because it seems like we are trying to optimize for maximum efficiency. You have the curve. But then as a result, there might be some fraction of people that always get left behind. Although overall, it might be the most optimal solution. AARON ROTH: Yeah. See, I think you've put your finger on one of the main weaknesses of these statistical notions of fairness. And we talked about this a little bit in the book. And it's actually one of the main focuses of our research.
这个问题该怎么解决？因为看起来我们是在追求最大效率的优化。你有那条曲线。但结果可能就是，总有一部分人会被落下。尽管整体来看，这也许是最优解。AARON ROTH：是的。你看，我觉得你正好点到了这些统计学公平性概念的主要弱点之一。我们在书里也稍微谈到了这一点。而且这实际上是我们研究的重点之一。
56:13
And by the way, this is why we think of maybe the fairness in the machine learning field as an academic field as 15 years behind privacy. So the claim is not that any of the existing definitions are very good. So I think what you're putting your finger on is, when you look at these statistical notions of fairness that say things like, well, I'd like the false rejection rate to be similar between-- like orange people and green people, say. Well, you have to-- like the first step of even enunciating that, was you had to say, OK, well, there are these two groups I care about, orange people and green people.
顺便说一句，这就是为什么我们觉得机器学习中的公平性作为一个学术领域，比隐私领域落后了 15 年。所以我们并不是说现有的这些定义有多好。我想你点出的问题是，当你看这些统计学的公平性概念时，它们会说这样的话：我希望误拒率在——比如说橙色人群和绿色人群之间——是相近的。比如说橙色人群和绿色人群。但问题是，光是要把这句话说出来，你就得先说：好吧，我关心的是这两个群体，橙色人群和绿色人群。
56:47
And usually it's not so easy. And just because I guarantee some notion of statistical equality in aggregate over two large groups doesn't mean that the solution that we come up with is fair in various technical senses to you as an individual, or even to large groups of people that you think of yourself as a member of, if they weren't the exact groups that we specified upfront. So let me just, without saying too much about it-- this is an active area of research. There are things you can do. There are fairness notions that are somewhat more satisfying than these.
而通常这并没有那么容易。而且，仅仅因为我在两个大群体上保证了某种总体层面的统计平等，并不意味着我们得出的方案在各种技术意义上对你个人是公平的，甚至对你自认为所属的那些大群体也未必公平，如果它们不是我们一开始所指定的那些群体的话。所以，我不多说太细——这是一个活跃的研究领域。是有一些办法可以做的。有一些公平性概念比这些要更令人满意一些。
57:30
They don't require enunciating a small number of prespecified, coarsely defined groups upfront. There are ways to talk about fairness at an individual level. And maybe we can talk a little bit offline. But this is sort of the research frontier, like we don't understand that much about methods that guarantee protections of this sort and their implications. So it's a very good question. And I'd say, like there's people thinking about it. You should go off and think about it. It's not a settled science yet.
它们不要求你一开始就指定少数几个预先设定、粗略划分的群体。也有办法在个体层面上谈论公平性。也许我们可以私下再聊一聊。但这算是研究前沿了，我们对能够保证这类保护的方法及其影响，了解得还不多。所以这是个非常好的问题。我想说，是有人在思考这件事的。你也应该去思考一下。这还不是一门已成定论的科学。
58:01
AUDIENCE: OK. EMILY DENTON: Thank you so much. [APPLAUSE]
观众：好的。EMILY DENTON：非常感谢。［掌声］
这期还没有生成总结。
这期还没有生成核心句型（制作精读 PDF 时会一并生成）。
这期还没有生成词汇表（制作精读 PDF 时会一并生成）。
这期还没有生成自测题（制作精读 PDF 时会一并生成）。
下载便签
手机：长按图片也可保存
← 上一期 · NO.119Fairness, part 1 - Moritz Hardt - MLSS 2020, Tübingen
下一期 · NO.121 →The Neurobiology of Beauty and its Implications - Prof. Semir Zeki
订阅苏菲周报
每周一封：本周入库的精读、一个值得带走的问题、一条苏菲按。免费，随时退订。
免费 · 每周一封 · 一键退订
苏菲拉底
THE SOPHIE LAB · ASK THE BEST MINDS THE BIG QUESTIONS
内容仅供学习 · thesophielab.com