算法也有性别歧视吗？奥德·伯恩海姆（Aude Bernheim）和弗洛拉·文森特（Flora Vincent）在他们合著的《人工智能，怎能缺少女性！》一书中曾指出，由人类编写出来用在人工智能上的算法没能摆脱成见的烙印。算法或许可以引发性别歧视或种族歧视，但也可以用来推动性别平等事业。
随着现代各项信息技术的发展，人们将在越来越多的领域应用现代技术进行决策，而在技术上如何保障性别上的平等，消除各种偏见，将是现代人类社会需要直面的问题之一。今天分享联合国教科文《信使》杂志的一篇文章，希望对你能有所启示。
本文转自：联合国科教文信使
原标题为：《我们必须改善算法》
阿涅丝·巴尔东（联合国教科文组织）担任采访
Q：您们是如何开始注意到人工智能（AI）中的性别问题的？
How did you become interested in the gender issue in artificial intelligence (AI)?
奥德·伯恩海姆：最初，我们关注的是性别平等与科学之间的关系。2013年，我们成立了科学体验协会(link is external)（WAX(link is external)），研究科研团队中缺乏性别多样性可能会对科技产品造成哪些潜在影响。我们在人工智能方面开展的工作正是源自这种想法。
Aude Bernheim: Originally, our thinking focused on the links between gender equality and science. In 2013, we founded the association WAX Science,(link is external) or WAX(link is external) (What About Xperiencing Science), to examine how the lack of gender diversity in scientific research teams could potentially affect the products of science and technology. Our work on AI stems from this reflection.
对于技术中的性别偏见，我们其实并不感到特别惊讶。许多其他领域都存在性别偏见，人工智能没理由成为例外。但性别偏见产生的后果是多方面的，不单局限于职业平等和薪资等常见问题。算法中所包含的成见对于求职申请的筛选方式（将女性排除在技术职位之外）、薪资建议，甚至医疗诊断都会产生负面影响。
Actually, we weren’t really surprised to find gender biases in these technologies because they exist in many other fields. There was no reason for AI to escape them. But the consequences are numerous, and go beyond the usual issues of professional equality or salaries. The stereotypes contained in the algorithms can have a negative impact on the way job applications are screened – by excluding women from technical positions – salary proposals, and even medical diagnoses.
弗洛拉·文森特：科研团队缺乏多样性，这一现象众所周知。但人们不太了解的是，这种情况已经影响到了如何开展研究和确定优先课题。美国科学史学家伦达·席宾格（Londa Schiebinger）近年来一直在研究这个问题。她指出，科研团队中女性成员越多，就越有可能在研究过程中考虑性别问题。
Flora Vincent: Scientific teams lack diversity – the phenomenon is well-known. What is not so well-known is that this has consequences on how research is developed and what subjects are given priority. An American science historian, Londa Schiebinger, has been working on this topic recently. She shows that the more women there are on a team, the more likely it is for the gender issue to be taken into account in the study itself.
科研中的这种歧视现象有很多实例。例如，药物实验中多采用雄鼠，这是因为雄鼠体内的荷尔蒙较少，测量药物副作用更容易。再如，碰撞测试使用的标准人体模型仿照的是男性的平均身材，身高1.70米，体重70千克，这就导致安全带的设计没有考虑到某些特殊情况，例如孕妇。
There are many examples of this discrimination in research. One example is that drugs are tested more on male rats because they have fewer hormones, and therefore it’s considered easier to measure side effects. Another example: for crash tests, standard 1.70-metre and seventy-kilogram dummies, modelled on the average size and build of a man, are used. As a result, the seatbelt does not take into account certain situations, such as pregnant women, for example.
Q：计算机科学是不是从一开始就是由男性主导的学科？
Has computer science been a predominantly male-dominated discipline from the outset?
伯恩海姆：以前并不是这样的。20世纪初，计算机科学需要进行大量繁琐而枯燥的计算，这部分工作当时通常是由女性完成。第一批计算机问世时，女性是这门学科的领导者。这在当时算不上是光鲜诱人的工作。一直到1984年，美国计算机行业的雇员中有37%是女性。但到了2018年，在法国学习计算机科学课程的女性只占10%，人工智能领域的女生估计只有12%。
Bernheim: No, that was not always the case. In the early twentieth century, computer science was a discipline that required a lot of rather tedious calculations. At the time, these were often done by women. When the first computers came along, women were in the lead. The work was not seen as prestigious at the time. As recently as 1984, thirty-seven per cent of those employed in the computer industry in the United States were women. By comparison, in France in 2018, only ten per cent of students in computer science courses were women; it is estimated that only twelve per cent of students in the AI sector are women.
待至20世纪80年代，随着个人计算机（PC）的出现，情况出现了重大变化。从那时起，计算机技术展现出了前所有未的经济重要性，计算机的娱乐性能也同时应运而生，围绕“极客”这一人物形象衍生出了典型的男性化文化意象。这种双重趋势的另一面是女性被边缘化。由此可见，男性对于计算机的喜爱并非与生俱来，这首先是一种人为构筑而成的文化现象。
In fact, a significant change took place in the 1980s, with the emergence of the personal computer (PC). From then on, computer technology acquired unprecedented economic importance. The recreational dimension of computers also emerged in those years, developing a very masculine cultural imagery around the figure of the geek. This dual trend was accompanied by the marginalization of women. This shows that boys' affinity for computers is not natural, but that it is, above all, cultural and constructed.
人们可能会认为算法在本质上是中立的，算法会在多大程度上复制性别偏见呢？
One might think that algorithms are neutral by nature. To what extent do they contribute to reproducing gender bias?
伯恩海姆：一些人很快就意识到并透漏出算法中存在偏见。例如，他们发现翻译软件（将其他语言译成法语，法语名词有阴阳性之分）往往会给职业加上性别色彩，比如将英文中的“doctor”译成“le docteur”（阳性），将“nurse”译成“l'infirmière ”（阴性）。语音助手（无论是Alexa、Siri，还是Cortana）出现之后，无一例外地都被冠以女性化的名字，而且即使是受到羞辱，也会以相当顺从的方式回应命令。
Bernheim: Some whistleblowers realized quite quickly that algorithms were biased. They found, for example, that translation software [into French, which has masculine and feminine nouns] tended to give professions a gender by translating the English “the doctor” into “le docteur” (masculine), and “the nurse” into “l'infirmière” (feminine). When voice assistants appeared – whether Alexa, Siri, or Cortana – they all had feminine names and responded to orders in a rather submissive manner, even when they were insulted.
2016年，麻省理工学院（MIT）非裔美国研究员乔伊·布兰维尼（Joy Buolamwini）对面部识别算法产生了兴趣。她在工作中发现，人们利用数据库来培训人工智能，而数据库中的大多数照片都是白人男性的照片。由此导致人工智能在识别黑人妇女或亚洲男子时的表现，要比在识别白人男性时差得多。可以设想，假如乔伊是算法研发团队的一员，情况可能会有所不同。
In 2016, Joy Buolamwini, an African-American researcher at the Massachusetts Institute of Technology (MIT), became interested in facial recognition algorithms. Her work showed that they [the AI] were trained on databases which contained mostly photos of white males. As a result, they were much less effective on [recognizing] black women or Asian men, than on white men. You can imagine that if she had been part of the team developing these algorithms, the situation would have been different.
文森特：编写算法就像是写文章。即便我们自认为是在客观如实地书写事实，但遣词造句的方法本身就带有一定程度的主观性。为找出偏见，我们的方法是逐一分解“性别歧视传播”的不同阶段。这是因为带有偏见的算法并非是单由某一个原因造成的，而是在算法开发的不同阶段介入的一系列因果关系共同作用的后果。实际上，假如编程、测试、控制和使用算法的人对于这些潜在的偏见全不知情，他们就会不断复制这些偏见。在大多数情况下，刻意的歧视并不存在。我们通常只不过是在重复生活和教育过程中无意形成的成见。
Vincent: Coding an algorithm is like writing a text. There’s a certain amount of subjectivity that manifests itself in the choice of words, the turns of phrases – even if we have the impression that we are writing a very factual text. To identify the biases, our approach consisted of dissecting the different stages of what we call “sexist contagion”. That’s because there isn’t a single cause that creates a biased algorithm, but rather, it’s the result of a chain of causality that intervenes at the different stages of its construction. In effect, if the people who code, test, control and use an algorithm are not aware of these potential biases, they reproduce them. In the vast majority of cases, there’s no wilful intention to discriminate. More often than not, we simply reproduce unconscious stereotypes forged in the course of our lives and education.
Q：人们是否意识到当今的某些人工智能产品带有偏见？
Is there an awareness of the bias in certain AI products today?
伯恩海姆：在人工智能领域，一切都在高速发展变化当中，这包括技术本身和人们对于使用技术的思考。与其他学科相比，歧视问题很早就显现出来了。在算法大行其道仅仅三年之后，就有敏锐者开始提醒人们注意某些算法中存在的区别对待。这在科学界已然成为一个课题，引发了一系列辩论，并且已经有人开始从伦理、数学和计算机科学的角度研究如何检测出算法中的偏见及其影响。近年来，主流媒体也体现出这种认识。并不是所有问题都得到了解决，但的确发现了问题。人们一旦确定了问题所在，就可以着手加以解决。
Bernheim: AI is a field where everything is evolving very quickly – the technology itself, but also the thinking about its use. Compared to other disciplines, the problem of discrimination emerged very early on. Barely three years after the onset of algorithm fever, whistleblowers started drawing attention to the differentiated treatment of certain algorithms. This is already a subject in its own right in the scientific community. It fuels many debates and has led to research work on the detection of bias and the implications of algorithms from an ethical, mathematical and computer science point of view. This awareness has also recently been reflected in the mainstream media. Not all the problems have been solved, but they have been identified and once they have been, solutions can be implemented.
Q：如何在算法中植入更多的平等思想？
How can algorithms be made more egalitarian?
伯恩海姆：我们必须先从数据库入手，让其中的数据能够充分体现出人口的多样性。一些公司已经行动起来了，正在研究如何让数据库能够顾及到性别、国籍和词法方面的差异。关于面部识别软件存在缺陷的研究成果发布之后，一些公司重新调整了算法，使之更具包容性。此外还出现了一些公司，专门开发算法评估工具和判断算法中是否存在偏见。
Bernheim: To begin with, we must act at the level of databases, so that they are representative of the population in all its diversity. Some companies are already doing this and are working on databases that take into account differences in gender, nationality or morphology. As a result of work published on the shortcomings of facial recognition software, some companies have retrained their algorithms to be more inclusive. Companies have also emerged that specialize in developing tools to evaluate algorithms and determine whether they are biased.
文森特：另一方面，科研界已经开始思考如何开展更加独立的评估，以及有无必要提高算法透明度。布兰维尼等一批专家提倡开发和推广包容性代码，其性质类似于包容性写作。
Vincent: At the same time, in the scientific and research community, there has been reflection on how to implement a more independent evaluation, and on the need for algorithmic transparency. Some experts, such as Buolamwini, advocate the development and generalization of an inclusive code, just as there is for inclusive writing.
在当前的各项举措中，不应忽略“善用数据”组织所做的工作。该组织正在研究如何让算法为大众利益服务，并为此起草了一份题为“数据科学家希波克拉底誓言”的伦理规范，提出了在实施一项算法之前需要逐项核查的一系列具体标准，以确保算法不带有歧视性。支持这类倡议是很重要的。
Among existing initiatives, we should also mention the work done by the collective Data for Good, which is thinking about ways to make algorithms serve the general interest. This collective has drafted an ethical charter called the Hippocratic Oath for Data Scientists, establishing a list of very concrete parameters to be checked before implementing an algorithm, to ensure it isn’t discriminatory. It is important to support this type of initiative.
Q：人工智能最终能够成为消除偏见的典范吗？
Could AI eventually become an example for how biases can be combated?
伯恩海姆：从某种意义上说是可以做到的，只要我们能够迅速意识到这些新技术可能导致偏见。人工智能正在彻底改变我们的社会，可以推动事物向积极的方向发展。有了人工智能的协助，人们可以管理和分析海量数据。2016年，谷歌公司利用人工智能开发出了一种算法，对美国主要电影作品中的女性说话时间进行量化计算，结果发现女性的代表性不足。另一方面，需要改善算法开发团队的性别均衡状况。不过，目前少有女性学习计算机科学，其中涉及很多原因，包括女孩在面对科学领域时的自我否定，以及高科技公司中盛行的性别歧视风气。要扭转这种趋势，还需要时间。
Bernheim: In a sense, yes, to the extent that we became aware fairly quickly of the biases these new technologies could induce. AI is in the process of revolutionizing our societies, so it can also make things evolve in a positive way. AI makes it possible to manage and analyze very large amounts of data. It enabled Google, in particular, to create an algorithm in 2016 to quantify the speaking time of women in major American film productions and show their under-representation. At the same time, the teams developing algorithms also need to become more gender-balanced. Today, however, for a number of reasons – including girls' self-censorship when it comes to scientific fields, and the sexism that reigns in high-tech companies – very few women study computer science. It will take time to reverse this trend.
文森特：算法固然需要完善，但单是改写几行代码并不足以解决问题。我们必须认识到，一个团队中如果没有女性成员，那在编写代码工作中就必然缺乏努力促进性别平等的意愿。
Vincent: Of course, the algorithms need to be educated, but changing a few lines of code will not be enough to solve the problems. We must bear in mind that there will be no willingness to code for equality if the teams involved do not include women.
亚洲浪潮，博鳌视野
立足亚洲 面向世界
传递论坛最新动态 促进亚洲深度合作
全球重要对话的传达者 亚洲共同发展的瞭望者
