There have been dozens of articles and news pieces about the need for “ethics in AI” or “ethical AI.”  This (apparently brand new) issue arises from a number of causes, including public concern over facial recognition, the use of automated decisionmaking, and the ongoing public fascination with the darker side of artificial intelligence (see: Black Mirror).  There have been some very interesting, nuanced, and thoughtful discussions of how to infuse ethics (or, at least, how to reduce bias) in AI.  There have also been way, way more hot takes and totally ill-informed talking heads.

Part of the problem is that we aren’t all having the same conversation.  As we’ve explained before, understanding AI is both more complicated and much simpler than you might believe, particularly when you understand that AI is one of those terms that get used as a substitute for actually understanding the subject matter.  Put another way, it’s a buzzword: you can use it seemingly without any idea what it actually means.  It’s the same with “the Cloud.”  When you hear someone say “we’ve uploaded all of your data to ‘the Cloud,'” all they’re actually saying is “we’ve uploaded all of your data on someone else’s computers.”  That’s it.  No actual clouds, no floating computers, no server room on Bespin (unfortunately).  Blockchain, too.  It’s just a distributed ledger.  You record transactions and the transactions are recorded and…and that’s it.

That’s not to say that AI doesn’t have incredible potential or that it’s smoke and mirrors.  On the contrary, we’re big believers in using machine learning (which is what drives the “thinking” behind AI).  We just think it’s important to be honest, realistic, and direct about why we’re using AI/ML, and what we mean when we say “ethics.”  It takes more than buzzwords to make a meaningful decision, and so we present a few challenges that are worth considering before diving into AI, or discussing what it means to make it ethical.

1. Whoever Decides Your AI “Ethic” Will Encode Their Biases

It’s just unavoidable.  Humans are, largely, incapable of recognizing just how biased they are, because it requires us to examine our behavior from outside of our own perspective, something akin to pulling yourself out of quicksand. Don’t confuse bias with bigotry, though: most biases are actually just errors in logic or shortcut thinking, rather than an outright animosity towards a particular group.  Take survivorship bias, for instance, which unreasonably compares successful or surviving models against historical averages as though they were normal, ignoring that the very fact of their survival or success might mean that they’re not normal.

In AI, survivorship bias might lead to your AI applying a kind of Horatio Alger, bootstrapping success model where those with better financial credentials are considered better candidates.  “It’s meritocratic!” you might say.  Perhaps.  But it would also almost certainly lead to reinforcement of existing biases in the financial sector, which disproportionately (and unnecessarily) downgrades individuals based on race, gender, or a host of other reasons (many of which are protected statuses under the law). AI won’t know that it’s doing something wrong because you didn’t tell it, and because you may not even know it either.

This means that the algorithm behind your AI tool won’t incorporate the requirements of the Civil Rights Act of 1964 or the European Convention on Human Rights unless you teach it, but you can’t incorporate the requirements of those laws in your algorithm unless the people creating your algorithm know about those laws, but those people can’t know about those laws unless you ensure that they do, which means that they’re encoding your/their own ignorance/bias into the algorithm, which, bringing it all home, is the title of this subsection.

2. You Can’t Be Ethical and “Creepy” Simultaneously

Ethical uses of AI also tend to conflate means with ends.  But AI is simply a tool, not an independent thinker who requires an ethos to guide behavior (not yet, anyway).  So there are two preliminary questions: is this an ethical use of AI, and is it deployed in service of an ethical purpose.  It’s the question that rarely comes up in Silicon Valley: Yeah, sure, you could do that, but should you?

Consider the AI-driven facial recognition technology that identifies, during loan interviews, micro-facial movements to determine the honesty, intelligence, and/or creditworthiness of applicants.  Or another loan system which examines social media platforms to identify whether an applicant has the kind of friends who pay back their own loans — “usually a good indicator” of whether they’ll pay back a loan themselves.  These systems rely on incredibly complex algorithms and technology to deliver their promise: a greater degree of insight into future human conduct.  They also begin to veer rapidly into the domain of the creepy.

Human intuition is a powerful tool, and it’s one we rely on frequently when we try to explain how to recognize when a proposed data processing goes from “oh wow” to “oh no.”  It’s never a perfectly clear demarcation, but somewhere between using AI to provide vastly more accurate and early diagnoses of neurodegenerative diseases and using AI to create increasingly sophisticated deepfake videos, we cross the frontier.  Remember: AI is entirely morally agnostic, it only operates within the parameters given.  The algorithms aren’t engaging in creepy conduct because they “felt” like it, because despite what you’ve heard, AI still can’t “feel” or “want” anything.

This means that any ethics of AI needs to be a kind of meta-ethics of AI usage.  That is going to be an extremely difficult task, because to be successful, it would require a fairly widespread agreement about the appropriateness of limits to begin with.  Consider the cloning controversies of the late 1990s.  Although there were debates about the ethics of cloning Dolly the sheep, the general consensus was that human cloning was out of the question.  Today, the analogue might be that, while automated financial decisions might have questionable aspects, handing the adjudication of criminal cases over to an AI “judge” is too far.  Except, you know, Estonia courts have already started to do that, and the indications are that it won’t be long before American courts do, too.

The point is not to demean the notion that AI will be useful in a variety of fields – it will.  The point is to ascertain when the uses of AI move beyond the helpful and morph into the dogmatic or, worse, the absurd.  The greater part of that task is recognizing that there is no such thing as an ethical way to do something unethical.  Resolving that higher order question can’t be done with AI, it can only be done with humans thinking critically.

3. There Are Unintended Consequences (or, the “Obi-Wan Principle”)

Ethical AI also has to account for the unknown, and anticipate the unexpected, even as it becomes more unknowable to us.  That isn’t the same as predicting the unknown or unexpected, but instead is about having the flexibility to address it when it arises.  Imagine that you’re in a city you’ve never visited and don’t speak the language, but you need to find a hospital.  What would you do?  Maybe you’d ask a stranger and gesticulate towards your injury (“ow” is pretty universal”), or maybe you’d look for a police officer, or maybe you’d even shout “help” in whatever languages you knew in the hopes that someone might understand.  That kind of plasticity of thinking is what characterizes human responses to the unknown: we improvise, but we do so in a way that draws upon our understanding of human nature to maximize the chances of success.

Something unknown and unpredictable is sometimes called an “unk-unk,” short for “unknown unknown.”  It comes from a sociological concept called the Johari Window, but you probably know it from the Donald Rumsfeld press conference where he talked about “known-knowns.”  Humans can sometimes get the response to unk-unks very wrong (which is basically the premise of E.T. and The Day the Earth Stood Still), but we are still vastly, vastly better at it than AI.  That’s because AI, as we’ve explained, is still really just a set of calculations performed on a given set of training data.  The more data AI has, the better able it is to identify patterns, but that does not equate, in any way, with the ability to glean meaning from the data.

Perfect example?  IBM’s Watson playing on Jeopardy.  Was it incredible to see a computer best Brad Rutter and KenJen?  Sure.  But the reason why Watson had the ability to answer so many questions correctly was that it had been trained on 200 million pages’ worth of potential questions and answers to draw upon when presented with a question.  Again, the fact that Watson could parse human language to answer those questions was little short of astonishing, and, yes, human learning is also partly a process of storing data and calling it back up.  But without the training data at its disposal, Watson would have been unable to answer the questions (that’s obvious: you can’t answer something you don’t know).

AI, then, can only analyze its training datasets and make its best estimate of the right answer.  When presented with something entirely new, it will be left without the necessary data to make its choice, leading to some very strange potential answers and outcomes.  It’s the same reason that a chatbot (which, as a computer program, is incapable of being sexist, racist, or any other kind of bigoted) can spew unbelievably offensive statements within hours of being exposed to human interaction: whatever we teach it, it will learn.  As a helpful graphical reminder, here’s some Gen-X nostalgia:

That covers the problems that occur when AI encounters the unexpected, or uncontrolled training data.  But what about the corollary problem, when we encounter the unknown in AI?  If you spend any time studying how AI platforms work, you’ll run into the same issue time and again: AI systems reaching inexplicable decisions or developing completely unplanned languages and operational activity.  It is the ultimate “black box” situation, in which the outcomes of the operation are completely untethered from the decisionmaking that produced them.

Facebook’s efforts at AI provide a good example.  In 2017, the Facebook Artificial Intelligence Research lab (hilariously, “FAIR”) allowed its chatbots to converse, and negotiate, with one another, to improve natural language skills and bargaining abilities.  The result?   Chatbots went, in a few hours, from simply having “Hello, I would like to buy five widgets” conversations that follow the training data scripts Facebook provided to the following:

Bob: i can i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i i can i i i everything else . . . . . . . . . . . . . .
Alice: balls have a ball to me to me to me to me to me to me to me
Bob: i . . . . . . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to
Bob: you i i i i i everything else . . . . . . . . . . . . . .
Alice: balls have 0 to me to me to me to me to me to me to me to me to
Bob: you i i i everything else . . . . . . . . . . . . . .
Alice: balls have zero to me to me to me to me to me to me to me to me to

Weird, I know.  But the striking part about the excerpt above is that, based on Facebook’s analysis, it represents hours’ worth of normal human conversation.  In other words, each letter, space, and period represents an entire thread of a conversation that would’ve taken a human far longer to say.  An interesting fact, but it also presents a substantial problem from an audit, control, and human perspective.  If we can’t recognize or decode the language AI platforms use when they communicate with one another or analyze data to reach a decision, how can we possibly understand if they’ve made a valuable, reasonable, or ethical decision?

GDPR, as we’ve discussed a number of times, makes an effort at curbing these kinds of excesses by guaranteeing a right for natural persons (viz.–humans, not corporations) to object to automated decisions, to know if AI platforms are making decisions, and to demand human intervention in the process.  But any honest assessment of Article 22 would concede that this involves a fairly easy workaround: programming the AI to provide an acceptable, pretextual answer to queries about the basis for a decision.  That shouldn’t be surprising, because humans do it all the time.

And so we reach the central problem of the inevitable opacity in artificial intelligence: it reacts in ways that we cannot always anticipate to make decisions we cannot always understand.  We will, almost to mathematical certitude, find ourselves facing decisions or reasoning that move outside the realm of normal human rationality into a process driven by 1) training data, 2) inherent biases, 3) uncontrolled access to other open-source datasets, and 4) the iterative process of machine learning.  It is a combination of factors that, regardless of our efforts to control, cabin, or quantify it, will be outside the scope of our understanding.

Think of Obi-Wan Kenobi in Star Wars, just before Vader kills him.  (Uh, spoilers, I guess?). He tells Vader:

It’s not that Vader has no understanding of power (he does) or that he doubts Obi-Wan’s skills (he doesn’t).  It’s that Vader cannot know that Kenobi will become something entirely new, and entirely unexpected: a cryptic, kind of mopey Force Ghost. When presented with the unknown, the unknowable, the opaque, humans often respond poorly; AI will sometimes not be able to respond at all, mute in the face of uncertainty, or perhaps groping for an analogue that isn’t there. That’s the ghost in the machine: uncertainty. It’s the ghost we’ll chase until AI — and we — learn to face the unknown with grace.

4. Collecting Valuable, Usable Training Data Can Create a Vicious Cycle

There’s a practical element to ethics as well.  Training data is extremely important, as we mentioned, because it’s what allows the machine learning process to understand and identify relationships and patterns.  But that very reliance on the training data means that the classic problems of data quality come into play when AI is making decisions and reaching conclusions without human input and guidance.

Poor data quality comes in a variety of forms, and each of them can disrupt or corrupt AI decisions.  Here are just two examples and some problems they raise:

Contradictory Data

You have data from a multiplicity of sources, and, many times, the data points will overlap with one another.  Say, for instance, that you have a client email list through your own operations (held, for instance, in MailChimp) and you also have purchased a huge number of data points from Dun & Bradstreet or another large-scale data provider.  If you’re dealing with a particular audience, it’s almost certain that you’ll have duplicates in the two datasets, and you will almost certainly see errors, conflicts, or contradictions between the two sets.  Sometimes that’s because the data subject themselves provided different information (e.g., they gave you one mailing address when they registered an account but D&B has an older address) and sometimes its because of simple data entry error.  Other times, it’s because the labels for datasets don’t match up (your entries for “Address” are for residence, while D&B uses business addresses, for instance).

It doesn’t really matter why, because it’s inevitable: the data will not always agree, which is why the merging of two datasets is, quite often, less valuable than the sum of its parts.  When you provide contradictory information to an AI platform, it has to decide what makes the most sense, and it will quite often get it very wrong.  AI is not designed to understand paradox or irrationality.  When faced with the contradictory data that inevitably works its way into datasets, AI will respond with a best guess which, often, is no better than what a human would do.

Incomplete or Outdated Data

Another very common problem is a dataset that lacks critical information.  If you’ve ever applied for a mortgage, you know how complicated it can be to assemble all the required documents, the necessary forms, and the supporting information.  But that’s only the front end.  Now imagine that the bank which originated your loan has sold it to another bank (and trust me, they did).  If the purchasing bank never got the complete file, or if some of the documents were scanned poorly, or if some of the physical paperwork was lost, the bank might not know for years (and trust me, they won’t).  Now imagine that the bank runs the data they have through an algorithm to identify, categorize, or reassess the credit risk profile of mortgagees or, perhaps, those who wish to secure a HELOC on the property they own.  AI could conclude that a missing file is the result of human error on the bank’s part, but is it likely that the bank will train its system to assign blame for errors to the bank, by default?  Not likely.  So the algorithm will either presume that the individual failed to provide what was necessary or that the individual does not meet the criteria the documentation would have shown.

Far fetched?  Perhaps.  But given what we know about how humans process loans, extend credit, and address the risks of a given financial profile, it’s hard to believe so.  And, remember, it isn’t just the financial sector that wants to use AI to make choices.  When datasets are incomplete or out of date, AI platforms will be grappling with the unknown and the incomplete simultaneously.  That can’t, and shouldn’t, inspire confidence.

What does all of this mean?  Let’s follow the logical course.  Ethical AI requires training data that is accurate, complete, fresh, and as free of contradiction as possible.  Collecting and maintaining datasets that have those characteristics requires a data integrity protocol that features systemic oversight, regularly refreshed data, and a quality control mechanism to weed out bad data.  Such a data integrity protocol demands data from new, different, and recent datasets in order to provide accuracy and the right kind of confidence intervals.  Those new datasets, must in turn, be verified and checked, which means yet more data is necessary.  Thus, ethical AI, in one sense, requires mass collection of data from multiple sources on a near-constant basis in order to perform as intended — which is the very situation that created the data hoarding of surveillance capitalism, a leading cause of the call for ethical AI in the first place.

But assume that we handle the biases, the purposes of the processing, factored the necessary flexibility into our AI platform, and kept the data accurate and updated.  Is that enough to resolve the problem of “ethical AI?”  I suppose it depends on how you frame the question.  In one way, there’s only so much you can expect of a system until it’s operational and, within parameters, there will be sufficiently ethical AI systems that are safe and desirable to deploy. You could virtually eliminate road fatalities if the speed limit were 5mph; we elect to take the risk of higher fatalities in the name of efficiency.  That seems perverse, but recall that, without an efficient mode of transportation, the economy would collapse.  So we’ve made the decision that, at a certain level, the risks are worth the cost.

That’s why we can’t possibly expect to create an artificial intelligence with encoded ethics that will satisfy all circumstances: we can’t create humans with a code of ethics like that either.  This is not a problem that we “solve,” it’s a dilemma that we have to manage, every day.  We can, and should, have discussions about how to import values into machine learning, and establish parameters and limits for our increasingly sophisticated (and opaque) algorithm-based decisions.  Yet we have to be clear about one thing: “ethical AI” is only shorthand for “ethical human uses of AI.”  It’s easier to elide our own responsibility, and our own duties, when we place the burdens on some undefined artificial intelligence.  But as long as it is humans who do the programming and humans who set the goals, it’s our own ethics we need to worry about first.