A Report from an AI Day in Hoboken

Two weeks ago, I attended an AI Day organized by SAIL at Stevens Institute of Technology in Hoboken, and attended talks by AI staff from Google, IBM, Verizon, SAP and Verisk Analytics.

Here are some notes that I want to share.

1. AI is hot

All these companies are putting significant resources into AI. The people working in this field are fanatically excited, and hungry for a future run by software. They presume the world will be changed completely, and for the better, as AI infiltrates practically all aspects of human life.

2. AI is deep learning

AI has become synonymous with deep learning. Speakers use those terms interchangeably throughout the day. Deep learning is a direct descendent of neural networks, research that can be traced back to at least the 1960s, and thus it represents an evolution rather than a revolution. It’s hard to reconcile how this limited technology would produce the grandly stated ambitions.

Two important articles about the risk of betting everything on deep learning have appeared recently, one by Michael Jordan, the Berkeley data science professor, and one by the Allen Institute by way of New York TImes.

3. Different definitions of AI

AI is most effective and useful in automating repetitive, tedious tasks. For example, the police in many jurisdictions are driving around taking photos of license plates to compile a database of drivers’ locations. Instead of having humans read off license plate numbers one by one, they deploy an AI tool to efficiently read those numbers.

A telling moment at the conference was, after one speaker presented an example of using AI to take over tedious tasks previously done by humans, two people in the audience proclaimed that what he presented is not AI.

Later, the speaker from IBM declared all of today’s AI “narrow AI”, and indicated that IBM is focused on making “broad AI”. Broad AI is defined as technologies that are generalized enough to perform different skills, not just one skill. For example, today’s deep learner is trained to do one thing well, like determining if an image contains a dog or not.

What is AI? It seems like everyone has an opinion.

4. We know bias, and we will eliminate it

Another moment of the day is when one speaker turned to the conference organizer and said “It’s become obvious that we need to have a bias seminar. Have a single day focused on talking about bias in AI.” That was his reaction to yet another question from the audience about “how to eliminate bias from AI”.

As a statistician, I was curious to hear of the earnest belief that bias can be eliminated from AI. Food for thought: let’s say an algorithm is found to use race as a predictor and therefore it is racially biased. On discovering this bias, you remove the race data from the equation. But if you look at the differential impact on racial groups, it will still exhibit bias. That’s because most useful variables – like income, education, occupation, religion, what you do, who you know – are correlated with race.

5. It’s too good it’s bad

When it comes to ethics and potential harms arising from AI, the prevailing mood seems to be that bad is the consequence of AI being “too good.” There isn’t talk of curtailing the technology – the hope is that whatever bad is eliminated (see point #4).

This reminds me of the advice we’re given when answering the interview question: what is your greatest weakness? We’re told that a good answer is something along the lines of “I work so hard I forget to sleep.” We’re told that this answer is great because it does not answer the question, and in fact, switches the topic to something positive.

6. Real challenges

Between the lofty visions presented by various speakers, we get a few glimpses of what is actually being worked on, and the real challenges in deploying AI.

One speaker summarized the three key obstacles in deploying AI in the real world as “not robust, not explainable, and not trusted.”

The speaker from Verizon described how they are training an AI system – a kind of chatbot – that handles Q&A with customers. The complexity of such a system is clear, consisting of some advanced AI technologies running alongside rules-based heuristics.

Someone in the audience asked how this chatbot learns from its mistakes. Actually, before learning from, it has to learn of. Imagine you’re using this automated Q&A tool. You asked a question and it returns an irrelevant answer. How does the chatbot know it made a mistake?

The naive answer is: just display a survey question at the bottom of the response, asking the user whether the response is useful or not. This is a standard feature of almost all such systems that I’ve seen. What’s the reality? Few people complete this survey (sparse, biased data), and some mis-use the survey to express their general unhappiness (noisy data).

Instead, the team at Verizon randomly selects 100 answers a day that they examine manually for accuracy. This suggests human input is still needed to calibrate these systems on a continuous basis.

7. Where is the user in this?

More than once, I wondered what is the role of the potential user in the development of these AI technologies. In the talks that I attended, I barely heard mention of user testing, gathering requirements from users, collecting feedback from users, measuring user satisfaction, etc. The prevailing philosophy is if you build them (well), they will come.

AI is obviously an important trend to keep track of. Comment below if you have reactions to what’s happening.

Comments

[Thomas Ball]

Kaiser-
Wish I had known of this event! But for all the bandwagon hype, smoke and mirrors was there any discussion -- beyond bias -- of the methodological limits of AI, DL, NNs, etc.?
Thanks...

[Antonio Rinaldi]

I remember a tweet about the double (un)known classification. Data is the known known. Sampling error is an example of known unknown. The bias speakers deal on is the known unknown. But there is also another bias, the unknown unknown, that cannot be repaired. The first one cannot be repaired only when assuming politically correctness as a dogma.

[Kaiser]

AR: Thanks for the add. I recall that comment. I think that quote is only approximately useful. "Data is the known known" doesn't cut it for me as it does not deal with fake data, bad data, missing data, etc. Sampling error is not a known unknown; it is a measure of variability caused by random sampling, which does not apply to "data exhaust" type of data.

I recommend people read Chapter 3 of my book Numbers Rule Your World - the section about dealing with racial bias in SAT testing - to understand what the issue of "bias" is about. Also, the Prologue of Numbersense also deals with this issue.

TB: That meeting is not the place to go to hear about limitations. Under the "Real Challenges" section, I collected the several episodes in which someone pointed out limitations. It does appear that the one problem acknowledged by all is "bias". The general mood in the room is that there is an engineering solution to all problems.