Big Data, Plainly Spoken (aka Numbers Rule Your World)

Is Waze a ruse?

Waze, Google Maps, and other GPS navigators presents themselves as solving an optimization problem that has been studied for decades: the shortest-path problem of navigating from a given starting location to a given ending location, while staying on legal roads on a given map, with the goal of reaching the destination in the shortest possible time.

The simplest version of this problem admits a static solution, that is to say, it gives the same route regardless of time of day. In this setting, traffic conditions are ignored, and speed of driving is assumed the same for all drivers and at all times. In practice, this static solution sets a benchmark, and a lower bound for the shortest possible time.

A more sophisticated solution removes those restrictions, which is possible only if some kind of sensor data is available, via sensors buried in the ground, embedded in mobile phones, or roadside cameras. With the ingestion of "real-time" data, the solution involves a new element - forecasting.

That's because knowing the traffic where the driver is currently is useless; it's dead on arrival, so to speak. What the navigator needs ideally is the perfect crystal ball - the future traffic conditions between the current location and the destination.

***

If you're a regular Waze user, you may have noticed Waze doesn't quite have the crystal ball. I have suspected this for a while, and a recent trip by a friend from Brooklyn to Manhattan on a Sunday morning motivated me to blog about this.

When he started the journey at 11:02 a.m., Waze estimated an arrival time of 11:32 a.m., so a travel time of 30 minutes. My friend sent me a link that allowed me to track his progress, with a note: "Waze shows only 30 minutes, doubt it... Let's see."

Needless to say, the 30-minute projection was wrong. Not slightly off, but way off.

At 11:17 a.m., about half way through the original schedule, the arrival time has shifted back 4 minutes to 11:36 a.m. Not bad -- except that it kept sliding. Ten minutes later, at 11:27 a.m., Waze now showed an estimated arrival time of 11:50 a.m. Wait, the new estimated travel time was almost 20 minutes more than the original estimate of 30 minutes. My friend texted over a photo of the traffic on the road.

Twelve minutes later, at 11:39 a.m., the estimated arrival time shifted further back to 11:54. So the passage of 12 minutes added another 4 minutes of delay. At 11:46 a.m., seven minutes later, the new arrival time was 11:56; thus, another 2 minutes of delay upon passage of 7 minutes. Another three minutes later, the estimated arrival time moved again, this time, it moved earlier by 1 minute. Oops, two minutes after saying he'd get there one minute faster, the app recomputed the arrival time again, adding two minutes, more than reversing the previous update.

In the end, he got there in 60 minutes, which is double the estimated travel time of 30 minutes at the start of the journey. At 11:50 a.m., 10 minutes before actual arrival, Waze gave an estimate of 5 minutes left so the actual remaining travel time was still double the estimated time towards the end of the trip. Pretty embarrassing stuff.

Note that this was a Sunday morning trip, not a particularly busy or unpredictable time. And the delay wasn't due to traffic accidents. My friend knew it was wrong at first sight.

***

My suspicion is that these navigators are not solving any kind of mathematical problem. It is not making any forecasts. It is purely reactive, responding to what's directly ahead of them.

Ever followed the progress bar on a file download of a large file? You'd probably notice that the estimated completion time fluctuates wildly, both up and down. This fluctuating phenomenon is most apparent at the start of many downloads, when there is a burst of speed. The progress bar will give an estimate, e.g. 5 minutes left. This number soon shots up to 8, 10, 15, 20 minutes... as the download speed slows down.

I have long suspected that the remaining time as displayed is estimated by a simple calculation: remaining amount of data to download divided by the current speed of download. Thus, if the download speed increases, the remaining completion time is shortened, and if the download speed slows, the remaining time is increased.

In this setting, there is no "forecasting" per se. The computation assumes that what's happening now is what will happen in the future. The current download speed will continue into the future until the file is completely downloaded.

Constant speed is a very poor assumption because the variability of the displayed remaining time is a direct result of variable speeds. If the download speed were constant, as assumed, then the remaining time would peel off at a linear rate consistent with real time. In other words, the estimated completion time would not change. In other words, the projection based on constant speed played out as expected.

***

The feeling I got from observing my friend's Waze on his recent trip reminds me of watching large files download onto my laptop. Could it be that Waze implements a similar, simple computation? The estimated arrival time is based on the remaining distance to the destination (assuming the driver follows the prescribed route) divided by the current speed of the driver?

It must be a little better than that. If it really used the instantaneous driving speed, then every time the car stopped, the estimated arrival time would jump to infiinty. Instead of the current speed, perhaps Waze takes the average speed of some duration, say 5 minutes.

Regardless, it appears that Waze's assumed driving speed does not match reality well at all.

Read that sentence again. Then, ask yourself: did Waze give you the shortest route to your destination at the start of the trip? Could it do so without accurate projections of your driving speed on that route?

 

 

 

Know your data 34: think before you scan

Quick, when was the last time you scanned a QR code? How frequently do you scan QR codes?

Source: Wikipedia

In recent years, QR codes have become omnipresent. They have been accepted by the public as a source of convenience. It's an elegant concept which led to an inspired piece of engineering. The camera on our mobile phone detects the QR code image, it converts the image into numbers then to text. The text is in the form of a hyperlink (https://..) The phone then calls that web link, the server on the other side sends the contents, e.g. the menu of a restaurant. In the meantime, the server collects data such as the owner of the phone, the location of the phone, etc. Advertisers and third-party trackers can insert themselves into this process.

A QR code is just a two-dimensional bar code. Usually, what's being encoded into the two-dimensional bar code is a hyperlink.

So, scanning a QR code is equivalent to clicking on a hyperlink.

Remember what cybersecurity people say about unsolicited emails with hyperlinks? They say never to click on such hyperlinks unless we inspect carefully the link itself. Something that starts with "https://www.citibank.com/" might be legitimate but if it looks like "https://citibank.bankingcenter.info/", that might be a pretender, and clicking on it will open up a communication channel with a server operated by a scammer.

***

What are we doing when we scan a QR code using our cell phone? We're clicking on a hyperlink - sight unseen.

Because the text is scrambled as a two-dimensional bar code, we can't inspect the link. This convenient piece of engineering opens us up to the kinds of scams that cybersecurity people have warned us about.

This scam is super easy to pull off. The scammer just needs to post a QR code outside a restaurant, with the words "Scan me to see the menu". Alternatively, the scammer can print their own QR code and place it on top of a legitimate one.

Think before you scan.

***
Here's more from the FTC warning (link).

P esteem, bosom peril, flag to clamor, and anti-tech tech

What drug am I on this morning? Let me tell you, it's the anti-tech tech drug.

As you may have guessed, many new technologies are prone to abuse e.g. using generative AI to produce deep fakes, or what I learned earlier this week, QR code scams (link). The tech consensus is that we don't see any problems because one can use technology to combat such risks. I call this the anti-tech tech drug.

In the academic community, the digitization of knowledge has had profound impacts. One such impact is the creation of "metrics" - a whole field called bibliometrics - that purportedly measure the influence of specific researchers and research papers. What did I say about metrics? See this post from long time ago called "The inevitable perversion of metrics", which later got expanded into Chapters 1-2 of Numbersense (link).

Someone decided it would be a good idea to measure research contribution just like we measure the value of online advertising. Both count things like citations, clicks, downloads, views, etc. Citations are like hyperlinks. Google, whose initial algorithm was entirely based on counting hyperlinks, quickly discovered that hyperlinking is easily gamed. From very early on, humans invented multiple schemes to defeat it as a measure of importance, e.g. link exchanges, barters, buying links, selling links.

 

Not surprisingly, smart researchers have used myriad schemes to game bibliometrics. According to this recent article in Significance magazine, some less-than-honest researchers are flooding the zone with "fake" papers, written by bots. Just imagine having generative AI generate large numbers of "new" papers by feeding it already published articles.

Some anti-tech tech companies say don't worry, we have technology to combat such abuse. What is this tech? It's a plagiarism detection tool that can find copied contents.

What happens? The tricksters adapted.

***

Plagiarism detectors look for segments of text that have been seen before. What kind of text will evade detection? Unusual phrases, what we might call statistically unlikely phrases. Well, someone already wrote software that injects unusual phrases to replace common ones. The tricksters use such software to continue gaming the metrics.

Thus, these "rephrasing tools" have invented "P esteem" instead of "p value", "bosom peril" in lieu of "breast cancer", "flag to clamor" which means "signal to noise".

***

Even so, the authors of the Significance article were too polite to say out loud what's really broken about the peer review process, which has failed to reject many such papers. They reported that more than a thousand papers retrieved from PubMed contain the words "statically significant" instead of "statistically significant" in its title or abstract.

The authors respond to this enormous red flag by musing that "somehow, these goobledegook sentences also pass through peer review."

I can think of two likely scenarios:

1. some peer reviewers did not even open the papers they were supposed to opine upon
2. some editors colluded with authors in a publishing scam