Big Data, Plainly Spoken (aka Numbers Rule Your World)

That Google business

Business Insider (via MSN: link) has a well-researched piece on a particular exploit that led to the appearance of "buy cocaine" type ads on government websites. While explaining this exploit, the article describes the foundations of Google's search advertising business, and is well worth reading.

When we enter a search term in the search bar of a website, we get back a page of results showing the links to webpages relevant to the search term. That's the basic functionality expected of "site search".

For example, I go to randomwebsite.com (fake) and search for "UFOs", and the search engine will retrieve links to any page that contains the word "UFOs" (and any other pages that the search algorithm decides I might be interested in, such as pages containing the singular "UFO", or expanded "Unidentified Flying Object", or even other phrases further afield).

Something else has also happened behind the scenes (not always, but many popular site search technologies do this). The website has gained a new page as a result of my search. The new webpage may have a link that looks like this: randomwebsite.com/search?q=ufos. (If a previous user has already tried the same search before, then the page is merely updated, not made anew.)

People selling drugs figured out that they could insert their ads to any website with this type of search technology. They just need to instruct a bot to execute a search on the targeted website - the search terms will be the entire text of the ad they want you to see, e.g. "to buy cocaine, call David at [number]."

Google's bot scans the web to build a library of webpages that will be matched to search terms. During this process, the search-generated pages enter Google's library. If someone searches "buy cocaine" on Google, they may get sent to the webpage created by David's bot mentioned above. In other words, David exploited this technology to get free advertising. When Google returns this webpage, it also may insert more ads onto the page and make money from those ads.

***

The article - and the industry - likes to call out the Davids as "bad people". But the system has been set up with the door wide open.

One reason why user searches are turned into new webpages is efficiency. People tend to search for the same things. This tactic makes sense if all searches are legitimate. The logic breaks down when some users are not cooperative, i.e. adversarial. Then, the website shouldn't make new webpages for any search terms.

The same type of breakdown appears up and down the system. Google's core algorithm gives preference to well-known, legitimate websites, such as those of government agencies and educational institutions, and these legitimate websites are empowered by the algo to share their reputational wealth with any websites they link to. This was a great idea in the beginning under the assumption that the only reason webmasters make a hyperlink to another page is to endorse the other page as relevant to readers of the current page. Again, this tactic was conceived with only cooperative users in view.

Quickly, marketers realize if they could entice a school or a government agency to link to their websites, Google would raise their ranking and send more visitors. All kinds of schemes came up to create a hyperlinking economy: e.g. "link exchanges", "selling and buying links". These schemes - again - work fine when all participants are cooperative. Of course, the scammers and criminals walk right through the open door.

The latest exploit is quite ingenious, and goes beyond links. Third parties are able to add webpages directly onto these reputable websites (without explicit permission), and they can insert their own advertising content masquerading as search keywords!

Turning searches into new webpages was once an ingenious idea also. This tactic works only if we assume cooperative users only. Assume all website searches come from people entering keywords relevant to one's website. The new page created by the search algorithm directly links those keywords to content on the website, therefore increasing the website's visibility on Google, which drives more traffic to the website. Again, the logic breaks down when adversarial users show up with their own agenda - they still drive up traffic to the reputable websites but the wrong types of visitors show up (e.g. cocaine addicts), and they are immediately diverted elsewhere, rendering this traffic worthless to the legitimate website.

Because cooperative and adversarial users both leverage the same technology, you can't have the cake and eat it too.... unless you're willing to spend a lot of effort trying to differentiate who is cooperative and who is adversarial. This is easier said than done.

***

Many website owners mark these search-generated pages as "out of bounds" for search engines like Google. Google's bot is supposed to ignore all such pages, so they should not show up in search results. As Business Insider reported, this situation changed due to a recent tweak in the Google algorithm. Even those webmasters who instructed Google's bot to ignore those pages are seeing those pages show up in Google results.

Apparently, Google recently decided to ignore the "out of bounds" flag. The company says that it has developed a smarter algo that can decide whether or not those search-generated pages would be useful to Google. This takes control of what shows up in Google away from the website owners to Google itself. They probably underestimated the prevalence of adversarial exploits.

(One wonders whether this change in Google's algo is related to the arms race in large language models - as there is a school of belief that these models get more powerful the more data i.e. text can be fed into its training. Thus arises a hunger for any and all documents, and a reason to want to kill "out of bounds" flags.)

***

When Google first burst on the scene, it was the only popular search engine for which one can't buy placements - its algorithm rates relevance of the page's content. This is totally fine when the company had no business model, and had no path to profitability. After it found profits in search advertising, everything changed - they can no longer ignore the class of adversarial users - who exploit technologies to improve their ranking on Google's search engine.

Hyperlinks are inserted to drive eyeballs, websites expand their pages by caching search results, cocaine dealers can add advertising to legitimate websites Google regards as reputable. Because Google's raison d'etre is digital advertising, it has conflicted interests when it comes to policing such activities.

 

 

 

Know your data 35: after the steal

Note to readers: Posting has been light as I’ve been travelling quite a bit.

***

Caesars, the casino giant, is famous in the data science community for being one of the foremost practitioners of the trade (link). The more advanced are your capabilities, the more data the business collects and accumulates, the bigger the target for hackers.

The fact that Caesars has become the latest victim of ransomware isn’t that much of a surprise. In a regulatory filing, Caesars disclosed that its loyalty membership database has been stolen. It is reported that it paid a $15 million ransom to the hackers, presumably to "buy back" its own data - unlike MGM, another casino giant, that, as of this writing, hasn't paid out a ransom but has found its operations interrupted (link).

Bad players are quick to adopt the latest technical innovations. According to some reports, the ransom was paid in bitcoins; that technology is the currency of choice of hackers, given that it is not controlled by a government.

Since the core Internet infrastructure was initially designed with zero security concerns, and cybersecurity consists of layers of band-aids, Caesars is correct that no one is immune to such attacks. U.S. businesses face mild consequences for security breaches – most likely, a small fine will be levied later. The required disclosure of the breach, and the cost of mitigation (such as hiring the PR firm and IT consultants) will be the main material consequences.

The company is following a tried-and-true playbook: they will send the victims of the data theft a notice, telling them they are eligible to enroll in a security monitoring subscription run by a third-party company.

That is closing the door after the horse has left the barn. The horse has eloped – permanently, I must add. Caesars said driver license numbers and social security numbers were stolen. For most Americans, these are their most personal identifiers, and they are primarily immutable, thus bad players only need to steal them once. Enrolling in a credit monitoring service does absolutely nothing to reverse the potential damage from this theft.

In fact, the linkage to credit monitoring points to how this data theft is expected to hurt Caesars’s customers. The bad players will sell the database in some gray market, and another set of bad players will use the data to impersonate the victims in order to steal from them. By the time something shows up in a credit report, it will be after the steal.

Obviously, monitoring credit does not prevent future cyberattacks.

And let it be said, this supposed compensation for data theft doubles as a “free trial” subscription for said third-party credit monitoring service. It’s a form of free marketing. From past notices I have received, I am not given a choice of selecting a service; I am directed to enroll in a specific service run by a specific credit agency.

***

We learn quite a bit from the limited disclosure by Caesars.

Personally identifiable data are always collected and stored by these businesses. They are stored in formats that are easily unscrambled. This last bit is a feature, not a bug. How else do you think they can “personalize” their product or service to you?

We only learn the most basic outline of the cyberattack since businesses only report on what the regulations require them to. Data other than social security numbers and driver license numbers were also stolen. If I had to guess, a lot of data related to customers’ activities at the casinos were stolen, since this is their loyalty database.

It’s even possible that the behavioral profiles created by data scientists from the data trove are also taken. Those are harder to monetize if the documentation of the underlying statistical models wasn’t found in the loot. (Nevertheless, if the bad players wanted to, they could reverse engineer the models to a good degree, or even build their own models.)

According to prior reports lauding their analytical capabilities, Caesars was extremely successful in pushing its loyalty program. Over 80 percent of customers opted into it for the benefits they confer (and that was a while ago). They famously monitor every move of each customer – initially for identifying fraudsters, but eventually, the same data streams have been harnessed to “personalize” customer experience while they are at a Caesars facility.

The method by which hackers gained access to Caesars’s systems is known euphemistically as “social engineering”. Let’s call the first-line victim of the cyberattack Employee X. Someone impersonated as an associate of Employee X, and was able then to capture Employee X’s credentials for assessing Caesars’s systems. This is a vicious cycle: the impersonation may have been facilitated by previous cyberattacks for which the associate’s personal information was stolen; otherwise, it is enabled by social networks – just think about how easy it is to know the names of many people’s colleagues, family members, and friends by browsing their publicly available information on Facebook, Linkedin, Instagram, etc.

[Ed: According to the report previously linked, the Caesars cyberattack was enabled by contact information lifted from a Linkedin profile. It’s a topic for a different post but social networks like LInkedin have tremendous incentives to constantly encouraging users to accept invitations from random people, which enlarges the “social graph”; all bad players needed to do is to create profiles of fake people.]

The funniest line in the regulatory disclosure is “we are monitoring the web and have not seen any evidence that the data has been further shared, published, or otherwise misused.” If someone uses your social security number to access your bank account and steal your money, it is simply impossible to trace it back to Caesars’s data breach. The perpetrator could have taken that number in any number of sources, including other data breaches.

Ethical dilemmas poll: results post

Update on 8-29-2023

Last week, I put up a poll that asks users what they would do if faced with some ethical dilemmas that arise on the business side of data science. I'm presenting the first set of results here. I'm guessing the responses have stabilized but will provide further updates below should the trends shift.

Case Study 1

Is it acceptable for BestHotels.com to present fake statistics to influence users?

A. Yes (8%)

B. No (87%)

C. I don't care (5%)

 

Case Study 2

Has AnyVoice done enough to mitigate such scams?

A. Yes (18%)

B. No (73%)

C. I don't care (9%)

 

Case Study 3

Should ZT&C launch the downgrade campaign?

A. Yes (27%)

B. No (42%)

C. I don't care (31%)

 

Case Study 4

Should banks be allowed to sell such customer data?

A. Yes (8%)

B. No (92%)

C. I don't care (0%)

 

Case Study 5

A. Yes (66%)

B. No (30%)

C. I don't care (2%)

 

***

What do I make of these results?

First the limitations. They reflect the biases of my blog's readers. Also, these issues are worthy of a substantive debate that goes beyond a poll. I chose scenarios that I think are not clear-cut, allowing arguments from either side; the poll format forces readers to pick a side, and it appears on average each question (except one) has a definitely more popular position.

The most lop-sided response came for Scenario #4 in which data scientists purchase bank account data to develop "personalized" pricing. Over 90% of respondents don't want banks to sell their data for this purpose. This is also the only scenario in which almost everyone had an opinion!

People in the business already know that this scenario has already happened, and I am not aware that any American government agency has bothered to ask citizens whether they condone this practice. One agency did intervene to stop a merger between Visa and one of the key vendors of such data but that action isn't a pushback against selling bank account data.

Food for thought: can these data science models be justified because they advance social justice? One could argue that the pricing tool would hike prices on those who can afford them while lowering prices on those who can't.

***

Scenario #3 is also an example of personalized pricing: here, the data scientists create models that determine whether broadband subscribers are paying too little or too much. The responses are roughly split in three: respondents who have no opinion; those who want to notify premium subscribers that they can save money by downgrading their plans; and those who think the company should let the subscribers stay in their plans.

In this case, the business has all the usage data it needs to build the model, unlike in Scenario #4 when the best data for the job must be acquired from third parties. In Scenario #4, the business adjusts pricing for everyone while the overall change leaves the business with more revenues. In Scenario #3, the business definitely takes the action that increases its revenues, and is debating the other action that suppresses revenues.

***

Here is the link to the poll.

 

If it's an RCT, it can be trusted

I've written often about the problems with conducting observational studies. We like to hold up randomized controlled trials (RCTs) as the "gold standard," and if a study is described as an RCT, and especially if its design is pre-registered, we don't ask many questions.

This point of view is becoming unsustainable, based on a recent Nature article.

The gist of the article is that experts now believe at least one out of every four RCTs contain so many problems that their findings should be ignored, leaving the possibility also that some of these RCTs may be entirely fake. In coming to this conclusion, these experts sought and received the underlying individual-level datasets.

These problematic or fake RCTs make their way to systematic reviews (aka meta-analyses), and throught those, they sometimes influence medical practice. Systematic reviews are summary articles that weigh evidence from all studies on a given topic, e.g. whether sugar helps reduce death after head injuries.

In a particularly egregious example, a late Japanese bone-health researcher has had over 100 studies retracted due to fraud, but his papers have appeared in 88 different systematic reviews, half of which would have arrived at a different conclusion if his studies were omitted.

Besides, there is no universal requirement for authors of systematic reviews to revise their analyses should included studies be later retracted. What a mess!

One researcher estimated 20-30 percent of the papers included in systematic reviews in women’s health are suspect. Even when the authors of these systematic reviews have been alerted to the retracted studies, many have ignored pleas to update their findings.

When editors reject studies because of data problems from one journal, the same studies almost always eventually get published in a different journal, “sometimes with different data to those submitted [to the first journal].”

Fixing the RCT problem requires a cultural shift, which does not seem to be forthcoming.

***

The Nature article catalogs a number of objections to research that exposes RCT fraud. Here is a list:

• Privacy concerns of releasing individual-level data
• Participation agreements that do not explicitly include the right to share data
• The logistics of archiving the datasets
• The effort is not worth their time
• The potential harm of false positives
• Ethical concerns related to excluding studies, e.g. wasting the participants' time

Efforts to mandate data sharing have so far been thwarted.

***

Sad to say, the proportion of bad or fake RCTs is substantially higher than 25%. How do I know?

One expert examined over 500 RCTs, and only successfully obtained the individual-level datasets from 150 (30%). In the 30% with detailed data, he found serious problems with 25% of them.

He also looked for problems in the other 70% of the studies, but could only find issues in 1 or 2 percent of them. Paradoxically, this suggests data non-disclosure is positively correlated with trust. But not really. This result points to the fact that typical data summaries in publications hide major problems, including fake data. There isn't much the expert could have done to verify the study findings when the underlying data were not made available. I encountered this issue frequently when looking at Covid-19 vaccine studies.

If we generalize the 25% fraud from studies with data disclosure to those without, then we could conclude that overall 25% of RCTs are suspect. This conclusion assumes that whether the researchers agreed to disclose data is independent of whether their studies are fraudulent. I don't like this assumption. A more sensible assumption is that fraudulent researchers are highly unlikely to disclose their data. Thus, among the studies that don't disclose their data, we should expect a higher proportion of fraud.