Big Data, Plainly Spoken (aka Numbers Rule Your World)

The Taylor Swift ticketing fiasco: causes

Last week, Shannon and I published a piece in Slate, analyzing some survey data about the pre-sales experience (link). The data indicated that certain segments of fans who supposedly received "boosts" might have experienced a lower success rate than others who didn't.  We also speculated about what might be causing this effect.

I call it speculation because the survey data can at best tell us what, but not really why.

In this post, I walk through our causal reasoning. This would be something data analysts should do for any causal studies, and then go find data to validate or refute the hypothetical causes.

Do the results have something to do with Ticketmaster’s much larger scale of operations?

This theory is a diseconomies of scale. Perhaps Ticketmaster did not provision enough server capacity to handle the much larger volume compared to Seatgeek.

Is it because of unbalanced distribution of tickets among time zones?

It appears that the process accommodates all time zones, which means the East coast pre-sale starts first. As a result, Ticketmaster must have some kind of quota system so that the tickets don't run out before the other time zones open up. If they allocated the tickets incorrectly, for example, underestimated the relative demand from East coast fans, then we could see a lower success rate for those in the East coast.

Because SeatGeek had a more efficient system in some way?

Just like any queuing system, the speed of servicing is another important input which determines how long the waits are, and the ultimate success of getting a ticket. See this recent discussion about self-checkout lines in supermarkets. (link)

Or is it that SeatGeek had less-demanding fans who would have taken any seat and date? Maybe if you’d had a “Lover Fest” ticket and were based on a coast, you were just pickier about tickets, and by the time you made it through the queue and into the seating arena view on Ticketmaster, you decided that no, you weren’t willing to click as quickly as you could to get absolutely any seats whatsoever into your cart.

Imagine you're in a store, and the person in front of you is bringing back three months of purchases to get price adjustments on every item. I have a theory that the "stans" are much more picky, want the best seats, want to sit close to other stans, etc., and therefore, the average processing time for each stan is longer, which means others would have been able to beat them to the checkout.

The “Lover Fest” tour hadn’t been scheduled to go to the Midwest, so it’s possible that fans who bought tickets to faraway shows in the first place were more willing to jump at whatever tickets they were offered, cost or logistics be damned.

Someone who's willing to take any seat will finish the process faster.

A nefarious read on this is that Ticketmaster deprioritized boosted fans.

It's nefarious if we attribute the deprioritization as a feature not a bug. It's possible that there was a software glitch that caused an unexpected reversal of priorities. It's more irritating if the "boost" was a marketing gimmick. The marketing materials might exploit people's wishful thinking, in which case it's harder to decide if the fans themselves were guilty of misinterpretation, or the marketers of misleading.

***

Do you have other theories? Tell me about them below.

If I'm in charge of investigating this, the next step is to go through each scenario, think about what data can be gathered to validate or refute each, and hopefully one of these leads will prove out!

The Taylor Swift ticketing fiasco

I have an article in Slate, coauthored with Shannon Palus, that just came out, on the recent controversy over pre-sales of Taylor Swift concert tickets (link). A Slate editor contacted me about a dataset that was collected by a Swiftie who was irate that her special status as a "verified fan" apparently did not bump up her chance of getting a ticket but might have even backfired. So the fan set up a survey to expand the sample size. A very good idea!

In the article, I presented my findings. There are a number of factors that affect a fan's chance of getting a ticket. The main takeaway is it's too simplistic to say having verified fan status is always bad. In statistical terms, interactions matter (keep reading if that term is unfamiliar).

The following chart summarizes the data analysis:

In my post, I'll briefly describe the analytical process.

First, I think about what tool I'd use for the job. In this case, I chose JMP, which is my go-to tool for quick, exploratory, one-time-only analyses.

I opened the raw data file, and inspected the rows (which correspond to individual responses to the survey). I removed a small number of rows from the dataset which could not be analyzed - e.g. rows with a time stamp but containing null responses to all questions. Also, from the response patterns, I could see that if someone said they did not have verified fan status, then no other responses were recorded. Those rows are also removed.

Next, I selected the columns of data that are relevant to the problem of predicting success rate based on some factors like having a verified-fan boost, time zone, ticketing platform, etc. For each factor, I made sure all answers are valid and coded missing values the way I like it. A small number of discretionary decisions has to be made. For example, the key outcome variable - whether someone was able to purchase the ticket - has a third possible answer of "Yes I got tickets but only through a friend who got through the line". I decided to count this as a "No" instead of a "Yes". Also, for reasons that I don't need to explain to modelers, I created duplicate columns, e.g. one column showing responses Yes/No, treated as text, and a duplicate showing response 0/1, treated as numbers or factors.

For the time being, I resisted renaming columns, which were long strings of the entire survey questions. I eventually succumbed and replaced those long strings with simple column names like "time zone". Having extremely long column names just make analytical results really hard to read.

I ran some standard analyses, including multiple regression and decision tree. For multiple regression, I screened for interaction effects. In particular, JMP has a really useful "profiler" feature that allows me to visualize the "response surface". I first ran the decision tree using the automated process. This methodology is very useful for assessing the potential interaction effects, and relative importance of the various factors. The regression equation also provides similar information: these analyses are complementary.

One of the reasons I chose JMP is that the decision tree procedure has a tree-building feature. Instead of using the automated rules specified by the inventors, I can iteratively build my own decision tree. The chart you saw above is the decision tree I built manually. It is not identical to what the automated procedure created. That's because I made decisions about how to structure the data (based on the learning I synthesized from both analyses I performed). The automated procedure always picks the strictly most discriminant split at each stage, but frequently, there are multiple splitting variables with similar degrees of discrimination. For example, the automated procedure would not have split the responses by ticketing platform first but not doing so vastly complicates the story.

I did one final transformation. The original question on boosting allows four choices: No boost, only Lover Fest boost, only Taylor Nation boost, and both Lover Fest and Taylor Nation boosts. Using this factor directly obscures the main and interaction effects, so I split this up into two variables: Lover Fest boost (yes/no) and Taylor Nation boost (yes/no). This rearrangement provides a more nuanced analysis. For example, for Ticketmaster users in the East Coast, having the Taylor Nation boost reduces success rate by a little but having the Lover Fest boost reduces success rate by a lot. Having both boosts is about as bad as having just the Lover Fest boost, thus we can't add up the two individual effects and get the aggregate effect. (This is what statisticians call "interaction" between two factors.) For this reason, the final tree diagram did not mention Taylor Nation boost for Ticketmaster users in the East time zone.

***

How does my analytical process deviate from what's presented in a textbook? I treat the results from standard analytical methods as references, not gospel. These standard methods spit out the best fitting models according to some criteria of "goodness". Such criteria almost never include maximizing human understanding of the (unknown) processes that generated the data. Chasing after tiny improvements to "R-squared" frequently over-complicates the model which reduces clarity to humans.

***

If you have any questions about the analytical process, just ask below.

 

 

 

 

 

 

 

 

Beneath the surface of recent Google, Elon, FCC news

Several recent news items caught my eye.

First, there's been endless drama coming out of Twitter since Elon Musk acquired it for an ungodly sum. One of his latest bombshells is to promise an email drop that "makes public all internal discussions about the decision to censor @NYPost's story on Hunter Biden's laptop" in order to "restore public trust" (link). It's just one example of how corporate executives can weaponize data.

Second, Wired reported that Google handed over the personally identifiable information (PII) and detailed location data of at least 1,500 people to FBI investigators (link), through "geofence warrants", which is a dragnet. Geofencing is a favorite technique also of marketers for finding targets to deliver their marketing messages: the marketer draws an area on the map, and the data providers hand over contact information of every cellphone that is known to have entered the geofenced area. In this case, the FBI requested information on anyone who entered the area around the Capitol with their cellphones on January 6, 2020.

Third, the FCC blocks several leading Chinese technology manufacturers from the U.S. market (link) in the name of protecting national security. This action is taken based on the "possibility" of "espionage, IP theft, blackail, foreign influence campaigns, and other nefarious activities".

***

These stories are all related to our data. What happens to our data. Who has our data. What can people do with our data. What have they done with our data.

Americans have allowed corporations to be custodians of our private data. This was originally cast as a grand bargain, in which corporations get our data for free while they offer us some service of value for free. Think back to the beginnings of Google and Facebook... scratch that, not the humble beginnings when the entrepreneurs lured us in by offering their services for free without a quid quo pro. It was after these companies went public when they realized they must make money, and eventually, every one of these outfits has become a hoarder of private data, which are sold to the highest bidders. Every new digital product now is conceived as a way to augment the profiles of the users. This includes paid services (e.g. Amazon and any number of subscription services) which also collect the same types of data, and trade them in an obscure market... without the grand bargain.

***

Many - most, perhaps - of us have decided to trust Google, Twitter and similar companies with our data. Some of us prefer to trust our corporations rather than our governments. But can we really trust corporations to act in our best interests?

What is our best interest? It depends. If you're one of those people who entered the Capitol on January 6, your best interest is for Google not to send your location data to the FBI. If you happen to be around the Capitol on January 6 but not involved in the election politics, you don't want investigators to be probing you just because you were in the neighborhood.

If you are one of the anti-government protestors in China, Iran, Russia, etc., your best interest is for Google not to send your location data to the investigators. Now, is it any different if you're protesting against a Western democratic government?

Closer to home, if you happen to live in one of those states that have banned abortion, and also encourage your neighbors to report on you to authorities, your best interest is for Google not to sell your location data to the highest bidder. If your neighbors are pro-life activists, their best interest is for Google to release geofenced location data around known abortion clinics so you can fulfil your civic duty to report on people and doctors who skirted the abortion ban.

Do we really believe that corporations are making these decisions based on our individual desires, or based on what their lawyers think is the best course of action for the corporate interest?

If the government entity is collecting our data, there are constitutional rights that govern what this entity can do. There are Freedom of Information Act (FOIA) provisions that citizens can use to discover if laws have been broken. There are no FOIA requirements that regulate corporate behavior. There is no punishment for companies issuing false statements when not under oath.

***

Elon Musk's behavior provides a glimpse at the answer. His threat to release a load of internal emails relating to Twitter's moderation process is a direct threat to weaponize data. He's not the first corporate executive to do this sort of thing: remember how a Uber executive casually told a reporter that she's being tracked during a time when another Uber executive threatened to dig dirt on journalists who wrote critical articles about Uber (link).

If Twitter will release internal emails to make a political point, it certainly has the means to release user data if an executive finds a reason to do so. This is selective, targeted disclosure of potentially private data. Musk is not releasing all emails about all banned accounts, but emails about one particular political figure that happens to belong to a political party he is on the record of opposing.

There is no way to verify that all emails related to that one situation would be released. No one outside of Twitter has access to this data. If there are any emails that could place Twitter in legal jeopardy, what's the chance that the corporate lawyers would allow them to be released? Would these lawyers play government and redact the documents?

Remember, besides the messages that Twitter users post publicly, there is a whole hidden universe of direct messages which are private conversations between people. Activists in authoritarian countries are known to use platforms like Twitter to hide communications from prying eyes.

Musk's announcement is much more than about Hunter Biden's laptop. It is an admission that private messages are not really private. They are in the hands of the companies which own these platforms. It only takes one CEO or executive with a grudge to weaponize someone's data. (Junior staff can also wage their own small-scale vendettas as many people have access to data as part of their work.)

***

The banning of Chinese tech companies is based on the possibility that the Chinese government may get access to private data from Chinese companies, and the possibility that such data would be used to harm people.

In Twitter's story, the CEO articulated a motive and a plan to selectively release data. He just hasn't pushed the button yet.

By contrast, Google pushed the button when they transferred data to the FBI. It's not anonymized data, either. It's personally identifiable information plus detailed maps of these people's locations. (While the Wired article singled out Google, it's likely that other big tech companies, like Microsoft and Apple, and telecoms companies, may have done something similar.)

 

P.S. [12/4/2022] In the evening of the day I put up this post and the next day, Elon Musk pushed the button and released the data so we now have two instances in which prominent U.S. corporations released data with the express purpose of harming the people who generated the data, plus an allegation that Chinese companies may do similar things.