Big Data, Plainly Spoken (aka Numbers Rule Your World)

The idol worship of objective data is damaging our discipline

In class last week, I discussed this New York Times article with the students. One of the claims in the article is that the U.S. News ranking of colleges is under threat by newcomers whose rankings are more relevant because they more directly measure outcomes such as earnings of graduates.

This specific claim in the article makes me head hurt: "If nothing else, earnings are objective and, as the database grows into the millions, reliable."

The entire Chapter 1 of Numbersense (link) is devoted to blowing apart the myth that school rankings are "objective." In fact, I go on to assert that even objective-sounding metrics like company revenues are not objective at all... if you know GAAP and the games accountants play with those numbers. If someone buys a car on eBay from another person for $20,000, does eBay book $20,000 in revenues or just x% of the revenues that the seller pays to eBay?

Objective implies there is a ground truth that can be verified. There is no true school ranking, nor is there true revenue.

Where does this "objective" earnings data come from? Apparently a company called Payscale, whose methodology is explained here. They say their data come from "individuals who fill out the PayScale Salary Survey." How did these people discover the survey? We don't really know. Are these people representative of the universe of employed people? Most likely not but we again do not know.

Do people give out their real salaries voluntarily? Not in my experience. In fact, none of my co-workers in my 15+ years in the corporate world has ever told me how much money they make. PayScale claims that the salary number "combines base annual salary or hourly wage, bonuses, profit sharing, tips, commissions, overtime, and other forms of cash earnings, as applicable." Do people have their aggregate salaries at their finger tips? I highly doubt it (unless they make just a base salary).

In return for filling out the surveys, the individuals receive a free salary report. Does this encourage people to make up fake data just to obtain the salary report? Take a guess.

PayScale claims that it "rigorously tests and verifies" the data. Given that most employers wouldn't even do salary verification for other employers, I don't believe the salary data can be verified, and absolutely not "every data point" as claimed in their marketing materials.

The other ludicrous claim is that the "reliability" of the data improves with scale. This is only true if the data is a proper random sample of the population of all salaries, which it clearly isn't. Dumping more garbage on top of garbage is still garbage.

Consider this sequence of scenarios: if

(a) you are the Devious Dean of Admissions at a college, and
(b) improving your college's ranking is on your annual performance management plan, and
(c) you know that the Economist ranking is largely based on the PayScale Salary Report, and
(d) PayScale's data come from the Salary Surveys which do not require explicit identity verification, and
(e) you have access to various devices that can access these Salary Surveys

why are you not sending in a bunch of fake reports of outsized compensation?

Oh, my Mom reads this blog so let me not promote unethical behavior. You don't need to fake data. You can just target a bunch of alumni who have had successful careers, and encourage them to send in their true reports.

***

The other key sentence in the article is: "[The Economist] took the College Scorecard earnings data and performed a multiple regression analysis to assess how much a school’s graduates earn compared with how much they might have made had they attended another school."

I asked the students what are the explanatory (X) variables that might be found in such a regression. Some of the answers were: job title, gender, location of job, GPA, college major, number of years of work experience, family background. Essentially the "all else equal" requires a lot of covariates.

It turns out PayScale has a product called MarketMatch (link) which gives us some hints. Here is a description of this product:

The MarketMatch algorithm uses a two-step process for producing compensation data in a PayScale report. The first step is to understand which of our more than 250 compensable factors are important when it comes to pricing a job and how that job's pay is affected by these compensable factors. This is done in order to define a pay distribution for this job. The mix of compensable factors and their effect on pay is highly dependent upon the job. For example, coding languages and locations are important compensable factors for a Software Developer, while average sales prices and annual sales are important for an Account Executive.

This description leads to a very complex multiple regression model, with "more than 250" covariates, and a host of interaction effects (e.g. allowing the effect of location to depend on the job title). This model has at least 250 main effects. If it has all pairs of 2-way interactions, the regression equation has over 31,000 more terms.

PayScale did earlier impress us with their 1.4 million salary profiles (which, for the following discussion, we assume to be objective and reliable.) This, they say, translate to anywhere from 50 to 4000 profiles per school. While the lower limit is 50 students, PayScale actually does not publish results for schools with fewer than 325 profiles.

Even with 4000 profiles, you can't estimate tens of thousands of regression coefficients with any resemblance of accuracy. If each of the 250 factors were binary ("Yes"/"No"), you would have created 2^250 unique types of individuals. That number has 75 zeroes in it, and you only have 4,000 observations. For overwhelming majority of these types of individuals for which you are issuing predicted salaries, you have zero data.

Is the resulting salary predictions "reliable"? You decide.

 

 

 

 

Reader's guide to the power pose controversy 2

Yesterday, I started a series of posts covering the "power pose" research controversy. The plan is as follows:

Key Idea 1: Peer Review, Manuscripts, Pop Science and TED Talks

Key Idea 2: P < 0.05, P-hacking, Replication Studies, Pre-registration

Key Idea 3: Negative Studies, and the File Drawer (Today)

Key Idea 4: Degrees of Freedom, and the Garden of Forking Paths

Key Idea 5: Sample Size

Here is a quick overview of the key documents:

I recently covered the power pose research controversy, ignited by an inflammatory letter by Susan Fiske (link). Dana Carney, one of the coauthors of the original power pose study, courageously came forward to disown the research, and explained the reasons why she no longer trusts the result. Here is her mea culpa. Her co-author, Amy Cuddy, then went to New York Magazine to publish her own corrective, claiming that the "primary" finding of power pose is confirmed. Here is the statement Cuddy released through a publicist.

***

Now a quick summary of post #1. There is a hierarchy of research studies based on its status within the publishing process, the publication and/or media outlet. Lower-quality research sometimes get popularized through non-traditional outlets such as pop science and TED talks. Even top journals may thirst for media attention. The old world of p < 0.05 is slowly unraveling, partly because it relies on researchers adhering to certain protocols, such as not engaging in p-hacking. The replication crisis is a warning sign to some critics that p-hacking is prevalent. P-hacking may be intentional or unintentional; in either case, the integrity of the science is compromised. Pre-registration and replication studies are potential remedies.

***

The concepts covered today are in the center of the current controversy.

Key Idea 3: Negative Studies, and the File Drawer

Now that replication studies have been legitimized, that leaves one type of studies that are missing from the scholarly literature: so-called negative studies, which are studies that fail to confirm the researcher’s hypothesis, which in turn means the experimental findings fail to hit p < 0.05. This publication bias is known colorfully as the “file drawer effect.” If one scours the literature for studies, such as when conducting a meta-analysis, one will find only studies that show p < 0.05. The other studies are left in the proverbial file drawer. The effect size estimated by such an analysis will be exaggerated.

The file drawer is a form of p-hacking. If only positive results are published, then one can repeatedly perform (substantively) the same experiment until one lands a positive result by chance. If readers of that study with p < 0.05 had known of the file drawer filled with negative studies, they would have treated the published study with appropriate skepticism. “If at first you don’t succeed, try, try and try again” is a positive way to live one’s life but a negative way to conduct science that relies on statistical significance.


Key Idea 4: Degrees of Freedom, Garden of Forking Paths

Developing a good experimental design is a key step in the scientific method. Table 12 in the Carney-Cuddy-Yap response to Ranehill, et. al.’s replication study is a useful reference to the types of decisions researchers make when conducting experiments. Whether the “power pose” is held for 2 minutes or 6 minutes is an example of a design choice. The experiment used in a given research project is a single configuration selected from a large set of possibilities.

Not all design decisions are equally important. The entries in Table 12 by Carney-Cuddy-Yap represent their opinion of which elements are important. The true impact of a specific design decision is not known unless the researcher explicitly tests multiple settings of that factor in the experimental design. Only a subset of the design decisions will materially affect the experimental outcome; some - perhaps many - of such decisions will be meaningless.

Imagine a researcher whose experiment failed to yield the desired p < 0.05. Instead of throwing out the hypothesis, the researcher may re-run the experiment using a different configuration of the design elements (e.g. hold the pose for a longer duration). Again, there are both legitimate and dubious reasons for doing this. Now there are many possible configurations, so this process can be repeated until p > 0.05.

Assume that the effect being measured is non-existent. In any run of the experiment, there is still a positive chance of seeing a positive signal. If one runs a sequence of experiments, eventually one will hit upon one that achieves p < 0.05. That study will be publishable, and thanks to the file-drawer effect (Key Idea 3 above), will be the only one that readers see. If the researcher isn’t careful, he or she might even conclude that the specific configuration of design choices contributed to the “success” of the experiment.

The current criticism of research practice is directed at this type of p-hacking, which some believe to have become all too common because many of the effects under study are likely to be only marginally positive. In addition, when negative studies are stacked away in the file drawer, the observed “path” of published studies represents a path of positive studies only. The problem is that such a path is a rare event akin to a series of consecutive Heads in a sequence of coin tosses. This is where the Cuddys of the world clash with the Gelmans.

In Cuddy’s world, the appearance of the next positive study in series is a triumphant event confirming the original result. In Gelman’s world, the observed path of positive studies is only one path in a “garden of forking paths,” and a  rare event shaped by journal editing decisions, and thus not likely to yield valid generalizations. The garden of forking paths contains all possible configurations of studies, including any configurations that might have been tried and yielded negative (p < 0.05) results.

I have only spoken of one of the “researchers’ degrees of freedom,” namely, the design of the experiment. Another common fertilizer in the “garden of forking paths” is the freedom related to choosing the target variable. As Cuddy’s note reveals, the “power pose” studies have used a variety of target variables, ranging from self-reported feeling of powerfulness, happiness, mood, hormones, cortisol, potassium levels, etc., etc.

Proper protocol calls upon the researchers to premeditate their target variable, and not vary from it even if the experimental results disappoint. In practice, there is a strong temptation to ask whether the treatment affected something else when the original target missed the p < 0.05 threshold. In the world of small, positive effects, the more target variables are inspected, the more likely one of them will show p < 0.05 by chance. In the statistics community, this “multiple testing” problem is long recognized, with a vast literature.

Another temptation is to ask whether the treatment affected a subset of the test subjects when its effect on the average subject is deemed not significant. Such analysis should also be pre-registered. Otherwise, there are countless ways to slice and dice the test subjects, and in the world of small, positive effects, one can easily find subsets that show p < 0.05 by chance. And again, because of publishing protocol, only the positive studies see the light of day, leaving outsiders none the wiser.

There are too many other causes to document here. I will mention one more, just because the New York Times recently published an article discussing it. The journalist (link) asked four pollsters to analyze the same underlying poll data, and they yielded somewhat different estimates. The divergent analyses came about because of subjective assumptions during the analytical process. It may be that some of these assumptions delineate p greater or less than 0.05.

Just like before, researchers are either fooling themselves or fooling others (or both). Unless the design is pre-registered, an outsider cannot tell if negative studies have been abandoned or suppressed. The lack of replication of many seminal studies is driving the narrative that most published studies are “false positives.”

This discussion ought to throw a little doubt on the utility of a systematic review or meta-analysis, which consolidates evidence from multiple studies addressing the same topic. Whether this consolidation work is qualitative or quantitative, it starts with identifying all published studies, and then culling the list down to “trustworthy” studies. But all published studies is a biased subset of all studies.

***

In the last part of this series, I will address the role of sample size in this controversy.

 

PS. For further reading on the garden of forking paths, see Gelman and Loken (link to PDF).

Reader's guide to the power pose controversy 1

I recently covered the power pose research controversy, ignited by an inflammatory letter by Susan Fiske (link). Dana Carney, one of the coauthors of the original power pose study, courageously came forward to disown the research, and explained the reasons why she no longer trusts the result. Here is her mea culpa. Her co-author, Amy Cuddy, then went to New York Magazine to publish her own corrective, claiming that the "primary" finding of power pose is confirmed. Here is the statement Cuddy released through a publicist.

The room is getting crowded. We have Cuddy and the TED Talk crowd on the one side, and co-researcher Carney, Andrew Gelman, and the research replication movement on the other side, each making their respective case. In order to make a judgment, you’re going to need to understand how scientific research and publishing works, and what the statistical critique of the established methodological approach is all about.

In a multi-part post, I provide some of the background information needed to understand what is going on. Here is the plan:

Key Idea 1: Peer Review, Manuscripts, Pop Science and TED Talks (Today)

Key Idea 2: P < 0.05, P-hacking, Replication Studies, Pre-registration (Today)

Key Idea 3: Negative Studies, and the File Drawer (Here)

Key Idea 4: Degrees of Freedom, and the Garden of Forking Paths (Here)

Key Idea 5: Sample Size

***

Key Idea 1: Peer Review, Manuscripts, Pop Science, and TED Talks

Researchers have long recognized the differential quality of research studies. That’s why we have “peer-reviewed” studies and published studies. Published studies may or may not be peer-reviewed. Peer-reviewed studies generally enjoy a higher status. As the number of journals multiplies, which publication puts out the study also matters.

Then, there are manuscripts. If a manuscript is “under review,” it has been submitted for peer review. Some proportion of manuscripts will be rejected by journal editors. Of those submissions that eventually get published, most will undergo revisions based on feedback from reviewers. Manuscripts that have been accepted may live in a queue for a while, in which case they are “accepted” and pending publication. Obviously, a published study carries more weight than a manuscript, especially if the latter is still under review.

Next, we have “pop science,” a relatively recent phenomenon. Malcolm Gladwell is among the earliest to achieve success in this genre. On the one hand, such authors are de facto marketers of scientific research. On the other hand, they may not be the best judges of good science. (Recall Gladwell’s “igon value” controversy.) Even more recently, TED talks emerge as an outlet highly coveted by scientists seeking publicity. Through this channel, science that is not mainstream can attain a mass following, usurping the conventional order of things.

“Pop science” thrives on the pop. In this vein, the team of Steven Levitt and Stephen Dubner, aka Freakonomics, entertained their readers with analyses of out-of-left-field topics, ranging from abortion and crime to geo-engineering solutions to global warming. Seemingly, the only subjects off limits are unemployment, inflation, growth, and other traditional themes in economics. The news media crave Freakonomics-style research. Editors of research journals soon joined the fray, crafting press releases to pitch journal articles with attention-grabbing headlines. Critics worry that “pop science” elevate the titillation factor above the quality of science. Journal editors may have relaxed standards for the sake of publicity. If such critique holds, the implication is that peer review has lost its luster.

 

Key Idea 2: P < 0.05, P-hacking, Replication Studies, Pre-registration

In social science, in order to get research published, the effect must be “statistically significant” with p < 0.05. This convention is intended to weed out findings that do not generalize beyond the one experiment. If a result is generalizable, it should replicate when the experiment is repeated (after an allowance for sampling error).

A replication study repeats a published experiment to validate it. Journal editors have long dismissed replication studies, deeming them unoriginal work. They believe that generalizability is guaranteed by ensuring p < 0.05. This line of thinking is currently under attack.

The edifice of p < 0.05 relies on researchers respecting accepted protocol. When such rules of practice are broken, p < 0.05 becomes meaningless. The researchers are either fooling themselves (unintentionally) or fooling others (intentionally). The most important piece of this protocol is Thou Shall Not P-Hack.

Imagine a researcher, after investing much time and effort into an experiment, discovers that the effect has a p-value of 0.055, just large enough to earn the journal editor’s ire? Because statistics is an inexact science, this situation presents much latitude for mischief. By removing some “outliers” or other “bad” data, or picking a different way of computing the p-value, or doing a host of other tricks, it is quite easy to nudge the offending p-value south of 0.05.

It doesn’t matter whether the intention of the researcher is to mislead. As Dana Carney noted, one can easily convince oneself that the aforementioned tricks represent good science rather than poor ethics. The publish-or-perish culture within the academic and research community is an enabler of this “p-hacking” phenomenon.

The same tricks used to p-hack can be used to solve statistical issues legitimately. That is one of the key ideas in my book Numbersense (link). The same methods can lead to either good or bad analyses. It boils down to a matter of trust and ethics. A replication study performed by a different lab is a great way to check the results for possible p-hacking. Those who trust the existing system will see no need to perform such studies, and may even feel that their ethics are being questioned by those who undertake such studies.

It is challenging if not impossible for an outsider, including possibly a co-investigator and a journal reviewer, to detect if a p-value has been hacked. (In her note, Amy Cuddy claimed no knowledge of Dana Carney’s work process.) Sometimes, one gets a hint based on disclosures of how the data have been processed and/or analyzed but most journals do not require authors to submit complete programming codes, nor do they require authors to detail every step taken in the data processing and/or analysis.

Besides, steps taken after peering at the p-values can be portrayed as part of the original design. This is why researchers nowadays are encouraged to “pre-register” their designs. If her experimental design were pre-registered, Carney might not have had to issue her mea culpa, in which she said she didn’t realize at the time she was p-hacking.

Recently, replication studies have come into fashion. Several careful large-scale studies failed to replicate seminal findings in many fields, including psychology and medicine, shaking the foundation of the science. The “replication crisis” means journal editors no longer can argue that p < 0.05 guarantees that the original finding is valid. At long last, some of this research have reached top journals.

Non-replication comes in two varieties. The effect in the replication study may have the same direction as the one in the original paper, albeit at a weaker magnitude that is indistinguishable from noise. This is a major sign of trouble because the replication study typically uses a larger sample than the original. Sometimes, the effect may even run in the opposite direction (which was the case in some of the Ranehill, et. al. replication studies).

***

More to come: File drawers, garden of forking path, and more!

Second post is up now (Here)

The plural of anecdote is not ...

One of my favorite statistics-related wisecracks is: the plural of anecdote is not data.

In today's world, the saying should really say: the plural of anecdote is not BIG DATA.

In class this week, we discussed a recent Letter to the Editor of top journal, New England Journal of Medicine, featuring a short analysis of weight data coming from a digital scale that, you guessed it, makes users consent to being research subjects by accepting its Terms and Conditions. (link to NEJM paper, covered by New York Times)

The "analysis" is succinctly summarized by this chart:

Their conclusion is that people gain weight around the major holidays.

How did the researchers come up with such a conclusion? They in essence took the data from the Withings scales, removed a lot of the data based on various criteria (explained in this Supplement), and plotted the average weight changes over time. Ok, ok, I hear the complaint that I'm oversimplifying. They also smoothed (and interpolated) the time series and "de-trended" the data by subtracting a "linear trend". The de-trending accomplished nothing, as evidenced by comparing the de-trended chart in the main article to the unadjusted chart in the Supplement.

Then, the researchers marked out several major holidays - New Year, Christmas, Thanksgiving (U.S.), Easter, and Golden Week (Japan) - and lo and behold, in each case the holidays coincided with a spike in weight gain, ranging from a high of about +0.8% (U.S.) to a low of +0.25% (Easter).

Each peak is an anecdote and the plural of these peaks is BIG DATA!

Why did I say that? Look for July 4th, another important holiday in the States. If this "analysis" is to be believed, July 4th is not a major holiday in the U.S. On average, people tend to lose weight (-0.1%) around Independence Day. There is also no weight change around Labor Day.

In a sense, this chart shows the power of data visualization to shape perception. Labeling those five holidays draws the reader's attention. Not labeling the other major holidays takes them out of the narrative. Part of having numbersense is to have ability and confidence to make our own judgment about the data. Once one notices the glaring problems around July 4th and Labor Day, one no longer can believe the conclusion.

There is also "story time" operating here. The researchers only had data on weight changes. They did not have, nor did they seek, data on food intake. But the whole story is about festive holidays leading to "increased intake of favorite foods" which leads to weight gain. Story time is when you lull readers with a little bit of data, and when they are dozing off, you feed them a huge dose of narrative going much beyond the data.

The real problem here relates to the research process. Traditionally, you come up with a hypothesis, and design an experiment or study to verify the hypothesis. Nowadays, you start with some found data, you look at the data, you notice some features in the data like the five peaks, you now create your hypothesis, and there really is little need to confirm since the hypothesis is suggested by the data. And yet, researchers will now run a t-test, and report p-values (in this weight change study, the p-values were < 0.005.)

Even if it's acceptable to form your hypothesis after peeking at the data, the researcher should have then formulated a regression model with all of the major holidays represented, and then the model will provide estimates of the direction and magnitude of each effect, and its statistical significance.

 

PS. Some will grumble that the analysis is not "big data" since it does not contain gazillion rows of data, far from it. However, almost all Big Data analyses that are done following the blueprint outlined above. Also, I do not define Big Data by its volume. Here is a primer to the OCCAM definition of Big Data. Under the OCCAM framework, the Withings scale data is observational, has no controls, is treated as "complete" by the researchers, and was collected primarily for non-research reasons.

PPS. Those p-values are hilariously tiny. The p-value is a measure of the signal-to-noise ratio in the data. The noise in this dataset is very high. In the Supplement, the researchers outlined an outlier removal procedure, in which they disclosed that the "allowable variation" is 3% daily plus an extra 0.1% for each following day between two observations. Recall the "signals" had sizes of less than 0.8%.

 

 

Why are GMAT scores going up?

In Chapter 1 of Numbersense (link), I went through an extensive list of shenanigans that can be used to trick the rankings of colleges and graduate schools. One of them is to allow students to submit the maximum of repeated sittings of GREs, LSATs, GMATs, etc.

This tactic is unabashedly headlined in a recent Wall Street Journal article, "Test Redos Give GMAT Scores a Lift." (link)

In the print edition, the article is printed on six columns. Not until the sixth column does the writer acknowledge that "the test costs $250 for each attempt." Allowing redos is an amazing business. So the stars are aligned: the college admissions people want to report higher GMAT averages; the applicants want to feel good about themselves; the test providers rake in the $$$; the test prep industry sucks even more of the $$$.

The casualty is accurate measurement of whatever the GMAT test is supposed to measure. In an accompanying chart, we learn that the average score of all exams is consistently at least 20 points lower than the average scores sent to MBA programs. A 20-point gap for the average test-taker is a huge difference. Part of this gap is selection: perhaps people who do not score above a certain threshold decide not to apply to business school. Part of this gap is surely due to allowing low scores to be dropped.

Here are the rules of the game:

• College admissions staff only looks at the maximum score (why don't they look at the average or median?)
• The test providers allow test takers to strike low scores from their record within 72 hours, thus preventing these scores from being seen by schools
• You can take the test two, three, four, ... times. There is almost no risk of re-taking the test since you can cancel lower scores, and retain higher scores

 

 

 

Two quick hits: how bad data analysis harms our discourse

I am traveling so have to make this brief. I will likely come back to these stories in the future to give a longer version of these comments.

I want to react to two news items that came out in the past couple of days.

First, Ben Stiller said that prostate cancer screening (the infamous PSA test) "saved his life". (link) So he is out there singing the praises of the PSA test, which has been disavowed even by its inventor (link), although still routinely used by many physicians.

One can't dispute that the PSA test result caused Ben Stiller to know about his cancer and he is better today because of that discovery.

However, imagine the following scenario: I invent my own screening test. The test consists of flipping a coin: heads, you have cancer, tails you don't. Amongst those people who came up heads, I can find one for whom he truly has cancer. I saved his life because my test alerted him to this fact. Because I saved this person's life, my test must be really good. (If one anecdote is too few, I could find a handful of people whose lives I have saved.)

***

Second, the FBI tells reporters that the Minnesota mall attacked "withdrew from friends in months before attack." (link)

Imagine that you are trying to predict who will be the next disgruntled attacker. Based on the FBI statement, you want to round up everyone who "withdrew from friends." How many people would that include? How many of them will eventually be attackers?

Same holds for all the other findings, such as "he converted to Islam recently", and "he posted something hateful on Facebook".

***
It is precisely when we want something badly, like information that saves our lives or that prevents terrorist attacks, when we become most susceptible to nonsense data analyses.

 

Data is the next frontier of equal rights

When I rented my first apartment in the U.S., there were two things about American society that left an impression. It was super easy to get credit. My parents had the foresight of making sure we build up a credit history while in college, which helped. But, from setting up cable TV to signing a rental lease, these businesses showed a lot of trust to someone who was earning a salary for the first time.

The other impression was an oddity concerning the phone service - in those days, we still used the land line. The strange piece of business was having to hand over 50 cents a month to be "unlisted" (from the phone books). The phone company knew my phone number, and I must pay them to keep it from the preening eyes of marketers, stalkers and the likes.

Fast forward to today. Lots of companies have our data, and in most cases, we don't even have the option to keep them private. (Whatsapp recently changed its privacy notice to send data back to mothership Facebook, and they instituted a 30-day window to do a partial "opt-out." You blink and you missed it.) Sometimes, we can pay to get a "premium" version of the service, which may be the same service but without the ads. That's the modern-day "unlisting."

Privacy comes at a cost. The rich and powerful will be able to buy their privacy; not so the average person. This is the new frontier of equal rights.

This battle is being played out in front of our eyes. See this New York Times article about RentLogic, the company that is attempting to create a rating of landlords in NYC. RentLogic uses data to rate landlords. They managed to get onto the rental listings of Citi Habitats, one of the large rental brokers in NYC, only to get kicked off after 10 days when the landlords applied pressure. Meanwhile, landlords have long had access to services that score tenants, and use these scores to disqualify prospective tenants. (Halstead, one of the landlords cited in the article who opposes landlord ratings, uses On-Site to rate tenants.)

The same battle is playing out elsewhere: increasingly, police is using omnipresent videos as evidence but they do not want citizens to point cameras at them.

Lots of websites collect data incessantly on users but they go to great lengths to stop users from scrapping data from the websites.

In each of these cases, the rich and powerful gets to collect/own/store others' data and they want to prevent others from collecting/owning/storing their data. Why should this relationship be asymmetric? It's an equal-rights issue of our times.