TSA gets into the randomization game

Surprise surprise. On my way to and from Raleigh last week, I became part of a randomized experiment! At the entrance to the security check, there was a tablet (iPad?) perched on a stand that randomized passengers into the left line or the right line. I encountered the same setup on the return leg.

On the outbound leg, I was assigned to the control group while on the trip back, I ended up in the test group, so I learned what the experiment was about.

In the test group, passengers followed the pre 9/11 rules. They called this "expedited": you don't have to take off your coats, not your shoes, not your scarves; and you don't need to take laptops out of your bags.

As usual, running real-life experiments is fraught with little annoyances. In the test group ("old style"), the agent has to explain to passengers that they don't need to take off shoes, etc. Some passengers are slightly confused and have started unloading coats, etc. Some probably ignored the agent's explanation, thinking that it is better not to risk being called back. If, say the throughput is one of the metrics being used to compare test to control, then this learning curve factor is surely going to disadvantage the "old style" protocol.

There was also an inconsistency in execution. On the away leg, the agent asked me to click on the screen while on the other leg, the agent clicked on my behalf and told me which line to enter.

I'n happy to see that TSA is getting serious about evaluating security policies using randomized tests. I wonder what metrics are being tracked. Any guesses?

Comments

[Ted]

Are you sure this is a randomized test? Maybe it's just a way to expedite screening by only giving some (random) passengers the full shake-down. Many countries do something very similar for their customs screening. Random checks are pretty effective at deterring crime and much more efficient.

[Glenn]

I am so using this in my AP Stats class. I am going to have the learners create a possible experimental design given the information in this post.
Thank you!

[Chris P]

My partner and I both travel a lot and have been using TSA Pre-check for the last year.

Pre-check is the old school security that you were randomized into. It was first offered to US Citizens who had gone through a background checks for immigration fast lanes (Global Entry or Nexus) plus ultra frequent flyers for major US carriers (50k miles or more initially) at select airports (SLC our home was one of the first).

It also had randomization involved in that when you got to the line check, your boarding card was scanned and 3 beeps meant Pre-check and 1 beep meant regular screening. From my experience this gave pre-check about 80% of the time and bumped to regular 20% of the time, so you always had to be prepared to switch lines and screening modes.

They may be practicing for higher volumes. Ninety percent of the time I go through the precheck line there is no one in the line so they may not know throughput.