Surprise surprise. On my way to and from Raleigh last week, I became part of a randomized experiment! At the entrance to the security check, there was a tablet (iPad?) perched on a stand that randomized passengers into the left line or the right line. I encountered the same setup on the return leg.
On the outbound leg, I was assigned to the control group while on the trip back, I ended up in the test group, so I learned what the experiment was about.
In the test group, passengers followed the pre 9/11 rules. They called this "expedited": you don't have to take off your coats, not your shoes, not your scarves; and you don't need to take laptops out of your bags.
As usual, running real-life experiments is fraught with little annoyances. In the test group ("old style"), the agent has to explain to passengers that they don't need to take off shoes, etc. Some passengers are slightly confused and have started unloading coats, etc. Some probably ignored the agent's explanation, thinking that it is better not to risk being called back. If, say the throughput is one of the metrics being used to compare test to control, then this learning curve factor is surely going to disadvantage the "old style" protocol.
There was also an inconsistency in execution. On the away leg, the agent asked me to click on the screen while on the other leg, the agent clicked on my behalf and told me which line to enter.
I'n happy to see that TSA is getting serious about evaluating security policies using randomized tests. I wonder what metrics are being tracked. Any guesses?