The BOTEC Uber-v.-taxi study in poor neighborhoods in Los Angeles, which I blogged about Monday, had some important limitations as a source of definitive answers:
1. It only covered L.A.
2. It didn’t address, e.g., service to people with disabilities.
3. It was paid for by Uber.
4. Uber had the final say on publishing the findings.
Of those, #4 was especially problematic. It’s well known that (before the FDA cracked down) pharma companies used to fund multiple safety-and-efficacy studies of new drugs and then cherry-pick which results to publish. I’m close to certain that wasn’t the case here; we only did the one study, and the idea came from us, not from Uber. Still, the question mark remains.
One way to cure that is for others to carry out similar studies. As I mentioned before, we’re happy to make our data available for re-analysis and our methods available for replication to anyone who asks.
#3 would be more of an issue had Uber been allowed to influence the study design or influence the data analysis, interpretation, or presentation. But that wasn’t the case, and the methods were entirely straightforward.
I agree with those who say that it would be better to have more studies not funded by Uber, and I’m aware of at least one independent group interested in trying. But the fact that Uber wanted the research done, while apparently the legacy taxi industry and is regulators don’t want it done, or at least don’t want it done enough to pay for it, suggests to me that everyone involved had the same intuition I did, and that was borne out by the study results: that Uber would turn out to be faster and cheaper in servicing low-income areas.
As to #2, of course service to those with disabilities is an important issue, one of many raised by the emergence of ridesharing. But no single study can cover all the issues.
The picture just got brighter with respect to both #1 (geographic scope) and #4 (the risk of cherry-picked results). Uber – motivated, no doubt, by the vote previously scheduled for tomorrow in the New York City Council on the Mayor’s proposal to cap the number of Uber drivers (but not privately-owned cars) in the name of reducing congestion – has decided to fund a similar study in New York.
That doesn’t entirely get rid of problem #1, but two research sites are better than one when it comes to generalizability.
Better yet, Uber agreed to fund the study but not claim ownership of the data, so we will be free to report whatever we find, thus avoiding problem #4 entirely.
The bad news is that, given the political calendar, data collection needed to be compressed into a single day. As I write, a small team is riding around New York, comparing UberX with both yellow taxis and what are called “boro taxis,” which operate only outside Manhattan (which accounts for 90% of all yellow-taxi trips).
So we will only be able to report pilot-scale data: a target of about 60 observations. If the differences turn out to be as dramatic and consistent in New York as they were in Los Angeles, that won’t matter, because such differences will show up clearly even on a small sample. If the differences are narrower, we may not be able to reach a firm conclusion at this stage. The plan is to do it again, more slowly, at a much larger scale.
Anyway, watch this space. The team expects to have the analysis done late tonight (California time), so there should be results up tomorrow.