I'm less concerned about privacy, although still concerned about that, than I am about the Brazil principle here. The sorts of people that end up in these border security jobs are, if nothing else, very good at "following orders" and if the computer says someone's a target who are they to think otherwise?
Will they arrest a five year old child with the same name as someone who's identified as a terrorist mastermind wanted for bombings in the 1980s? I'd bet money they would! Will someone be arrested for looking like Osama Bin Ladin because for some reason he's still in the database? Undoubtedly.
They'll be rounding up people that simply look like other people, or by people mis-identified due to software bugs or broken, badly implemented features. What if all people who have really dark skin are classified as an immediate threat because of a single entry in the database that caused the identifier to over-fit for a particular set of inputs?
The failure rate on this is bound to be high. Even a 99.9% accurate system is going to identify nearly a million people as threats if there's 900 million trips per year, which is a typical year in the US. If it's 99.8% or 99.5% the numbers grow to the level of pure absurdity.
Will they arrest a five year old child with the same name as someone who's identified as a terrorist mastermind wanted for bombings in the 1980s? I'd bet money they would! Will someone be arrested for looking like Osama Bin Ladin because for some reason he's still in the database? Undoubtedly.
They'll be rounding up people that simply look like other people, or by people mis-identified due to software bugs or broken, badly implemented features. What if all people who have really dark skin are classified as an immediate threat because of a single entry in the database that caused the identifier to over-fit for a particular set of inputs?
The failure rate on this is bound to be high. Even a 99.9% accurate system is going to identify nearly a million people as threats if there's 900 million trips per year, which is a typical year in the US. If it's 99.8% or 99.5% the numbers grow to the level of pure absurdity.