If the underlying code behind the neural network is open source, and a user has rights to access their own data, then doesn't that comply in every meaningful way? Considering that Facebook has already embraced open source AI and Google has given users near complete control in viewing and purging their history, I think that these companies are already gearing up for compliance.
Part of that data may be the personal data of other people, equally bad to hand over. The whole neural net and would have to be deleted, along with any software component, database, cache etc. resulting from that which can personally identify people.
Neural nets trained on personal data may be illegal wholesale, now or very soon.
> I bet this conundrum never crossed the minds of any of the people writing the regulation.
I expect it has, but they feel the ability for people to have an explanation for why they were denied insurance/credit/a job is more important than allowing companies to use processes they can't explain to make important decisions about your life.
And I bet you haven't trained a model in your life.
You don't need the data. You need the model. And regulations like these are already in effect in the US. In banking, for example, it is prohibited to use protected classes (like race for example) or attributes that highly correlate with them, in your models.
What is it with hacker news commenters and assuming other people don't know things?
The comment I was responding to was talking about training sets, I riffed on that. Depending on your definition of "full explanatory power", the model itself might very well not be enough, especially in the case of neural networks. Could you take a set of weights in a 5-deep neural network, look at an input vector, and have any kind of intuition about the output? It could be that there's some really interesting research that I'm unaware of here, please let me know if there's something I'm missing.
You'll need additional descriptive statistics at the very least, combined with some stochastic meta-models to relate a given record to the "whys" of their output. It's doable without access to training sets, but convincing to us and convincing to a lawyer reading the letter of the law regarding "all inputs to the decision" are 2 different things.
Where ANNs make life-affecting decisions? I can't say with certainty there aren't companies doing it, but if there are just regulate them out and require transparent models.
Speaking from experience, there is a massive shift away from ANNs in decision making. Companies (esp. in marketing) drank the Kool-Aid of some data scientists but figured out that they aren't all that useful. Turns out people care a lot more about inference and are willing to take a hit on prediction in order to be able to interact with the model. I'm not too familiar with ANNs but from the talks I've seen, I'm not even convinced they perform any better than other models.
So from what I've seen, ANNs have very limited application area which does not overlap with decision making, so I don't think there is a problem with them. And if there is, just ban them in decision making. That's not being luddite. We ban all sort of technologies because they are not appropriate.
I'm not sure what the boundary is for "life-affecting", if you're talking about loan origination or anything, then like you said, there's already a boatload of regulations there.
For marketing optimization, FWIW, my experience is the opposite. For a big enough application, any 0.5% improvement is hugely welcome, as long as it works reliably and you're not just gaming the metrics (or, to be honest, even if you are, yay corporate).
