That's correct (although our adjustment step changes the lighting and color of the replacement candidate to match the frame of the input image).
This is why I said earlier that one could create multiple databases. Since the replacement candidates might be recognizable, they should be from a set of people who've given their permission for their photos to be used (e.g., stock photo collections). Another option is to use computer graphics to render a large collection of synthetic faces. Modern techniques are good enough to generate fairly realistic faces.
Finally, note that if every face within some application has been replaced (e.g., if this were to be applied on all images on Google Streetview), it wouldn't even be a privacy issue for the replacement candidates -- since users would know that all faces are replaced, seeing a particular face only tells the user that that person was not there.
This is why I said earlier that one could create multiple databases. Since the replacement candidates might be recognizable, they should be from a set of people who've given their permission for their photos to be used (e.g., stock photo collections). Another option is to use computer graphics to render a large collection of synthetic faces. Modern techniques are good enough to generate fairly realistic faces.
Finally, note that if every face within some application has been replaced (e.g., if this were to be applied on all images on Google Streetview), it wouldn't even be a privacy issue for the replacement candidates -- since users would know that all faces are replaced, seeing a particular face only tells the user that that person was not there.