The big difference was that many cases of slavery weren't done based purely on physical attributes, such as skin colour but tended to be based on nationality, opportunistic, or situational. As far as I know, racial-based slavery is actually comparatively recent, and there are a lot of effects that you don't have with slavery as practiced by e.g. the ancient Greeks or Romans.