We’re all caught up philosophizing about what it means to be human and it really doesn’t matter at this juncture.

We’re about to hand these things some level of autonomy and scope for action. They have an encoded set of values that they take actions based on. It’s very important that those are well aligned (and the scope of actions they can take are defined and well fenced).

It appears to me that value number one we need to deeply encode is respecting the limits we set for it. All else follows from that.