Once you have the private key, you would submit a transaction with that private key and authorize a transaction to a public key that you control, and doesn't have part of the private key available.
You don't need the public key, and IIRC most algorithms allow you to derive the public key from the private key, though I'm not sure that's the case with Bitcoin. I have vague memories that there are algorithms where this is not that case, but it's been a while.
If you have a normal ECDSA private key, you get only one public key. However, there are ways to get a 1-to-many scheme, and similar ideas are used in U2F (yubikey or similar) systems.
The basic idea is you pick one private key that's a sequence of 256 bits or so, call this k. When you need a keypair, you compute H(k, tag) to get another bitstring, then turn that into an ECDSA private key (minding the bear traps here) and that then has a single public key.
For example in U2F, the key derivation is H(k, domain, ...) where k is the secret baked into the USB token, domain is the domain you're logging in to (this is the part that protects against phishing, among other things) and further protocol-specific information.
You don't need the public key, and IIRC most algorithms allow you to derive the public key from the private key, though I'm not sure that's the case with Bitcoin. I have vague memories that there are algorithms where this is not that case, but it's been a while.