antibodies disapear quickly, but the B-memory cells that produce them are still in the body ready to be mobilized.

"Common cold" and "flu" is not 1 virus, but hundreds of different viruses which is why you get them repeatedly. You do build immunity though. It's why children always have a cold and adults a lot less common.