the way these LLMs work, there is a weight for each parameter? 13 billion weights? what is an example of a parameter?

A parameter is a variable for which a weight (a floating point value) is the concrete value.

a weight is an example of a parameter

so is a bias, and presumably the biases are also in the same file with the weights