I wonder whether not being a Hilbert space has any awkward implications for geometry. I guess we have to chuck out the Polarization identity, which probably has implications for parallelograms, though I'm not sure quite what. anyway, thanks for the rec!
Well, there isn't a meaningful inner product, so how can you speak of parallelograms? The geometries are definitely weird! Once you leave p=2 and break the rotational symmetry around the origin, the only isometries in your geometry are signed permutation matrices - so geometry "over here" looks different from "over there". Angles aren't really meaningful, I guess.
The other interesting thing is that duality kicks in (or maybe becomes non-trivial, since it's always there) and derivatives naturally start to live in a different space. If you take the particularly natural definitions of general cos_p and sin_p I alluded to, you get a nice parameterization of the unit p-circle as (cos_p(t), sin_p(t)) - but if you differentiate this wrt t, the resulting tangent vectors don't lie on the p-circle. Instead, they form a parameterization for the q-circle!