From some quick searching around, volume on the exchanges is around 500,000/day while volume on the blockchain is over a million. If that’s accurate (and it may not be, I’m not an expert on this and the sources I found might suck) then most volume is on the blockchain by a pretty good margin.
I don’t know how much gold is actually shifted around physically, but the trading volume in a year is several times the actual amount of gold that has been mined, so I suspect the proportion of that which involves physical movement is tiny.