Increases in either heart rate or blood pressure are major cardiovascular risks. There were a couple drugs that went through the FDA a couple years ago that didn't get approved because they increased blood pressure.

Exercise does increase blood pressure, but it's not associated with an increased risk of cardiovascular events. Don't know the details about caffeine, but I would assume the risk isn't that great.