What phone are you using? How much system memory do you have left? There's a lot of reasons for lag beyond just a new OS. Some of those things could be exacerbated by the new OS, but for another datapoint, I'm running an iPhone 5 with 3.1/16GB used and I'm not experiencing any lag. The animations are a delight.
I am, however, not a fan of some of the graphical choices they made. A few subtleties like a thin 1px inset color on the NavBar text would do wonders for UI and contrast - but I'm not hired by them to make those decisions.
Which is funny after seeing Apple bash fragmentation at the WWDC. Apple has their own version of fragmentation, newer OS versions don't work on older devices.
I, and everyone I know has had a flawless upgrade experience. There have been a couple of exceptions -- the iPhone 3G (not 3GS) suffered with some point releases, but it was a horrifically underpowered device anyway.
When we talk about fragmentation (as developers), we mean API/display size fragmentation. My iPhone 4 doesn't have Siri, but developers can safely use iOS 6 APIs, knowing I can upgrade.
THe iPhone 5 and the 4" display was a step backwards in that regards, but still, you can safely target iOS 5+ and reach about 95% of active, current app-buying iOS users, or target iOS 6+ and reach 90%. But you can't do that with Android (yet) - targeting 4.0+ would lose you a lot of sales.
This will be fixed in the Android market in the next couple of years though, but until then, iOS users/developers/lovers can rightfully boast about "less fragmentation".
I am, however, not a fan of some of the graphical choices they made. A few subtleties like a thin 1px inset color on the NavBar text would do wonders for UI and contrast - but I'm not hired by them to make those decisions.