Usually home improvement stores now carry them, I bought several and very pleased.
Although if you want super bright, go for the 10mm 100000mcd+ ones off ebay, 50 of them easily replaces a 70W light. I bought 1000 of them, cheap in bulk, and their a lot of fun. If you're making your own LED series don't forget to add resistor to limit the current.
I'd be suspicious of super cheap LEDs on ebay -- they may be cheap but they're unbranded and you don't know how long they'll last. And I'd definitely stay away from the ones that consist of 50 normal LEDs all pasted together. Normal LEDs aren't made to deal with the high current and heat you get from packing them together, so their lifetime is anyone's guess.
I've been putting in LED spots for my office (instead of 25W halogen MR16 lights). I've tried some off of ebay that (claim to) have Luxeon or Nichia LEDs. I've also gotten some from http://www.ledwaves.com/. All are pretty nice, but I haven't decided what color temp I like so currently there's a mishmash of cool and warm ones...
I bought an item (an audio connector) from Ebay and when I got it, it was clear it was from Maplin (like Radio Shack in the UK). But it was cheaper than Maplin, and a great buy, and I respect him for beating Maplin on postage (and they do charge too much) by combining orders and posting out at cost.
I have seen led lights - but they all are extremely expensive. If you want to use to replace incandecent bulbs, CFLs would be a lot cheaper (IMHO).
Some of my friends built their own lighting system and they ordered the LEDS directly from a mail order electronics store. This would probably be the cheapest.
Where in NY is a good spot to shop for LED lights?