By making the ecosystem useful enough that the benefits of doing it your own way are often outweighed by the benefits of being conformant to a standard. If I have a nice dashboard to ties together all the items in my house, giving me easy access to them all, nice status info at a glance, and summaries of usage and/or problems, any device I want to buy that doesn't tie into that nicely is going to really have to justify its existence through some really nice features.
As for who would buy into a system like this initially? Lots of Asian manufacturers that aren't interesting in being software companies, and would rather give you some standard firmware that you can tie into some other management console or replace with what you want.
I agree it often does. That's not to say that concerted effort can't make a change though. The internet itself works this way, partially because there wasn't a lot of concerted commercial interest initially, and partially because there existed standards, and that's cheaper.
It's hard for any one company to compete with Google or Apple when they decide to make their own standard and it's semi (or totally) proprietary, but then there's hardware and manufacturing it's harder to be totally dominant. If there's hundreds of companies that all do it a standard way, there's a lot less use in being the one company that's not compliant. There just needs to be something for all those other companies to target.
That's not to say it succeed, but if there's a chance, I think it's worth it to try.
As for who would buy into a system like this initially? Lots of Asian manufacturers that aren't interesting in being software companies, and would rather give you some standard firmware that you can tie into some other management console or replace with what you want.