I had a professor, way back in the early 00s, who was helping solve this problem. We had no textbook in the class. Instead, our readings were all recently published papers. On the class website he linked to all the papers so we could download and print them if we wanted to. He also negotiated with the author of each paper for the rights to reprint the paper. You could buy the "textbook", which was just a bound photocopy of the papers, for $10 (the cost of printing).
If more professors did this, then the problem would be solved.
I'm doing this with a class I'm teaching, but I think it works better for small research/discussion-oriented seminars than larger introductory courses, where recent papers are most relevant. For intro courses, coming up with good intro materials is a fairly hard problem. If I were teaching an intro course on theory of computation, for example, I would probably still use Michael Sipser's Theory of Computation, because it's good, and I don't think I would be likely able to put together an equally good replacement out of freely available materials. I would, though, try to make sure that the course didn't prevent students from using the previous edition (which is now available cheaply).
This would only apply to upper level classes. The thing about this, there can some specialized classes where reading source papers makes sense. But honestly, even for those classes, usually a good survey book can be better, more concise than the source papers. It depends. But I don't think this would make sense of Algebra or Calculus ... or probably any class besides more specialized ones. I mean, you could teach a Distributed Systems class with source papers; I think that would be OK but it would probably be good to have a textbook as a reference.
My husband, who starts back to university tomorrow, just spent $230 on a new textbook, when I found a used copy on Amazon for $48, because he wanted a new book. I spent a while banging my head against the desk and sighing loudly over that one.
On the other hand, he's excited about finishing his degree, which he has put off for many years now, so if this is what does it for him, it's a small price to pay.... at least that's what I keep telling myself. I suspect that once the excitement of the first couple of semesters of "hey, I'm actually doing this" wears off, he'll be buying used books like the rest of us.
To be fair, used textbooks have highlighting in them. I gave up on used books in law school because the existing highlighting and margin notes were often wrong and distracting, and got in the way of my own highlighting which was a big part of my workflow.
I'm sure this was part of his reasoning. Now that you mention it, I'm not sure I would buy one from Amazon for this reason. I bought used textbooks from the university bookstore all the time when I was in school, because I could flip through them and find one that wasn't badly marked up, but you can't exactly do that with an online retailer.
The reality is that people stop buying books that aren't useful. I knew a lot of folks in grad school who just refused to buy books for classes that didn't use them for weekly problem sets. By and large they got away with it.
Find the right incentives. Whatever your solution, it has to make (financial) sense for the school and the professor, otherwise you're not going anywhere.
I even knew some college students that looked "down" on buying used books, like you were buying someone's used underwear.