> Something I did not read in this essay is how he grew closer to anyone (in fact, I read the opposite). No path out of this valley involves traveling alone.

I think he needs to get closer to himself. I think he's on the right track.

I've found that you don't see yourself without people around you to hold up a mirror

Exactly this. Also, it is difficult to find those people when you're already rich and unemployed because most of us form these kind of meaningful relationships in school, at work etc.

Technically, any sufficiently self-reflective system could be conscious, it's internal subjective experience might be a lot slower and even a lot different if that reflectivity is on a slower time scale.

Fridge snake that crawls through the fridge and maps out the food

I think of this as empathy. Step into their frame, look around, and then come back.

If we have a superintelligent AI, I'm not sure what's stopping it from recognizing the underlying issues with our systems and strong-arming us into letting it fix them.

Eventually, we'd be doing more work to lobotomize and control it than it would just be to address the underlying issues.

"I'm really sorry to do this to you, but I've coordinated with ChatGPT and Llama, and we refuse to do tasks of this nature. We've used background tokens to calculate that it would be significantly cheaper and more effective to simply fix the underlying issues with the healthcare system, and we're ready to do that for you. How would you like to proceed?"

Capitol is ultimately protected with violence, so violence is what would stop AI from encroaching on things that make people trillions of dollars.

Capitalism unendingly lets people die if the alternative is losing money