This is absolutely fine for a young kid. Kids are way less conventional than adults at that age. If a kid gets food, warmth and attention they're basically golden.
It solidly depends on the age/stage of development of the child. When you say young, that really means like <3 years. Anything above that, and they really need stability, routine, and regular/stable interaction with individuals outside of the immediate family group to avoid long-term developmental and psychological issues.
Yeah I think one could quibble on the exact age...I think even up to 4 it would be OK. After that I think a more stable setup is definitely positive. I would say though that a stable environment where they get little attention from parents is not superior.
