Yeah, compared to my contrived example, U and V were usually differentiated for typographical reasons and not perfectly interchangeable (kind of like the S and the "long S" that looks like a lowercase "f", e.g. in the phrase "Congrefs of the United States" in the original Bill of Rights).
Another weird gray area between ligature and letter are the "æ" and "œ" characters, which are mainly used as ligatures for the British spellings of "foetus" and "encyclopaedia" but were, at some times, seen as actual letters. American spelling reforms erased this particular weirdness from our dialect.
I liked the story of young web developer who suggested a new glyph for 'Th' since it was clearly missing, blizzfully unaware of the letter Thor that had been used in English until printing became available.
Ðere was also ðe letter "ðæt" ("that"), a later invention ðat began as a stylistic alternative to þorn in Old English manuscripts, but fell out of favour early in þe Middle English period (during þe 1200s or so).
Þorn continued to be used to a limited extent in early printed works of English. The <th> digraph started to supplant it already in manuscripts during the Middle English period, with <þ> being retained only in limited contexts well before the introduction of the press in Europe (contexts being primarily common sigla (scribal abbreviations) like <þᵉ> for "the" and <þᵗ> for "that", and occasionally when the scribe felt like they might run out of room on the line).
Another weird gray area between ligature and letter are the "æ" and "œ" characters, which are mainly used as ligatures for the British spellings of "foetus" and "encyclopaedia" but were, at some times, seen as actual letters. American spelling reforms erased this particular weirdness from our dialect.