NLP via word associations might get you closer to capturing meaning in a Wittgensteinian conception than other methods (I happen to think the PI should be required reading for people studying NLP). But he would probably say the inputs into an NLP model would need to capture a much wider range of human culture and behavior beyond co-occurrences and symbols.
And even if you had a very complete model - for him "meaning" something through an utterance was likely something inextricably linked with being human engaged in social acts. (See: the private language argument https://en.wikipedia.org/wiki/Private_language_argument)
One thing I learned from reading Wittgenstein is that language is far more than speech and words. If you want to understand language, or rather "communication" in a Wittgensteinian-way, I think you'd have better luck analysing YouTube videos, where body language, environment, context, intonation, prosody, implicit emotion, etc is far more visible, rather than just analysing text.
It's such a weird, but somewhat freeing perspective on language, where I once thought that the for instance the whistle language is absolutely fascinating and perplexing, it just makes perfect normal sense to me now. As long as you can recognize the human through the medium, there is room for language, and that's so cool! There's so much room for playing with communication and interaction I never thought about thanks to his writing, it also really makes me understand Jazz as a "language" way better, I never understood the almost pretentious way people talked about Jazz, "it's different, hard to explain if you never experienced it for yourself!", it really, truly is a conversation in music, super interesting.
> he would probably say the inputs into an NLP model would need to capture a much wider range of human culture and behavior
I get what you're saying - that something deserving the name "language" way outstrips the simple word embedding strategies in TFA, right? and I wholeheartedly agree - but my point is that the position of PI, at least as I read it, is fundamentally opposed to the idea that modeling - what he calls "a mental mechanism" - is how language proceeds. This is one of the major points of divergence between PI and the Tractatus (which posits an explicit correspondence or modeling relation between language and the world.)
The most explicit formulation of this that comes to mind is in PI #689:
> "There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM."
> Certainly such a connexion exists. Only not as you imagine it: namely by means of a mental mechanism.
The point is very much that PI's emergent view of meaning is quite in tune with the connectionist sentiment that a bunch of simple elements (i.e. neurons, either in the brain or in some AI project) can exhibit the rule learning and rule following capacities at the centre of PI's treatment of language, without any model of the elements of language needing to be explicitly encoded.
The degree to which such a model may or may not exist, embedded within the parameterisation of a language-capable set of neural components, is an interesting question. But even granting its "existence" I think the point would be that the basic properties of the set of components, coupled with the training and samples provide the capability and the "model," if we want to grant its existence, is effectively a by-product - epiphenomenal if you like. :)
And even if you had a very complete model - for him "meaning" something through an utterance was likely something inextricably linked with being human engaged in social acts. (See: the private language argument https://en.wikipedia.org/wiki/Private_language_argument)