I will say that Japanese specifically feels like there is a large gap between the vocabulary required to understand dialogue which includes both dialogue in books/manga/games/movies and actual conversation and that required to understand general prose (descriptive etc.) which is mostly required for reading books, but the relevant words can of course pop up in other contexts as well.
In addition there is a large gap between being able to have comfortable conversations about most topics (potentially by just talking around words you don’t know) and knowing the specific vocabulary required for understanding every line of dialogue in a random fictional work or indeed any random sentence someone might throw at you.
This is of course true for most languages, but I feel like you can get away with knowing less Japanese vocabulary and still have a relatively high-level conversation compared to English, but maybe I just don’t remember what it was like to know less English and at the time I started learning English I was much less likely to speak with native speakers so I probably cannot compare fairly. I might also be misremembering how difficult topics I could actually discuss when I was worse at Japanese, mixing it up with later experiences.
In this sense I understand why some people want to focus on getting good at communicating quickly, especially when living in Japan (certainly my main focus at first was simply becoming able to communicate, although studying the written language is still very useful for this in my experience). But it can absolutely give a false sense of security, where you think you are much better at Japanese than you are and when you open a book or a newspaper you get a serious reality check.
I don’t think that it is ever a “waste” to learn more specific or uncommon vocabulary, however, as it can still come up in conversation and it will definitely pop up in other contexts if you end up trying to read stuff. Sometimes “strange” fantasy and sci-fi terminology can have slightly different meanings in other contexts, but familiarity with the words are still helpful when learning the latter. Weird phrases/talking patterns you might see in some manga can absolutely come up in casual conversation if simply in a joking manner. Overall, I’ve always thought the old internet advice of “never learn Japanese from manga/anime” was rather silly, it will teach you plenty of stuff and a bit of common sense and a few months interacting with natives will quickly allow you to see which parts are weird in daily conversation. And of course you need to know this stuff anyways if one of your reasons for studying Japanese is wanting to read manga.
Also the more Japanese vocabulary you yourself use, the more your conversation partner is liable to use. I definitely get the feeling that people have started using more complicated words with me over time, simply because I myself use a wider vocabulary and they trust I can understand more. In this sense part of the gap I was mentioning may also be related to Japanese people adjusting their level of conversation downwards when speaking with someone who they sense have a more shallow vocabulary. Not always of course! I have definitely talked with some people who were quite hard to understand due to using relatively obscure (at least to me at the time) vocabulary.
However, the gap between dialogue and general text is definitely real, it is so much easier to understand 90% of a random Japanese movie/anime set in modern day Japan (as long as they don’t go crazy with yakuza slang or whatever) than any moderately complex written text.
