The written word has long been my favorite medium for storytelling specifically because of how interpretive it is. When reading I can change the style I use to visualize things and it can completely change the experience of a book. Once you see an adaptation of a book that is lost, or at least heavily damaged. Having literally scene what the characters look like and how scenes play out I find myself having to actively avoid falling back on the movie or comic or whatever for my visualization, even when they go expressly against the text. For example, here's a bunch of artwork for The Hobbit that predates any movie adaption. There were so many interpretations of things like Gollum before the movies, and now there's only one. The recent Gollum game has (among many far more valid criticisms) been widely criticized for changing Gollum's face from what the movie went with. Only the face, the rest is just a slight variation of the movie design. In addition, the nature of writing leads it to being the most detailed medium of storytelling by far. You can look into peoples heads without breaking the flow, showing an inner monologue in a movie would be criticized for telling not showing; you can explain action with all the wonderful flavor of English connotation; the difference between lumbering or striding or marching to describe somebody walking for example (this is also a very interpretive part of writing, as connotation is naturally subjective); you can go into absurd world building detail, as seen with The Silmarillion and the Lord of the Rings appendices. All of these aspects are lost when you turn a book into a movie. This isn't to disparage other mediums for storytelling, only to enumerate some of the strengths of writing and explain why it is my favorite. Some of these strengths are even shared with other forms, but never all. For this reason I am pretty against adapting books, and adaptations in general. I feel like stories should be left in their original medium, you can make connected stories across mediums, but again you should perhaps not; a movie can be a squeal to a book, but again the visualization aspect will be hurt by showing exactly how the characters look. I want to hear more thoughts and perspectives about adaptions, though I don't want this to devolve into arguing about how faithful adaptions should be and which adaptions are best; I've had that conversation plenty of times.
I don't personally have a favorite medium for story telling, I think there are great and terrible examples for all of them. I do find myself becoming more easily emotionally attached to writing, however. I'm not sure why, but I think it's probably because having a visual lets me detach from it and add a degree of separation between me and the characters. I wouldn't say writing is necessarily "better" than a movie, although I don't get the sense that's what you're trying to say, I think that certain methods of story telling suit one format better and more easily than the other.
I have to say I'm iffy on adaptations. On one hand, It allows someone to see a very fleshed out version of someone else's interpretation. It's also condenses the book into a more digestible package for most, especially if the book is a difficult read or not one that the moviegoer may otherwise read. And generally, they get the main plot points correct. On the other hand, many details and nuances are left out entirely. Not mention that movies aren't just one person, they're essentially a game of telephone when it comes to communicating the interpretation of a book.
While adaptations do pigeon-hole people into seeing only one interpretation and only seeing it as "correct," I've never really had a problem with it. The way I see it, adaptations are very much for some people and not others. I just don't view others' adaptations to not influence myself too much. Or I'll view as many as possible to "give myself options" as far as visualizing text goes. And as far as influencing other adaptations or expansions of the story, such as the Gollum game example you gave, I view it as two different versions of the same story. I know the OG book version and it's description, and other's know the movie one. Like if I read one translation of a book while most people read a version with a different translator.
TL;DR: They're not my cup of tea, and I don't think they should stop being made, mostly because I don't consider an adaptation truly the same story as the source.
In my childhood I had similar feelings as OP, I even could not stand illustrations in books very much, because in my opinion they were always wrong, that is different from my personal interpretation.
But as I grew older, I began to see adaptations (or at least the good ones among them) more as works in their own right, inspired by but still fundamentally different from the adapted work. Although, of course, I must admit that there are genres that are clearly less suited to adaptation than others.
The basic problem, therefore, in my eyes, is much more the modern film industry, (Yes, exactly -industry!) whose goal is not to create a work of art, but merely mindless entertainment. Of course, nothing of quality will come from them.