I'd have to disagree with you on that. For me personally, I've been exposed to a lot of modern music, especially not that I'm a conservatory student. And while I am by no means an expert, I wouldn't say it is my ignorance that causes me to dislike most modernist/avant garde styles; it's just the simple fact that I don't like them, for various reasons. A few of those reasons; I don't understand them, not from a theory perspective (although that is also true at times) but from a "logical listening" one. The "language" of modern music to me seems like gibberish. While a fan of modern music might say that such and such a piece sound beautiful, to me it is complete randomness, that for me seems to have no sense of logic or direction.
I don't think it is my lack of knowledge that makes me dislike modern music, but my experience with it. For the most part, the modern music I've heard sounds like madness. There are exceptions of course, but on the whole it doesn't "click" with me.
I think it's a valid point. I don't think people are presuming anything as you say. They listen to the music, and based on what they here make a guess on what could have inspired such music. As I said, for me the music sounds random and illogical, so I wonder whether the composer just wrote down random scribblings. Someone else might find the work to be logical, and I would probably ask them to explain, because truthfully I just simply don't understand a lot of modern music, in the sense that I don't understand how anyone could like that, or want to WRITE like that. I don't mean that offensively, I just mean that it's hard for me to see, or rather hear, the same things that modernist fans hear.
A question I want to pose to you: I've never heard anyone say that an early Classical work seems illogical, or random (i.e. Haydn, Mozart, or farther back to Salieri, Vivaldi, etc.). However, we hear it all the time about new works. So is it because we haven't yet become adapted to the newer styles, or is it because the styles themselves are inherently "random"?
Personally I would say that it is because modern Classical music, or more precisely the method of writing it, has shifted from putting the listener at the center to the composer. I only mean this somewhat literally; early music was built on a system of consonance over dissonance, tonality, and especially form. All these things are clearly recognizable, even to those who only have a minute knowledge of Classical music. However, newer music TENDS to focus on things that we hardly notice in music, and in some cases are impossible to notice, such as pieces written using a particular mathematical formula. Also, today the idea of "dissonance" among modernists has been obliterated. Everything is treated as a consonance, even such "obvious" dissonances as the m2.