The problem with reading books is applying what you learn, and therein lies the difficulty. I have gotten hold of many books on harmony and counterpoint as well as form. But when I come to composing I don't think in terms of the rules I have learned but in terms of what sounds good and what my inner inspirational voice dictates. The problem is in applying what you read from books. How do you do that? When you are inspired to compose a piece, you cannot stop the process to go and look at books for the rules. And when you are reading the books and learning the rules, you can't force yourself to compose according to them. Maybe you have to do exercises. And maybe you have to somehow learn WHILE composing (maybe by getting feedback) and improving upon your previous work in your next.
Regarding avoiding formal training, I don't avoid it because it's hard or boring. Rather, I like to keep composition at the level of an enjoyable hobby and not transform it into work or a job which would make me possibly dread it. Other than that, I also am afraid that formal training would dry up inspiration, originality, and my unique voice and cause me to compose pieces in a boring and academic style.