Yeah I know what you mean. In my experience their usability is very context-dependent - you just have to try and see if they work and adjust as needed.
For example, I use Spitfire Symphonic Orchestra for most of the stuff I make. They tend to be pretty good for simpler stuff and passages that are scalar in nature, but as soon as you try to use them for arpeggio's and lines with alot of skips, they quickly start to get a little messy - the solo trumpet and some of the woodwinds are particularly bad. I honestly find myself using the "long" patches on the woodwinds more often than not, since they tend to sound more legato than the actual "legato" patches lol.
FWIW, I recently picked up VSL's Synchron Strings and Synchron Brass. It seems to me like they took kind of a hybrid approach with these libraries, where they have lyrical legato patches that use real samples, while they also have "fast" legato patches that seem to be more simulated. It may not be a "true" legato, but since you're using it for fast scales or arpeggio's you can't really tell anyway.