Ideally you would not think either of those things, you would know the chord/scale relations to the point where you no longer have to do the arithmetical operations in your head and just intuitively play in a musical way. When it comes to improvisation you need to have all that stuff known on a sub-conscious level. Keep practicing and if you are diligent you will be able to be able to internalize a lot more than hitting basic chord tones. Force yourself to play in unfamiliar keys until they become memorized.