From what I've heard, the bagpipe is the most difficult... I'm not sure why though lol.
But out of everything I've played, the hardest is probably voice. While organ is insanely difficult, it's just a matter of doing the right thing at the right time while doing a bunch of other things at once, but with voice you have to get all of your diction right (I had to basically re-learn how to speak english to sing :P), keep yourself resonating on the shelf no matter what, always keep the perfect length of tongue sticking out even though it changes for every note, change your lips/teeth brightness ratio, chest vibrating, diapragm supporting, pitch, and all that crazy stuff.
Not to mention you can't even SEE 95% of the instrument that you're working with. :lol: