The M&P hands down! Ergonomics, a true combat gun (no external safety's) fit function the .40 cal M&P has about the same recoil as most 9 mm, they are really great guns! Hope this helps! and good luck with your quest!
I carried an XD (not XDM) .40 for 3 years. I put a few thousand rounds through it and was very proficient with it. I picked up my buddies M&P .40 one day and shot a box of 50 through it. I now own 0 XD's and 4 M&P's. Never went back to the XD after shooting his.
Ergonomically it is the shnit. Fits like a glove and feels like it belongs in your hand. I was never truly comfortable with the XD, but shot it enough that I became adapt to it. The M&P recoils straight back as to where the XD seems to try and jump out of your hand. I dont know why, but the upward recoil of the XD was far more than that of the M&P.
Most of the EDC guns that I own are in .40. I like the caliber, I have fine tuned a lot of carry/competition/plinking loads in this caliber and have found that the M&P will handle them the best. Even the hotter major power factor USPSA loads shoot smoother through it.
All I can offer you is what I have come across. Yes, the triggers are garbage and it takes an $89 Apex kit to fix them but it is worth it to me. If you leave the stock trigger setup in there it is something that you will be able to become accustom to with minimal range time. I just like to tinker with everything I own, so I picked up a few kits for them.