I've been shooting APSC sensor Nikons since 2009. First a D60 then a D5300. Both were great cameras, but I finally invested in a D750 this winter and it is a big step up. Still just scratching the surface of its capabilities. Having both side by side, I can say if you can afford the investment, you won't miss the crop sensor. I have barely touched my crop sensor cameras since getting the full frame beast.
Also consider glass... do you have a lot invested in DX glass? A big consideration for me to go full frame was that I wanted to invest in real quality glass down the road, and I figured that if I was really going to go for it with glass I'd better go with full frame glass. Of course, if you're going with full frame glass you'd better get the camera that matches!
All that being said, for backpacking the little mirrorless crop sensor cameras are AWESOME. Excellent image quality and so much less weight and size. So, that's another option to consider.