What if you only had arms? Then everything would be climbing to some extent.
Climbing is expressed in two methods; first, to define what one does when one explains it to another individual.
Second; to use a single word to sum up an entire adventure, i.e.-"I went climbing this weekend"
Although skiing can be climbing, it really isn't because when you sum it up in one sentence it becomes, "I went skiing". Without the skis it would normally be considered climbing.
I think it relates to the method of communication from the individual and how the individual tries to relate the experience to another individual.
Bushwhacking for a flyfisherman is still flyfishing.
Bushwhacking for a kayaker is still kayaking.