My kids are on the cusp of leaving the nest. Your post takes me back...
My advice is simple: separate your workout/fun desires from activities with your kids and you'll be more likely to make everyone happy, including yourself. So get fit enough to get out with your friends once or twice a month. It'll be good for your body and soul. Cover parenting so your partner can have a similar opportunity. And do family-focused outdoor fun whenever you can. Just being in nature is good for everyone, even small kids. For family fun, keep in mind that often staying close to the trailhead is easier. You and your kid can have more fun playing in the snow bank across from the Alpental parking lot or near Gold Creek Pond than walking two miles in a crowded trail to Source Lake basin.
If kids have a great time doing stuff outdoors with you they might be more inclined to keep doing it - with or without you - as they grow older.
Be kind to others. Be kind to yourself. Have fun!