I was up at Skaha a couple of weekends ago and had the exact opposite experience. There was hardly anyone there, we had access to any route we wanted, and had a fantastic time.
It's unfortunate that you were up there on a holiday, because when that place is more chill, it is a very peaceful place to climb. Even on a busy day, you can get away from the crowds by hiking to the areas that are about and hour from the parking lot. The best climbs are back there too!
I hope you give it another chance, because I think it is a great climbing area. By the way, the guide book (which is six years old) indicates that at least 300 routes require at least some gear placements. Some of the walls that are further from the lot are all trad.
Did you notice that there was poison ivy EVERYWHERE? I have never seen so much in one area.
At least you enjoyed a nice