I've found that having the brakes on makes it quite a bit more difficult to rotate the turret between touring modes easily on the fly (i.e. with ski-pole, not stooping down). I think it both adds friction to the system and provides more bits to get packed with wet snow.
I started with the brakes becuase I wanted them to be able to do double duty in-bounds, but have since bought a new alpine set-up so will probably take them off and shave a few ounces. I think from a avvy safety perspective, brakes are safer since you would get free from the skiis making for easier swimming to stay afloat.
Conversely, leashes would prevent you from losing a ski completely on a steep descent.