Leaving the CB up will in fact reduce your risk of capsizing.
Here is a diagram:
[click to enlarge]As the wind pushes on your sail, the CB resists the hull drifting to leeward. Your weight helps the boat stay level, but if the wind is too strong, the boat can "trip" over the CB. With the CB raised, the boat would simply slide to leeward.
That's half the story.
The other half is that the rudder needs the CB as a "pivot" point around which to turn the boat. With the CB up, there's no longer that central pivot and the bow will freely slide downwind.
Now, going downwind, you don't use the CB (even when it is working properly). You want the bow to point downwind, after all, and for small angles to either side of downwind, the immersed part of the hull can make up for that missing pivot. If you leave the CB down, it will slow you a bit (you might not care if you don't race) but it will also be a bit of a trip hazard, should a wind-gust come unexpectedly more from the side, but in lighter winds that would be secondary.
However, most importantly, without using your CB you are unlikely to be able to sail across the wind or upwind at all.
In fact, if you keep your boat on a dock or mooring, leaving the CB fixed in the down position would be the most versatile (except that it's not that good for the CB to be down when not in use).