Your vibration could be from the rudder, or secondarily transmitted from vibration elsewhere.
For the first, check your rudder profile. Is it smooth or is there a reason to assume the rudder might create turbulence? If it does, that can set up vibrations.
Next, same for the CB. On a foil, it's the trailing edge that sheds vortexes and causes vibrations. I cleaned up my CB and never had vibration issues since.
I used 3M High Strength Marine Filler to add material to both leading and trailing edges.

Minimal, quick fix to the original "profile" on my CB. The edges were pretty banged up, and instead of just smoothing them, I added a bit more material as indicated. The CB has stopped fluttering/humming, still fits into the trunk and doesn't exceed max width under class rules (although, if I wanted to race under those rules, I'd buy a "real" CB to be competitive).
There should be a thread started around 2011 where I describe the process in detail.