Anthony,
In my humble opinion, I think you should have it redone. For various reasons, rivets derive strength by holding two parts (bracket and mast) in direct contact around the rivet. As with any fastener, PROPERLY DONE, friction between the parts takes some shear load off the fastener, the parts cannot flex and cause tension variation, fastener cannot bend, and neither part can easily yield and rupture at the hole. This is not true if the parts are separated. Also when properly done, compression load on the bracket is distributed on the surface of the bracket and mast underneath. Whatever is holding the bracket off the mast (its edges, or the rivets themselves) is too concentrated to be a good thing. It sounds like just a poor riveting job, easily redone.
Good luck.
