1972 Roamer 32v electrical issues

I have a 1972 55 foot Chris-Craft aluminum hull Roamer that is having 32 V electrical issues. All of the sudden my 12 V instruments (newer Garmin displays, ship to shore radio and stereo system) will shut down and turn off and at the same time my 32 Volt house system immediately drops to around 12 volts so all 32 V lights will dim and heads barely flush. I will then go to engine room and shut off my 12 V battery positive switch - then immediately see the 32 V System restored to see all lights and heads back to working. This happens several times a day . I have two banks of 8 volt batteries. Four batteries hooked on port side and four on starboard side.
Batteries are two years old maintain well checking water and load test check out fine. Same with 12 V battery system however I replace those again this year. I’ve checked wires and cleaned all battery terminal connections, Engine block ground wires and the connections on my 12 volt rectifier/charger. Has anyone ever come across this issue or have suggestions on what I should check for?

Thanks for your question Capt Ralph, I appreciate any advice you could offer me. Yes the vessel has separate 12 and 32 volt banks of batteries and all share the same common ground to the Detroit diesel engine block. The 32v Does have a 12 v converter in the line as well and I am not sure how that works, but appears to be in the same line as my 12 V batteries. The vessel has 32 V and 12 V charger when I’m plugged in at the dock. And while underway the engines alternators charge the 32 V batteries. The 12 V batteries also start my generator and the generator charges them underway. Then the same thing happens where the 32 V house lights go dim and 32 V heads barely flush and the only way to reset is to turn off my 12 V battery switch to the off position and that action puts the 32 V back to full power and lights are bright again. This situation happens about three times a day sitting plugged in at the dock. And when underway the same situation happens after about two hours.