1. if motor starts in 4 seconds with one battery you are great.
2. if it takes two batteries to start in four seconds ... that is a sign that something is wrong or about to get worse.
3. with batteries every time you flip a switch the voltage will drop ... depends on how much you are drawing how much the voltage will drop. I have yet to get the same voltage on the master fuse panel that is available at the battery ... do this test if you have a voltage readout on your gps ... see if it comes anywhere near the voltage you read at the battery.
4. on larger boats they often have a bypass pushbutton switch in parallel to the starter button to basically do what you are doing with a screwdriver. That is generally near the motor access so you can easily start the motor at the motor. The ignition still must be turned on before you start.
5. I am waiting for this winter for Bill to give me an analysis on Vee Jay 505's original starter ... that Hitachi starter took me though all the steps last winter and finally was resolved with a cheap chinese replacement starter via Ebay (new). Now one season old and doing fine. Probably not "marine" grade ... but cheap.
6. anything with an in line glass fuse is a weak link.
7. Analog voltage meters are near worthless on a boat.
8. Standard marine ignition switch works fine. no special Yanmar wizardry in that piece except that it mechanically fits into the hole better. and you get a real key with the non Yanmar switch.