Here is a trick that may work. Pull the backstay down and as you do, put the tip of the tiller inside the triangle and pull the backstay down a bit more. This will hold the tiller dead straight.
Some people use this technique to keep the tiller from banging around when they are on a mooring.
