I then tried to put a PWM stage with the output from my iPod and the buffer stage. In the process of doing that I managed to knock some critical wires loose and spent the next few hours trying to work out what had changed to stop it working. During this process I realized that even with all of the capacitors in my collection in use there was still ripple on the power rails in time with the bottom of the sawtooth. The other thing I realized that to get the PWM working with Audio frequencies I'm going to need to up the frequency.
To solve the first problem I'm going to go into Jaycar tomorrow and pick up some LM7905's if they have them and possibly some generic BJT's in case I need to make a constant current supply for the Sawtooth oscillator.
To solve the second problem I have just substituted a smaller capacitor in the oscillator. This has definetly increased the frequency by an order of magnitude; so hopefully that problem has been solved.
If anyone wants some diagrams or pictures of my setup let me know in the comments and I'll put some up.