You don't need to reverse the 1646 output. Only the "From Tape" needs to be reversed to account for it being inverted on the way out to tape.
Otherwise, I don't see where you could get dry / wet signal bleeding in. In the extreme positions, the wiper should completely shunt that signal.
It could be that the pot is crappy and the wiper doesn't really make full contact with the legs. If you measure the resistance with a meter between the wiper and each leg of the pot in the extreme positions, do you get zero (a few tenths of an ohm is not abnormal)?
Or maybe your ground isn't quite right. Did you use a proper bipolar supply or a virtual ground? If you measure the resistance between the wiper of the feedback pot and the ground on the summing op amp (IC1D) do you get zero? If you measure the resistance between the wiper and the ground at the power supply, do you get zero?
Or maybe the dry signal is coming from the tape machine. Meaning it's somehow mixing in some dry. Try listening to the output of the tape machine directly.
As for the "phasy" sound, it could be that the tape is just running too fast. At 15 IPS, it would sound doubled. It needs to be at more like 3.75 IPS to get more of a traditional delay / echo.