It's not a perfect solution (there may be a better server-side solution), but if you use something like
IPCop that uses FTP Connection Tracking, I believe the router will actually re-write the address in the packet. Since switching to that, I've never had a problem (although, sadly, I don't remember how I specifically solved the problem (though, it may have been solved by switching to IPCop).