terminal connection from a different local IP other than the primary IP

view story

http://serverfault.com – I have ~20 public IP addresses defined on my server. Now, when I connect to a remote server via terminal, it gets connected from the primary IP of my server. But I want to use terminal from my another IP address. Is this possible? (HowTos)