How can I compress a compressed file?

view story

http://unix.stackexchange.com – I have access to a server via SSH, I download files from it over SFTP or HTTP. I want to pull down a 4.4GB .mkv file but I have limited bandwidth. I have used zip, gzip and p7zip, but the file is always 4.4GBs after the compression process. After fiddling with some of the CLI args I managed on one attempt to get the file down to 4.3GBs. I understand that the video file I have is a compressed h.264 file. I don't understand why I can't compress this further though (perhaps a question for another Stack-exchange forum) Is there some way I can compress at a byte level so the compression tool woul (HowTos)