3

Deploying Files To Multiple Servers

view full story
linux-howto

http://serverfault.com – We have a central repository of files in say server-1 at /srv/www. Then we have N number of servers as well. We want server-1 to be able to deploy its files in /srv/www to all servers as quickly and efficiently as possible. Is there something like rsync, but instead of specifying a single target, specifying a cluster (N servers) of targets? I was thinking git may work, but can we specify multiple remotes to push too? What is the best solution, assuming N servers may eventually reach hundreds. (HowTos)

taskli's picture
Submitted by taskli on 02/11/2012 – Made popular on 02/11/2012

Stories similar to Deploying Files To Multiple Servers

first thing: Sorry for my bad english ;)

I have to handle many (1.000 - 30.000) big files with a size from 200mb up to 2gb. The demand of this files is diffrent (0 - 300 downloads/file).

I'm hoping for a high level discussion of this problem I know is quickly approaching my application.

I have a server that binds on a specific port and manages TCP Sockets from my clients.

Sorry for the long message, I wanted to be as clear as possible. I have to add hundreds of new servers to hundreds of configuration files in Nagios.

I'm currently managing a cluster of PHP-FPM servers, all of which tend to get out of sync with each other.

If I have two web servers, what would the best way to keep their files in sync?

Unfortunately the sites themselves cannot be altered significantly. Most of them are Wordpress based, but there are over 200 of them. There are also some custom sites.

This setup is going from Web server + DB Server to Varnish load balancer + 2x web servers + DB Server.

We have a number of Database severs running MongoDB on Debian plus a number of Application servers also on Debian. The db servers hold replicating db clusters, so they need to talk to each other. Application servers need to talk to all db servers (for reasons of fault tolerance). The servers are potentially spread across multiple hosting centers, so we need secure channels between all servers.

I will be providing "File-Hosting" service and I need something that works with me along the line and allow the project to expand. At first I was thinking about one big server with RAID 5 as a start-up, but then I thought using multiple servers would be better as it's easier to expand in the future and should be cheaper, but I have absolutely no idea how these servers should/can be linked and if

In a lot of documentation I've seen recently, servers are referred to by internal hostnames, such as production-1, production-2, db-1.

I have several web application servers that i'd like to convert to using https instead of http. My organization has a purchased wildcard certificate that I have asked to use.