1

How long does it take in practice to warm up large in-memory databases?

view story
linux-howto

http://serverfault.com – Companies such as Peak Hosting are offering 64 core machines with 512Gb RAM for $2K/month. This is a very interesting choice for in-memory databases such as Memcached/Redis as well as databases whose performance degrades rapidly when the data & indexes don't fit in RAM, such as MongoDB. My main concern with monster machines such as these is the time it takes to warm up an in-memory database. In my experience, theoretical metrics, e.g., that SATA can load 100Mb/sec, fall short of what happens in practice. Even at that rate, 100Mb/sec means that loading up 512Gb RAM machine from SATA disks (HowTos)

dayu's picture
Submitted by dayu on 10/21/2012

Stories similar to How long does it take in practice to warm up large in-memory databases?

I have a Redis server running version 2.4.5 and with a dump.rdb of 11GB loaded into memory. It is running on EC2 on a high memory 4x extra large instance (70GB total memory). However, turns out Redis is already taking up 50GB of memory and is just growing more and more. My dataset is still gonna grow larger, probably to around 20GB, so clearly 70GB memory wont be enough.

We have recently received a new Dell PowerEdge R820 system. It comes with 4 CPU sockets and 512GB of memory.

In-Memory DBMS 48 weeks 4 days ago

Thanks to powerful hardware, in-memory databases run without accessing mass memory devices, which means they handle transactions and evaluations at high speed, introducing a paradigm shift in the database market.

I have a SQL Server 2008 R2 Express instance that runs 3 databases.

Two of the databases are for a program I wrote, the third one is for an Oracle program.

My databases are approximately 100mb, and the Oracle database fluctuates in size between 2-3 GB.

Within the last day or so, the Oracle database intermittently drops into recovery mode, while the other databases on the server continue running

I am using XenServer with several virtual machines having local postgres databases.

Shared Hosting using Linux 1 year 24 weeks ago

I am planning to set up a server hosting multiple websites on a single high-end Linux server on Virtualization-enabled Hardware using a LAMP stack.

I have decided to go with Centos 6.2 as a KVM host with two Virtual Machines running on top of it, of similar Centos Version. One to act a Web Server and other act as a MySQL database server.

I was thinking of turning this on some large databases not just mail files - we have around 8 - 10GB of large databases as well as small databases of couple of hundred MB in size.

But after reading this post I am not too sure:

http://www-10.lotus.com/ldd/nd85forum.nsf/4b9931b774db788c85256bf0006b5e...

Can anyone confirm whether this is true?

I have two databases. First one is local and the other is at Internet, in my domain. I want to synchronous two databases. Is it possible to do so ?

This is for a SLES 10 database server.

One of my servers runs three databases and three app servers; I am analyzing how their shared memory segments grow and shrink to avoid intermittent out-of-memory scenarios.