AMD Demos Live Migration Across Three Opterons

bigwophh writes "Advanced Micro Devices has just revealed to the public the first video and images demonstrating live migration across three generations of AMD Opteron processors on VMware ESX 3.5, including the six-core AMD Opteron processor, often referred to as 'Istanbul.' For those unaware of the strains in a server environment, live migration of virtual machines across physical servers is crucial to providing flexibility for managing data centers. AMD is also taking this opportunity to highlight its continued, cooperative development efforts with Microsoft as evidenced in Windows Server 2008 R2 Hyper-V, which just also happens to be available today in beta form, that adds support for AMD-V technology with Rapid Virtualization Indexing."

Nothing new?

[morgan_greywolf]

VMWare ESX has been able to do live migrations for a while now. I'm not sure what makes the Opteron special in this regard.

Re:

[Enry]

Xen supports live migrations too. Not sure what the big deal is.

Re:Nothing new?

[rachit]

What is new is live migration across different CPU generations.

I believe its using some features in the newer CPUs to turn off certain features / CPUID bits such that all CPUs look the same to the guest OS / applications running on top of it.

Re:

[Anonymous Coward]

No, the CPU features (the CPUID bits) are masked out with ESX, your can fine-tune this in the settings of every VM.

Re:Nothing new?

[Domint]

The crucial part is the 'across three generations' bit - I can tell you from first hand experience that VMWare ESX has problems performing live migrations across CPUs with different steppings even within the same generation, so the fact that AMD pulled it off across 3 distinct generations without having to utilize cobbed-together solutions like EVC is a pretty big deal. Or, at least it is to me.

Re:

[morgan_greywolf]

without having to utilize cobbed-together solutions like EVC

I was just about to say 'EVC' when I got that part of your post. The thing is if you are already running EVC, then it doesn't matter much. The hard part of EVC is getting the first couple of hosts running. Once that's going, EVC is cake.

Re:

[Natales]

It's not that ESX has issues. It's that the Guest OS is flat out not designed for "something" to change your CPU on the fly, which is the way the guest sees it... In fact we (yes, I work for VMware) used to allow that back in the ESX 2.0 days, but the guest bluescreened/panicked after a while as the microcode is loaded at boot time. The only way we can pull these tricks today is by masking some specific instructions of newer chip generations vs. the previous ones. Hard, but not rocket science. Get me a gues

Re:

[morgan_greywolf]

Get me a guest that adapt to those changes live and you'll have what you want...

Generally speaking, Linux distros will move between different processors as long as they are the same generation...x86-64 will move to other x86-64s, for instance.

Re:

[WaxMan0]

Pretty certian the is marketing bumpf, Is this not what the AMD-v and intels equivilent (cant remember of the op of my head) allow. Realy wish i kept my sources to refernace but from what i remember the different levels of virtuilisation include emulation where the complete pc is emulated and the OS sees this as a native hardware this is the Slow method and used by older desktop virtual pc applications. Another 2 one of which I have completely forgotten the other uses a set of cpu instructions to allow t

Yay?

[eln]

They can call me when they've demonstrated seamless live migration between Intel and AMD chips, not just generations of their own hardware. Nobody wants to build a large-scale cloud if they're going to be locked to one vendor forever once they get started.

Re:Yay?

[CannonballHead]

Yeah, what's with the wanting to show how their hardware is better than Intel's, anyways.

Re:

[Phantom_Linux]

Didn't they do that already?

http://www.youtube.com/watch?v=EuhU6jJjpAQ [youtube.com]

Re:

[Agent ME]

I thought KVM or qemu did this?

Re:What about networks?

[Chris Burke]

I was curious how they migrate active network connections though. Does the old host act as a proxy/router? Can anyone shed some light?

Its been around 9 years since I did the project in college, but it is possible to transfer active network connection state from one computer to another. It was a "connection-aware seamless backup server", where our hacked linux kernels would exchange state about an active TCP/IP connection at regular intervals, and when the primary dropped (in our demo we yanked the ethernet cable out of the hub in the middle of streaming an mp3), the other would take over, pretending to be the same IP and picking up where the other left off. The best part was that TCP/IP already deals with redundant, missing, or out of order packets so anything sent or received since the last update to the backup would be handled automagically.

That was just as stupid semester project on a 3-computer ethernet LAN, but I imagine the big boys have figured out how to make it work. Besides, they're literally transfering the memory image of the guest OS over to the other machine so all the state update is already done. The hard part is probably making the IP migrate along with, but I'm sure they've figured that out too.

Re:What about networks?

[cowbutt]

That was just as stupid semester project on a 3-computer ethernet LAN, but I imagine the big boys have figured out how to make it work

Checkpoint's Firewall-1 product has been able to transfer the current firewall state to a backup firewall for quite some time (at least 1998 or so), but it originally required the use of RIP to implement failover which introduced delays. In about 1999, Stonesoft produced an add-on product, Stonebeat, which added ARP spoofing, so the failover was virtually instantaneous. These days, the failover is done using VSRP.

Re:

[omfglearntoplay]

Supposedly Double-Take software is good at "mirroring" one box with another... and then when box1 dies box2 very quickly takes over, stealing the IP, etc. I have the software, but I haven't had the balls or the weekend time to test it for real yet.

Re:

[Domint]

Nothing that complex. It simply passes the TCP state from one nic to the other. I don't know the gory details of how that all happens behind the scenes, but it's pretty impressive to watch it happen - in our lab at work we tested this functionality on multiple guest OSes simultaneously and didn't lose a single packet, even when pulling the power chords out of a number of components (including one of the ESX servers).

Re:

[chrome]

as far as the nics are concerned, its all just ethernet packets. There is no state to handle. The state is maintained by the guest OS, and is transferred when the OS memory is transferred.