Hypervisor Brief Intro

View of Avi Kivity By Irfan Habib, http://www.linuxjournal.com/article/9764?page=0,1

In many ways, VMware is a ground-breaking technology. VMware manages to fully virtualize the notoriously complex x86 architecture using software techniques only, and to achieve very good performance and stability. As a result, VMware is a very large and complex piece of software. KVM, on the other hand, relies on the new hardware virtualization technologies that have appeared recently. As such, it is very small (about 10,000 lines) and relatively simple. Another big difference is that VMware is proprietary, while KVM is open source.

Xen is a fairly large project, providing both paravirtualization and full virtualization. It is designed as a standalone kernel, which only requires Linux to perform I/O. This makes it rather large, as it has its own scheduler, memory manager, timer handling and machine initialization.

KVM, in contrast, uses the standard Linux scheduler, memory management and other services. This allows the KVM developers to concentrate on virtualization, building on the core kernel instead of replacing it.

QEMU is a user-space emulator. It is a fairly amazing project, emulating a variety of guest processors on several host processors, with fairly decent performance. However, the user-space architecture does not allow it to approach native speeds without a kernel accelerator. KVM recognizes the utility of QEMU by using it for I/O hardware emulation. Although KVM is not tied to any particular user space, the QEMU code was too good not to use—so we used it.

Comments off

Decided to Switch KVM

Choosing a hypervisor in #Cloud projects always costs me time, between Xen & Kernel-based Virtual Machine (KVM). After an interesting discussion with one of Cloud Computing architect @ IBM, we get a deal – use KVM for future #Cloud implement. My considerations:

  1. IBM announced to boost KVM @ Cloud initiatives since March this year. This indicates IBM’s strategy @ hypervisor on Intel machines leans to KVM, rather than Xen. And more official development, test and support are coming after.
  2. Virtio project to support better network IO performance. http://kbase.redhat.com/faq/docs/DOC-25441

More resources:
KVM vs Xen, dates back in 2006, but telling some basic > http://www.gridvm.org/xen-vs-kvm.html
Xen vs KVM by Xen > http://blog.xen.org/index.php/2010/05/07/..kvm-linux-..community/
KVM @ IBM InfoCenter > http://publib.boulder.ibm.com/../kvminstall/liaaikvminstallstart.htm
Securing KVM @ IBM InfoCenter > http://publib.boulder.ibm.com/../kvmsec/kvmsecguest.htm
Benchmark by Xen > http://wiki.xensource.com/..Quantitative+Comparison+of+Xen+and+KVM.pdf
KVM vs VMware > http://www.workswithu.com/2009/04/27/kvm-vs-vmware-a-case-study/

They’re all @ http://delicious.com/j3ffyang/kvm

Notice: since I only consider #OpenSource virtualization solution @ my #Cloud projects on Intel, there is #proprietary #VMware in my solution design. But integration with VMware is always available.

Comments off

Enable VT @ BIOS

One of my customer asked me how to reuse the old fashioned PC in their Cloud. The question leads to “How to enable VT @ BIOS?”. I wrote the following message to my customer:

Go to BIOS setting, then “Enable” or “Turn On” “Virtualization Technology” like

Source & Credit: http://publib.boulder.ibm.com/../kvminstall/kvmadvancedsetup.jpg

In Linux, here are steps to enable VT

(source: http://www.redhat.com/..VT_in_BIOS.html)

Verify the virtualization extensions are enabled in BIOS. The BIOS settings for Intel® VT or AMD-V are usually in the Chipset or Processor menus. The menu names may vary from this guide, the virtualization extension settings may be found in Security Settings or other non standard menu names.

Procedure 34.1. Enabling virtualization extensions in BIOS

Reboot the computer and open the system’s BIOS menu. This can usually be done by pressing the delete key, the F1 key or Alt and F4 keys depending on the system.

Select Restore Defaults or Restore Optimized Defaults, and then select Save & Exit.

Power off the machine and disconnect the power supply.

Enabling the virtualization extensions in BIOS

Note: BIOS steps

Many of the steps below may vary depending on your motherboard, processor type, chipset and OEM. Refer to your system’s accompanying documentation for the correct information on configuring your system.

Power on the machine and open the BIOS (as per Step 1).

Open the Processor submenu The processor settings menu may be hidden in the Chipset, Advanced CPU Configuration or Northbridge.

Enable Intel Virtualization Technology (also known as Intel VT) or AMD-V depending on the brand of the processor. The virtualization extensions may be labeled Virtualization Extensions, Vanderpool or various other names depending on the OEM and system BIOS.

Enable Intel VTd or AMD IOMMU, if the options are available. Intel VTd and AMD IOMMU are used for PCI passthrough.

Select Save & Exit.

Power off the machine and disconnect the power supply.

Run cat /proc/cpuinfo | grep vmx svm. If the command outputs, the virtualization extensions are now enabled. If there is no output your system may not have the virtualization extensions or the correct BIOS setting enabled.

Comments off

Fix Virtualized Windows XP on VirtualBox 3.0 over Ubuntu 9.04

Configuration:

  • Host OS > Ubuntu 9.04
  • Guest OS > Windows XP
  • VirutalBox > 3.0.0

Symptom: Windows XP crashes very frequently and the dump shows

virtbox_crash3

As well as the message >

DRIVER_UNLOADED_WITHOUT_CANCELLING_PENDING_OPERATIONS

Solution:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Processor

Or

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Intelppm

And changing the ‘Start’ value to ’4′.

The detail @ http://blogs.msdn.com/virtual_pc…

Comments off

What does RackSpace offer?

Part of the following comes from: http://www.comparewebhosts.com/…htm

Rackspace’s cloud strategy is supported by three core offerings, all part of Mosso, Rackspace’s Cloud Hosting Division. The company’s existing Hosting Cloud and CloudFS storage offerings have been re-branded to reflect the company’s newly integrated approach:

· Cloud Sites — Rackspace’s flagship cloud offering, The Hosting Cloud, is now Cloud Sites. Developed by Mosso, Rackspace’s cloud division, it offers a scalable platform for handling huge traffic spikes and a pay-as-you-grow pricing model. Cloud Sites is a heterogeneous environment, supporting both Windows and Linux.

· Cloud Files — Rackspace’s internet-based storage service, CloudFS, is now Cloud Files. Cloud Files gives developers instant access to an enterprise-grade storage infrastructure and reduces overall investment and IT costs while providing infinite scalability. Cloud Files offers an industry leading SLA and a highly competitive pricing model with replicated storage starting at $0.15/GB. Also later this year, Limelight Networks will team with Rackspace to allow developers to easily distribute content to millions of end users around the world and bring scalable content delivery and application acceleration services to the masses. While continuing to support the Amazon S3 platform, Jungle Disk will port to Rackspace’s Cloud Files system in the coming months. Jungle Disk comes in both desktop and workgroup editions across the Windows, Mac and Linux platforms.

· Cloud Servers — This new hosting solution, which will deliver on-demand server capacity to businesses of all sizes, will leverage key technology developed by Slicehost, which uses Xen virtualization software. Slicehost will remain as the company’s developer brand, creating innovative new features driven through shared intellectual property in conjunction with development initiatives from Rackspace. As part of the announcement, Slicehost also announced new, larger slices for high performance computing, lower prices as well as IP sharing for high availability computing.

To understand Mosso > http://www.mosso.com

Mosso architectures > http://www.mosso.com/cloud.jsp > look at “Benefit” @ bottom left.

The selling point to our customer >
Step One: Load it up
You can create a new Mosso-powered site – and even add some email accounts – in less than five minutes. Load-Balancing, clustering, and redundant storage are all inherited by your application automatically, without any effort.

Step Two: Watch it scale
From the first byte served, your site is hosted on advanced clustered technology designed for high- traffic, high- performance websites. When your site grows bigger than what it’s included, you pay inexperience scale pricing for exactly what you use and nothing more.

  • Scale your bandwidth
  • Starts at 500GB/mo. As much as you need for 25¢ per GB
  • Scale your SAN storage
  • Starts at 50GB/mo. As much as you need for 50¢ per GB
  • Scale your compute
  • Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle

Closer watch with screenshots > http://www.mosso.com/screenshots.jsp > this describes the ease of process when a client requests a host.

=-=-

http://www.slicehost.com is equal to Cloud Servers. This pretty much looks like VPS – virtual private server on Xen.

  • Included in Every Slice™
  • Full root access and rebooting
  • Choice of Linux distro
  • Dedicated IP address and Tier-1 redundant bandwidth
  • RAID-10 disk storage
  • Reserved RAM
  • Guaranteed CPU share and more when available
  • 4-core servers running Xen virtualization instances
  • Slicehost management portal for reboots and software installs
  • Mobile management portal for smartphones
  • Ajax console access
  • Bootable rescue mode
  • Private IPs for inter-slice communication
  • HA capabilities via shared IPs
  • Machines running with fixed usage limits, below full capacity

=-=-

JungleDisk http://www.jungledisk.com/ uses Hadoop with friendly UI to provide backup/ redundancy. For security, all personal data is encrypted by personal key known only to you.

Comments off

Q&A: Citrix exec says cloud to carry Xen against VMware

Source: http://news.cnet.com/8301-1001_3-10076377-92.html

Author: Peter Judge

Copyright: ZDNet

I’m looking for the comparison between Xen and VMWare. I believe this interview tells some result and I highlight the key points in italic and bold from my understanding.

=-=-=-=-=-=-=-

Citrix aims to beat VMware at virtualization. A year ago it bought XenSource, the company created by the founders of the Xen open-source hypervisor, and switched the Citrix business focus to virtualization.

Ian Pratt

Ian Pratt, vice president, Citrix

Citrix made XenServer, the commercial system based on Xen, central to its strategy, and applied a Xen brand to other Citrix products involved in delivering applications to desktops. XenSource staff gained senior positions at Citrix and have been setting the company’s future direction.

Ian Pratt, the original project leader of Xen and a founder of XenSource, remains a Fellow of King’s College, Cambridge, but is now also vice president for special products at Citrix–and remains chairman of Xen.org.

ZDNet UK spent a day at Citrix’s U.K. headquarters with Pratt and his colleague, Simon Crosby, who has moved from chief technology officer of XenSource to become chief technology officer of Citrix. After lengthy briefings on Citrix products and the future of virtualization, we sat down with Pratt to understand where Citrix is going and why.

In the first of what will be a two-part interview, Pratt discusses how Citrix hopes to make headway in the virtualization market.

At the moment there is a lot of publicity for VMware and Microsoft Hyper-V. Is there a danger XenWare will be overlooked–especially as it is difficult to measure market share in virtualization? Could XenWare become the Liberal Democrats of virtualization?
Pratt: In the market, there is obviously a big incumbent player, VMware, and Microsoft has a very basic product that covers the low end. And then there is XenServer, which is going head to head with VMware, with an enterprise feature set.

If you look toward the cloud, all the cloud vendors use Xen. It gets used in all the largest deployments, by folks like Amazon and other large providers, because of all the features it offers.

It is very hard to judge what the market share is. With VMware, you just look at VMware’s bank account.

But if you are looking at market share for Xen in general, you’ve got XenServer, Virtual Iron, XenApps, and products from other companies. And then there are all the Linux distributions that include Xen, most of which are free, and no one really has any idea of how many copies are in use.

Even with something like XenServer, because there’s a free version, we keep stumbling into customers–particularly people doing software as a service, where it all runs on XenServer Express Edition (the free unsupported version of XenServer).

They haven’t paid us any money as yet and they have thousands of servers running it. But we’d much rather they were running our stuff than VMware, because it’s an opportunity for us.

With the Cloud Edition we recently announced, there will be lots of people wanting to pay for support and get features added.

If cloud providers are mostly on the free XenServer, is this the whole reason for the Cloud Edition–to turn them into “real” customers and start getting some money in?
Pratt: Simon Crosby has this analogy, that Xen is an engine and needs a car built around it. (The point of the analogy is that the Xen hypervisor is open-source, with a GPL license, so users can make additional technology outside that hypervisor and keep it proprietary, building commercial products that work with Xen.)

If you are a cloud provider or a big software-as-a-service (SaaS) vendor, you can download the open-source engine and build your own car around it. That’s fine if you have the engineering resource to do that, but a lot of this stuff is going to become more commoditized. SaaS vendors don’t all have very particular needs. They don’t have to do this for themselves–they have just had to do it that way until now.

Cloud Edition gives them a standard framework, so they can just concentrate on the value-added bits that they are interested in.

So it’s just like many software developments in the past, where it’s become obvious that everyone is doing the same thing, and a supported version of that is produced?
Pratt: Yes. And they just switch over to the supported version.

So VMware is the big competitor, then. What is the state of things between you and VMware at the moment?
Pratt: VMware has been really successful as a virtualization vendor. But virtualization as a category will disappear. The basic use of virtualization–server consolidation–is now a commodity.

Virtualization will be included in every operating system and on every server. XenSource Express is built into every HP and Dell server, on a USB stick soldered into the box. Users can run multiple virtual machines on those machines out of the box.

This is the only thing VMware does, and it has 100,000 customers. VMware is preparing for this to happen, by building management tools. But this puts them in direct competition with huge established players, like Tivoli and HP OpenView.

Citrix’s approach is to concentrate on application delivery. The function of an IT department is to deliver applications, and we are doing it end to end, from the data center to the client device.

We don’t need to do systems management, and we don’t need to compete head on. It is all about application delivery. People won’t buy virtualization–but they will buy high availability and fault tolerance.

Citrix has 200,000 customers. That’s a pretty good beachhead to deliver more Citrix stuff to customers.

What is distinctively better about Xen’s approach compared with VMware’s?
Pratt: We don’t want to create a class of people called virtualization administrators who you need to manage your virtual machines. That’s how VMware works.

VMware is operating system virtualization–or hardware virtualization. That puts a lot of effort into a problem that is no longer there (since modern hardware from Intel and AMD has evolved to support virtualization).

What we did was to start out with the idea that hardware should support virtualization and the virtual machine should be aware. We call that para-virtualization, and Microsoft calls it enlightenment. That’s marketing.

If you employ virtualization to get a separation not just in the hardware layer, then you can compose things dynamically. That’s the way to bring down the real cost of IT.

We want to be as much of an appliance as possible. We’ve always seen Xen as an appliance that hosts virtual appliances. You want it to be an appliance and manage it like an appliance. It’s like a Netgear router–you just plug it in and go. Adding a new machine to a XenServer pool should be as easy as that.

It is also quite hard to establish the relative performance of VMware’s hypervisor and XenServer because of VMware’s licensing terms. Are you working on a way round that problem?
Pratt: The VMware EULA (end-user licensing agreement) prohibits the publication of any benchmark results to a third party. We tried to publish results in 2002, and that clause has been in the VMware EULA ever since.

As XenSource, we might have had fun and games around the policy, but as Citrix, we have to be more circumspect. It’s possible to publish comparisons against “Hypervisor A” and “Hypervisor B,” though.

How about comparisons with Microsoft? Pretty soon, Microsoft will be able to claim that all the people who have Server 2008 have Hyper-V, won’t it?
Pratt: Yes, but then there will be the question of how many people are using it, and how many people have the bits. If we wanted to measure Xen market share like that, we would be in great shape, because every Linux distro has Xen included in it.

Comments off

Is Amazon Ready For The Enterprise?

Source: http://www.nytimes.com/…is-amazon-ready-for-the-enterprise-26615.html

Author: ALISTAIR CROLL

Copyright: GigaOm
With a flurry of announcements in recent weeks, Amazon has extended its cloud computing lead. The beta label’s gone. It can run Windows applications. By investing in firms like Elastra, it’s tackling enterprise deployment. And there’s a 99.95 percent uptime guarantee.

Much of this is a pre-emptive strike at Microsoft’s upcoming cloud offering. Microsoft has a huge advantage: It owns the stack from OS and virtual machine through to application. Amazon wants to compete on reliability and performance, rather than software suites and licensing. But there are still some things missing before enterprises will really embrace it.

Back in May, most of the people we asked were more likely to trust Amazon than Microsoft with their enterprise applications. But while enterprise customers are using Amazon already, in many cases that use is limited to a department or a short-term project. If Amazon wants to capture entire IT departments, it needs to prove it’s as good or better than in-house infrastructure. And that means delivering responsive, highly available applications, not just an SLA.

To accomplish this, Amazon needs to tackle performance and availability at an architectural level. When companies build their own applications, they rely on building blocks like load-balancing, WAN acceleration, managed DNS and redundant data centers. Fortunately, this is where much of Amazon’s roadmap leads.

  • Network performance: Amazon’s CDN will get static content closer to users. With availability zones, Amazon can also get computation near the edge. All of this reduces the time it takes to deliver bits to users. But it can be faster still: Modern enterprises squeeze every millisecond out of the network. Amazon should also add route optimization, HTTP and TCP optimization to really address network delay.
  • Amazon’s plans for integrated scaling, monitoring, and load balancing in EC2

  • Processing performance: Internet architects improve server performance with load-balancing. First send the request to the fastest data center, then send it to the fastest machine in that data center. If there aren’t any fast machines, the newly announced dynamic scaling will make new ones. All that’s missing (though hinted at) is the ability to measure user experience so EC2 knows when to add new servers. Amazon needs a complete load balancing/monitoring/scaling strategy — with proper controls so IT staff can manage it — to make elastic computing a reality. While they’re at it, a performance SLA would be great, too.
  • Network availability: Those same load balancing technologies improve uptime, using DNS or BGP to bypass unreachable data centers. Amazon needs to launch a SimpleDNS service, tied to availability zones and performance, that gives operators more control. It’s going to have to deal with DNS when it launches its CDN anyway. This looks less like managed DNS (Amazon uses UltraDNS already) and more like products from F5, Citrix or others. Amazon also needs to open up about its carriers and peering arrangements for enterprises to feel comfortable.
  • Processing availability: Big Internet sites don’t achieve high uptime with machines that always work. Instead, they monitor for failure and then have the load balancers take out bad servers. That way, overall availability can be high, even when individual components are broken. Amazon should add load testing and profiling capabilities – particularly since EC2 doesn’t give users deep visibility into the platform — to ensure that applications work worldwide under stress.

As Amazon CTO Werner Vogels pointed out, enterprises like cloud computing for its economics, its elastic capacity and its ability to deliver high reliability. With this roadmap, Amazon goes after Microsoft’s weak spots. But it’s not there yet.

Comments off