Decided to Switch KVM

Choosing a hypervisor in #Cloud projects always costs me time, between Xen & Kernel-based Virtual Machine (KVM). After an interesting discussion with one of Cloud Computing architect @ IBM, we get a deal – use KVM for future #Cloud implement. My considerations:

  1. IBM announced to boost KVM @ Cloud initiatives since March this year. This indicates IBM’s strategy @ hypervisor on Intel machines leans to KVM, rather than Xen. And more official development, test and support are coming after.
  2. Virtio project to support better network IO performance. http://kbase.redhat.com/faq/docs/DOC-25441

More resources:
KVM vs Xen, dates back in 2006, but telling some basic > http://www.gridvm.org/xen-vs-kvm.html
Xen vs KVM by Xen > http://blog.xen.org/index.php/2010/05/07/..kvm-linux-..community/
KVM @ IBM InfoCenter > http://publib.boulder.ibm.com/../kvminstall/liaaikvminstallstart.htm
Securing KVM @ IBM InfoCenter > http://publib.boulder.ibm.com/../kvmsec/kvmsecguest.htm
Benchmark by Xen > http://wiki.xensource.com/..Quantitative+Comparison+of+Xen+and+KVM.pdf
KVM vs VMware > http://www.workswithu.com/2009/04/27/kvm-vs-vmware-a-case-study/

They’re all @ http://delicious.com/j3ffyang/kvm

Notice: since I only consider #OpenSource virtualization solution @ my #Cloud projects on Intel, there is #proprietary #VMware in my solution design. But integration with VMware is always available.

  • Share/Bookmark

Comments off

Enable VT @ BIOS

One of my customer asked me how to reuse the old fashioned PC in their Cloud. The question leads to “How to enable VT @ BIOS?”. I wrote the following message to my customer:

Go to BIOS setting, then “Enable” or “Turn On” “Virtualization Technology” like

Source & Credit: http://publib.boulder.ibm.com/../kvminstall/kvmadvancedsetup.jpg

In Linux, here are steps to enable VT

(source: http://www.redhat.com/..VT_in_BIOS.html)

Verify the virtualization extensions are enabled in BIOS. The BIOS settings for Intel® VT or AMD-V are usually in the Chipset or Processor menus. The menu names may vary from this guide, the virtualization extension settings may be found in Security Settings or other non standard menu names.

Procedure 34.1. Enabling virtualization extensions in BIOS

Reboot the computer and open the system’s BIOS menu. This can usually be done by pressing the delete key, the F1 key or Alt and F4 keys depending on the system.

Select Restore Defaults or Restore Optimized Defaults, and then select Save & Exit.

Power off the machine and disconnect the power supply.

Enabling the virtualization extensions in BIOS

Note: BIOS steps

Many of the steps below may vary depending on your motherboard, processor type, chipset and OEM. Refer to your system’s accompanying documentation for the correct information on configuring your system.

Power on the machine and open the BIOS (as per Step 1).

Open the Processor submenu The processor settings menu may be hidden in the Chipset, Advanced CPU Configuration or Northbridge.

Enable Intel Virtualization Technology (also known as Intel VT) or AMD-V depending on the brand of the processor. The virtualization extensions may be labeled Virtualization Extensions, Vanderpool or various other names depending on the OEM and system BIOS.

Enable Intel VTd or AMD IOMMU, if the options are available. Intel VTd and AMD IOMMU are used for PCI passthrough.

Select Save & Exit.

Power off the machine and disconnect the power supply.

Run cat /proc/cpuinfo | grep vmx svm. If the command outputs, the virtualization extensions are now enabled. If there is no output your system may not have the virtualization extensions or the correct BIOS setting enabled.

  • Share/Bookmark

Comments off

Thoughts on Cloud Architecture in Open Source

Recently one of my friends I met from LinkedIn sent me several questions, about Cloud Computing, which looks like a Request for Proposal (RFP). I post my very personal response here for a mindshare.

About Cloud, Cluster/ Load Balancing, Data Center and Open Source, I could talk day and night without stop. I’m giving my personal but confident thoughts upon my friend’s email in brief.


Cloud Computing is a big topic. People have their own definitions depending on their view and objective.

- Open Source / Operating System / Framework
I’m Unix/ Linux guy. Other than Unix, Linux is the only operating system in Cloud management and Cloud resource pool, in terms of efficiency, green security, cost, and manageability. No Windows, No Mac… Certainly Windows could be one of computing services in resource pool as a kind of computing resource, that Infrastructure_as_a_Service (IaaS) provides and delivers.

Microsoft .Net framework doesn’t make much sense to Cloud @ IaaS, as Cloud is Open. That’s a defacto rule in Cloud. Windows isn’t “royal free” either, nor brings satisfied ROI in fact.

Most/ major components in Cloud are supposed to be operating system independent, like Java, Xen, Apache, MySQL, Linux/ Unix, JDBC, Mozilla/ Firefox, Eclipse, Tomcat… etc. As long as a software depends on a specific OS as a pre- requisite, it doesn’t fit in Cloud.

- Cloud Architecture
Several open source projects to recommend
http://www.opennebula.org > this is being used @ http://nebula.nasa.gov/
http://open.eucalyptus.com > this is a very AWS EC2- like open source cloud which I personally installed several time for Proof_of_Concept (PoC).

April 23rd, 2009, Ubuntu released 9.04 Server edition, which includes OpenNebula and Eucalyptus open source projects and supports Cloud environment >
http://doc.ubuntu.com/ubuntu/serverguide/C/opennebula.html
http://doc.ubuntu.com/ubuntu/serverguide/C/eucalyptus.html

5 cost- efficient flexible open source resources for Cloud

http://ostatic.com/blog/5-cost-efficient-flexible-open-source-resources-for-cloud…

Cloud hosting & storage toolbox

http://www.webresourcesdepot.com/cloud-hosting-storage-toolbox-options-tools/

- Virtualization and Virtual Machine Manager (VMM)
Virtualization is a key player in Cloud, but not all in Cloud. Virtualization technology is a carrier of computing resource dynamically managed and delivery in Cloud. In my opinion, its importance has been exaggerated in market. The keys of Virtualization are (1) standard (2) simple (3) easy to manage. You can’t deploy and manage multiple virtualization technologies within one Cloud environment.

Hyper-V doesn’t comply with the rule I mentioned earlier – Cloud is open. VMWare solution is expensive, isn’t it? Xen / XenSource is open source / free and recommended. Xen has been widely adopted in Cloud service provider in Internet. Xen is also compatible with monitoring system to be discussed below.

I’ve a blog describing the comparison between XenSource and VMWare > http://tr.im/mFFa

- Monitor
The ability of monitor determines how precisely and automatically Cloud detects from Cloud, and how fine- grained (granular) computing resource could be deployed to customer.

I recommend Ganglia and Nagios that are popular in Cloud, data center, server farm/ cluster, and widely adopted in many enterprises.

developerWorks > http://www.ibm.com/developerworks/opensource/library/l-ganglia-nagios-1/index.html
Nagios homepage > http://www.nagios.org/

- Request Driven Deployment / Provisioning
Deployment driven by request means the end user decides how much computing resource s/he needs from Cloud. Such as you specify number of CPU, # of gigabyte of memory and size of disk (priced @ http://aws.amazon.com/ec2 ) from Amazon Web Services (AWS) Elastic Compute Cloud (EC2). Very simple. We may learn a lot from AWS architecture > http://highscalability.com/amazon-architecture

Provisioning means that a specific software, like JVM or middleware software, or database, to be installed and configured in an deployed / existing virtual system on demand.

The scenario: eg. in AWS, you’re given a Linux operating system with 2 CPUs and 4G memory. But you can do nothing in a blank OS. You might need an Oracle database 11 or a middleware like JBoss or perhaps WebSphere. Then provisioning function will install and configure on the deployed virtual machine unattended.

For now, in that case @ AWS, you have to install by your own or you can select one available image from AWS image pool, with pre-loaded software you need, in which if someone else made one and published previously. Amazon doesn’t have provisioning service so far. That results that AWS has to manage thousand images in its image repository so that implies it is a challenge.

- SLA / Service Policy / Automation
An example of Service Policy: when an instance is running 80% disk utilization, a pre- defined service policy triggers a configured action – to add another 100G disk into such instance.

This is done by Service Policy, not by system operator and administrator. We can see how this differentiate from mainframe’s virtualization since 60′s.

In another word, Service Policy = Automation!

This kind of policy could be very specific in many and many scenarios. It does require customization effort to fit requirement. I do have some experience/ skill of elastic JVM/ JavaEE computing resource as being engaged with a banking customer now, @ Platform as a Service (PaaS) project.

- Load balancing (not F5)
Load balancing is a must in large enterprise and busy traffic internet. F5 is popular. But F5 throws URL (a request from web) as a token to go into web server cluster. In fact, URLs are not equal. F5 doesn’t know how much resource behind one URL might need from JVM in middleware. So F5 can round- robin requests / URLs, but not granularly down to CPU utilization level.

Cloud Computing @ each layers – IaaS, PaaS & SaaS – needs monitoring to know how much resource remains, how much needed, how much to deliver, how much to remove after use.

- Security/ Authentication / LDAP
Security is a big concern in Cloud, as well as alert and audit, in term of authentication, authority, data integrity, encryption/ decryption, etc.

LDAP (OpenLDAP) could be used @ Cloud portal where end user applies computing resource in resource pool, while Cloud operator manages in management portal. As long as a computing resource is deployed, end user is received an authority to access the delivered resource and doesn’t need LDAP authentication.

- Languages
Java is preferred as global languange
C is used in Eucalyptus Cloud Management
Ruby on Rail used to display @ Portal & UI
XML in data transaction, Python, etc.

- Approach
Have to have Proof_of_Concept – to build a small system to prove all works prior to production

  • Share/Bookmark

Comments off

What does RackSpace offer?

Part of the following comes from: http://www.comparewebhosts.com/…htm

Rackspace’s cloud strategy is supported by three core offerings, all part of Mosso, Rackspace’s Cloud Hosting Division. The company’s existing Hosting Cloud and CloudFS storage offerings have been re-branded to reflect the company’s newly integrated approach:

· Cloud Sites — Rackspace’s flagship cloud offering, The Hosting Cloud, is now Cloud Sites. Developed by Mosso, Rackspace’s cloud division, it offers a scalable platform for handling huge traffic spikes and a pay-as-you-grow pricing model. Cloud Sites is a heterogeneous environment, supporting both Windows and Linux.

· Cloud Files — Rackspace’s internet-based storage service, CloudFS, is now Cloud Files. Cloud Files gives developers instant access to an enterprise-grade storage infrastructure and reduces overall investment and IT costs while providing infinite scalability. Cloud Files offers an industry leading SLA and a highly competitive pricing model with replicated storage starting at $0.15/GB. Also later this year, Limelight Networks will team with Rackspace to allow developers to easily distribute content to millions of end users around the world and bring scalable content delivery and application acceleration services to the masses. While continuing to support the Amazon S3 platform, Jungle Disk will port to Rackspace’s Cloud Files system in the coming months. Jungle Disk comes in both desktop and workgroup editions across the Windows, Mac and Linux platforms.

· Cloud Servers — This new hosting solution, which will deliver on-demand server capacity to businesses of all sizes, will leverage key technology developed by Slicehost, which uses Xen virtualization software. Slicehost will remain as the company’s developer brand, creating innovative new features driven through shared intellectual property in conjunction with development initiatives from Rackspace. As part of the announcement, Slicehost also announced new, larger slices for high performance computing, lower prices as well as IP sharing for high availability computing.

To understand Mosso > http://www.mosso.com

Mosso architectures > http://www.mosso.com/cloud.jsp > look at “Benefit” @ bottom left.

The selling point to our customer >
Step One: Load it up
You can create a new Mosso-powered site – and even add some email accounts – in less than five minutes. Load-Balancing, clustering, and redundant storage are all inherited by your application automatically, without any effort.

Step Two: Watch it scale
From the first byte served, your site is hosted on advanced clustered technology designed for high- traffic, high- performance websites. When your site grows bigger than what it’s included, you pay inexperience scale pricing for exactly what you use and nothing more.

  • Scale your bandwidth
  • Starts at 500GB/mo. As much as you need for 25¢ per GB
  • Scale your SAN storage
  • Starts at 50GB/mo. As much as you need for 50¢ per GB
  • Scale your compute
  • Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle

Closer watch with screenshots > http://www.mosso.com/screenshots.jsp > this describes the ease of process when a client requests a host.

=-=-

http://www.slicehost.com is equal to Cloud Servers. This pretty much looks like VPS – virtual private server on Xen.

  • Included in Every Slice™
  • Full root access and rebooting
  • Choice of Linux distro
  • Dedicated IP address and Tier-1 redundant bandwidth
  • RAID-10 disk storage
  • Reserved RAM
  • Guaranteed CPU share and more when available
  • 4-core servers running Xen virtualization instances
  • Slicehost management portal for reboots and software installs
  • Mobile management portal for smartphones
  • Ajax console access
  • Bootable rescue mode
  • Private IPs for inter-slice communication
  • HA capabilities via shared IPs
  • Machines running with fixed usage limits, below full capacity

=-=-

JungleDisk http://www.jungledisk.com/ uses Hadoop with friendly UI to provide backup/ redundancy. For security, all personal data is encrypted by personal key known only to you.

  • Share/Bookmark

Comments off

Q&A: Citrix exec says cloud to carry Xen against VMware

Source: http://news.cnet.com/8301-1001_3-10076377-92.html

Author: Peter Judge

Copyright: ZDNet

I’m looking for the comparison between Xen and VMWare. I believe this interview tells some result and I highlight the key points in italic and bold from my understanding.

=-=-=-=-=-=-=-

Citrix aims to beat VMware at virtualization. A year ago it bought XenSource, the company created by the founders of the Xen open-source hypervisor, and switched the Citrix business focus to virtualization.

Ian Pratt

Ian Pratt, vice president, Citrix

Citrix made XenServer, the commercial system based on Xen, central to its strategy, and applied a Xen brand to other Citrix products involved in delivering applications to desktops. XenSource staff gained senior positions at Citrix and have been setting the company’s future direction.

Ian Pratt, the original project leader of Xen and a founder of XenSource, remains a Fellow of King’s College, Cambridge, but is now also vice president for special products at Citrix–and remains chairman of Xen.org.

ZDNet UK spent a day at Citrix’s U.K. headquarters with Pratt and his colleague, Simon Crosby, who has moved from chief technology officer of XenSource to become chief technology officer of Citrix. After lengthy briefings on Citrix products and the future of virtualization, we sat down with Pratt to understand where Citrix is going and why.

In the first of what will be a two-part interview, Pratt discusses how Citrix hopes to make headway in the virtualization market.

At the moment there is a lot of publicity for VMware and Microsoft Hyper-V. Is there a danger XenWare will be overlooked–especially as it is difficult to measure market share in virtualization? Could XenWare become the Liberal Democrats of virtualization?
Pratt: In the market, there is obviously a big incumbent player, VMware, and Microsoft has a very basic product that covers the low end. And then there is XenServer, which is going head to head with VMware, with an enterprise feature set.

If you look toward the cloud, all the cloud vendors use Xen. It gets used in all the largest deployments, by folks like Amazon and other large providers, because of all the features it offers.

It is very hard to judge what the market share is. With VMware, you just look at VMware’s bank account.

But if you are looking at market share for Xen in general, you’ve got XenServer, Virtual Iron, XenApps, and products from other companies. And then there are all the Linux distributions that include Xen, most of which are free, and no one really has any idea of how many copies are in use.

Even with something like XenServer, because there’s a free version, we keep stumbling into customers–particularly people doing software as a service, where it all runs on XenServer Express Edition (the free unsupported version of XenServer).

They haven’t paid us any money as yet and they have thousands of servers running it. But we’d much rather they were running our stuff than VMware, because it’s an opportunity for us.

With the Cloud Edition we recently announced, there will be lots of people wanting to pay for support and get features added.

If cloud providers are mostly on the free XenServer, is this the whole reason for the Cloud Edition–to turn them into “real” customers and start getting some money in?
Pratt: Simon Crosby has this analogy, that Xen is an engine and needs a car built around it. (The point of the analogy is that the Xen hypervisor is open-source, with a GPL license, so users can make additional technology outside that hypervisor and keep it proprietary, building commercial products that work with Xen.)

If you are a cloud provider or a big software-as-a-service (SaaS) vendor, you can download the open-source engine and build your own car around it. That’s fine if you have the engineering resource to do that, but a lot of this stuff is going to become more commoditized. SaaS vendors don’t all have very particular needs. They don’t have to do this for themselves–they have just had to do it that way until now.

Cloud Edition gives them a standard framework, so they can just concentrate on the value-added bits that they are interested in.

So it’s just like many software developments in the past, where it’s become obvious that everyone is doing the same thing, and a supported version of that is produced?
Pratt: Yes. And they just switch over to the supported version.

So VMware is the big competitor, then. What is the state of things between you and VMware at the moment?
Pratt: VMware has been really successful as a virtualization vendor. But virtualization as a category will disappear. The basic use of virtualization–server consolidation–is now a commodity.

Virtualization will be included in every operating system and on every server. XenSource Express is built into every HP and Dell server, on a USB stick soldered into the box. Users can run multiple virtual machines on those machines out of the box.

This is the only thing VMware does, and it has 100,000 customers. VMware is preparing for this to happen, by building management tools. But this puts them in direct competition with huge established players, like Tivoli and HP OpenView.

Citrix’s approach is to concentrate on application delivery. The function of an IT department is to deliver applications, and we are doing it end to end, from the data center to the client device.

We don’t need to do systems management, and we don’t need to compete head on. It is all about application delivery. People won’t buy virtualization–but they will buy high availability and fault tolerance.

Citrix has 200,000 customers. That’s a pretty good beachhead to deliver more Citrix stuff to customers.

What is distinctively better about Xen’s approach compared with VMware’s?
Pratt: We don’t want to create a class of people called virtualization administrators who you need to manage your virtual machines. That’s how VMware works.

VMware is operating system virtualization–or hardware virtualization. That puts a lot of effort into a problem that is no longer there (since modern hardware from Intel and AMD has evolved to support virtualization).

What we did was to start out with the idea that hardware should support virtualization and the virtual machine should be aware. We call that para-virtualization, and Microsoft calls it enlightenment. That’s marketing.

If you employ virtualization to get a separation not just in the hardware layer, then you can compose things dynamically. That’s the way to bring down the real cost of IT.

We want to be as much of an appliance as possible. We’ve always seen Xen as an appliance that hosts virtual appliances. You want it to be an appliance and manage it like an appliance. It’s like a Netgear router–you just plug it in and go. Adding a new machine to a XenServer pool should be as easy as that.

It is also quite hard to establish the relative performance of VMware’s hypervisor and XenServer because of VMware’s licensing terms. Are you working on a way round that problem?
Pratt: The VMware EULA (end-user licensing agreement) prohibits the publication of any benchmark results to a third party. We tried to publish results in 2002, and that clause has been in the VMware EULA ever since.

As XenSource, we might have had fun and games around the policy, but as Citrix, we have to be more circumspect. It’s possible to publish comparisons against “Hypervisor A” and “Hypervisor B,” though.

How about comparisons with Microsoft? Pretty soon, Microsoft will be able to claim that all the people who have Server 2008 have Hyper-V, won’t it?
Pratt: Yes, but then there will be the question of how many people are using it, and how many people have the bits. If we wanted to measure Xen market share like that, we would be in great shape, because every Linux distro has Xen included in it.

  • Share/Bookmark

Comments off

Xen versus VMware

Source: http://searchservervirtualization.techtarget.com/…

Author: Bernard Golden

Even though VMware and Xen both provide an enterprise-focused server virtualization product, Xen’s approach to implementing virtualization has made for some challenges. VMware bases its product on hardware emulation, in which VMware provides a software layer that “looks” like an x86-based machine to a guest operating system. VMware cleverly patches the running guest operating system so that it interacts with the hypervisor, which in turn mediates between the guest operating system and the underlying hardware. This is a powerful technique that allows unmodified operating systems to run as guest machines; however, it takes a toll on performance due to the hardware emulation the hypervisor provides.

Xen’s product, by contrast, operates more like a traffic cop, multiplexing access to the underlying hardware resources. Xen dubs this approach “paravirtualization,” and one of the primary benefits is that the hypervisor is a very skinny piece of code which imposes little overhead. Tests run against paravirtualized guest operating systems indicate a trivial amount of virtualization performance hit on the order of less than 5%.

A drawback to Xen’s approach is that the thin hypervisor requires modification of the guest operating systems so that they run as paravirtualized guests. Specifically, this requires patching the kernels of the guest operating systems to allow interaction with the control structures of the Xen hypervisor. Another drawback to Xen’s thin architecture is that underlying services must be provided by a privileged guest operating system. (In Xen parlance, a privileged operating system is called a Dom0, and a regular guest operating system is called a DomU.) The privileged guest requires a patched kernel as well, since it must access the same Xen control structures to pass data back and forth with DomUs. In addition, the privileged guest requires multiplex access to underlying resources—such as the processor, memory, network and storage—on behalf of DomUs.

  • Share/Bookmark

Comments off