CASE STUDY: VMWARE
ESX Server: VMware’s type 1 hypervisor.
ESX Server (unlike VMware Workstation) required users to install a new system
image on a boot partition.
Despite the drawbacks, the trade-off made sense for dedicated deployments of
virtualization in data centers, consisting of hundreds or thousands of physical ser-
vers, and often (many) thousands of virtual machines. Such deployments are some-
times referred today as private clouds. There, the ESX Server architecture provides
substantial benefits in terms of performance, scalability, manageability, and fea-
tures. For example:
The CPU scheduler ensures that each virtual machine gets a fair share
of the CPU (to avoid starvation). It is also designed so that the dif-
ferent virtual CPUs of a given multiprocessor virtual machine are
scheduled at the same time.
The memory manager is optimized for scalability, in particular to run
virtual machines efficiently even when they need more memory than
is actually available on the computer. To achieve this result, ESX Ser-
ver first introduced the notion of ballooning and transparent page
sharing for virtual machines (Waldspurger, 2002).
The I/O subsystem is optimized for performance. Although VMware
Workstation and ESX Server often share the same front-end emula-
tion components, the back ends are totally different. In the VMware
Workstation case, all I/O flows through the host operating system and
its API, which often adds overhead. This is particularly true in the
case of networking and storage devices. With ESX Server, these de-
vice drivers run directly within the ESX hypervisor, without requiring
a world switch.
4. The back ends also typically relied on abstractions provided by the
host operating system. For example, VMware Workstation stores vir-
tual machine images as regular (but very large) files on the host file
system. In contrast, ESX Server has VMFS (Vaghani, 2010), a file
VIRTUALIZATION AND THE CLOUD
system optimized specifically to store virtual machine images and
ensure high I/O throughput. This allows for extreme levels of per-
formance. For example, VMware demonstrated back in 2011 that a
single ESX Server could issue 1 million disk operations per second
5. ESX Server made it easy to introduce new capabilities, which re-
quired the tight coordination and specific configuration of multiple
components of a computer. For example, ESX Server introduced
VMotion, the first virtualization solution that could migrate a live vir-
tual machine from one machine running ESX Server to another ma-
chine running ESX Server, while it was running. This achievement re-
quired the coordination of the memory manager, the CPU scheduler,
and the networking stack.
Over the years, new features were added to ESX Server. ESX Server evolved
into ESXi, a small-footprint alternative that is sufficiently small in size to be
pre-installed in the firmware of servers. Today, ESXi is VMware’s most important
product and serves as the foundation of the vSphere suite.
7.13 RESEARCH ON VIRTUALIZATION AND THE CLOUD
Virtualization technology and cloud computing are both extremely active re-
The research produced in these fields is way too much to enumerate.
Each has multiple research conferences.
For instance, the Virtual Execution Envi-
ronments (VEE) conference focuses on virtualization in the broadest sense.
will find papers on migration deduplication, scaling out, and so on.
ACM Symposium on Cloud Computing (SOCC) is one of the best-known venues
on cloud computing.
Papers in SOCC include work on fault resilience, scheduling
of data center workloads, management and debugging in clouds, and so on.
Old topics never really die, as in Penneman et al. (2013), which looks at the
problems of virtualizing the ARM in the light of the Popek and Goldberg criteria.
Security is perpetually a hot topic (Beham et al., 2013; Mao, 2013; and Pearce et
al., 2013), as is reducing energy usage (Botero and Hesselbach, 2013; and Yuan et
With so many data centers now using virtualization technology, the net-
works connecting these machines are also a major subject of research (Theodorou
et al., 2013).
Virtualization in wireless networks is also an up-and-coming subject
(Wang et al., 2013a).
One interesting area which has seen a lot of interesting research is nested virtu-
alization (Ben-Yehuda et al., 2010; and Zhang et al., 2011).
The idea is that a vir-
tual machine itself can be further virtualized into multiple higher-level virtual ma-
chines, which in turn may be virtualized and so on.
One of these projects is appro-
priately called ‘‘Turtles,’’ because once you start, ‘‘It’s Turtles all the way down!’’