Weevil: a Distributed System Experiment Automation Tool
Overview
This research project addresses the problem of experimenting with
highly distributed systems. We use the term highly distributed
system to refer to a system capable of delivering a service to
many clients through a large number of distributed access points. A
highly distributed system usually consists of a network of components,
executing independent and possibly heterogeneous tasks, that
collectively realize a coherent service. Examples of such systems are
various forms of general-purpose communication systems,
application-level overlays, networks of caching servers, peer-to-peer
file-sharing systems, distributed databases, replicated file systems,
and distributed middleware systems in general.
One difficulty software engineers face later in distributed system
development process is how to evaluate and tune the alternative
designs and implementations. The only solution is systematic,
repeatable experimentation with executable prototypes in realistic
execution environments. Thus, many large-scale or emulated networks
come into being to provide the testbed for such large-scale
experiments. However, the large-scale and complicated nature of
large-scale network environments and distributed system usage
scenarios makes the control of such experiments difficult.
Our tool, Weevil, is to ease the difficulty in programming
distributed experiment control scripts by generating them based on
some easy-to-write configuration files. It covers a two-phase
experimentation process.
- Workload generation: A distributed system workload is a
time-ordered list of service calls from clients to system access
points. While there are many ways to create such workloads, they
are still not flexible enough and unavailable for some scenarios.
Our approach to workload generation is to perform an offline
simulation of expected client behavior and record the resulting
service calls. During an experiment run, these service calls are
issued as inputs into the subject system.
- Experiment deployment and execution: This activity
applies a workload to a system and gathers runtime data. To achieve
this, the subject system, the workload, and the framework codes are
first deployed across an experimentation testbed and then
configured. Next, the system is started and stimulated by the
execution of service calls at the times and locations dictated by
the workload. When the experiment terminates, output and diagnostic
data are collected from across the testbed to facilitate efficient
data analysis and reduction.
Weevil consists of a synthetic-workload generator coupled with
an environment for managing the deployment and execution of
experiments. Weevil is intended to facilitate experimentation
activities for distributed systems by providing engineers with a
flexible, configurable, automated and, thus, repeatable process for
evaluating their software on a networked testbed. Further, Weevil is
not targeted at a specific system or application model, but rather it
is a generic, programmable tool. A prototype
implementation of
Weevil is available.
LICENCE AND CREDITS
Copyright © 2004-2006 University of Colorado.
Weevil is free software. you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your
option) any later version.
Weevil is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
People
News
- Oct 15, 2006 Released version 1.2.0 with testbed
actors.
- Oct 10, 2006 Released version 1.1.1 with enhanced
workload format to support conditional and reactive workload.
- May 1, 2006 Released version 1.1.0 with robustness
support. In detail, it includes the following new features:
- If network failure happens during experiment deployment, the
experiment terminates immediately. You can fix the network and
continue the experiment from the failure point with the command:
${execPath}/weevil resume-WVL_Experiment_Name.
- If some testbed hosts fail during experiment deployment, the
experiment terminates immediately. You can replace the failed hosts
with other hosts specified in the original configuration file. The
replacement can be implemented by only providing another supplemental
configuration file. Please refer to the details in the user manual.
- If testbed host fails during experiment execution, the experiment
terminates immediately. Weevil can automatically find out a host to
replace the failed host, re-generate part of the experiment scripts,
re-deployment these updated scripts, and re-execute the experiment
from the beginning.
- Dec 28, 2004 Released version 1.0.1 with parallel
communication between master and testbed hosts
- Dec 13, 2004 Released version 1.0.0
Packages Download
The following .tar.gz
files are tar archives compressed with gzip.
Documents
User Manual
Reference Manual
Workload Generator Reference Manual
Experiment Environment Reference Manual
PUBLICATIONS
Thesis
Conference Proceeding
Yanyan Wang, Matthew
J. Rutherford, Antonio Carzaniga,
Alexander L. Wolf,
Automating Experimentation on Distributed Testbeds, In
Proceedings of the 20th IEEE/ACM International Conference on Automated
Software Engineering (ASE 2005)
, Long Beach, California, November 7-11, 2005. (Acceptance rate <
10%, nominated for the Best Paper Award)
Technical Report
Acknowledgements
We extended a parallel tool, vxargs, to
allow more arguments included in an arbitrary command (instead of only
allowing IP address as arguments in the original vxargs). We thank its
auther, Yun Mao, for providing this useful tool. You can download our
extended version of vxargs here.
Feedback
Your feedbacks are very valuable to us. Please don't hesitate to send
me email (Yanyan.Wang@colorado.edu) if you
have any problems, corrections, or suggestions. Thanks!
Last Updated: Oct 15, 2006;
Copyright 2004-2006 Yanyan Wang
