A High-Performance Cluster-Based Web Server with
With the ever-growing web traffic, cluster-based web server is becoming more and more important to the Internet’s infrastructure. Making the best use of all the available resources in the cluster to achieve high performance is thus a significant research issue. In this paper we introduce Cyclone, a cluster-based web server that can achieve nearly optimal throughput. Cyclone makes use of a novel network support mechanism called Socket Cloning (SC) together with the concept of hot object replication to obtain high performance. SC allows an opened socket to be moved efficiently between cluster nodes. With SC, the processing of HTTP requests can be migrated to the node that has a cached copy of the requested document, thus bypassing any cache transfer between cluster nodes. To achieve better load balancing, frequently accessed documents (hot objects) are replicated to other cluster nodes by forwarding cached copies. Trace-driven benchmark test using http_load shows that Cyclone outperforms existing approaches and can achieve a throughput of 14575 requests/s (89.5 MBytes/s), which is 98% efficiency of the available network bandwidth, with 8 web server nodes.
Socket Cloning is an efficient network support mechanism that enables an opened socket to move to another machine for communication on its behalf. When a web server node decides not to handle a particular HTTP request by some load balancing policies, it can clone the socket that corresponds to the request to a more suitable node in the cluster. For example, a node can clone a socket to a node that has the cached copy of the requested document, instead of performing local disk access to serve the request. A socket, called clone, will be set up in the chosen node with the same states of the original socket after cloning. Messages (HTTP requests) received by the original socket are also placed in the clone’s buffer. The web server software in this node will then treat the clone as an ordinary socket and process the requests in the clone’s buffer.
the following discussion, it is assumed that a Layer-4/2 dispatcher is used to
distribute requests to the web server nodes in the cluster. Any other
lightweight distribution mechanism can also be used.
are three components in Socket Cloning, which are shown in Figure
SC Client provides a system call interface to the web server
software in the node. When the web server decides to let another node to handle
the request, it issues the system call provided by SC Client to clone the
socket. SC Client then packs all the relevant information of the socket and
sends this out to the SC Server in the remote node through a persistent
connection. The whole message is called SC Message. When the cloning system call
returns, the web server treats the request as has been served.
SC Server is responsible for reconstructing the socket and all
its relevant states. When SC Server receives an SC Message, it will create a
socket, called clone, in its node according to the information contained in the
message. The states of the socket and the protocol stacks are reconstructed
after the clone is created. The clone is now native to this node and subsequent
packets will go through its normal network protocol stack. There is no extra
overhead in process the packets. Outgoing packets of the clone will be sent
directly to the client. Upon successful creation of the clone, the SC Server
will send an acknowledgement back to the SC Client. Knowing that the remote
clone has been set up, the SC Client will inform the Packet Router to route
subsequent packets for that socket to the clone’s node and an entry is added
to the Packet Router’s table. As a result, packets from the client will first
reach the original node and be routed to the clone’s node while packets to the
client are sent directly from the clone’s node. A triangular routing path is
thus established. Furthermore, client packets that contain non-zero TCP payload
are passed to the network stack of the original node, as well as routed to the
clone if the connection has an entry in the Packet Router’s table.
After cloning, the original socket remains in its node. It
will not be destroyed until the connection is closed. The original socket will
handle further messages received in the connection after cloning in persistent
HTTP, while the clone only handles the first request that it has in its buffer,
which may contain several requests in persistent HTTP. Figure
2 shows how a series of non-persistent HTTP requests are handled in the
The client is not aware of the cloning and hence there is no
synchronization between the client and the cluster node. Such transparency to
the clients allows an efficient mechanism for Socket Cloning.
comparison to other content-aware approaches,
show the logical flow of how the same series of HTTP requests are
processed in relaying and TCP handoff respectively. In both cases, the clients
first have to set up a connection with the dispatcher, which then parses the
requests one by one. After that, the connection is relayed or handed off to the
chosen web server. This sequential request processing together with heavyweight
connection handoff and relaying impose a great limit in a parallel cluster-based
web server. In SC, requests are distributed by Layer-4 dispatcher (or other
lightweight mechanisms, such as DNS approach
), which has very small overhead. Clients set up connection with the web server
nodes directly and therefore the processing of different requests can be carried
out in parallel.
have implemented a prototype of Socket Cloning and hot object replication in Linux,
kernel version 2.4.2. The network stack has been modified so that the clone
(a socket) can be created without a real connection. A flag is also added to the
socket’s structure to differentiate a normal socket from a cloned socket. This
is to avoid a clone to clone itself again. Normal network operations are not
affected and applications are unaware of the change.
SC Server, SC Client, and the Packet Router are all implemented as kernel
modules. These modules have to be loaded in all the cluster nodes before any
application can clone a socket. When the system starts, SC Client connects to
the SC Servers in the other nodes of the cluster. All the SC Messages and cache
copies are sent through these connections without the need to start a new one
for each message. We have also modified kHTTPd, a kernel-based web
server, to make use of the SC facilities.
You can download the source code here: http://www.cs.utexas.edu/users/yfsit/cyclone.tar.bz2