Cyclone

Mr. Yiu-Fai Sit

With the ever-growing web traffic, cluster-based web server is becoming more and more important to the Internet’s infrastructure. Making the best use of all the available resources in the cluster to achieve high performance is thus a significant research issue. In this paper we introduce Cyclone, a cluster-based web server that can achieve nearly optimal throughput. Cyclone makes use of a novel network support mechanism called Socket Cloning (SC) together with the concept of hot object replication to obtain high performance. SC allows an opened socket to be moved efficiently between cluster nodes. With SC, the processing of HTTP requests can be migrated to the node that has a cached copy of the requested document, thus bypassing any cache transfer between cluster nodes. To achieve better load balancing, frequently accessed documents (hot objects) are replicated to other cluster nodes by forwarding cached copies. Trace-driven benchmark test using http_load shows that Cyclone outperforms existing approaches and can achieve a throughput of 14575 requests/s (89.5 MBytes/s), which is 98% efficiency of the available network bandwidth, with 8 web server nodes.

Socket Cloning is an efficient network support mechanism that enables an opened socket to move to another machine for communication on its behalf. When a web server node decides not to handle a particular HTTP request by some load balancing policies, it can clone the socket that corresponds to the request to a more suitable node in the cluster. For example, a node can clone a socket to a node that has the cached copy of the requested document, instead of performing local disk access to serve the request. A socket, called clone, will be set up in the chosen node with the same states of the original socket after cloning. Messages (HTTP requests) received by the original socket are also placed in the clone’s buffer. The web server software in this node will then treat the clone as an ordinary socket and process the requests in the clone’s buffer.

In the following discussion, it is assumed that a Layer-4/2 dispatcher is used to distribute requests to the web server nodes in the cluster. Any other lightweight distribution mechanism can also be used.

Figure 1. Socket Cloning: System Architecture

SC Client provides a system call interface to the web server software in the node. When the web server decides to let another node to handle the request, it issues the system call provided by SC Client to clone the socket. SC Client then packs all the relevant information of the socket and sends this out to the SC Server in the remote node through a persistent connection. The whole message is called SC Message. When the cloning system call returns, the web server treats the request as has been served.

SC Server is responsible for reconstructing the socket and all its relevant states. When SC Server receives an SC Message, it will create a socket, called clone, in its node according to the information contained in the message. The states of the socket and the protocol stacks are reconstructed after the clone is created. The clone is now native to this node and subsequent packets will go through its normal network protocol stack. There is no extra overhead in process the packets. Outgoing packets of the clone will be sent directly to the client. Upon successful creation of the clone, the SC Server will send an acknowledgement back to the SC Client. Knowing that the remote clone has been set up, the SC Client will inform the Packet Router to route subsequent packets for that socket to the clone’s node and an entry is added to the Packet Router’s table. As a result, packets from the client will first reach the original node and be routed to the clone’s node while packets to the client are sent directly from the clone’s node. A triangular routing path is thus established. Furthermore, client packets that contain non-zero TCP payload are passed to the network stack of the original node, as well as routed to the clone if the connection has an entry in the Packet Router’s table.

After cloning, the original socket remains in its node. It will not be destroyed until the connection is closed. The original socket will handle further messages received in the connection after cloning in persistent HTTP, while the clone only handles the first request that it has in its buffer, which may contain several requests in persistent HTTP. Figure 2 shows how a series of non-persistent HTTP requests are handled in the system.

The client is not aware of the cloning and hence there is no synchronization between the client and the cluster node. Such transparency to the clients allows an efficient mechanism for Socket Cloning.

Figure 2. Socket Cloning in Cluster-Based Web Server

For comparison to other content-aware approaches, Figure 3 and Figure 4 show the logical flow of how the same series of HTTP requests are processed in relaying and TCP handoff respectively. In both cases, the clients first have to set up a connection with the dispatcher, which then parses the requests one by one. After that, the connection is relayed or handed off to the chosen web server. This sequential request processing together with heavyweight connection handoff and relaying impose a great limit in a parallel cluster-based web server. In SC, requests are distributed by Layer-4 dispatcher (or other lightweight mechanisms, such as DNS approach [10] ), which has very small overhead. Clients set up connection with the web server nodes directly and therefore the processing of different requests can be carried out in parallel.

Figure 3. Relaying and Its Variants

Figure 4. TCP Handoff

We have implemented a prototype of Socket Cloning and hot object replication in Linux, kernel version 2.4.2. The network stack has been modified so that the clone (a socket) can be created without a real connection. A flag is also added to the socket’s structure to differentiate a normal socket from a cloned socket. This is to avoid a clone to clone itself again. Normal network operations are not affected and applications are unaware of the change.

The SC Server, SC Client, and the Packet Router are all implemented as kernel modules. These modules have to be loaded in all the cluster nodes before any application can clone a socket. When the system starts, SC Client connects to the SC Servers in the other nodes of the cluster. All the SC Messages and cache copies are sent through these connections without the need to start a new one for each message. We have also modified kHTTPd, a kernel-based web server, to make use of the SC facilities.

Cyclone: A High-Performance Cluster-Based Web Server with Socket Cloning