A Study of a Dynamic Safety Net for Server Software

As server software is getting bigger and more complex, two major problems that server software involves in terms of safety come to the surface. One is an attack against server software and the other is its instability. Most of these problems are caused by software flaws. Since finding all software flaws is difficult, unknown flaws are left in a lot of server software. Therefore, there are serious demands on the facility that minimizes damages in case that server software becomes insane. We call this facility safety net. Since server software must handle requests from various users simultaneously and performance is important as well as safety, it needs to dynamically change the range of a safety net depending on the situations. However, it is not easy to achieve such a safety net in terms of security and performance. This dissertation studies a dynamic safety net that enables server software to securely change the range of the safety net and achieves good performance. We have developed a system to provide such a dynamic safety net. The system consists of two mechanisms: an access control mechanism for user-level servers and a fail-safe mechanism for operating system modules. (1) Our access control mechanism allows a server process to impose appropriate access restrictions on it depending on the clients. To avoid risks involved in changing the access restrictions, this mechanism uses a new technique called process cleaning. Process cleaning recovers even a compromised server to be sane before changing access restrictions. (2) Our fail-safe mechanism, which we call multi-level protection, allows running each operating system module separate from the kernel so that misbehavior of particular modules due to software flaws does not affect the whole system. For performance improvement, the multi-level protection enables the users to lower the protection level of the modules without any modifications. We have implemented these access control mechanism and fail-safe mechanism on Linux and NetBSD, respectively, and thereby showed that our ideas can be implemented with reasonable performance. For process cleaning, we experimented on the Apache web server and confirmed that the overhead is less than 35%. For the multi-level protection, we experimented on file system modules and network modules and confirmed that the overhead of this mechanism is less than 12% at the minimum protection level.