Configuration Testing: Testing Configuration Values Together with Code Logic
share
This paper proposes configuration testing as a key reliability engineering discipline for configuration management in large-scale production systems. We advocate that configuration changes should be systematically tested at the same level as code changes. Furthermore, we argue that configuration testing should be conducted by putting the configuration values under test in the context of the code that uses the values, in order to exercise the semantics and effects of the configuration. We demonstrate how to create configuration tests and show the promises of reusing and repurposing existing software tests by transforming them into configuration tests. We present the preliminary results that show the effectiveness of configuration testing in capturing real-world configuration errors. We also discuss the measures of test adequacy and quality, continuous configuration testing for incremental configuration changes, and a number of open problems towards effective configuration testing.
READ FULL TEXT