Language-level Support for Software Configuration

A while back, I wrote a small load testing application where each test declared what about itself could be configured, and when the tests were loaded, a central manager tracked those declarations and ensured that they were satisfied at runtime. This seems like an approach that would be useful if it were baked in at the language level. I think such a feature should have the following characteristics:

First, the language should provide syntactic support for configuration. For instance, if your language declared variables “var foo = ‘bar’”, you could instead say “conf foo = ‘bar’” (where ‘bar’ would be interpreted as a default value). If you hate this example, please don’t run away yet, I’m not advocating any ‘one true syntax’ for this. The important bit is that it should be as convenient as possible for the programmer to declare directly in the source code what’s available for end-user configuration.

Second, the configuration surface of a program should be discoverable. Users shouldn’t have to pore through the code to find all the little bits they can tweak, and they shouldn’t have to fight with possibly out-of-date documentation. The language system should be able to reveal every little knob that can be fiddled with for a given body of software.

Third, the language should not specify the end-user configuration format. Instead, it should provide a swappable configuration input system, so that the end user can switch between command-line parameters, different file formats, remote files, databases, etc. (this implies some bootstrapping – how do you configure which configuration system you’re going to use?).

Finally, configured variables should be safe. This could mean several things: that visibility for configured variables is limited to the module that declared that variable. That the system ensures that variables without a default are actually configured or else throws an error. That type or value constraints for variables are enforced. That configured values can be ‘locked down’ externally if they need to be. Again, it really depends on what’s appropriate for the language.

I’ve thrown together a starting approximation of how this might work for Python. Please don’t take this as the full expression of the idea, it’s just a nudge in that direction, to see what’s possible. Thanks are due to Thermal Noise for getting me pointed in the “right” direction for the bytecode stuff.

#!/usr/bin/python
"""
A first pass proof-of-concept at a quasi-language-level 
configuration system for Python.  
 
This defines the conf function; any time you want a variable that 
can be configured externally, simply assign it the output value 
of conf(), which will look up the external configuration for the 
variable, fallback to the default value given as an argument to 
conf(), or throw an error.
 
There are some serious limitations to this.  First, there's no way 
to externally define configuration, there's only the hint of future 
mechanisms for this.  Second, there's no way to expose what 
can be configured until the conf() function is evaluated for an 
assignment (this could be addressed if conf were more integrated
into the core language). Third, there's no care taken to mitigate 
misuse of conf().  I'm sure there are many more.
 
For the sake of simplicity, this is limited to module-level 
variables.
"""
import inspect
import dis
 
_undef = []
external_config = {}
def conf(default=_undef, process=None):
    confkey = makeconfkey(inspect.currentframe().f_back)
    if confkey in external_config:
        confval = external_config[confkey]
        return process(confval) if process is not None else confval
    elif default is not _undef:
        return default
    else:
        raise Exception('No config value provided')
 
def makeconfkey(frame):
    varname = findvarname(frame)
    varmodule = inspect.getmodulename(inspect.getfile(frame))
    return '%s.%s' % (varmodule, varname)
 
def findvarname(frame):
    """Look forward from the frame's current op to find the name 
    of the variable we're assigning to.  I'm sure this is horribly 
    broken in some or many ways."""
    varname = None
    bytecode = frame.f_code.co_code
    for i in xrange(frame.f_lasti, len(bytecode)):
        op = dis.opname[ord(bytecode[i])]
        if op == 'STORE_NAME':
            varname = frame.f_code.co_names[ord(bytecode[i+1])]
            break
    return varname
 
##EXAMPLES
#regular assignment with a default
foo = conf(1)
print foo
 
#overriding default with configured value
external_config['conf.bar'] = '2'
bar = conf(1.0, float)
print bar
 
#oops, no default, no config
try:
    baz = conf()
except:
    print 'No config for baz'

29 March 2009