config

New in version 1.1.0.

User configuration.

Arbitrary configuration file utility.

Configuration file syntax

The configuration syntax is JSON-compliant and should represent a dictionary:

"interval": 12.7,
"database": {
  "connection": "mysql://[email protected]/grek",
  "auto_reload": true
},
"sockets": [
  "tcp://127.0.0.1:1234",
  "tcp://127.0.0.1:1235",
  "tcp://127.0.0.1:1236"
],
"logging": {
  "levels": {
    "": WARN,
    "some.module": DEBUG
  }
}

The configuration file must contain a literal JSON dictionary and might leave out the outer curly brackets ({ and }), as demonstrated in the example above.

JavaScript/JSON-style comments are supported:

"interval": 12.7, /* A value in the range [0,123)  */

"database": {
  "connection": "mysql://[email protected]/grek",

  /* Enables automatic reloading of entites: */
  "auto_reload": true
},

Includes

Files can be included using the special key @include

"interval": 12.7, /* A value in the range [0,123) */
"@include": "another/file.conf",

Multiple files can be included at once by specifying a list of paths:

"interval": 12.7, /* A value in the range [0,123) */
"@include": ["another/file.conf", "/yet/another/file.conf"],

Paths can be expanded using glob, so another way of including multiple file is using a glob pattern:

"interval": 12.7, /* A value in the range [0,123) */
"@include": "conf.d/*.conf",

Glob patterns can be included in lists too:

"interval": 12.7, /* A value in the range [0,123) */
"@include": ["conf.d/*.conf", "other/*/*.conf"],

Paths deduced from a glob pattern are loaded in ascending alphabetical order. This enables variable configuration directories, like those of Apache HTTPd and LigHTTPd. Consider the following file layout:

some-path/
  my-app.conf
  conf.d/
    001-users.conf
    002-database.conf
    321-extras.conf

Now consider my-app.conf to contain the following configuration:

"interval": 12.7, /* A value in the range [0,123) */
"@include": "conf.d/*.conf",

It’s fully predictable what happens:

  1. my-app.conf is loaded and applied
  2. 001-users.conf is loaded and applied
  3. 002-users.conf is loaded and applied
  4. 321-users.conf is loaded and applied

In other words, files included (using @include) overrides the parent configuration. — Or: — Files inheriting another file is based on the other file.

Note

Relative paths are always relative to the file which is defining them. If file /foo/bar.conf defines "@include": "more/abc.conf", /foo/more/abc.conf is loaded. If /foo/more/abc.conf defines "@include": "more/xyz.conf", /foo/more/more/xyz.conf is loaded.

@inherit

Another including directive, or special key, is @inherit, which work much like @include, with the difference in what gets applied first (what configuration might override the other).

Let’s consider the previous example, but instead using the @inherit directive:

"@inherit": "conf.d/*.conf",
"interval": 12.7, /* A value in the range [0,123)  */

This is the order in which files are loaded and applied:

  1. my-app.conf is loaded
  2. 001-users.conf is loaded and applied
  3. 002-users.conf is loaded and applied
  4. 321-users.conf is loaded and applied
  5. my-app.conf is applied

In other words, files inherited (using @inherit) is overridden by the parent configuration.

Note that @inherit is not the inverse or reverse of @include, but rather a hybrid of a reverse @include and a normal @include.

@inherit is comparable to class inheritance in Python.

Logging

Logging (the standard library logging module) can be configured based on a dictionary passed to configure_logging():

{
  'stream': 'stdout',
  'filename': '/var/log/myapp.log',
  'filemode': 'a',
  'syslog': {'socket': '/var/run/syslog'},
  'format': '%(asctime)s.%(msecs)03d %(levelname)-8s %(message)s',
  'datefmt': '%H:%M:%S',
  'levels': {
    '': WARN,
    'some.module': DEBUG
  }
}
stream

If present, the root logger will be configured with a StreamHandler, writing to stream sys.stream.

Two streams are available:

  • stdout — Standard output
  • stderr — Standard error
filename, filemode
If present, the root logger will be configured with a FileHandler, writing to the file denoted by filename, using mode filemode (or “a” if filemode is not set).
syslog

If present, the root logger will be configured with a SysLogHandler.

If any true value except a dict is passed as value, simply adds a SysLogHandler() (default options). Otherwise the value should be a dict which might contain any of: host, port, facility.

format, datefmt
If present, the handler of the root logger will be configured to use a Formatter based on this format.
levels

A dictionary with logging levels keyed by logger name.

Note that the root logger level is set by associating a level with the empty string. I.e.:

'levels': {
  '': WARN,
}

Note

Logging is automatically configured by Configuration after some configuration has been loaded (if Configuration.logging_key is exists in the loaded configuration).

see:configure_logging()
see:Configuration.logging_key

Symbols

A set of basic symbols, meant to simplify syntax (and to make configuration files compatible with Python repr), are available through Configuration.default_symbols. During call-time, you can also pass an extra set of symbols, being combined with and overriding default_symbols when eval ing configurations.

from smisk.config import config
config.default_symbols['foo'] = 'Foo!'
config.loads('"some_key": foo')
print config['some_key']
# Foo!
config.loads('"some_key": foo', symbols={'foo':'BAR'})
print config['some_key']
# BAR
config.loads('"some_key": foo')
print config['some_key']
# Foo!

Predefined symbols

Symbol Python value
true True
false False
null None
CRITICAL logging.CRITICAL
FATAL logging.FATAL
ERROR logging.ERROR
WARN logging.WARN
WARNING logging.WARNING
INFO logging.INFO
DEBUG logging.DEBUG
NOTSET logging.NOTSET
critical logging.CRITICAL
fatal logging.FATAL
error logging.ERROR
warn logging.WARN
warning logging.WARNING
info logging.INFO
debug logging.DEBUG
notset logging.NOTSET

Practical use

Normally, you use the shared instance config

from smisk.config import config
config('my-app')
print config['some_key']

If your system have different default configuration directories than the default ones, these might be added module-wide by modifying config_locations

from smisk.config import config_locations, config
config_locations[0:0] = ['/etc/spotify/default', '/etc/spotify']
config('my-app')
# loading /etc/spotify/my-app.conf
print config['some_key']

In the case you need several sets of configurations in parallel, Configuration can be used to create new configuration dictionaries:

from smisk.config import Configuration
config1 = Configuration(some_key='default value')
config2 = Configuration()
config1('my-app1')
config2('my-app2')
print config1['some_key']
print config2['something_else']

Attributes

smisk.config.config
Shared Configuration.
smisk.config.config_locations
List of default directories in which to look for configurations files, effective when using Configuration.__call__().
smisk.config.LOGGING_FORMAT
Default logging format
smisk.config.LOGGING_DATEFMT
Default logging date format

Functions

smisk.config.configure_logging(conf)

Configure the logging module based on conf dictionary.

This function is automatically applied by Configuration after configuration has been loaded and if Configuration.logging_key is set (which it is by default).

See:Logging

Classes

class smisk.config.Configuration(dict)

Configuration dictionary.

Example use:

from smisk.config import Configuration
cfg = Configuration()
cfg('my-app')
print cfg['some_key']
defaults

Default values.

If you modify this dict after any configuration has been loaded, you need to call reload() afterwards, in order to actually apply the defaults.

To set or update single, specific default values, considering using set_default() instead, or simply assign a new dictionary to defaults. That way reloading is done automatically for you.

Default:{}
default_symbols

Default symbols.

See:Symbols
sources

Ordered list of sources used to create this dict.

Each entry is a tuple with two items:

( string <path or string hash>, dict configuration )

<path or string hash> is used to know where from and configuration is the unmodified, non-merged configuration this source generated.

Every Configuration instance contains a list of all sources (string and files) used to create the configuration dictionary. This information is used by Configuration.reload() in order to correctly update and merge options.

from smisk.config import config
config('my-app')
print 'Sources:', config.sources
Default:[]
filters

List of filters which are applied after configuration has been loaded.

A filter receives the Configuration instance calling it and should not return anything:

def my_filter(conf):
  if 'my_special_key' in conf:
    something_happens(conf['my_special_key'])
config.add_filter(my_filter)

Filters are automatically applied both when initially loading and reloading configuration.

Default:[]
See:add_filter()
filename_ext

Filename extension of configuration files

Default:".conf"
logging_key

Name of logging key

Default:"logging"
See:Logging
input_encoding

Character encoding used for reading configuration files.

Default:"utf-8"
max_include_depth

How deep to search for (and load) files denoted by a “@include”.

A value of 0 or lower disables includes.

Default:7
__init__(*args, **defaults)
Create a new Configuration, optionally setting defaults.
__call__(name, defaults=None, locations=[], symbols={}, logging_key=None)

Load configuration files from a series of pre-defined locations.

defaults is added to (and might override) defaults

By default, will look for these files in the following order:

/etc/default/<name>.conf
/etc/<name>.conf
/etc/<name>/<name>.conf
./<name>.conf
./<name>-user.conf
~/<name>.conf
set_default(key, value)
Assign a default value to key.
load(path, symbols={}, post_process=True) → dict

Load configuration from file denoted by path.

Returns the configuration loaded from path.

loads(string, symbols={}, post_process=True) → dict

Load configuration from string.

Returns the configuration loaded from string.

reload()

Reload all sources, effectively reloading configuration.

You can for example register a signal handler which reloads the configuration:

from smisk.config import config
import signal
signal.signal(signal.SIGHUP, lambda signum, frame: config.reload())
config('my_app')
import os
os.kill(os.getpid(), signal.SIGHUP)
# config.reload() called
reset(reset_defaults=True)

Reset this configuration dictionary.

Causes sources, filters and possibly defaults to be cleared as well as the configuration dictionary itself.

add_filter(filter)

Add a filter.

Simply does this:

if filter not in self.filters:
  self.filters.append(filter)
See:filters