Configuring Output¶

This part discusses how to configure twiggy’s output of messages. You should do this once, near the start of your application’s __main__.

Quick Setup¶

To quickly configure output, use the quick_setup function. Quick setup is limited to sending all messages to a file or sys.stderr. A timestamp will be prefixed when logging to a file.

twiggy.quick_setup(min_level=<LogLevel DEBUG>, file=None, msg_buffer=0)

Quickly set up emitters.

Parameters:	min_level (LogLevel) – lowest message level to cause output file (string) – filename to log to, or `sys.stdout`, or `sys.stderr`. `None` means standard error. msg_buffer (int) – number of messages to buffer, see `outputs.AsyncOutput.msg_buffer`

twiggy_setup.py¶

Twiggy’s output side features modern, loosely coupled design.

By convention, your configuration lives in a file in your application called twiggy_setup.py, in a function called twiggy_setup(). You can of course put your configuration elsewhere, but using a separate module makes integration with configuration management systems easy. You should import and run twiggy_setup near the top of your application. It’s particularly important to set up Twiggy before spawning new processes.

A twiggy_setup function should create ouputs and use the add_emitters() convenience function to link those outputs to the log.

from twiggy import add_emitters, outputs, levels, filters, formats, emitters # import * is also ok
def twiggy_setup():
    alice_output = outputs.FileOutput("alice.log", format=formats.line_format)
    bob_output = outputs.FileOutput("bob.log", format=formats.line_format)

    add_emitters(
        # (name, min_level, filter, output),
        ("alice", levels.DEBUG, None, alice_output),
        ("betty", levels.INFO, filters.names("betty"), bob_output),
        ("brian.*", levels.DEBUG, filters.glob_names("brian.*"), bob_output),
        )

# near the top of your __main__
twiggy_setup()

add_emitters() populates the emitters dictionary:

>>> sorted(emitters.keys())
['alice', 'betty', 'brian.*']

In this example, we create two log destinations: alice.log and bob.log. alice will receive all messages, and bob will receive two sets of messages:

messages with the name field equal to betty and level >= INFO
messages with the name field glob-matching brian.*

Emitters can be removed by deleting them from this dict. filter and min_level may be modified during the running of the application, but outputs cannot be changed. Instead, remove the emitter and re-add it.

>>> # bump level
... emitters['alice'].min_level = levels.WARNING
>>> # change filter
... emitters['alice'].filter = filters.names('alice', 'andy')
>>> # remove entirely
... del emitters['alice']

We’ll examine the various parts in more detail.

Outputs¶

Outputs are the destinations to which log messages are written (files, databases, etc.). Several implementations are provided. Once created, outputs cannot be modified. Each output has an associated format.

Asynchronous Logging¶

Many outputs can be configured to use a separate, dedicated process to log messages. This is known as asynchronous logging and is enabled with the msg_buffer argument. Asynchronous mode dramatically reduces the cost of logging, as expensive formatting and writing operations are moved out of the main thread of control.

Formats¶

Formats transform a log message into a form that can be written by an output. The result of formatting is output dependent; for example, an output that posts to an HTTP server may take a format that provides JSON, whereas an output that writes to a file may produce text.

Line-oriented formatting¶

LineFormat formats messages for text-oriented outputs such as a file or standard error. It uses a ConversionTable to stringify the arbitrary fields in a message. To customize, copy the default line_format and modify:

# in your twiggy_setup
import copy
my_format = copy.copy(formats.line_format)
my_format.conversion.add(key = 'address', # name of the field
                         convert_value = hex, # gets original value
                         convert_item = "{0}={1}".format, # gets called with: key, converted_value
                         required = True)

# output messages with name 'memory' to stderr
add_emitters(('memory', levels.DEBUG, filters.names('memory'), outputs.StreamOutput(format = my_format)))

Filtering Output¶

The messages output by an emitter are determined by its min_level and filter (a function which take a Message and returns bool). These attributes may be changed while the application is running. The filter attribute of emitters is intelligent; you may assign strings, bools or functions and it will magically do the right thing. Assigning a list indicates that all of the filters must pass for the message to be output.

e = emitters['memory']
e.min_level = levels.WARNING
# True allows all messages through (None works as well)
e.filter = True
# False blocks all messages
e.filter = False
# Strings are interpreted as regexes (regex objects ok too)
e.filter = "^mem.*y$"
# functions are passed the message; return True to emit
e.filter = lambda msg: msg.fields['address'] > 0xDECAF
# lists are all()'d
e.filter = ["^mem.y$", lambda msg: msg.fields['address'] > 0xDECAF]