API Reference

Global Objects


the magic log object


InternalLogger for reporting errors within Twiggy itself


InternalLogger for use by developers writing extensions to Twiggy


the global emitters dictionary, tied to the log



Add multiple emitters

tuples should be (name_of_emitter, min_level, filter, output). The last three are passed to Emitter.


Configure twiggy logging via a dictionary

Parameters:config – a dictionary which configures twiggy’s outputs and emitters. See TWIGGY_CONFIG_SCHEMA for details of the format of the dict.

See also

dict_config() for a thorough explaination of the outputs and emitters concepts from the dictionary

twiggy.quick_setup(min_level=<LogLevel DEBUG>, file=None, msg_buffer=0)

Quickly set up emitters.

quick_setup() quickly sets up logging with reasonable defaults and minimal customizablity. Quick setup is limited to sending all messages to a file, sys.stdout or sys.stderr. A timestamp will be prefixed when logging to a file.

  • min_level (LogLevel) – lowest message level to cause output
  • file (string) – filename to log to, or sys.stdout, or sys.stderr. None means standard error.
  • msg_buffer (int) – number of messages to buffer, see outputs.AsyncOutput.msg_buffer


Optional additions of logging functionality


Logging feature to add information about process, etc.


Adds the following fields:

Hostname:current hostname
Pid:current process id
Thread:current thread name


Logging feature to add information about a socket

twiggy.features.socket.socket(self, s)

Adds the following fields:

ip_addr:numeric IP address
port:port number
host:peer hostname, as returned by getnameinfo()
service:the human readable name of the service on port
Parameters:s (socket) – the socket to extract information from
twiggy.features.socket.socket_minimal(self, s)

Like socket, but only log ip_addr and port


twiggy.filters.filter(msg : Message) → bool

A filter is any function that takes a Message and returns True if it should be emitted.

twiggy.filters.msg_filter(x) → filter

create a filter intelligently

You may pass:

None, True:the filter will always return True
False:the filter will always return False
string:compiled into a regex
regex:match() against the message text
callable:returned as is
list:apply msg_filter to each element, and all() the results
Return type:filter function
twiggy.filters.names(*names) → filter

create a filter, which gives True if the messsage’s name equals any of those provided

names will be stored as an attribute on the filter.

Parameters:names (strings) – names to match
Return type:filter function
twiggy.filters.glob_names(*names) → filter

create a filter, which gives True if the messsage’s name globs those provided.

names will be stored as an attribute on the filter.

This is probably quite a bit slower than names().

Parameters:names (strings) – glob patterns.
Return type:filter function
class twiggy.filters.Emitter

Hold and manage an Output and associated filter()


only emit if greater than this LogLevel


arbitrary filter() on message contents. Assigning to this attribute is intelligent.


Output to emit messages to. Do not modify.


Formats are single-argument callables that take a Message and return an object appropriate for the Output they are assigned to.

class twiggy.formats.LineFormat(separator=':', traceback_prefix='\nTRACE', conversion=line_conversion)

string to separate line parts. Defaults to :.


string to prepend to traceback lines. Defaults to \nTRACE.

Set to '\\n' (double backslash n) to roll up tracebacks to a single line.


ConversionTable used to format fields. Defaults to line_conversion


format the text part of a message


format the fields of a message


format the traceback part of a message


a default line-oriented ConversionTable. Produces a nice-looking string from fields.

Fields are separated by a colon (:). Resultant string includes:

time:in iso8601 format (required)
level:message level (required)
name:logger name

Remaining fields are sorted alphabetically and formatted as key=value


a default LineFormat for output to a file. Sample output.

Fields are formatted using line_conversion and separated from the message text by a colon (:). Traceback lines are prefixed by TRACE.


a default line-oriented ConversionTable for use in the shell. Returns the same string as line_conversion but drops the time field.


a default LineFormat for use in the shell. Same as line_format but uses shell_conversion for fields.


Levels include (increasing severity): DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, DISABLED

class twiggy.levels.LogLevel(name, value)

A log level. Users should not create new instances.

Levels are opaque; they may be compared to each other, but nothing else.

class twiggy.levels.LogLevelMeta

Metaclass that aids in making comparisons work the same in Python2 and Python3

Python3 raises TypeError when unorderable types are compared via lt, gt, le, ge. Python2 picks an order but it doesn’t always make much sense.

In Python3, we only need the rich comparison operators to get this behaviour.

In Python2, we use the __cmp__ function to raise TypeError for lt, gt, le, and ge. We define __eq__ and __ne__ on their own since those should just say that a LogLevel is never equal to a non-LogLevel.


return a LogLevel from a case-insensitve string



convert time to ISO 8601 format - it sucks less!

Parameters:gmtime (time.struct_time) – time tuple. If None, use time.gmtime() (UTC)

XXX timezone is not supported


return the name of the current thread


class twiggy.lib.converter.Converter(key, convert_value, convert_item, required=False)

Holder for ConversionTable items

  • key – the key to apply the conversion to
  • convert_value (function) – one-argument function to convert the value
  • convert_item (function) – two-argument function converting the key and converted value
  • required (bool) – is the item required to present. Items are optional by default.

return the value unchanged

twiggy.lib.converter.same_item(k, v)

return the item unchanged

twiggy.lib.converter.drop(k, v)

return None, indicating the item should be dropped

New in version 0.5.0: Add same_value, same_item, drop.

class twiggy.lib.converter.ConversionTable(seq)

Convert data dictionaries using Converters

For each item in the dictionary to be converted:

  1. Find one or more corresponding converters c by matching key.
  2. Build a list of converted items by calling c.convertItem(item_key, c.convertValue(item_value)). The list will have items in the same order as converters were supplied.
  3. Dict items for which no converter was found are sorted by key and passed to generic_value / generic_item. These items are appended to the list from step 2.
  4. If any required items are missing, ValueError is raised.
  5. The resulting list of converted items is passed to aggregate. The value it returns is the result of the conversion.

Users may override generic_value/generic_item/aggregate by subclassing or assigning a new function on a ConversionTable instance.

Really, it’s pretty intuitive.

Parameters:seq – a sequence of Converters

You may also pass 3-or-4 item arg tuples or kwarg dicts (which will be used to create Converters)


do the conversion

Parameters:d (dict) – the data to convert. Keys should be strings.

convert values for which no specific Converter is supplied

generic_item(key, value)

convert items for which no specific Converter is supplied


aggregate list of converted items. The return value of convert


make an independent copy of this ConversionTable


return the first converter for key


return a list of all converters for key

add(*args, **kwargs)

Append a Converter.

args & kwargs will be passed through to its constructor


delete the all of the converters for key


Loggers should not be created directly by users; use the global log instead.

class twiggy.logger.BaseLogger(fields=None, options=None, min_level=None)

Base class for loggers


dictionary of bound fields for structured logging. By default, contains a single field time with value time.gmtime(). This function will be called for each message emitted, populating the field with the current time.struct_time.


dictionary of bound options.


minimum LogLevel for which to emit. For optimization purposes only.

fields(**kwargs) → bound Logger

bind fields for structured logging. kwargs are interpreted as names/values of fields.

fields_dict(d) → bound Logger

bind fields for structured logging. Use this instead of fields if you have keys which are not valid Python identifiers.

Parameters:d (dict) – dictionary of fields. Keys should be strings.
options(**kwargs) → bound Logger

bind options for message creation.

trace(trace='error') → bound Logger

convenience method to enable traceback logging

name(name) → bound Logger

convenvience method to bind name field

struct(**kwargs) → bound Logger

convenience method for structured logging. Calls fields() and emits at info

struct_dict(d) → bound Logger

convenience method for structured logging. Use instead of struct if you have keys which are not valid Python identifiers.

Parameters:d (dict) – dictionary of fields. Keys should be strings.

The following methods cause messages to be emitted. format_spec is a template string into which args and kwargs will be substitued.

debug(format_spec='', *args, **kwargs)

Emit at DEBUG level

info(format_spec='', *args, **kwargs)

Emit at INFO level

notice(format_spec='', *args, **kwargs)

Emit at NOTICE level

warning(format_spec='', *args, **kwargs)

Emit at WARNING level

error(format_spec='', *args, **kwargs)

Emit at ERROR level

critical(format_spec='', *args, **kwargs)

Emit at CRITICAL level

class twiggy.logger.Logger(fields=None, options=None, min_level=None)

Logger for end-users. The type of the magic log


Filter on format_spec. For optimization purposes only. Should have the following signature:

func(format_spec : string) → bool

Should the message be emitted.

classmethod addFeature(func, name=None)

add a feature to the class

  • func – the function to add
  • name (string) – the name to add it under. If None, use the function’s name.
classmethod disableFeature(name)

disable a feature.

A method will still exist by this name, but it won’t do anything.

Parameters:name (string) – the name of the feature to disable.
classmethod delFeature(name)

delete a feature entirely

Parameters:name (string) – the name of the feature to remove
class twiggy.logger.InternalLogger(output, fields=None, options=None, min_level=None)

Special-purpose logger for internal uses

Sends messages directly to output, bypassing emitters.

Variables:output (Output) – an output to write to

a decorator that emits at level after calling the method. The method should return a Logger instance.

For convenience, decorators for the various levels are available as emit.debug, emit.info, etc..


class twiggy.message.Message(level, format_spec, fields, options, args, kwargs)

A logging message. Users never create these directly.

Changed in version 0.4.1: Pass args/kwargs as list/dict instead of via */** expansion.

The constructor takes a dict of options to control message creation. In addition to suppress_newlines, the following options are recognized:

trace:control traceback inclusion. Either a traceback tuple, or one of the strings always, error, in which case a traceback will be extracted from the current stack frame.
style:the style of template used for format_spec. One of braces, percent, dollar. The aliases {}, % and $ are also supported.

Any callables passed in fields, args or kwargs will be called and the returned value used instead. See dynamic messages.

All attributes are read-only.


dictionary of structured logging fields. Keys are string, values are arbitrary. A level item is required.


should newlines be escaped in output. Boolean.


a stringified traceback, or None.


the human-readable message. Constructed by substituting args/kwargs into format_spec. String.

__init__(level, format_spec, fields, options, args, kwargs)
  • level (LogLevel) – the level of the message
  • format_spec (string) – the human-readable message template. Should match the style in options.
  • fields (dict) – dictionary of fields for structured logging
  • args (tuple) – substitution arguments for format_spec.
  • kwargs (dict) – substitution keyword arguments for format_spec.
  • options (dict) – a dictionary of options to control message creation.


class twiggy.outputs.Output(format=None, close_atexit=True)

a callable taking a Message and formatting it for output. None means return the message unchanged.


Class variable, indicating that locks should be used when running in a synchronous, multithreaded environment. Threadsafe subclasses may disable locking for higher throughput. Defaults to True.

__init__(format=None, close_atexit=True)
  • format (format) – the format to use. If None, return the message unchanged.
  • close_atexit (bool) – should close() be registered with atexit. If False, the user is responsible for closing the output.

New in version 0.4.1: Add the close_atexit parameter.


Finalize the output.

The following methods should be implemented by subclasses.


Acquire any resources needed for writing (files, sockets, etc.)


Release any resources acquired in _open


Do the work of writing

Parameters:x – an implementation-dependent object to be written.
class twiggy.outputs.AsyncOutput(msg_buffer=0)

An Output with support for asynchronous logging.

Inheriting from this class transparently adds support for asynchronous logging using the multiprocessing module. This is off by default, as it can cause log messages to be dropped.

Parameters:msg_buffer (int) – number of messages to buffer in memory when using asynchronous logging. 0 turns asynchronous output off, a negative integer means an unlimited buffer, a positive integer is the size of the buffer.
class twiggy.outputs.FileOutput(name, format, mode='a', buffering=1, msg_buffer=0, close_atexit=True)

Output messages to a file

name, mode, buffering are passed to open()

class twiggy.outputs.StreamOutput(format, stream=sys.stderr)

Output to an externally-managed stream.

The stream will be written to, but otherwise left alone (i.e., it will not be closed).

class twiggy.outputs.NullOutput(format=None, close_atexit=True)

An output that just discards its messages

class twiggy.outputs.ListOutput(format=None, close_atexit=True)

an output that stuffs messages in a list

Useful for unittesting.

Variables:messages (list) – messages that have been emitted

Changed in version 0.4.1: Replace DequeOutput with more useful ListOutput.