API Reference¶
Global Objects¶
-
twiggy.
log
¶ the magic log object
-
twiggy.
internal_log
¶ InternalLogger
for reporting errors within Twiggy itself
-
twiggy.
devel_log
¶ InternalLogger
for use by developers writing extensions to Twiggy
Configuration¶
-
twiggy.
add_emitters
(*tuples)¶ Add multiple emitters
tuples
should be(name_of_emitter, min_level, filter, output)
. The last three are passed toEmitter
.
-
twiggy.
dict_config
(config)¶ Configure twiggy logging via a dictionary
Parameters: config – a dictionary which configures twiggy’s outputs and emitters. See TWIGGY_CONFIG_SCHEMA
for details of the format of the dict.See also
dict_config() for a thorough explaination of the outputs and emitters concepts from the dictionary
-
twiggy.
quick_setup
(min_level=<LogLevel DEBUG>, file=None, msg_buffer=0)¶ Quickly set up
emitters
.quick_setup() quickly sets up logging with reasonable defaults and minimal customizablity. Quick setup is limited to sending all messages to a file,
sys.stdout
orsys.stderr
. A timestamp will be prefixed when logging to a file.Parameters: - min_level (LogLevel) – lowest message level to cause output
- file (string) – filename to log to, or
sys.stdout
, orsys.stderr
.None
means standard error. - msg_buffer (int) – number of messages to buffer, see
outputs.AsyncOutput.msg_buffer
Features¶
Optional additions of logging functionality
procinfo¶
Logging feature to add information about process, etc.
-
twiggy.features.procinfo.
procinfo
(self)¶ Adds the following fields:
Hostname: current hostname Pid: current process id Thread: current thread name
socket¶
Logging feature to add information about a socket
-
twiggy.features.socket.
socket
(self, s)¶ Adds the following fields:
ip_addr: numeric IP address port: port number host: peer hostname, as returned by getnameinfo()
service: the human readable name of the service on port
Parameters: s (socket) – the socket to extract information from
Filters¶
-
twiggy.filters.
filter
(msg : Message) → bool¶ A filter is any function that takes a
Message
and returns True if it should beemitted
.
-
twiggy.filters.
msg_filter
(x) → filter¶ create a
filter
intelligentlyYou may pass:
None, True: the filter will always return True False: the filter will always return False string: compiled into a regex regex: match()
against the message textcallable: returned as is list: apply msg_filter
to each element, andall()
the resultsReturn type: filter
function
-
twiggy.filters.
names
(*names) → filter¶ create a
filter
, which gives True if the messsage’s name equals any of those providednames
will be stored as an attribute on the filter.Parameters: names (strings) – names to match Return type: filter
function
Formats¶
Formats are single-argument callables that take a Message
and return an object appropriate for the Output
they are assigned to.
-
class
twiggy.formats.
LineFormat
(separator=':', traceback_prefix='\nTRACE', conversion=line_conversion)¶ -
separator
¶ string to separate line parts. Defaults to
:
.
-
traceback_prefix
¶ string to prepend to traceback lines. Defaults to
\nTRACE
.Set to
'\\n'
(double backslash n) to roll up tracebacks to a single line.
-
conversion
¶ ConversionTable
used to formatfields
. Defaults toline_conversion
-
format_text
(msg)¶ format the text part of a message
-
format_fields
(msg)¶ format the fields of a message
-
format_traceback
(msg)¶ format the traceback part of a message
-
-
twiggy.formats.
line_conversion
¶ a default line-oriented
ConversionTable
. Produces a nice-looking string fromfields
.Fields are separated by a colon (
:
). Resultant string includes:time: in iso8601 format (required) level: message level (required) name: logger name Remaining fields are sorted alphabetically and formatted as
key=value
-
twiggy.formats.
line_format
¶ a default
LineFormat
for output to a file. Sample output.Fields are formatted using
line_conversion
and separated from the messagetext
by a colon (:
). Traceback lines are prefixed byTRACE
.
-
twiggy.formats.
shell_conversion
¶ a default line-oriented
ConversionTable
for use in the shell. Returns the same string asline_conversion
but drops thetime
field.
-
twiggy.formats.
shell_format
¶ a default
LineFormat
for use in the shell. Same asline_format
but usesshell_conversion
forfields
.
Levels¶
Levels include (increasing severity): DEBUG
, INFO
, NOTICE
, WARNING
, ERROR
,
CRITICAL
, DISABLED
-
class
twiggy.levels.
LogLevel
(name, value)¶ A log level. Users should not create new instances.
Levels are opaque; they may be compared to each other, but nothing else.
-
class
twiggy.levels.
LogLevelMeta
¶ Metaclass that aids in making comparisons work the same in Python2 and Python3
Python3 raises TypeError when unorderable types are compared via lt, gt, le, ge. Python2 picks an order but it doesn’t always make much sense.
In Python3, we only need the rich comparison operators to get this behaviour.
In Python2, we use the __cmp__ function to raise TypeError for lt, gt, le, and ge. We define __eq__ and __ne__ on their own since those should just say that a LogLevel is never equal to a non-LogLevel.
Library¶
-
twiggy.lib.
iso8601time
(gmtime=None)¶ convert time to ISO 8601 format - it sucks less!
Parameters: gmtime (time.struct_time) – time tuple. If None, use time.gmtime()
(UTC)XXX timezone is not supported
-
twiggy.lib.
thread_name
()¶ return the name of the current thread
Converter¶
-
class
twiggy.lib.converter.
Converter
(key, convert_value, convert_item, required=False)¶ Holder for
ConversionTable
itemsVariables: - key – the key to apply the conversion to
- convert_value (function) – one-argument function to convert the value
- convert_item (function) – two-argument function converting the key and converted value
- required (bool) – is the item required to present. Items are optional by default.
-
twiggy.lib.converter.
same_value
(v)¶ return the value unchanged
-
twiggy.lib.converter.
same_item
(k, v)¶ return the item unchanged
-
twiggy.lib.converter.
drop
(k, v)¶ return None, indicating the item should be dropped
New in version 0.5.0: Add same_value
, same_item
, drop
.
-
class
twiggy.lib.converter.
ConversionTable
(seq)¶ Convert data dictionaries using
Converters
For each item in the dictionary to be converted:
- Find one or more corresponding converters
c
by matching key. - Build a list of converted items by calling
c.convertItem(item_key, c.convertValue(item_value))
. The list will have items in the same order as converters were supplied. - Dict items for which no converter was found are sorted by key and passed to
generic_value
/generic_item
. These items are appended to the list from step 2. - If any required items are missing,
ValueError
is raised. - The resulting list of converted items is passed to
aggregate
. The value it returns is the result of the conversion.
Users may override
generic_value
/generic_item
/aggregate
by subclassing or assigning a new function on a ConversionTable instance.Really, it’s pretty intuitive.
-
__init__
(seq=None)¶ Parameters: seq – a sequence of Converters You may also pass 3-or-4 item arg tuples or kwarg dicts (which will be used to create
Converters
)
-
convert
(d)¶ do the conversion
Parameters: d (dict) – the data to convert. Keys should be strings.
-
copy
()¶ make an independent copy of this ConversionTable
-
get
(key)¶ return the first converter for key
-
get_all
(key)¶ return a list of all converters for key
-
delete
(key)¶ delete the all of the converters for key
- Find one or more corresponding converters
Logger¶
Loggers should not be created directly by users; use the global log
instead.
-
class
twiggy.logger.
BaseLogger
(fields=None, options=None, min_level=None)¶ Base class for loggers
-
_fields
¶ dictionary of bound fields for structured logging. By default, contains a single field
time
with valuetime.gmtime()
. This function will be called for each message emitted, populating the field with the currenttime.struct_time
.
-
fields
(**kwargs) → bound Logger¶ bind fields for structured logging.
kwargs
are interpreted as names/values of fields.
-
fields_dict
(d) → bound Logger¶ bind fields for structured logging. Use this instead of
fields
if you have keys which are not valid Python identifiers.Parameters: d (dict) – dictionary of fields. Keys should be strings.
-
trace
(trace='error') → bound Logger¶ convenience method to enable traceback logging
-
name
(name) → bound Logger¶ convenvience method to bind
name
field
-
struct
(**kwargs) → bound Logger¶ convenience method for structured logging. Calls
fields()
and emits atinfo
-
struct_dict
(d) → bound Logger¶ convenience method for structured logging. Use instead of
struct
if you have keys which are not valid Python identifiers.Parameters: d (dict) – dictionary of fields. Keys should be strings.
The following methods cause messages to be emitted.
format_spec
is a template string into whichargs
andkwargs
will be substitued.-
debug
(format_spec='', *args, **kwargs)¶ Emit at
DEBUG
level
-
info
(format_spec='', *args, **kwargs)¶ Emit at
INFO
level
-
notice
(format_spec='', *args, **kwargs)¶ Emit at
NOTICE
level
-
warning
(format_spec='', *args, **kwargs)¶ Emit at
WARNING
level
-
error
(format_spec='', *args, **kwargs)¶ Emit at
ERROR
level
-
critical
(format_spec='', *args, **kwargs)¶ Emit at
CRITICAL
level
-
-
class
twiggy.logger.
Logger
(fields=None, options=None, min_level=None)¶ Logger for end-users. The type of the magic
log
-
filter
¶ Filter on
format_spec
. For optimization purposes only. Should have the following signature:-
func
(format_spec : string) → bool Should the message be emitted.
-
-
classmethod
addFeature
(func, name=None)¶ add a feature to the class
Parameters: - func – the function to add
- name (string) – the name to add it under. If None, use the function’s name.
-
classmethod
disableFeature
(name)¶ disable a feature.
A method will still exist by this name, but it won’t do anything.
Parameters: name (string) – the name of the feature to disable.
-
classmethod
delFeature
(name)¶ delete a feature entirely
Parameters: name (string) – the name of the feature to remove
-
Message¶
-
class
twiggy.message.
Message
(level, format_spec, fields, options, args, kwargs)¶ A logging message. Users never create these directly.
Changed in version 0.4.1: Pass args/kwargs as list/dict instead of via
*
/**
expansion.The constructor takes a dict of
options
to control message creation. In addition tosuppress_newlines
, the following options are recognized:trace: control traceback inclusion. Either a traceback tuple, or one of the strings always
,error
, in which case a traceback will be extracted from the current stack frame.style: the style of template used for format_spec
. One ofbraces
,percent
,dollar
. The aliases{}
,%
and$
are also supported.Any callables passed in
fields
,args
orkwargs
will be called and the returned value used instead. See dynamic messages.All attributes are read-only.
-
fields
¶ dictionary of structured logging fields. Keys are string, values are arbitrary. A
level
item is required.
-
suppress_newlines
¶ should newlines be escaped in output. Boolean.
-
traceback
¶ a stringified traceback, or None.
-
text
¶ the human-readable message. Constructed by substituting
args
/kwargs
intoformat_spec
. String.
-
__init__
(level, format_spec, fields, options, args, kwargs)¶ Parameters: - level (LogLevel) – the level of the message
- format_spec (string) – the human-readable message template. Should match the
style
in options. - fields (dict) – dictionary of fields for structured logging
- args (tuple) – substitution arguments for
format_spec
. - kwargs (dict) – substitution keyword arguments for
format_spec
. - options (dict) – a dictionary of options to control message creation.
-
Outputs¶
-
class
twiggy.outputs.
Output
(format=None, close_atexit=True)¶ -
_format
¶ a callable taking a
Message
and formatting it for output. None means return the message unchanged.
-
use_locks
¶ Class variable, indicating that locks should be used when running in a synchronous, multithreaded environment. Threadsafe subclasses may disable locking for higher throughput. Defaults to True.
-
__init__
(format=None, close_atexit=True)¶ Parameters: - format (format) – the format to use. If None, return the message unchanged.
- close_atexit (bool) – should
close()
be registered withatexit
. If False, the user is responsible for closing the output.
New in version 0.4.1: Add the
close_atexit
parameter.-
close
()¶ Finalize the output.
The following methods should be implemented by subclasses.
-
_open
()¶ Acquire any resources needed for writing (files, sockets, etc.)
-
_write
(x)¶ Do the work of writing
Parameters: x – an implementation-dependent object to be written.
-
-
class
twiggy.outputs.
AsyncOutput
(msg_buffer=0)¶ An
Output
with support for asynchronous logging.Inheriting from this class transparently adds support for asynchronous logging using the multiprocessing module. This is off by default, as it can cause log messages to be dropped.
Parameters: msg_buffer (int) – number of messages to buffer in memory when using asynchronous logging. 0
turns asynchronous output off, a negative integer means an unlimited buffer, a positive integer is the size of the buffer.
-
class
twiggy.outputs.
FileOutput
(name, format, mode='a', buffering=1, msg_buffer=0, close_atexit=True)¶ Output messages to a file
name
,mode
,buffering
are passed toopen()
-
class
twiggy.outputs.
StreamOutput
(format, stream=sys.stderr)¶ Output to an externally-managed stream.
The stream will be written to, but otherwise left alone (i.e., it will not be closed).
-
class
twiggy.outputs.
NullOutput
(format=None, close_atexit=True)¶ An output that just discards its messages
-
class
twiggy.outputs.
ListOutput
(format=None, close_atexit=True)¶ an output that stuffs messages in a list
Useful for unittesting.
Variables: messages (list) – messages that have been emitted
Changed in version 0.4.1: Replace DequeOutput
with more useful ListOutput
.