gluetool.log module

Logging support.

Sets up logging environment for use by gluetool and modules. Based on standard library’s logging module, augmented a bit to support features loke colorized messages and stackable context information.

Example usage:

# initialize logger as soon as possible
logger = Logging.create_logger()

# now it's possible to use it for logging:
logger.debug('foo!')

# or connect it with current instance (if you're doing all this
# inside some class' constructor):
logger.connect(self)

# now you can access logger's methods directly:
self.debug('foo once again!')

# find out what your logging should look like, e.g. by parsing command-line options
...

# tell logger about the final setup
logger = Logging.create_logger(output_file='/tmp/foo.log', level=...)

# and, finally, create a root context logger - when we create another loggers during
# the code flow, this context logger will be in the root of this tree of loggers.
logger = ContextAdapter(logger)

# don't forget to re-connect with the context logger if you connected your instance
# with previous logger, to make sure helpers are set correctly
logger.connect(self)

class gluetool.log.BlobLogger(intro, outro=None, on_finally=None, writer=None)

Bases: object

Context manager to help with “real time” logging - some code may produce output continuously, e.g. when running a command and streaming its output to our stdout, and yet we still want to wrap it with boundaries and add a header.

This code:

with BlobLogger('ls of root', outro='end of ls'):
    subprocess.call(['ls', '/'])

will lead to the output similar to this:

[20:30:50] [+] ---v---v---v---v---v--- ls of root
bin  boot  data  dev ...
[20:30:50] [+] ---^---^---^---^---^--- end of ls

Note

When you already hold the data you wish to log, please use gluetool.log.log_blob() or gluetool.log.log_dict(). The example above could be rewritten using log_blob by using subprocess.check_output() and passing its return value to log_blob. BlobLogger is designed to wrap output whose creation caller don’t want to (or cannot) control.

Parameters:

• intro (str) – Label to show what is the meaning of the logged data.
• outro (str) – Label to show by the final boundary to mark the end of logging.
• on_finally (callable) – When set, it will be called in __exit__ method. User of this context manager might need to flush used streams or close resources even in case the exception was raised while inside the context manager. on_finally is called with all arguments the __exit__ was called, and its return value is returned by __exit__ itself, therefore it can examine possible exceptions, and override them.
• writer (callable) – A function which is used to actually log the text. Usually a one of some logger methods.

class gluetool.log.ContextAdapter(logger, extra=None)

Bases: logging.LoggerAdapter

Generic logger adapter that collects “contexts”, and prepends them to the message.

“context” is any key in extra dictionary starting with ctx_, whose value is expected to be tuple of (priority, value). Contexts are then sorted by their priorities before inserting them into the message (lower priority means context will be placed closer to the beggining of the line - highest priority comes last.

Parameters:

• logger (logging.Logger) – parent logger this adapter modifies.
• extras (dict) – additional extra keys passed to the parent class. The dictionary is then used to update messages’ extra key with the information about context.

connect(parent)

Create helper methods in parent, by assigning adapter’s methods to its attributes. One can then call parent.debug and so on, instead of less readable parent.logger.debug.

Simply instantiate adapter and call its connect with an object as a parent argument, and the object will be enhanced with all these logging helpers.

Parameters:parent – object to enhance with logging helpers.

process(msg, kwargs)

Original process overwrites kwargs['extra'] which doesn’t work for us - we want to chain adapters, getting more and more contexts on the way. Therefore update instead of assignment.

class gluetool.log.Logging

Bases: object

Container wrapping configuration and access to logging infrastructure gluetool uses for logging.

static _close_output_file()

If opened, close output file used for logging.

This method is registered with atexit.

static create_logger(output_file=None, level=20, sentry=None, sentry_submit_warning=None)

Create and setup logger.

This method is called at least twice:

• when gluetool.glue.Glue is instantiated: only a stderr handler is set up, with loglevel being INFO;
• when all arguments and options are processed, and Glue instance can determine desired log level, whether it’s expected to stream debugging messages into a file, etc. This time, method only modifies propagates necessary updates to already existing logger.

Parameters:

• output_file (str) – if set, new handler will be attached to the logger, streaming messages of all log levels into this this file.
• level (int) – desired log level. One of constants defined in logging module, e.g. logging.DEBUG or logging.ERROR.
• sentry (bool) – if set, logger will be augmented to send every log message to the Sentry server.
• sentry_submit_warning (callable) – if set, it is used by warning methods of derived loggers to submit warning to the Sentry server, if asked by a caller to do so.

Return type:

logging.Logger

Returns:

a logging.Logger instance, set up for logging.

static get_logger()

Returns a logger instance.

Expects there was a call to create_logger() method before calling this method that would actually create and set up the logger.

Return type:logging.Logger Returns:a logging.Logger instance, set up for logging, or None when there’s no logger yet.

logger = None

Logger singleton - if anyone asks for a logger, they will get this one. Needs to be properly initialized by calling create_logger().

output_file = None

If enabled, handles output to catch-everything file.

output_file_handler = None

stderr_handler = None

Stream handler printing out to stderr.

class gluetool.log.LoggingFormatter(colors=True, log_tracebacks=False)

Bases: logging.Formatter

Custom log record formatter. Produces output in form of:

[stamp] [level] [ctx1] [ctx2] ... message

Parameters:

• colors (bool) – if set, colorize output. Enabled by default but when used with file-backed destinations, colors are disabled by logging subsystem.
• log_tracebacks (bool) – if set, add tracebacks to the message. By default, we don’t need tracebacks on the terminal, unless its loglevel is verbose enough, but we want them in the debugging file.

static _format_exception_chain(exc_info)

_level_color = {40: <function <lambda>>, 50: <function <lambda>>, 20: <function <lambda>>, 30: <function <lambda>>}

Colorizers assigned to loglevels

_level_tags = {5: 'V', 40: 'E', 10: 'D', 50: 'C', 20: '+', 30: 'W'}

Tags used to express loglevel.

format(record)

Format a logging record. It puts together pieces like time stamp, log level, possibly also different contexts if there are any stored in the record, and finally applies colors if asked to do so.

Parameters:record (logging.LogRecord) – record describing the event. Return type:str Returns:string representation of the event record.

class gluetool.log.ModuleAdapter(logger, module)

Bases: gluetool.log.ContextAdapter

Custom logger adapter, adding module name as a context.

Parameters:

• logger (logging.Logger) – parent logger this adapter modifies.
• module (gluetool.glue.Module) – module whose name is added as a context.

gluetool.log.format_dict(dictionary)

Format a Python data structure for printing. Uses json.dumps() formatting capabilities to present readable representation of a given structure.

gluetool.log.log_blob(writer, intro, blob)

Log “blob” of characters of unknown structure, e.g. output of a command or response of a HTTP request. The blob is preceded by a header and followed by a footer to mark exactly the blob boundaries.

Note

For logging structured data, e.g. JSON or Python structures, use gluetool.log.log_dict(). It will make structure of the data more visible, resulting in better readability of the log.

Parameters:

• writer (callable) – A function which is used to actually log the text. Usually a one of some logger methods.
• intro (str) – Label to show what is the meaning of the logged blob.
• blob (str) – The actual blob of text.

gluetool.log.log_dict(writer, intro, data)

Log structured data, e.g. JSON responses or a Python list.

Note

For logging unstructured “blobs” of text, use gluetool.log.log_blob(). It does not attempt to format the output, and wraps it by header and footer to mark its boundaries.

Note

Using gluetool.log.format_dict() directly might be shorter, depending on your your code. For example, this code:

self.debug('Some data:\n{}'.format(format_dict(data)))

is equivalent to:

log_dict(self.debug, 'Some data', data)

If you need more formatting, or you wish to fit more information into a single message, using logger methods with format_dict is a way to go, while for logging a single structure log_dict is more suitable.

Parameters:

• writer (callable) – A function which is used to actually log the text. Usually a one of some logger methods.
• intro (str) – Label to show what is the meaning of the logged structure.
• blob (str) – The actual data to log.

gluetool.log.log_xml(writer, intro, element)

Log an XML element, e.g. Beaker job description.

Parameters:

• writer (callable) – A function which is used to actually log the text. Usually a one of some logger methods.
• intro (str) – Label to show what is the meaning of the logged blob.
• element – XML element to log.

gluetool.log.verbose_adapter(self, message, *args, **kwargs)

gluetool.log.verbose_logger(self, message, *args, **kwargs)

gluetool.log.warn_sentry(self, message, *args, **kwargs)

Beside calling the original the warning method (stored as self.orig_warning), this one also submits warning to the Sentry server when asked to do so by a keyword argument sentry set to True.