furemcape.transformer package

Transforms raw log data to hitdb entries.

class furemcape.transformer.Transformer(*args, **kwargs)

Bases: furemcape.shared.obj.BaseAttrObject

Feeds log data into the hitdb.

cfg

Configuration dict.

Type

dict

default_system

Fallback system definition.

Type

SystemDefinition

system_prototypes

Dict of prototypes.

Type

dict

systems

List of system definitions.

Type

list

hit_to_log

If set, hits are sent to this logger instead of hitdb.

Type

Logger

ignore_duplicate_hits

True to warn on duplicate hits, instead of die.

Type

boolean

LOG = <Logger furemcape.transformer.transformer (WARNING)>
SYSTEM_PROTOTYPES = {}
feed(data)

Transforms and inserts the specified log-data into the hitdb.

Parameters

data (dict) – Log-data to transform and insert.

hit(data)

Inserts the specified log-data into the hitdb.

Parameters

data (dict) – Log-data to insert.

load_config(path)

Replaces configuration with specified config file.

Parameters

path (str) – Path to config file.

transform(data)

Transforms the specified log-data.

Parameters

data (dict) – Log-data to transform.

Returns

Updated log data.

Return type

dict

validate_db()

Raises exception if db not available.

Raises
  • AttributeError – Database not initialized. # noqa: DAR402

  • DoesNotExist – Required migrations not run. # noqa: DAR402

Submodules

furemcape.transformer.config module

Utilities for loading transformer configuration.

furemcape.transformer.config.load_config(transformer, path)

Replaces transformer configuration with specified config file.

Parameters
  • transformer (Transformer) – Transformer object.

  • path (str) – Path to Furem Cape config file.

furemcape.transformer.config.load_default_config(transformer)

Replaces transformer configuration with default config file.

Parameters

transformer (Transformer) – Transformer object.

furemcape.transformer.config.set_config(transformer, cfg)

Replaces transformer configuration with specified config dict.

Parameters
  • transformer (Transformer) – Transformer object.

  • cfg (dict) – Furem Cape config dict.

furemcape.transformer.pipeline_step_definition module

Definition of a log-data transformation step.

class furemcape.transformer.pipeline_step_definition.PipelineStepDefinition(*args, **kwargs)

Bases: furemcape.shared.obj.BaseAttrObject

Definition of a log-data transformation step.

id

Step ID.

Type

str

step

Step number.

Type

int

processor

Step processor class name.

Type

str

processor_instance

Step processor instance.

Type

BaseProcessor

LOG = <Logger furemcape.transformer.pipeline_step_definition (WARNING)>
process(data)

Processes the specified log-entry data.

Parameters

data (dict) – Dict of log-entry data.

Returns

Updated dict of log-entry data.

Return type

dict

furemcape.transformer.system_definition module

Definition of log-data transformations for a particular system.

class furemcape.transformer.system_definition.SystemDefinition(*args, **kwargs)

Bases: furemcape.shared.obj.BaseAttrObject

Definition of log-data transformations for a particular system.

id

System definition ID (not necessarily the system ID).

Type

str

prototype

Definition ID of prototype to use for definition defaults.

Type

str

keys

List of log-data keys to match to determine when to use this definition.

Type

list

key

Single log-data key to use if keys not specified.

Type

str

match

Regex string against which to match keys to determine when to use this definition.

Type

str

match_re

Regex against which to match keys to determine when to use this definition.

Type

re

pipeline

Dict of pipeline configuration, keyed by pipeline step ID.

Type

dict

pipeline_steps

Ordered list of pipeline steps.

Type

list

LOG = <Logger furemcape.transformer.system_definition (WARNING)>
matches(data)

True if this definition matches the specified log-data.

Parameters

data (dict) – Dict of log-entry data.

Returns

True if this definition matches the specified log-data.

Return type

bool

furemcape.transformer.transformer module

Feeds log data into the hitdb.

class furemcape.transformer.transformer.Transformer(*args, **kwargs)

Bases: furemcape.shared.obj.BaseAttrObject

Feeds log data into the hitdb.

cfg

Configuration dict.

Type

dict

default_system

Fallback system definition.

Type

SystemDefinition

system_prototypes

Dict of prototypes.

Type

dict

systems

List of system definitions.

Type

list

hit_to_log

If set, hits are sent to this logger instead of hitdb.

Type

Logger

ignore_duplicate_hits

True to warn on duplicate hits, instead of die.

Type

boolean

LOG = <Logger furemcape.transformer.transformer (WARNING)>
SYSTEM_PROTOTYPES = {}
feed(data)

Transforms and inserts the specified log-data into the hitdb.

Parameters

data (dict) – Log-data to transform and insert.

hit(data)

Inserts the specified log-data into the hitdb.

Parameters

data (dict) – Log-data to insert.

load_config(path)

Replaces configuration with specified config file.

Parameters

path (str) – Path to config file.

transform(data)

Transforms the specified log-data.

Parameters

data (dict) – Log-data to transform.

Returns

Updated log data.

Return type

dict

validate_db()

Raises exception if db not available.

Raises
  • AttributeError – Database not initialized. # noqa: DAR402

  • DoesNotExist – Required migrations not run. # noqa: DAR402