furemcape.issuedb package

Stores and queries hit issues.

furemcape.issuedb.init(config_file='', url='', **params)

Initializes the issuedb.

Parameters
  • config_file (str) – Path to config file (defaults to default config file locations).

  • url (str) – DB connection URL (defaults to parameters specified in config file, or ‘postgresext://issuedb:issuedb@localhost:5432/issuedb’).

  • **params (dict) – Other kwargs are passed as connection parameters to DB.

furemcape.issuedb.migrate(migrate_dir='migrations')

Runs issuedb migrations.

Parameters

migrate_dir (str) – Path to migrations directory (defaults to ‘migrations’).

furemcape.issuedb.require(name)

Raises exception if issuedb migration not run.

Parameters

name (str) – Name of migration to check.

Raises

DoesNotExist – Migration not run. # noqa: DAR402

furemcape.issuedb.validate(migrate_dir='migrations')

Raises exception if not all issuedb migrations have been run.

Parameters

migrate_dir (str) – Path to migrations directory (defaults to ‘migrations’).

class furemcape.issuedb.Issue(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Individual issue entry.

id

Primary key.

Type

int

definition

Reference to issue_definition entry.

Type

Hit

hit

Reference to issue_hit entry.

Type

Hit

danger

Danger level (1-9).

Type

int

DoesNotExist

alias of IssueDoesNotExist

classmethod create_from_dict(d)

Creates a new issue entry from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved issue entry.

Return type

Hit

classmethod create_from_issues(issues, hit)

Creates new issue entries and other related data within an transaction.

Parameters
  • hit (IssueHit) – Hit model.

  • issues (list) – List of issue dicts, with at least “danger” and “definition” properties.

Returns

Newly saved issue entry.

Return type

Issue

danger = <IntegerField: Issue.danger>
definition = <ForeignKeyField: Issue.definition>
definition_id = <ForeignKeyField: Issue.definition>
hit = <ForeignKeyField: Issue.hit>
hit_id = <ForeignKeyField: Issue.hit>
id = <BigAutoField: Issue.id>
to_dict()

Returns dict of field values.

Returns

Field values of this entry.

Return type

dict

class furemcape.issuedb.IssueDefinition(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Issue definition.

id

Primary key.

Type

int

elements

List of elements (eg [‘ip’, ‘user’]).

Type

list

elements_flags

Bitset of elements (eg ip = 1, user = 4, etc). Use “elements” property instead.

Type

int

system

System ID.

Type

str

base

Base issue type (eg ‘not_seen_before’).

Type

str

variant

Custom variant of issue (eg ‘non_anon’).

Type

str

name

Display name of custom variant (eg ‘Logged in user’).

Type

str

description

User-defined description/notes.

Type

str

title

Full display title of variant (eg ‘Ip+User Not Seen Before: Logged in user’).

Type

str

DoesNotExist

alias of IssueDefinitionDoesNotExist

base = <TextField: IssueDefinition.base>
classmethod build_title(flags=0, base='', variant='', name='')

Builds display title from components.

Parameters
  • flags (int) – Elements bitset (eg 9).

  • base (str) – Base issue type (eg ‘not_seen_before’).

  • variant (str) – Custom variant of issue (eg ‘non_anon’).

  • name (str) – Display name of custom variant (eg ‘Logged in user’).

Returns

Display title (eg ‘Group+Ip Not Seen Before: Logged in user’).

Return type

str

classmethod create_from_dict(d)

Creates a new issue definition from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved issue definition.

Return type

Hit

description = <TextField: IssueDefinition.description>
element_flags = <BitField: IssueDefinition.element_flags>
static element_flags_to_list(flags)

Builds elements list from elements bitset.

Parameters

flags (int) – Elements bitset (eg 9).

Returns

Elements list (eg [‘group’, ‘ip’]).

Return type

list

classmethod element_flags_to_string(flags)

Builds elements string from elements bitset.

Parameters

flags (int) – Elements bitset (eg 9).

Returns

Elements string (eg ‘group_ip’).

Return type

str

property elements
id = <AutoField: IssueDefinition.id>
is_action = <peewee.Expression object>
is_batch = <peewee.Expression object>
is_block = <peewee.Expression object>
is_clump = <peewee.Expression object>
is_cluster = <peewee.Expression object>
is_error = <peewee.Expression object>
is_group = <peewee.Expression object>
is_ip = <peewee.Expression object>
is_resource = <peewee.Expression object>
is_session = <peewee.Expression object>
is_suite = <peewee.Expression object>
is_user = <peewee.Expression object>
issues
static list_to_element_flags(elements)

Builds elements bitset from elements list.

Parameters

elements (list) – Elements list (eg [‘group’, ‘ip’]).

Returns

Elements bitset (eg 9).

Return type

int

name = <TextField: IssueDefinition.name>
classmethod string_to_element_flags(elements)

Builds elements bitset from elements string.

Parameters

elements (str) – Elements string (eg ‘group_ip’).

Returns

Elements bitset (eg 9).

Return type

int

system = <TextField: IssueDefinition.system>
property title
to_dict()

Returns dict of field values.

Returns

Field values of this definition.

Return type

dict

variant = <TextField: IssueDefinition.variant>
class furemcape.issuedb.IssueHit(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Individual log entry.

id

Primary key (matches id of corresponding hit from hitdb).

Type

int

at

Entry timestamp.

Type

DateTime

system

System ID.

Type

str

session

Session token (eg ‘abc123==’ etc).

Type

str

ip

User IP address (eg ‘1.2.3.4’, ‘1234::abcd’, etc).

Type

str

user

User ID (eg ‘bob’, ‘bob@example.com’, etc).

Type

str

resource

Resource path/ID (eg ‘/documents/doc123.doc’ etc).

Type

str

action

Action ID (eg ‘GET’, ‘readDocument’, etc).

Type

str

error

Error ID (eg ‘404’, ‘Not Found’, etc).

Type

str

DoesNotExist

alias of IssueHitDoesNotExist

action = <TextField: IssueHit.action>
at = <DateTimeTZField: IssueHit.at>
classmethod create_from_dict(d)

Creates a new hit entry from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved hit entry.

Return type

Hit

error = <TextField: IssueHit.error>
id = <BigIntegerField: IssueHit.id>
ip = <InetField: IssueHit.ip>
issuehitresponded_set
issues
static next_to_respond(after='epoch')

Returns the next issue_hit entry to respond to.

Parameters

after (DateTime) – Limit to hits after the specified datetime (defaults to the beginning of the epoch).

Returns

Next issue_hit entry to respond to, or None.

Return type

Hit

resource = <TextField: IssueHit.resource>
session = <TextField: IssueHit.session>
system = <TextField: IssueHit.system>
to_dict()

Returns dict of field values.

Returns

Field values of this hit.

Return type

dict

user = <TextField: IssueHit.user>
class furemcape.issuedb.IssueHitResponded(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Records whether an individual issue has been responded to yet.

hit

Reference to issue_hit entry (1:1 with issue_hit table).

Type

Hit

at

Hit timestamp.

Type

DateTime

responded

True if responded.

Type

bool

DoesNotExist

alias of IssueHitRespondedDoesNotExist

at = <DateTimeTZField: IssueHitResponded.at>
hit = <ForeignKeyField: IssueHitResponded.hit>
id = <ForeignKeyField: IssueHitResponded.hit>
classmethod next_to_respond(after='epoch')

Returns the next issue_hit_responded entry.

Parameters

after (DateTime) – Limit to hits after the specified datetime (defaults to the beginning of the epoch).

Returns

Next issue_hit_responded entry, or None.

Return type

Hit

responded = <BooleanField: IssueHitResponded.responded>

Submodules

furemcape.issuedb.db module

Functions for setting up and accessing the issuedb.

class furemcape.issuedb.db.BaseModel(*args, **kwargs)

Bases: peewee.Model

Base model class for issuedb peewee models.

DoesNotExist

alias of BaseModelDoesNotExist

id = <AutoField: BaseModel.id>
furemcape.issuedb.db.db = <peewee.DatabaseProxy object>

Peewee Database object for issuedb.

furemcape.issuedb.db.init(config_file='', url='', **params)

Initializes the issuedb.

Parameters
  • config_file (str) – Path to config file (defaults to default config file locations).

  • url (str) – DB connection URL (defaults to parameters specified in config file, or ‘postgresext://issuedb:issuedb@localhost:5432/issuedb’).

  • **params (dict) – Other kwargs are passed as connection parameters to DB.

furemcape.issuedb.db.migrate(migrate_dir='migrations')

Runs issuedb migrations.

Parameters

migrate_dir (str) – Path to migrations directory (defaults to ‘migrations’).

furemcape.issuedb.db.require(name)

Raises exception if issuedb migration not run.

Parameters

name (str) – Name of migration to check.

Raises

DoesNotExist – Migration not run. # noqa: DAR402

furemcape.issuedb.db.validate(migrate_dir='migrations')

Raises exception if not all issuedb migrations have been run.

Parameters

migrate_dir (str) – Path to migrations directory (defaults to ‘migrations’).

furemcape.issuedb.issue module

Issuedb models.

class furemcape.issuedb.issue.Issue(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Individual issue entry.

id

Primary key.

Type

int

definition

Reference to issue_definition entry.

Type

Hit

hit

Reference to issue_hit entry.

Type

Hit

danger

Danger level (1-9).

Type

int

DoesNotExist

alias of IssueDoesNotExist

classmethod create_from_dict(d)

Creates a new issue entry from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved issue entry.

Return type

Hit

classmethod create_from_issues(issues, hit)

Creates new issue entries and other related data within an transaction.

Parameters
  • hit (IssueHit) – Hit model.

  • issues (list) – List of issue dicts, with at least “danger” and “definition” properties.

Returns

Newly saved issue entry.

Return type

Issue

danger = <IntegerField: Issue.danger>
definition = <ForeignKeyField: Issue.definition>
definition_id = <ForeignKeyField: Issue.definition>
hit = <ForeignKeyField: Issue.hit>
hit_id = <ForeignKeyField: Issue.hit>
id = <BigAutoField: Issue.id>
to_dict()

Returns dict of field values.

Returns

Field values of this entry.

Return type

dict

class furemcape.issuedb.issue.IssueDefinition(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Issue definition.

id

Primary key.

Type

int

elements

List of elements (eg [‘ip’, ‘user’]).

Type

list

elements_flags

Bitset of elements (eg ip = 1, user = 4, etc). Use “elements” property instead.

Type

int

system

System ID.

Type

str

base

Base issue type (eg ‘not_seen_before’).

Type

str

variant

Custom variant of issue (eg ‘non_anon’).

Type

str

name

Display name of custom variant (eg ‘Logged in user’).

Type

str

description

User-defined description/notes.

Type

str

title

Full display title of variant (eg ‘Ip+User Not Seen Before: Logged in user’).

Type

str

DoesNotExist

alias of IssueDefinitionDoesNotExist

base = <TextField: IssueDefinition.base>
classmethod build_title(flags=0, base='', variant='', name='')

Builds display title from components.

Parameters
  • flags (int) – Elements bitset (eg 9).

  • base (str) – Base issue type (eg ‘not_seen_before’).

  • variant (str) – Custom variant of issue (eg ‘non_anon’).

  • name (str) – Display name of custom variant (eg ‘Logged in user’).

Returns

Display title (eg ‘Group+Ip Not Seen Before: Logged in user’).

Return type

str

classmethod create_from_dict(d)

Creates a new issue definition from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved issue definition.

Return type

Hit

description = <TextField: IssueDefinition.description>
element_flags = <BitField: IssueDefinition.element_flags>
static element_flags_to_list(flags)

Builds elements list from elements bitset.

Parameters

flags (int) – Elements bitset (eg 9).

Returns

Elements list (eg [‘group’, ‘ip’]).

Return type

list

classmethod element_flags_to_string(flags)

Builds elements string from elements bitset.

Parameters

flags (int) – Elements bitset (eg 9).

Returns

Elements string (eg ‘group_ip’).

Return type

str

property elements
id = <AutoField: IssueDefinition.id>
is_action = <peewee.Expression object>
is_batch = <peewee.Expression object>
is_block = <peewee.Expression object>
is_clump = <peewee.Expression object>
is_cluster = <peewee.Expression object>
is_error = <peewee.Expression object>
is_group = <peewee.Expression object>
is_ip = <peewee.Expression object>
is_resource = <peewee.Expression object>
is_session = <peewee.Expression object>
is_suite = <peewee.Expression object>
is_user = <peewee.Expression object>
issues
static list_to_element_flags(elements)

Builds elements bitset from elements list.

Parameters

elements (list) – Elements list (eg [‘group’, ‘ip’]).

Returns

Elements bitset (eg 9).

Return type

int

name = <TextField: IssueDefinition.name>
classmethod string_to_element_flags(elements)

Builds elements bitset from elements string.

Parameters

elements (str) – Elements string (eg ‘group_ip’).

Returns

Elements bitset (eg 9).

Return type

int

system = <TextField: IssueDefinition.system>
property title
to_dict()

Returns dict of field values.

Returns

Field values of this definition.

Return type

dict

variant = <TextField: IssueDefinition.variant>
class furemcape.issuedb.issue.IssueHit(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Individual log entry.

id

Primary key (matches id of corresponding hit from hitdb).

Type

int

at

Entry timestamp.

Type

DateTime

system

System ID.

Type

str

session

Session token (eg ‘abc123==’ etc).

Type

str

ip

User IP address (eg ‘1.2.3.4’, ‘1234::abcd’, etc).

Type

str

user

User ID (eg ‘bob’, ‘bob@example.com’, etc).

Type

str

resource

Resource path/ID (eg ‘/documents/doc123.doc’ etc).

Type

str

action

Action ID (eg ‘GET’, ‘readDocument’, etc).

Type

str

error

Error ID (eg ‘404’, ‘Not Found’, etc).

Type

str

DoesNotExist

alias of IssueHitDoesNotExist

action = <TextField: IssueHit.action>
at = <DateTimeTZField: IssueHit.at>
classmethod create_from_dict(d)

Creates a new hit entry from the specified dict of field values.

Parameters

d (dict) – Dict of field values.

Returns

Newly saved hit entry.

Return type

Hit

error = <TextField: IssueHit.error>
id = <BigIntegerField: IssueHit.id>
ip = <InetField: IssueHit.ip>
issuehitresponded_set
issues
static next_to_respond(after='epoch')

Returns the next issue_hit entry to respond to.

Parameters

after (DateTime) – Limit to hits after the specified datetime (defaults to the beginning of the epoch).

Returns

Next issue_hit entry to respond to, or None.

Return type

Hit

resource = <TextField: IssueHit.resource>
session = <TextField: IssueHit.session>
system = <TextField: IssueHit.system>
to_dict()

Returns dict of field values.

Returns

Field values of this hit.

Return type

dict

user = <TextField: IssueHit.user>
class furemcape.issuedb.issue.IssueHitResponded(*args, **kwargs)

Bases: furemcape.issuedb.db.BaseModel

Records whether an individual issue has been responded to yet.

hit

Reference to issue_hit entry (1:1 with issue_hit table).

Type

Hit

at

Hit timestamp.

Type

DateTime

responded

True if responded.

Type

bool

DoesNotExist

alias of IssueHitRespondedDoesNotExist

at = <DateTimeTZField: IssueHitResponded.at>
hit = <ForeignKeyField: IssueHitResponded.hit>
id = <ForeignKeyField: IssueHitResponded.hit>
classmethod next_to_respond(after='epoch')

Returns the next issue_hit_responded entry.

Parameters

after (DateTime) – Limit to hits after the specified datetime (defaults to the beginning of the epoch).

Returns

Next issue_hit_responded entry, or None.

Return type

Hit

responded = <BooleanField: IssueHitResponded.responded>