pcvs.testing.test module

class pcvs.testing.test.Test(**kwargs)[source]

Bases: object

Smallest component of a validation process.

A test is basically a shell command to run. Depending on its post-execution status, a success or a failure can be determined. To handle such component in a convenient way, more information can be attached to the command like a name, the elapsed time, the output, etc.

In order to make test content flexible, there is no fixed list of attributes. A Test() constructor is initialized via (*args, **kwargs), to populate a dict _array.

Variables:

Timeout_RC (int) – special constant given to jobs exceeding their time limit.
NOSTART_STR (str) – constant, setting default output when job cannot be run.

MAXATTEMPTS_STR = b'This test has failed to be scheduled too many times. Discarded.'

NOSTART_STR = b'This test cannot be started.'

SCHED_MAX_ATTEMPTS = 50

class State(value)[source]

Bases: IntEnum

Provide Status management, specifically for tests/jobs.

Defined as an enum, it represents different states a job can take during its lifetime. As tests are then serialized into a JSON file, there is no need for construction/representation (as done for Session states).

Variables:

WAITING (int) – Job is currently waiting to be scheduled
IN_PROGRESS (int) – A running Set() handle the job, and is scheduled for run.
SUCCEED (int) – Job successfully run and passes all checks (rc, matchers…)
FAILED (int) – Job didn’t suceed, at least one condition failed.
ERR_DEP (int) – Special cases to manage jobs descheduled because at least one of its dependencies have failed to complete.
ERR_OTHER (int) – Any other uncaught situation.

ERR_DEP = 4

ERR_OTHER = 5

EXECUTED = 6

FAILURE = 3

IN_PROGRESS = 1

SUCCESS = 2

WAITING = 0

Timeout_RC = 127

property basename: str

been_executed()[source]

Cehck if job has been executed (not waiting or in progress).

Returns:: False if job is waiting for scheduling or in progress.
Return type:: bool

property combination

Getter to the test combination dict.

Returns:: test comb dict.
Return type:: dict

property command

Getter for the full command.

This is a real command, executed in a shell, coming from user’s specificaition. It should not be confused with invocation_command.

Returns:: unescaped command line
Return type:: str

classmethod compute_fq_name(label, subtree, name, combination=None, suffix=None)[source]: Generate the fully-qualified (dq) name for a test, based on : - the label & subtree (original FS tree) - the name (the TE name it is originated) - a potential extra suffix - the combination PCVS computed for this iteration.

display()[source]: Print the Test into stdout (through the manager).

property encoded_output: bytes

evaluate()[source]: TODO:

executed(state=None)[source]

Set current Test as executed.

Parameters:

state – give a special state to the test, defaults to FAILED
state – Test.State, optional

extract_metrics()[source]: TODO:

first_incomplete_dep()[source]

Retrive the first ready-for-schedule dep.

This is mainly used to ease the scheduling process by following the job dependency graph.

Returns:: a Test object if possible, None otherwise
Return type:: Test or NoneType

from_json(test_json)[source]

Replace the whole Test structure based on input JSON.

Parameters:: json (test-result-valid JSON-formated str) – the json used to set this Test
Return type:: None

from_minimal_json(json)[source]

generate_script(srcfile)[source]

Serialize test logic to its Shell representation.

This script provides the shell sequence to put in a shell script switch-case, in order to reach that test from script arguments.

Parameters:: srcfile (str) – script filepath, to store the actual wrapped command.
Returns:: the shell-compliant instruction set to build the test
Return type:: str

get_dep_graph()[source]

get_dim(unit='n_node')[source]

Return the orch-dimension value for this test.

The dimension can be defined by the user and let the orchestrator knows what resource are, and how to ‘count’ them’. This accessor allow the orchestrator to exract the information, based on the key name.

Parameters:: unit (str) – the resource label, such label should exist within the test
Returns:: The number of resource this Test is requesting.
Return type:: int

classmethod get_jid_from_name(name)[source]

get_raw_output(encoding='utf-8')[source]

Return type:: bytes

has_completed_deps()[source]

Check if the test can be scheduled.

It ensures it hasn’t been executed yet (or currently running) and all its deps are resolved and successfully run.

Returns:: True if the job can be scheduled
Return type:: bool

has_failed_dep()[source]

Check if at least one dep is blocking this job from ever be scheduled.

Returns:: True if at least one dep is shown a Test.State.FAILURE state.
Return type:: bool

property invocation_command

Getter for the list_of_test.sh invocation leading to run the job.

This command is under the form: sh /path/list_of_tests.sh <test-name>

Returns:: wrapper command line
Return type:: str

property jid: str

Getter for unique Job ID within a run.

This attribute is generally set by the manager once job is uploaded to the dataset. :return: the job id :rtype: an positive integer of -1 if not set

property job_depnames

Getter to the list of deps, as an array of names.

This array is emptied when all deps are converted to objects.

Returns:: the array of dep names
Return type:: list

property job_deps

“Getter to the dependency list for this job.

The dependency struct is an array, where for each name (=key), the associated Job is stored (value) :return: the list of object-converted deps :rtype: list

property label

Getter to the test label.

Returns:: the label
Return type:: str

property mod_deps

Getter to the list of pack-manager rules defined for this job.

There is no need for a _depnames version as these deps are provided as PManager objects directly.

Returns:: the list of package-manager based deps.
Return type:: list

property name

Getter for fully-qualified job name.

Returns:: test name.
Return type:: str

not_picked()[source]

property output: str

property output_info: dict

pick()[source]: Flag the job as picked up for scheduling.

pick_count()[source]

res_scheme = <pcvs.helpers.system.ValidationScheme object>

resolve_a_dep(name, obj)[source]

Resolve the dep object for a given dep name.

Parameters:

name (str) – the dep name to resolve, should be a valid dep.
obj (Test) – the dep object, should be a Test()

property retcode

save_final_result(rc=0, time=None, out=b'', state=None)[source]

Build the final Test result node.

Parameters:

rc (int, optional) – return code, defaults to 0
time (float, optional) – elapsed time, defaults to 0.0
out (bytes, optional) – standard out/err, defaults to b’’
state (Test.State, optional) – Job final status (if override needed), defaults to FAILED

save_raw_run(out=None, rc=None, time=None)[source]: TODO:

save_status(state)[source]

property state

Getter for current job state.

Returns:: the job current status.
Return type:: Test.State

property subtree

Getter to the test subtree.

Returns:: test subtree.
Return type:: str.

property tags

Getter for the full list of tags.

Returns:: the list of of tags
Return type:: list

property te_name

Getter to the test TE name.

Returns:: test TE name.
Return type:: str.

property time: TODO:

property timeout

Getter for Test timeout in seconds.

It cumulates timeout + tolerance, this value being passed to the subprocess.timeout.

Returns:: an integer if a timeout is defined, None otherwise
Return type:: int or NoneType

to_json(strstate=False)[source]

Serialize the whole Test as a JSON object.

Returns:: a JSON object mapping the test
Return type:: str

to_minimal_json()[source]