pcvs.orchestration.publishers module

class pcvs.orchestration.publishers.BuildDirectoryManager(build_dir='.')[source]

Bases: object

This class is intended to serve a build directory from a single entry point. Any module requiring to deal with resources from a run should be compliant with this interface. It provides basic mechanism to load/save any past, present or future executions.

add_cache_entry(idx=0)[source]

clean(*args)[source]

Prepare the build directory for a new execution by removing anything not relevant for a new run.

Please not this function will erase anything not relative to PCVS. As an argument, one may specify a specific prefix to be removed. Paths should relative to root build directory.

Return type:: None

clean_archives()[source]

Prepare the build directory for a new execution by moving any previous archive to the backup directory named after NAME_BUILD_ARCHIVE_DIR.

Return type:: None

property config: MetaConfig

Return the configuation associated with the current build directory

Returns:: config struct
Return type:: class:MetaConfig

create_archive(timestamp=None)[source]

Generate an archive for the build directory.

This archive will be stored in the root directory..

Parameters:: timestamp (Datetime, optional) – file suffix, defaults to current timestamp
Returns:: the archive path name
Return type:: str

finalize()[source]

Close & release the current instance.

It should not be used to save tests after this call.

get_cache_entry(idx=0)[source]

get_config()[source]

Return the loaded configuration for the current build directory.

Returns:: a dict representantion of yaml config
Return type:: dict

init_results(per_file_max_sz=0)[source]

Initialize the result handler.

This function is not called directly from the __init__ method as this isntance may be used for both reading & writing into the destination directory. This function implies storing a new execution.

Parameters:: per_file_max_sz (int, optional) – max file size, defaults to unlimited

load_config()[source]

Load config stored onto disk & populate the current instance.

Returns:: the loaded config
Return type:: class:MetaConfig

classmethod load_from_archive(archive_path)[source]

Populate the instance from an archive.

This object is initially built to load data from a build directory. This way, the object is mapped with an existing archive.

Warning

This method does not support (yet) to save tests after an archive has been loaded (as no output directory has been configured).

Parameters:: archive_path (_type_) – _description_
Returns:: _description_
Return type:: _type_

property prefix

Get the build directory prefix

Returns:: the build path
Return type:: str

prepare(reuse=False)[source]

Prepare the dir for a new run.

This function is not included as part of the __init__ function as this instance may be used both for reading & writing into the destination directory. This function implies all previous results be be cleared off.

Parameters:: reuse (bool, optional) – keep previously generated YAML test-files, defaults to False

property results

Getter to the result handler, for direct access

Returns:: the result handler
Return type:: class:ResultFileManager

save_config(config)[source]

Save the config & store it directly into the build directory.

Parameters:: config (class:MetaConfig) – config
Return type:: None

save_extras(rel_filename, data='', dir=False, export=False)[source]

Register a specific build-relative path, to be saved into the directory.

The only entry-point to save a resource into it. Resources can be files (with or without content) or directory.

If export is set to True, resource (file or whole directory) will also be copied to the final archive.

Parameters:

rel_filename (str) – the filepath, relative to build dir.
data (Any, optional) – data to be saved into the target file, defaults to “”
dir (bool, optional) – is it a directory to save, defaults to False
export (bool, optional) – should the target be also exported in final archive, defaults to False

Return type:

None

property scratch_location

Returns where third-party artifacts must be stored

Returns:: the scratch directory
Return type:: str

property sid: Optional[int]

Return the run ID as per configured with the current build directory.

If not found, this function may return None

Returns:: the session ID
Return type:: int

use_as_global_config()[source]

class pcvs.orchestration.publishers.ResultFile(filepath, filename)[source]

Bases: object

A instance manages a pair of file dedicated to load/store PCVS job results on disk.

A job result is stored in two different files whens given to a single ResultFile: * <prefix>.json, containing metadata (rc, command…) * <prefix>.bz2 BZ2-compressed job data.

a MAGIC_TOKEN is used to detect file/data corruption.

MAGIC_TOKEN = 'PCVS-START-RAW-OUTPUT'

close()[source]: Close the current instance (flush to disk)

property content

property count

Get the number of jobs in this handler.

Returns:: job count
Return type:: int

extract_output(offset, length)[source]

Return type:: str

flush()[source]: Sync cache with disk

load()[source]: Load job data from disk to populate the cache.

property metadata_prefix

Getter to the actual metadata file name

Returns:: filename
Return type:: str

property prefix

Getter to the build prefix

Returns:: build prefix
Return type:: str

property rawdata_prefix

Getter to the actual rawdata file name

Returns:: file name
Return type:: str

retrieve_test(id=None, name=None)[source]

Find jobs based on its id or name and return associated Test object.

Only id OR name should be set (not both). To handle multiple matches, this function returns a list of class:Test.

Parameters:

id (int, optional) – job id, defaults to None
name (str, optional) – test name (full), defaults to None

Returns:

A list of class:Test

Return type:

list

save(id, data, output)[source]

Save a new job to this instance.

Parameters:

id (int) – job id
data (dict) – metadata
output (bytes) – raw output

property size

Current rawdata size

Returns:: lenght of the rawdata file.
Return type:: int

class pcvs.orchestration.publishers.ResultFileManager(prefix='.', per_file_max_ent=0, per_file_max_sz=0)[source]

Bases: object

Manages multiple class:ResultFile. Main purpose is to manage files to ensure files stored on disk remain consistent.

browse_tests()[source]

Iterate over every job stored into this build directory.

Returns:: an iterable of Tests
Return type:: List of tests
Yield:: Test
Return type:: Iterator[Test]

build_bidir_map_data()[source]

Rebuild global views from partial storage on disk.

For optimization reasons, information that may be rebuilt are not stored on disk to save space.

Return type:: None

create_new_result_file()[source]

Initialize a new result file handler upon request.

Return type:: None

discover_result_files()[source]

Load existing results from prefix.

Return type:: None

file_format = 'jobs-{}'

finalize()[source]

Flush & close the current manager.

This instance should not be used again after this call.

flush()[source]

Ensure everything is in sync with persistent storage.

Return type:: None

increment = 0

map_id(id)[source]

Comnvert a job ID into its class:Test representation.

Parameters:: id (int) – job id
Returns:: the associated Test object or None if not found
Return type:: class:Test or None

property maps

Returns available views from the current instance.

Returns:: the maps
Return type:: dict

reconstruct_map_data()[source]

Return type:: None

reconstruct_view_data()[source]

Return type:: None

register_view(name)[source]

Initialize a new view for this result manager.

Parameters:: name (str) – the view name
Return type:: None

register_view_item(view, item)[source]

Initialize a single item within a view.

Parameters:

view (str) – the view name (created if not exist)
item (str) – the item

Return type:

None

retrieve_test(id)[source]

Build the Test object mapped to the given job id.

If such ID does not exist, it will return None.

Parameters:: id (_type_) – _description_
Returns:: _description_
Return type:: List[Test]

retrieve_tests_by_name(name)[source]

Locate a test by its name.

As multiple matches could occur, this function return a list of class:Test

Parameters:: name (str) – the test name
Returns:: the actual list of test, empty if no one is found
Return type:: list

save(job)[source]

Add a new job to be saved to the result directory.

May not be flushed righ away to disk, some caching may be used to improve performance. While adding the Test to low-level manager, this function also updates view & maps accordingly.

Parameters:: job (class:Test) – the job element to store

property status_view

Returns the status view provided by PCVS.

Returns:: a view
Return type:: dict

subtree_view(subtree)[source]

Get a subset of the ‘tree’ view. Any LABEL/subtree combination is valid.

Parameters:: subtree (str) – the prefix to look for
Returns:: the dict mapping tests to the request
Return type:: dict

property tags_view

Get the tags view provided by PCVS.

Returns:: a view
Return type:: dict

property total_cnt

Returns the total number of jobs from that directory (=run).

Returns:: number of jobs
Return type:: int

property tree_view

Get the tree view, provided by default.

Returns:: a view
Return type:: dict

property views

Returns available views for the current instance.

Returns:: the views
Return type:: dict