The Scheduler Module

This module is used to deal with anything related to test scheduling/partitioning in stestr.

stestr.scheduler.generate_worker_partitions(ids, worker_path, repository=None, group_callback=None, randomize=False)

Parse a worker yaml file and generate test groups

Parameters:

• ids (list) – A list of test ids too be partitioned

• worker_path (path) – The path to a worker file

• repository – A repository object that will be used for looking up timing data. This is optional, and also will only be used for scheduling if there is a count field on a worker.

• group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker. This is optional and also will only be used for scheduling if there is a count field on a worker.

• randomize (bool) – If true each partition’s test order will be randomized. This is optional and also will only be used for scheduling if there is a count field on a worker.

Returns:

A list where each element is a distinct subset of test_ids.

stestr.scheduler.get_dynamic_test_list(test_ids, repository=None, group_callback=None, randomize=False)

Group the test ids for the dynamic scheduler.

The ids are returned as a list of groups of test ids, sorted from the slowest to the fastest estimated group. Each group is the unit of work a dynamic scheduler worker picks from the queue: with a group_callback (or a group_regex config) a group is what the non-dynamic scheduler keeps on a single worker, e.g. a test class, so that the class level fixtures are only run once per worker like in the non-dynamic mode.

Parameters:

• test_ids (list) – The list of test_ids to be scheduled

• repository – A repository object used for looking up the timing data of the tests. Optional.

• group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept together on a worker. Optional.

• randomize (bool) – If true the order of the groups is randomized

Returns:

A list where each element is a distinct list of test_ids (a group), and the union of all the elements is equal to set(test_ids).

stestr.scheduler.local_concurrency()

Get the number of available CPUs on the system.

Returns:

An int for the number of cpus. Or None if it couldn’t be found

stestr.scheduler.partition_tests(test_ids, concurrency, repository, group_callback, randomize=False)

Partition test_ids by concurrency.

Test durations from the repository are used to get partitions which have roughly the same expected runtime. New tests - those with no recorded duration - are allocated in round-robin fashion to the partitions created using test durations.

Parameters:

• test_ids (list) – The list of test_ids to be partitioned

• concurrency (int) – The concurrency that will be used for running the tests. This is the number of partitions that test_ids will be split into.

• repository – A repository object that

• group_callback – A callback function that is used as a scheduler hint to group test_ids together and treat them as a single unit for scheduling. This function expects a single test_id parameter and it will return a group identifier. Tests_ids that have the same group identifier will be kept on the same worker.

• randomize (bool) – If true each partition’s test order will be randomized

Returns:

A list where each element is a distinct subset of test_ids, and the union of all the elements is equal to set(test_ids).