Skip to content

update: Increase complexity x2. Add randomization in duration of the test. Suppress warnings.#16

Open
IgorPelevanyuk wants to merge 3 commits intoDIRACGrid:masterfrom
IgorPelevanyuk:master
Open

update: Increase complexity x2. Add randomization in duration of the test. Suppress warnings.#16
IgorPelevanyuk wants to merge 3 commits intoDIRACGrid:masterfrom
IgorPelevanyuk:master

Conversation

@IgorPelevanyuk
Copy link

Suppress #9
Fix #15

@fstagni fstagni requested a review from aldbr October 8, 2025 14:46
@chaen
Copy link

chaen commented Oct 9, 2025

I did not look at the implementation details, but just a general remark: LHCb relies heavily on db12 to asses the job duration, and the correlation between db12 and our application is important to preserve. So if changes to the output results are introduced, we will need to find a more "parametrizable way" to do

@IgorPelevanyuk
Copy link
Author

Considering increased complexity - results would be just more accurate, since test will run twice more time.
Randomization of time also will not change things drasticaly, especially with prolonged duration of the test. The change in results value is much smaller than the accuracy of DB12 benchmark.

Copy link
Collaborator

@aldbr aldbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • For the randomization and 2x complexity: see my comments in #15 (comment)

  • For the factors.json file, we have been using python3.11 for a few years and haven't noticed any issue even though there was no correction.
    I see 2 solutions:

  • we make the study to get the correction factors and we apply them (does it make sense now?). But we would need to do it every time we update the python version, which is cumbersome.

  • we remove this factors.json file and we stop relying on it. Here it means that we will drift away the original scores without noticing it (though in the current state it's already the case). On a higher level point of view, we might consider replicating the code in a language like Rust to avoid that kind of issue in the future.

Ideally, it would be better to separate these 2 aspects in different PRs I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem with uniformity of DB12 results

3 participants