cutechess -> pentanomial

The scripts bayes_opt and cutechess_batch make an incorrect assumption in the calculation of the pentanomial frequencies. Namely that the game pairs are consecutive in a cutechess match. That's not the case with concurrency>1. E.g.:

Finished game 1 (test vs base): 0-1 {Black wins by adjudication}
Finished game 2 (base vs test): 1/2-1/2 {Draw by 3-fold repetition}
Finished game 3 (test vs base): 1-0 {White wins by adjudication}
Finished game 5 (test vs base): 1-0 {White wins by adjudication}
Finished game 4 (base vs test): 1-0 {Black loses on time}
Finished game 8 (base vs test): 1-0 {White wins by adjudication}
Finished game 9 (test vs base): 1-0 {White wins by adjudication}
Finished game 6 (base vs test): 0-1 {White loses on time}
Finished game 10 (base vs test): 1-0 {White wins by adjudication}
Finished game 7 (test vs base): 0-1 {White loses on time}
Finished game 11 (test vs base): 1-0 {White wins by adjudication}
Finished game 12 (base vs test): 0-1 {Black wins by adjudication}
Finished game 13 (test vs base): 0-1 {Black wins by adjudication}
Finished game 15 (test vs base): 0-1 {Black wins by adjudication}
Finished game 16 (base vs test): 1-0 {White wins by adjudication}
Finished game 17 (test vs base): 1-0 {White wins by adjudication}
Finished game 14 (base vs test): 0-1 {White loses on time}
Finished game 18 (base vs test): 1/2-1/2 {Draw by 3-fold repetition}
Finished game 19 (test vs base): 1-0 {White wins by adjudication}
Finished game 20 (base vs test): 1-0 {White wins by adjudication}

The following should work:

    # Convert Cutechess-cli's result into W/L/D (putting game pairs together)
    lines = []
    for line in output.decode("utf-8").splitlines():
        if line.startswith('Finished game'):
           lines.append(line)

    lines.sort(key=lambda l: float(l.split()[2]))

    score = []
    for line in lines:
        if line.find(": 1-0") != -1:
          if line.find("test vs base") != -1:
            score.append('w')
          if line.find("base vs test") != -1:
            score.append('l')
        elif line.find(": 0-1") != -1:
          if line.find("test vs base") != -1:
            score.append('l')
          if line.find("base vs test") != -1:
            score.append('w')
        elif line.find(": 1/2-1/2") != -1:
          score.append('d')
        else:
            sys.stderr.write('the game did not terminate properly\n')
            return 2

Navigation Menu

cutechess -> pentanomial #1

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

cutechess -> pentanomial #1

Description

Activity

vondele commented on Jan 1, 2020

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions