test: run unit tests for each built wheel

[2bndy5]

Uses cibuildwheel to run unit tests on each binary wheel built (natively, without qemu).

This could replace the current tests.yml workflow as it more accurately represents user experience during tests.

If removing test.yml in favor of this patch, then I would also adjust the CI triggers for wheels.yml (to run in PR sync events). See also #1366 for additional proposal.

[2bndy5]

I had to fix an assertion that was too broad for the test requirements. I also have to skip testing wheels built when qemu is involved (ppc64le).

Otherwise, the tests are running fine on native built wheels (x86, x64, arm). 🎉

[jdavid]

This is cool. Now as you said we could replace the tests workflow, and run this instead.
To do this I think we need two changes:

• Most of the time is spent in the ppc build, this should only run in the master branch.
• The tests workflow runs for the s390x platform too, but they fail. I think we should keep the tests workflow but run it only in the s390x platform, and only in master. If in the future the tests are fixed for this platform, then move it to the wheels. I would keep the workflow tests.yml so it does not add a new entry, and just change the name to "Tests (s390x)".

What do you think? Will you make a new PR?

[2bndy5]

What do you think? Will you make a new PR?

Sure. That sounds rather easy enough.

---

BTW, What's the procedure for updating the CHANGELOG? Is that dependent on maintainers? Is there interest in auto-creating a GitHub release (when a tag is pushed) with auto-generated notes?

[jdavid]

I maintain the changelog, so far it's not much of a burden, so I didn't bother with automating it.

[2bndy5]

Yeah, automating the changelog is best done with conventional-commit messages (the format I've been using for all my PR titles).

It would be easy to add a step to wheels.yml pypi job:

    - name: Create GitHub Release
      env:
        GITHUB_TOKEN: ${{ github.token }}
        TAG: ${{ github.ref_name }}
        REPO: ${{ github.repository }}
      # https://cli.github.com/manual/gh_release_create
      run: >-
        gh release create ${TAG} 
        --verify-tag
        --repo ${REPO}
        --title ${TAG}
        --generate-notes

That way, people watching this repo will be notified when you push a new tag.