Skip to content

Add benchmarking tool#16

Draft
bruno-f-cruz wants to merge 21 commits intomainfrom
feature-add-benchmarking-tool
Draft

Add benchmarking tool#16
bruno-f-cruz wants to merge 21 commits intomainfrom
feature-add-benchmarking-tool

Conversation

@bruno-f-cruz
Copy link
Member

This PR adds a new tool to run hardware tests of Harp devices against the Harp protocol specification.

The main features:

  • Integrates with the toolkit using the benchmark command: dotnet run --project Harp.Toolkit benchmark --port COM3 --report here.html
  • Takes as single required input the port of the device to test
  • Outputs output to the console using ANSI (also supports verbose mode to render all test results via a --verbose flag)
  • Using RazorLight templates, can also render the results to a html file that can be used to exchange results

glopesdev and others added 2 commits January 30, 2026 20:29
The aim is to make it easier to add and maintain individual features.

Common options have been refactored into separate classes for reuse.
@bruno-f-cruz
Copy link
Member Author

@glopesdev was this the integration you had in mind for new tools?
Mostly ok with it. Just a few questions:

  • Should we create new namespaces to not clutter the top level one (e.g.: Harp.Toolkit.Benchmark) or leave everything as Harp.Tookit
  • Are you ok with adding these two extra dependencies? (RazorLight and Ansi support)

@glopesdev
Copy link
Contributor

Should we create new namespaces to not clutter the top level one (e.g.: Harp.Toolkit.Benchmark) or leave everything as Harp.Tookit

@bruno-f-cruz Happy to create a namespace for particularly complex commands requiring many helper classes.

An alternative is to create a separate library for the command backend and then add it as a dependency (e.g. I am going to do this with the reflex generator).

Are you ok with adding these two extra dependencies? (RazorLight and Ansi support)

In principle not opposed to any dependencies, since this will be a self-contained dotnet tool, so it will be deployed and run completely independent from any bonsai project dependency, etc.

But let's discuss the details on the benchmarks and reflex generator PRs.

@bruno-f-cruz
Copy link
Member Author

Sounds good. I guess the best way to go about this is to expose a single method (e.g.: https://github.com/harp-tech/Harp.Benchmark/blob/bb0529326aee7a12ab9b33d2dfeec573f8bbd2ac/src/Harp.Benchmark/Program.cs#L43) as the entry point of the tool and create a single command subclass in this repository that wraps it?

glopesdev and others added 19 commits February 10, 2026 15:21
The aim is to make it easier to add and maintain individual features.

Common options have been refactored into separate classes for reuse.
Refactor repository layout to allow multiple projects
* Import device template project into toolkit
* Move version to common build props
Improve compliance with standardized repository layout and CI workflows.
Major highlights include:

* Consistent build and DocFX setups
* Everything builds and tests all the time
* Packages are published on GitHub continuously
* Packages are published to NuGet.org upon release
* Documentation website can be published on release

Migration notes include:

* Normalize license file formatting
* Exclude project files inside template for docs
* Avoid including symbols for template package

Template packages contain only content and no compiled code. Trying to
generate a symbol package will result in error NU5017.
The backend is implemented using the new Harp.Generators API so that
generation can run directly from the toolkit without additional
dependencies.
Allows automatically generating device.yml and ios.yml metadata files
from the legacy XLS device description format.
This way the template can evolve separately from other components and
also keep increasing its own version as guidelines evolve.
Error reporting is handled via the result object to avoid returning long
stack traces.
Describes intended usage of the new code generation commands. It has
been kept separate from the main README quick start guide for now.
Brings improvements to documentation and generator firmware naming
conventions.
Brings improvements to generator firmware naming conventions.
The aim is to make it easier to add and maintain individual features.

Common options have been refactored into separate classes for reuse.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants