adsp: discussion on umpf for patchwork

[sipraga]

Accidentally hijacked Jie's PR #3376 with a discussion on umpf. @nunojsa let's continue the discussion from your last comment #3376 (comment) here below.

[nunojsa]

Accidentally hijacked Jie's PR #3376 with a discussion on umpf.

Ehehehe :)

[sipraga]

@nunojsa wrote #3376 (comment):

Yeah, I think @pamolloy brought this tool not that long ago. If not Philip, someone in the org did :). Looks interesting and something that might be usable for ADSP.

When I first looked at, I did wondered about the xilinx based trees we have but my feeling for now is that is a can of worms that I'm not sure I want to open (given how big and messy our git history is for those branches).

I don't know enough to say for sure, but I guess if there isn't much cross-pollination between Xilinx and ADSP trees then a different process isn't out of the question. Just as long as it doesn't break things for non-ADSP stuff.

From a utag, umpf also supports generating a bitbake, buildroot, nix, or possibly other patch series in-situ within the meta or br2 repository. At B&O we would do our patchwork, push a tag, generate the new bitbake patch series, and essentially review .patches like on the kernel mailing list. Once approved he would then push his affected topic branches (we never bothered with automation here, although it's quite doable). The nice thing is that once the change was approved, it got put in the meta layer, a

The above does look interesting but sounds like a bunch of work for say, doing one single driver.

Almost all of it can be automated so long as there is an umpf tag - which would be needed for CI anyway. But specific process aside, I think umpf will only really be practical in ADSP if there is a "most up to date kernel" utag. Currently the most up to date is just the working git branch I guess.

It might be something to consider for the upstream patch series though. I think, ideally, we get into a place (at least I think that's the goal) where we upstream our platforms, cleanup a bunch of technical debt and then just go with an "upstream first" mindset. Then, hopefully, we just have a small set of out of tree code (I suspect we'll always have some - even in the ideal world).

Yes agree. I think the umpf workflow doesn't work very well when there is a lot of churn, particularly spanning various subsystems (which could mean juggling many topic branches for even minor janitorial changes).

We should also bear in mind that a lot of effort was put on current CI we have right now so I guess that any reviewing should still use the same infra we have now and this is very geared towards the github workflow (also generating artifacts from pushes and merges). Maybe trivial to adapt it for what you're suggesting, don't know :)

Also agree

...and that became the authoritative HEAD of our working kernel. Developers would then run a script to parse the tag from yocto, pull the tag, and generate a umerge from the utag to continue their own work.

I would also pushback a bit on the above depending on whether we want to have the concept of stable releases/trees or not. Example, a stable tree does not get any new driver or feature - just fixes.

I guess this also goes back to your point about aiming to upstream what we have when it's ready, and switching to a similar model as upstream.

[sipraga]

Almost all of it can be automated so long as there is an umpf tag - which would be needed for CI anyway. But specific process aside, I think umpf will only really be practical in ADSP if there is a "most up to date kernel" utag. Currently the most up to date is just the working git branch I guess.

Just to clarify, by "most up to date kernel", I just mean that people are able to agree on what is the common list of topic branches to be used. I can always regenerate my own utag/umerge from the topic branches on github, but I need to know which topics to use so that I can test my changes against the same things other people are using. There can obviously be multiple kernels in development at a time.

[nunojsa]

Yes agree. I think the umpf workflow doesn't work very well when there is a lot of churn, particularly spanning various subsystems (which could mean juggling many topic branches for even minor janitorial changes).

That kind of rules out xilinx stuff already. On those branches is like upstream + bunch of xilinx out of tree + bunch of ADI out of tree. So, very nice indeed :).

Almost all of it can be automated so long as there is an umpf tag - which would be needed for CI anyway

To give a bit context to my statement is that my default mindset for a single driver is to first mainline it. So my workflow is pretty much just work against an upstream tree (note that we also have some mirrors/* branches so when one opens a PR against it - for internal review - our CI runs and, in fact, it is more strict when targeting an upstream branch). Push it upstream with b4 and then when done, backport it opening a single PR with patches cherry picked from upstream. I guess the backport part could be done with umpf (making it more straight with build systems like yocto and buildroot)? Also depends on how likely it will be to backport for ADSP (considering a point in time where our platforms are upstream). For xilinx that happens a lot because those branches are also the place where various drivers for various chips (that don't have any dependence on xilinx platforms) land.