feat: restructure repository by akihikokuroda · Pull Request #40 · generative-computing/mellea-contribs

akihikokuroda · 2026-03-18T19:01:39Z

jakelorocco

Is it possible to make the workflows dynamic? Ie instead of having to create a new one for each subpackage, we just mandate all folders under mellea_contribs are subpackages and require some sort of structure?

Even if we don't have that, we should make sure to have a version of the quality check that ensures new packages have tests. For example, if I make a new subpackage and don't add tests / quality workflows right now, will I be able to merge? Or there's some necessary check still?

I'm also not certain how these differently named checks interact with github branch protections? For the mellea repo, we have to specifically select checks.

akihikokuroda · 2026-03-20T16:34:19Z

@jakelorocco A new CI that checks if each subpackage has its own ci can be added. The quality of each ci is up-to the subpackage though. Is it enough as a minimum?

jakelorocco · 2026-03-20T17:09:09Z

@jakelorocco A new CI that checks if each subpackage has its own ci can be added. The quality of each ci is up-to the subpackage though. Is it enough as a minimum?

I think that makes sense. We should vet CI at merge time, but a check will ensure nothing gets through accidentally.

psschwei · 2026-03-23T13:17:10Z

Once this merges, we'll also need to update the required checks in the branch protections ( as the quality checks no longer exist)

psschwei · 2026-03-23T13:20:25Z

.github/workflows/ci.yml

 on:
-  pull_request:
-    types: [opened, reopened, synchronize]
+  workflow_dispatch:


this would need to be run manually right? would this be more for auditing or do you see another purpose?

Yes, this must be run manually. I just left it for convenience. It can be deleted.

So we can't have checks on this repo then? Or the checks just need to be manually kicked off for a given branch / PR?

I also don't know how branch protections interact with this. And also, if this is the case, why are the github action runners showing that checks ran below in the PR?

psschwei · 2026-03-23T13:25:27Z

.github/workflows/ci-crewai-backend.yml

+  push:
+    branches:
+      - main
+    paths:


since all the backends depend on mellea-integration-core, there is the chance that if we change something in the core that it would break the backends... should we also trigger a run of the backends if the core changes?

That's right. I'll add mellea-integration-core change in all integration paths.

I don't think we should create cross-dependencies in this repo. See my comment below, but if we update mellea-integration-core and it forces us to update other packages, then we've just inherited a ton of maintenance work.

I took out the cross-dependencies build.

psschwei · 2026-03-23T15:41:17Z

mellea_contribs/mellea-integration-core/pyproject.toml


 dependencies = [
-    "mellea>=0.3.2",
+    "mellea==0.3.2",


do we want to bump this to 0.4.0?
longer-term, we probably need to sync our update strategy here to the release process
cc @jakelorocco

I only flagged one, but there are multiple places we may need to update if we do want to bump

Some tests failed with 0.4.0. Bumping 0.4.0 will be follow up PR.

I think it's fine for these to lag behind mellea (and expected since we don't want to own all the maintenance here). What happens if we bump mellea-integration-core? Do the internal packages that rely on it in here also have to get bumped? If so, that seems untenable.

Integration projects can pin to a release of mellea-integration-core. The release numbering of the mellea-integration-core needs some rule that related to the compatible Mellea release and API version of the integration-core.

thinking back on this, I'm wondering if we should revert this change and just set a minimum mellea version rather than pinning to one specifically?

psschwei

I think I am largely good with the changes here. would be good to have a review from the core team as well

jakelorocco

I think this is looking good and on the right track. The main purpose of my comments is to make longterm support from the Mellea team much less burdensome in this repo. To me, that means automating a large amount of the testing process and ensuring subpackages can be updated independently when needed.

jakelorocco · 2026-03-24T13:08:43Z

run_tests.sh

+# Test reqlib_package (may have dependency issues)
+echo "=========================================="
+echo "Testing: reqlib_package"
+echo "=========================================="
+echo -e "${YELLOW}Note: reqlib_package requires additional dependencies (citeurl, eyecite, playwright)${NC}"
+cd "$REPO_ROOT/mellea_contribs/reqlib_package"
+if PYTHONPATH="src:$PYTHONPATH" python3 -m pytest test/ -v --tb=short 2>&1; then
+    echo -e "${GREEN}✓ reqlib_package tests PASSED${NC}"
+    PASSED_PROJECTS+=("reqlib_package")
+else
+    echo -e "${YELLOW}⚠ reqlib_package tests SKIPPED (missing dependencies)${NC}"
+    FAILED_PROJECTS+=("reqlib_package (missing deps)")
+fi
+echo ""
+
+# Test tools_package (may have dependency issues)
+echo "=========================================="
+echo "Testing: tools_package"
+echo "=========================================="
+echo -e "${YELLOW}Note: test_mprogram_robustness.py requires 'benchdrift' package (skipped)${NC}"
+cd "$REPO_ROOT/mellea_contribs/tools_package"
+if PYTHONPATH="src:$PYTHONPATH" python3 -m pytest test/test_double_round_robin.py test/test_top_k.py -v --tb=short 2>&1; then
+    echo -e "${GREEN}✓ tools_package tests PASSED (2 tests, test_mprogram_robustness.py skipped - requires benchdrift)${NC}"
+    PASSED_PROJECTS+=("tools_package")
+else
+    echo -e "${YELLOW}⚠ tools_package tests FAILED${NC}"
+    FAILED_PROJECTS+=("tools_package")
+fi
+echo ""


We should mandate that all projects have the same structure. We shouldn't need to specify different commands here than from the run_project_tests above.

Projects have the same structure now. This file was made for convenience. It probably causes confusion and maintenance issue. So I removed it.

jakelorocco · 2026-03-24T13:10:29Z

mellea_contribs/mellea-integration-core/pyproject.toml


 dependencies = [
-    "mellea>=0.3.2",
+    "mellea==0.3.2",


I think it's fine for these to lag behind mellea (and expected since we don't want to own all the maintenance here). What happens if we bump mellea-integration-core? Do the internal packages that rely on it in here also have to get bumped? If so, that seems untenable.

jakelorocco · 2026-03-24T13:11:30Z

.github/workflows/ci-crewai-backend.yml

+  push:
+    branches:
+      - main
+    paths:


I don't think we should create cross-dependencies in this repo. See my comment below, but if we update mellea-integration-core and it forces us to update other packages, then we've just inherited a ton of maintenance work.

jakelorocco · 2026-03-24T13:12:16Z

mellea_contribs/tools_package/README.md

@@ -0,0 +1 @@
+README.md


I think we should either remove the README.md or add something here (even if it's auto generated). There's another README.md with the same problem.

I took out this file.

jakelorocco · 2026-03-24T13:13:07Z

.github/workflows/ci.yml

I might have missed it, but does this introduce the auto-detect packages without testing that we talked about?

This file was left for convenience. It may cause confusion so I removed this file.

jakelorocco · 2026-03-24T13:14:13Z

.github/workflows/ci.yml

+  detect-changes:
+    runs-on: ubuntu-latest
+    outputs:
+      mellea-integration-core: ${{ steps.changes.outputs.mellea-integration-core }}
+      crewai-backend: ${{ steps.changes.outputs.crewai-backend }}
+      dspy-backend: ${{ steps.changes.outputs.dspy-backend }}
+      langchain-backend: ${{ steps.changes.outputs.langchain-backend }}
+      reqlib-package: ${{ steps.changes.outputs.reqlib-package }}
+      tools-package: ${{ steps.changes.outputs.tools-package }}


I feel like instead of having to manually add a bunch of boilerplate for any new package that gets added to this repo, we should enforce a required package structure. Then, we don't need to duplicate this code and can just auto-process any new additions and automatically handle new packages in CI runs.

This file was left for convenience. It may cause confusion so I removed this file.

jakelorocco · 2026-03-24T13:14:51Z

.github/workflows/ci.yml

 on:
-  pull_request:
-    types: [opened, reopened, synchronize]
+  workflow_dispatch:


So we can't have checks on this repo then? Or the checks just need to be manually kicked off for a given branch / PR?

akihikokuroda · 2026-03-24T13:29:29Z

@jakelorocco I'm taking out ci.yml, README.md and run_test.sh. They are not necessary and may be causing issues later. I also take back the changes mellea_integration_core kicking the 3 integrations ci.

akihikokuroda · 2026-03-24T13:59:19Z

:-(

jakelorocco · 2026-03-27T17:09:20Z

Hi @akihikokuroda, I see that you re-requested my review. Based on the changes since my previous review, I don't think my review comments have been addressed yet. Please let me know if you have questions / disagreements with my suggestions.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

Add detect-changes job that checks which subpackages have been modified and outputs flags for each. Each subpackage workflow now only runs when: - Its files in mellea_contribs/ are changed, OR - Its corresponding quality workflow file is changed This prevents unnecessary CI runs when only unrelated subpackages are modified. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda · 2026-03-29T14:57:28Z

@jakelorocco I removed the ".github/workflows/ci.yml" that is not unnecessary and may cause confusion.

I think I addressed all your comments. Thanks!

akihikokuroda · 2026-03-29T14:59:33Z

Once this PR is merged, I'll work on to make the integrations work with Mellea 0.4.0.

psschwei

We should also add some documentation on how the repo is now structured (readme, contribution docs).

psschwei · 2026-03-30T14:28:47Z

.github/workflows/quality-crewai-backend.yml

+    strategy:
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+      max-parallel: 1


Is it possible to run these tests in parallel rather than sequentially? (will speed up CI time)

It is suggested that they may cause conflicts especially LLM access. I'm not sure but I made safe setting.

psschwei · 2026-03-30T14:30:28Z

.github/workflows/quality-crewai-backend.yml

+          cd mellea_contribs/crewai_backend
+          if ! command -v uv &> /dev/null; then
+            echo "Installing uv..."
+            pip install uv


the old action used astral-sh/setup-uv@v5 to setup uv... any reason why we're doing that manually now?

It had some issues with the action. I believe that the reason were integration-core is not published yet and the the it is sub-project in the repo. I would revisit once the core is published.

akihikokuroda · 2026-03-30T14:45:41Z

We should also add some documentation on how the repo is now structured (readme, contribution docs).

@psschwei once this is finalized. I'll work on CD and docs.

jakelorocco

Were you able to investigate how github PR checks would be able to handle having differently named checks? I believe you have to specify each check and I'm not certain there's a way to set selection criteria for which checks are needed.

If there's not a way to do that, that's another reason why I think we should just have one check that runs the relevant tests and have the PR pass status checks based off that.

jakelorocco · 2026-03-30T18:16:47Z

.github/workflows/quality-tools-package.yml

All of the ci-.yml and quality-.yml files are almost equivalent. I had previously mentioned if it would be possible to just check that tests exist in a given location and run those tests. Is that possible?

Then, for any subpackage, we can detect changes to that subpackage, and automatically run the tests without having to add boilerplate.

We can just force all tests to be runnable with pytest <subpackage>/tests or something like that.

I don't think we should force all subpackages to update this boilerplate code. If something ever changes in how we want to orchestrate, it will be a much larger change.

jakelorocco · 2026-03-30T18:20:21Z

.github/workflows/ci-subpackage-validation.yml

+    paths:
+      - 'mellea_contribs/**'
+      - '.github/workflows/ci-*.yml'
+      - '.github/workflows/quality-*.yml'
+      - '.github/workflows/ci-subpackage-validation.yml'
+  push:
+    branches:
+      - main
+    paths:
+      - 'mellea_contribs/**'
+      - '.github/workflows/ci-*.yml'
+      - '.github/workflows/quality-*.yml'
+      - '.github/workflows/ci-subpackage-validation.yml'


Have you thought about how we should handle changes to files outside of this set (like pyproject.toml, etc...)? Do changes to those files require testing any of the subpackages?

This CI is for checking if all of the sub project have its own CI. The CI for the sub project should be triggered any file changes in it. The file changes in the root or directries other than mellea_contribs are not monitored.

Replace 6 package-specific quality workflows with a single reusable generic workflow that auto-discovers tests. This eliminates 240+ lines of duplicated test logic and reduces friction when adding new subpackages. Benefits: - Single source of truth for test execution logic - One file to update when CI requirements change - Automatic test discovery (tests/ or test/ directories) - Consistent behavior across all subpackages - 38% fewer workflow files (13 → 8) All CI wrappers now pass subpackage path and package-specific config (timeout, Ollama requirement) to the generic workflow. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

The CI was trying to run tests for mellea_contribs/tools which was deleted and renamed to mellea_contribs/tools_package. Added --diff-filter=ACMRTU to git diff to exclude deleted files from the discovery process, preventing the CI from trying to test non-existent directories. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda · 2026-03-31T01:10:52Z

@jakelorocco @psschwei The ci.yml and quality_generic.yml files updated. Please take a look.

akihikokuroda marked this pull request as draft March 18, 2026 19:01

akihikokuroda mentioned this pull request Mar 18, 2026

feat: multi packages #37

Closed

akihikokuroda marked this pull request as ready for review March 19, 2026 20:38

akihikokuroda requested review from a team, avinash2692, jakelorocco and psschwei March 19, 2026 20:42

jakelorocco reviewed Mar 20, 2026

View reviewed changes

psschwei reviewed Mar 23, 2026

View reviewed changes

jakelorocco reviewed Mar 24, 2026

View reviewed changes

akihikokuroda requested review from jakelorocco and psschwei March 25, 2026 22:07

psschwei mentioned this pull request Mar 27, 2026

docs: update readme #44

Open

akihikokuroda added 7 commits March 29, 2026 10:56

move integrations under mellea_contribs

61dd0ee

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

move reqlib and tools src

979c6c9

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

separate and move tests and docs

cb0eadb

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

duplucate pyproject.toml and uv.lock to tools amd reqlib packages

99e877e

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

duplucate pyproject.toml and uv.lock to tools amd reqlib packages

df13888

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

adjust imports and fix tests

f9340ac

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

update ci and fix some file path

4450ea2

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda and others added 9 commits March 29, 2026 10:56

fixing tools reqlib

c909e8e

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

avoid conflict

84463d8

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

separate ci

2db3aae

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

add ci checking ci coverage

2c3d00d

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comments

94fcc18

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comments

c2a479c

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

disable unstable test

b858814

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

add empty README.md

4f21cf4

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda force-pushed the reporestructure branch from 0d40973 to 4f21cf4 Compare March 29, 2026 14:56

github-actions bot added the enhancement New feature or request label Mar 29, 2026

psschwei reviewed Mar 30, 2026

View reviewed changes

jakelorocco reviewed Mar 30, 2026

View reviewed changes

akihikokuroda and others added 12 commits March 30, 2026 15:42

update test_mprogram_robustness

3d2d8ee

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

update quality-generic.yml

a2f7d72

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

update ci-tools-package.yml

2e8c955

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

mellea_contribs/langchain_backend/pyproject.toml

df47526

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

.github/workflows/quality-generic.yml

1899acc

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

mellea_contribs/crewai_backend/pyproject.toml

e4c30e9

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

ci.yml

58e41ca

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

fix concurrency group

ecc2d7a

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

delete .github/workflows/ci-subpackage-validation.yml

cbc0631

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

delete CI_REFACTOR_IMPLEMENTATION.md

e34a188

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda mentioned this pull request Mar 31, 2026

chore: add governance doc #45

Merged

Conversation

akihikokuroda commented Mar 18, 2026

Uh oh!

jakelorocco left a comment

Choose a reason for hiding this comment

Uh oh!

akihikokuroda commented Mar 20, 2026

Uh oh!

jakelorocco commented Mar 20, 2026

Uh oh!

psschwei commented Mar 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

psschwei left a comment

Choose a reason for hiding this comment

Uh oh!

jakelorocco left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akihikokuroda commented Mar 24, 2026

Uh oh!

akihikokuroda commented Mar 24, 2026

Uh oh!

jakelorocco commented Mar 27, 2026

Uh oh!

akihikokuroda commented Mar 29, 2026

Uh oh!

akihikokuroda commented Mar 29, 2026

Uh oh!

psschwei left a comment

Choose a reason for hiding this comment