Skip to content

feat(plugin): allow disabling the tenant plugin model providers cache - #39632

Merged
wylswz merged 3 commits into
mainfrom
feat/esq1-190-gate-provider-caches
Jul 28, 2026
Merged

feat(plugin): allow disabling the tenant plugin model providers cache#39632
wylswz merged 3 commits into
mainfrom
feat/esq1-190-gate-provider-caches

Conversation

@GareArc

@GareArc GareArc commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Plugins assigned to a workspace from an enterprise admin console never appear in that workspace's model provider list (ESQ1-190, Zendesk #3547).

PluginService.fetch_plugin_model_providers caches the per-tenant provider list in Redis for 24h. Enterprise installs plugins by calling the plugin daemon directly, bypassing this API, so nothing here ever learns the cache is stale and the newly assigned plugin stays invisible until the TTL expires.

A cache is only correct if the system that owns the writes can invalidate it. This adds PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED so deployments where something else installs plugins can turn it off and read through to the daemon, which does own the writes. The daemon-side replacement cache is langgenius/dify-plugin-daemon#780.

Defaults to true, so nothing changes for anyone installing plugins through this API — including all OSS deployments, where the existing invalidate_plugin_model_providers_cache call on plugin lifecycle keeps the cache correct.

The early return sits before the generation read and the refresh lock, so the disabled path touches no Redis at all rather than degrading to a slower cached path.

Second commit

Reverts b6099d0 (#39468), the inner-API endpoint that let an external system invalidate this cache. That approach was abandoned: it required wiring the dify endpoint and inner API key into every enterprise deployment, coupled enterprise to dify's cache internals, and did nothing for any other installer. Turning the cache off where it cannot be invalidated is the simpler contract.

What this deliberately does not do

An earlier revision also added PLUGIN_MODEL_SCHEMA_CACHE_ENABLED. That has been dropped.

The model schema cache has a related but much narrower problem: its key is {tenant_id}:{provider}:{model_type}:{model}:{user_id}, and provider carries no version, so a plugin upgrade leaves a stale schema behind. But it self-limits — PLUGIN_MODEL_SCHEMA_CACHE_TTL is 1h, versus 24h for the provider list — and it only bites if the affected model is exercised inside that window. Most upgrades are additive, so the visible symptom is "the new parameter has not shown up yet".

More to the point, there is no daemon-side replacement for it, so a switch there would only let operators delete a cache and get nothing back. Making the key version-aware is the right fix, but the plugin build is discarded in _to_provider_entity before anything is cached, and every way of threading it back through was worse than the 1h staleness it would prevent. Left for separate work.

Testing

  • test_fetch_plugin_model_providers_bypasses_redis_when_cache_disabled asserts the daemon is the only source when disabled, and that get/mget/setex/lock are all untouched.
  • uv run pytest tests/unit_tests/services/plugin/ tests/unit_tests/core/plugin/ — 572 passed.
  • ruff clean.

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 55.10% 55.11% +0.01%
Strict coverage 54.59% 54.60% +0.01%
Typed symbols 35,801 35,799 -2
Untyped symbols 29,447 29,433 -14
Modules 3078 3076 -2

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.09%. Comparing base (9fd1fea) to head (35ad555).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #39632      +/-   ##
==========================================
- Coverage   86.10%   86.09%   -0.02%     
==========================================
  Files        5076     5073       -3     
  Lines      287044   286890     -154     
  Branches    57241    57200      -41     
==========================================
- Hits       247168   247003     -165     
- Misses      35124    35136      +12     
+ Partials     4752     4751       -1     
Flag Coverage Δ
api 86.31% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

GareArc added 2 commits July 27, 2026 02:18
The provider list is cached here for 24h, but plugins can be installed by a
system other than this one — enterprise installs them through the plugin
daemon directly. Nothing then tells this cache it is stale, so a newly
assigned plugin stays invisible to the workspace until the TTL expires.

Add PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED so those deployments can turn the
cache off and read through to the daemon, which owns the writes and can
invalidate correctly. Defaults to true, so nothing changes for anyone
installing plugins through this API.

Extract _fetch_plugin_model_providers_uncached so the disabled path and the
caching path share one daemon fetch. The early return sits before the
generation read and the refresh lock, so the off path touches no Redis at all.
…iders cache (#39468)"

This reverts commit b6099d0.

The endpoint existed so an external installer could drop a tenant's provider
cache after installing plugins behind this service's back. Disabling the
cache outright removes the need: there is nothing to invalidate, and the
endpoint has no caller.

PluginService.invalidate_plugin_model_providers_cache stays. It predates the
endpoint and is still called from this service's own plugin lifecycle paths.
@GareArc GareArc changed the title feat(plugin): allow disabling the plugin model provider and schema caches feat(plugin): allow disabling the tenant plugin model providers cache Jul 27, 2026
@GareArc
GareArc force-pushed the feat/esq1-190-gate-provider-caches branch from eef9194 to cc177d6 Compare July 27, 2026 09:19
@GareArc
GareArc marked this pull request as ready for review July 27, 2026 10:28
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 27, 2026
@wylswz wylswz added this to the 1.16.1 milestone Jul 27, 2026
@wylswz
wylswz requested a review from Copilot July 27, 2026 23:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Ready to approve

The functional change is well-scoped and covered by a unit test; only a small docstring clarification is recommended.

Note: this review does not count toward required approvals for merging.

Pull request overview

This PR adds an operator-controlled switch to disable the per-tenant plugin model providers Redis cache so enterprise deployments (where plugins may be installed out-of-band via the daemon) don’t serve stale provider lists for up to 24 hours.

Changes:

  • Add PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED (default true) and document it in env examples.
  • Add an uncached fetch path in PluginService.fetch_plugin_model_providers that bypasses Redis entirely when disabled.
  • Remove the previously-added inner API endpoint (and its unit tests) used to invalidate this cache externally.
File summaries
File Description
docker/envs/core-services/shared.env.example Adds the new env flag to the docker deployment example.
api/.env.example Adds the new env flag to the backend env example.
api/configs/feature/init.py Introduces PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED in PluginConfig.
api/core/plugin/plugin_service.py Implements the Redis-bypass path and refactors uncached fetch into a helper.
api/tests/unit_tests/services/plugin/test_plugin_service.py Adds a unit test asserting no Redis calls occur when the cache is disabled.
api/controllers/inner_api/workspace/plugin_model_providers.py Removes the inner API invalidation endpoint.
api/controllers/inner_api/init.py Removes the namespace import/export for the deleted endpoint.
api/tests/unit_tests/controllers/inner_api/workspace/test_plugin_model_providers.py Removes unit tests for the deleted endpoint.

Review details

  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Low

Note

Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread api/core/plugin/plugin_service.py
@wylswz
wylswz enabled auto-merge July 27, 2026 23:53
@wylswz
wylswz added this pull request to the merge queue Jul 27, 2026
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jul 27, 2026
Merged via the queue into main with commit a57b0b9 Jul 28, 2026
42 checks passed
@wylswz
wylswz deleted the feat/esq1-190-gate-provider-caches branch July 28, 2026 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants