Google provider base-depends on google-cloud-aiplatform[evaluation], force-installing litellm/scikit-learn for every user

[brunohelloprint]

Repo: apache/airflow · Labels: kind:bug, provider:google

---

Title

Google provider base-depends on google-cloud-aiplatform[evaluation], force-installing litellm / scikit-learn for every user

Apache Airflow Provider(s) version

apache-airflow-providers-google 22.0.0 — also confirmed on latest 22.2.0.

What happened

apache-airflow-providers-google declares google-cloud-aiplatform[evaluation] as an unconditional base dependency (no extra marker):

• 22.0.0: google-cloud-aiplatform[evaluation]>=1.145.0
• 22.2.0: google-cloud-aiplatform[evaluation]>=1.155.0

The [evaluation] extra transitively installs a full ML/eval stack that has nothing to do with the core Google hooks/operators:

apache-airflow-providers-google
└── google-cloud-aiplatform[evaluation]
    ├── litellm            → huggingface-hub → tokenizers → tqdm
    ├── scikit-learn
    └── ruamel-yaml

Anyone who installs the provider only for, say, GCSHook / BigQueryHook still gets litellm, scikit-learn, and friends.

What you think should happen instead

The Vertex AI evaluation feature set should be opt-in, not forced on all provider users. Either:

1. Move it behind a provider extra (e.g. apache-airflow-providers-google[vertex-eval]), or
2. Base-depend on google-cloud-aiplatform without the [evaluation] extra (users who need eval add it themselves).

Why it matters

• Image-size bloat: litellm + scikit-learn + the HF/tokenizers chain is a large install for users who never touch Vertex evaluation.
• CVE noise: litellm ships a steady stream of proxy-server CVEs (e.g. auth bypass, sandbox RCE, privilege escalation) that surface in pip-audit for every provider user, even when litellm is never imported and its proxy is never run — each one has to be triaged/ignored downstream.

How to reproduce

pip install apache-airflow-providers-google==22.2.0
pip show litellm scikit-learn      # both present, transitively via the [evaluation] extra

Anything else

The evaluation extra of google-cloud-aiplatform is the sole source of litellm/scikit-learn/ruamel-yaml in a provider install. Making it optional would let downstreams that use google-genai (the newer unified SDK) for Vertex AI avoid the old aiplatform eval SDK entirely.

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[gingeekrishna]

Opened PR #69399 with a fix.

Approach:

1. : drop [evaluation] from the base dep (→ google-cloud-aiplatform>=1.155.0), add a new vertex-ai-evaluation optional extra that includes google-cloud-aiplatform[evaluation]>=1.155.0
2. generative_model.py: guard the vertexai.preview.evaluation import with try/except ImportError; get_eval_task() and run_evaluation() raise a clear ImportError pointing to pip install apache-airflow-providers-google[vertex-ai-evaluation] when the extra is missing

Users of GCSHook/BigQueryHook etc. get no litellm/scikit-learn; users who need the Rapid Evaluation API add the explicit extra.