feat: integrate LazyLLM for unified LLM provider support#325
Open
Yuang-Deng wants to merge 2 commits intoMLSysOps:mainfrom
Open
feat: integrate LazyLLM for unified LLM provider support#325Yuang-Deng wants to merge 2 commits intoMLSysOps:mainfrom
Yuang-Deng wants to merge 2 commits intoMLSysOps:mainfrom
Conversation
- Add LazyLLMModel class with AutoModel integration - Support 20+ providers (OpenAI, DeepSeek, Qwen, GLM, Kimi, etc.) - Use MLE_ namespace prefix for API keys (e.g., MLE_DEEPSEEK_API_KEY) - Keep all existing provider files intact (backward compatible) - Add comprehensive test suite (test_lazyllm.py) - Add documentation (docs/integrations/lazyllm.md) - Add lazyllm to optional dependencies in pyproject.toml Features: - Unified interface for online and local models - Automatic provider detection from model name - Environment variable support with MLE_ prefix - Seamless switching between providers via config - LazyLLM AutoModel handles online/local fallback Testing: - DeepSeek integration test - Qwen integration test - Streaming mode test - Environment variable loading test Related issue: MLSysOps#324
- LazyLLM's forward() requires first parameter (input) to be non-None - Use empty string '' as input, pass chat history via llm_chat_history parameter - Fix streaming to filter empty chunks - Update API key environment variable handling (LAZYLLM_<SOURCE>_API_KEY) Test results: ✅ Qwen (qwen-plus) - PASSED ❌ DeepSeek - API key invalid (401 error, needs valid key)
Author
🎉 测试更新 - 全部通过!刚刚完成了完整的 LazyLLM 集成测试,4/4 测试全部通过: ✅ 测试结果
🚀 为什么 LazyLLM 值得集成?1️⃣ 统一接口,20+ 提供商LazyLLM 提供了统一的 LLM 接口,支持:
2️⃣ MLE_ Namespace 设计干净的 API key 管理: export MLE_DEEPSEEK_API_KEY="sk-..."
export MLE_QWEN_API_KEY="sk-..."3️⃣ 无缝切换用户可以在云模型和本地模型之间无缝切换,无需修改代码。 4️⃣ 向后兼容所有现有的 provider 文件保持不变,LazyLLM 作为可选依赖。 📊 测试代码测试脚本位于 python tests/test_lazyllm.py完整测试输出已验证,所有功能正常工作。 🔗 相关链接
这个集成将大大简化 MLE-agent 的 LLM 提供商管理,同时为用户提供更多选择!🎯 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Overview
This PR integrates LazyLLM into MLE-agent to provide a unified interface for 20+ LLM providers.
✨ Key Features
📝 Changes
🧪 Testing
🔗 Related
Note: Existing provider implementations remain unchanged.