Base import requirements for processors - #48522
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
|
||
| class Emu3ProcessorKwargs(ProcessingKwargs, total=False): | ||
| text_kwargs: Emu3TextKwargs | ||
| images_kwargs: Emu3ImageProcessorKwargs |
There was a problem hiding this comment.
is resolved automatically via self.image_processor.valid_kwargs - this line is redundant
| class Gemma4UnifiedVideoProcessorKwargs(VideosKwargs, total=False): | ||
| """ | ||
| patch_size (`int`, *optional*): | ||
| Size of each image patch in pixels. | ||
| max_soft_tokens (`int`, *optional*): |
There was a problem hiding this comment.
should've never been here, a bug in modular that is fixed now
|
[For maintainers] Suggested jobs to run (before merge) run-slow: emu3, fuyu, gemma4, gemma4_unified, pixtral |
CI recapDashboard: View test results in Grafana |
vasqu
left a comment
vasqu
left a comment
There was a problem hiding this comment.
In general yes aligned here, would like the audio team for requirements potentially (torchaudio)
I wouldn't put torch as basic requirement for now, it could be used to prepare data on a super lightweight container (torch is quite big, even cpu only)
| "feature_extractor": [], | ||
| "audio_processor": [], |
There was a problem hiding this comment.
Iirc, torchaudio is no longer maintained anymore. Maybe we should start pivoting when we can
There was a problem hiding this comment.
not across all models yet, some really old ones are in numpy though I think Eustache is planning to get torchaudio as a proper backend for all transforms
| for check, requirements in BASE_FILE_REQUIREMENTS.items(): | ||
| if check(module_name, file_content): | ||
| base_requirements = requirements | ||
| if isinstance(requirements, Callable): |
There was a problem hiding this comment.
Isn't everything a callable now?
There was a problem hiding this comment.
no, they return a tuple of requirements based on file name, and only processors need to check with file content to see supported attributes/modalities
3 participants
What does this PR do?
Wanted to do it long time ago, a processor can't be initiated anyway if one of the subprocessor can't. So an error will be raised at run-time and after this PR it is raised at import time
Processor base requirements are inferred from its subprocessors - vision always requires PIL, while videos also require torch since that is the only backend supported. I think for audio we can add
torchaudioas well after the big refactor landsActually, why don't we make all processors rely on
torchat some point, we don't support any other framework anymore and I don't think processors are used to returnlist/numpyanyway?