Split tests into a separate package, plus rattler-build #107

mgorny · 2024-12-24T16:58:29Z

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

split tests into a separate package as suggested in Recipe improvements #22, reducing the main package size
removed obsolete code for CUDA < 12

…onda-forge-pinning 2024.12.24.08.00.50

Eliminate the recipe conditions for cuda_major, since only CUDA 12+ is supported now. Replace the jinja `{% if %}` conditions with inline selectors, since the latter are more compatible with different tools (e.g. conda-recipe-manager).

After moving the tests out of the main package, its size reduces to 1.5M. The tests package itself is 9M. Proposed in issue conda-forge#22.

…onda-forge-pinning 2024.12.24.08.00.50

conda-forge-admin · 2024-12-24T16:59:58Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12484953648. Examine the logs at this URL for more detail.}

hmaarrfk · 2024-12-24T22:49:38Z

Thanks I think reduced download sizes are definitely welcome!

hmaarrfk · 2024-12-24T22:50:27Z

recipe/meta.yaml

+outputs:
+  - name: torchvision
+  - name: torchvision-tests
+    test:


you might still need a requirements section no?

I feel like we have very few of these -tests packages so it might be good to make this a good example!

To be honest, I don't think so. This package has no build steps, so no build requirements are needed — at least conda-build handles it pretty much immediately:

Packaging torchvision-tests WARNING: No files or script found for output torchvision-tests number of files: 0 Fixing permissions Packaged license file/s. INFO :: Time taken to mark (prefix) 0 replacements in 0 files was 0.00 seconds

As for run dependencies — I don't think having any is correct either. After all, the package doesn't install any files.

So the all dependencies needed are test requirements, and these are listed.

I think you are right. I just didn’t look at the failure

Oh, I haven't looked at all the jobs but it's definitely failing because of the new output. To be honest, I'm not sure we should actually be publishing that extra package.

I don’t know if we have the tooling to not publish it.

For pillow I used a giant if statement that had to manually be turned on. But I haven’t run the test in months due to that manual step

Yeah, it's probably not worth making things more fragile over this. Should I file a request to allow this new output, or wait for other maintainers to look at it first?

I think you can push through with the request.

My review should be fine

h-vetinari doesn’t mind multi output recipiez

I’m typically more apprehensive due to the increased complexion of debugging multi output recipies locally.

Given the file size savings. I’m ok with this.

…onda-forge-pinning 2024.12.24.08.00.50

mgorny · 2024-12-25T16:05:57Z

recipe/recipe.yaml

+    ${{ 'skip test_url_is_accessible instead of hitting 20+ servers per run, since' if 0 }}
+    ${{ 'each server might be occasionally unresponsive and end up failing our CI' if 0 }}
+    test_url_is_accessible
+    ${{ 'spurious failure because upstream skip (Image.__version__ >= "7") does not trigger for Pillow "10"' if 0 }}
+    or (test_transforms and test_adjust_saturation)


This is basically the best I could come up with, for a way to intersperse strings and comments here. Of course, I'm open to better suggestions.

Can’t review so easily on my phone. But my main goal was to make it easy to expand and add new tests one line at a time to make the diff obvious.

This seems like it follows that spirit!

conda-forge-admin · 2024-12-25T16:05:59Z

Hi! This is the friendly automated conda-forge-linting service.

I failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint --conda-forge . from the recipe directory. You can also examine the workflow logs for more detail.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/12495694887. Examine the logs at this URL for more detail.}

mgorny · 2024-12-25T16:07:30Z

I've added a rattler-build conversion, and filed output request in conda-forge/admin-requests#1248. FWICS conda-smithy crashes on it, I'm going to look into that later today.

As discussed in conda-forge/torchvision-feedstock#107.

…onda-forge-pinning 2024.12.24.08.00.50

mgorny · 2024-12-25T18:26:22Z

I'm going to guess that the osx failure is a fluke.

mgorny · 2024-12-25T18:44:14Z

The conda-smithy problems are conda-forge/conda-smithy#2165 and conda-forge/conda-smithy#2196. The former I've been able to workaround by avoiding nested conditions.

hmaarrfk · 2024-12-25T22:41:59Z

I came to check in, and saw

test_decode_gif[True-treescap]

failed due to connection issues. I always feel bad since conda-forge hammers webservers which are meant to be cached locally for data.

Should we skip this test?

mgorny added 7 commits December 24, 2024 15:42

MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.45.1, and c…

b8898a5

…onda-forge-pinning 2024.12.24.08.00.50

MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.45.1, and c…

28910f7

…onda-forge-pinning 2024.12.24.08.00.50

Eliminate cuda_major >= 12 conditions

4b19200

Eliminate the recipe conditions for cuda_major, since only CUDA 12+ is supported now. Replace the jinja `{% if %}` conditions with inline selectors, since the latter are more compatible with different tools (e.g. conda-recipe-manager).

Remove unused code for old CUDA versions

c2c3f8b

Move tests to a torchvision-tests output

2dccfd1

After moving the tests out of the main package, its size reduces to 1.5M. The tests package itself is 9M. Proposed in issue conda-forge#22.

MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.45.1, and c…

ff3237b

…onda-forge-pinning 2024.12.24.08.00.50

Bump the buid number

c31ac3f

mgorny requested review from h-vetinari, hmaarrfk and nehaljwani as code owners December 24, 2024 16:58

Move pip requirement to the correct test group

477803a

hmaarrfk reviewed Dec 24, 2024

View reviewed changes

mgorny mentioned this pull request Dec 25, 2024

Multi-output package with pin_subpackage in test requirements fails to build prefix-dev/rattler-build#1142

Open

mgorny added 2 commits December 25, 2024 17:02

Convert to rattler-build

70ef339

MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.45.1, and c…

ef345c9

…onda-forge-pinning 2024.12.24.08.00.50

mgorny mentioned this pull request Dec 25, 2024

torchvision: Allow torchvision-tests output conda-forge/admin-requests#1248

Merged

19 tasks

mgorny changed the title ~~Split tests into a separate package, plus minor cleanup~~ Split tests into a separate package, plus rattler-build Dec 25, 2024

mgorny commented Dec 25, 2024

View reviewed changes

hmaarrfk pushed a commit to conda-forge/admin-requests that referenced this pull request Dec 25, 2024

torchvision: Allow torchvision-tests output

489c02b

As discussed in conda-forge/torchvision-feedstock#107.

mgorny added 2 commits December 25, 2024 17:43

Fix preprocessing for cross-aarch64 jobs

39add14

MNT: Re-rendered with conda-build 24.11.2, conda-smithy 3.45.1, and c…

318322e

…onda-forge-pinning 2024.12.24.08.00.50

Avoid nested conditions to workaround bug in conda-smithy

8b56af1

hmaarrfk added the automerge Merge the PR when CI passes label Dec 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split tests into a separate package, plus rattler-build #107

Split tests into a separate package, plus rattler-build #107

mgorny commented Dec 24, 2024

conda-forge-admin commented Dec 24, 2024 •

edited

Loading

hmaarrfk commented Dec 24, 2024

hmaarrfk Dec 24, 2024

mgorny Dec 25, 2024

hmaarrfk Dec 25, 2024

mgorny Dec 25, 2024

hmaarrfk Dec 25, 2024

mgorny Dec 25, 2024

hmaarrfk Dec 25, 2024

hmaarrfk Dec 25, 2024

mgorny Dec 25, 2024

hmaarrfk Dec 25, 2024

conda-forge-admin commented Dec 25, 2024 •

edited

Loading

mgorny commented Dec 25, 2024

mgorny commented Dec 25, 2024

mgorny commented Dec 25, 2024

hmaarrfk commented Dec 25, 2024

Split tests into a separate package, plus rattler-build #107

Are you sure you want to change the base?

Split tests into a separate package, plus rattler-build #107

Conversation

mgorny commented Dec 24, 2024

conda-forge-admin commented Dec 24, 2024 • edited Loading

hmaarrfk commented Dec 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

conda-forge-admin commented Dec 25, 2024 • edited Loading

mgorny commented Dec 25, 2024

mgorny commented Dec 25, 2024

mgorny commented Dec 25, 2024

hmaarrfk commented Dec 25, 2024

conda-forge-admin commented Dec 24, 2024 •

edited

Loading

conda-forge-admin commented Dec 25, 2024 •

edited

Loading