Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It's unclear why the pipeline type is specified both in the pipeline interface and the looper config #465

Closed
nsheff opened this issue Feb 16, 2024 · 2 comments
Milestone

Comments

@nsheff
Copy link
Contributor

nsheff commented Feb 16, 2024

Right now, a pipeline interface can say pipeline_type: sample or pipeline_type: project.

Then, in the .looper.yaml, you say something like:

pipeline_interfaces:
  sample: ../pipeline/pipeline_interface.yaml

So, you're specifying the type of the pipeline there, too. What if these don't match? Like, you say sample: piface.yaml but that actually says pipeline_type: project inside?

Why can't looper just read the pipeline_type from the piface in the first place? We could simplify the looper config to:

pipeline interfaces:
  - ../pipeline/piface.yaml
@donaldcampbelljr
Copy link
Contributor

This work is now complete with the above PR. It also maintains backwards compatibility.

Now, the user can give a list of pifaces in the looper config and Looper will assign them to sample or project pifaces given the pipeline_type within the piface:

pep_config: project/project_config.yaml # local path to pep config
output_dir: "results"
pipeline_interfaces:
  - pipeline/pipeline_interface.yaml
  - pipeline/pipeline_interface_project.yaml

@donaldcampbelljr
Copy link
Contributor

When working on the above changes, I had forgotten that, for Looper 2.0, we are considering placing both sample and project-level interfaces into one pipeline interface: #493

@donaldcampbelljr donaldcampbelljr modified the milestones: v2.0.0, v1.9.0 Jun 24, 2024
@nsheff nsheff added this to PEP Jun 25, 2024
@github-project-automation github-project-automation bot moved this to Done in PEP Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants