deploy: b782e93

.gitignore

@@ -0,0 +1,32 @@
+README.html
+README_files/
+*.DS_Store
+*__pycache__
+*.h5ad
+changelogs
+
+# IDE ignores
+/.idea/
+/.vscode/
+
+# repo specific ignores
+output_bash
+
+# R specific ignores
+.Rhistory
+.Rproj.user
+*.Rproj
+
+# viash specific ignores
+docker_output/
+
+check_results/
+log.txt
+.viash*
+/resources/
+/resources_test/
+
+# nextflow specific ignores
+/.nextflow*
+/work
+output

CODE_OF_CONDUCT.md

@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[INSERT CONTACT METHOD].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

CONTRIBUTING.md

@@ -0,0 +1,24 @@
+# Contributing to OpenProblems
+
+[OpenProblems](https://openproblems.bio) is a community effort, and
+everyone is welcome to contribute. This project is hosted on
+[github.com/openproblems-bio/openproblems](https://github.com/openproblems-bio/openproblems).
+
+You can find a full in depth guide on how to contribute to this project
+on the [OpenProblems website](https://openproblems.bio/documentation/).
+
+## Code of conduct
+
+We as members, contributors, and leaders pledge to make participation in
+our community a harassment-free experience for everyone, regardless of
+age, body size, visible or invisible disability, ethnicity, sex
+characteristics, gender identity and expression, level of experience,
+education, socio-economic status, nationality, personal appearance,
+race, caste, color, religion, or sexual identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open,
+welcoming, diverse, inclusive, and healthy community.
+
+Our full [Code of Conduct](CODE_OF_CONDUCT.md) is adapted from the
+[Contributor Covenant](https://www.contributor-covenant.org), version
+2.1.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 OpenProblems
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,14 @@
+[![](https://openproblems.bio/images/heros/home_hero_text.png)](https://openproblems.bio)
+
+------
+
+Open Problems is a living, extensible, community-guided benchmarking platform.
+
+Useful links:
+
+* [Introduction to Open Problems](https://openproblems.bio)
+* [Our benchmarks](https://openproblems.bio/results)
+* [Our datasets](https://openproblems.bio/datasets)
+* [Our team and community](https://openproblems.bio/team)
+* [Planned and past events](https://openproblems.bio/events)
+* [How to contribute](https://openproblems.bio/documentation)

_viash.yaml

@@ -0,0 +1,14 @@
+viash_version: 0.8.6
+
+source: src
+target: target
+
+config_mods: |
+  .functionality.version := 'dev'
+  .platforms[.type == 'docker'].target_registry := 'ghcr.io'
+  .platforms[.type == 'docker'].target_organization := 'openproblems-bio/openproblems'
+  .platforms[.type == 'docker'].target_image_source := 'https://github.com/openproblems-bio/openproblems'
+  .platforms[.type == "nextflow"].directives.tag := "$id"
+  .platforms[.type == "nextflow"].auto.simplifyOutput := false
+  .platforms[.type == "nextflow"].config.labels := { lowmem : "memory = 20.Gb", midmem : "memory = 50.Gb", highmem : "memory = 100.Gb", lowcpu : "cpus = 5", midcpu : "cpus = 15", highcpu : "cpus = 30", lowtime : "time = 1.h", midtime : "time = 4.h", hightime : "time = 8.h", veryhightime : "time = 24.h" }
+  .platforms[.type == "nextflow"].config.script := "process.errorStrategy = 'ignore'"

main.nf

@@ -0,0 +1,3 @@
+workflow {
+    print("This is a dummy placeholder for pipeline execution. Please use the corresponding nf files for running pipelines.")
+}

nextflow.config

@@ -0,0 +1 @@
+process.container = 'nextflow/bash:latest'

scripts/sync_resources.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+set -e
+
+viash run src/common/sync_test_resources/config.vsh.yaml

src/common/check_dataset_schema/config.vsh.yaml

@@ -0,0 +1,45 @@
+functionality:
+  name: check_dataset_schema
+  namespace: common
+  description: Checks if the dataset has the necessary slots that are predefined in a schema.
+  argument_groups:
+    - name:  Inputs
+      arguments:
+        - name: --input
+          type: file
+          required: true
+          description: A h5ad file.
+        - name: --schema
+          type: file
+          required: true
+          description: A schema file for the h5ad object.
+    - name: Arguments
+      arguments:
+        - name: --stop_on_error
+          type: boolean
+          default: false
+          description: Whether or not to stop with exit code 1 if the input file does not adhere to the schema.
+    - name: Output
+      arguments:
+        - name: --output
+          type: file
+          required: true
+          description: If specified, this file will contain a structured log of which checks succeeded (or not).
+          example: checks.json
+          direction: output
+  resources:
+    - type: python_script
+      path: script.py
+  test_resources:
+    - path: /resources_test/common/pancreas
+    - type: python_script
+      path: test.py
+platforms:
+  - type: docker
+    image: openproblems/base_python:1.0.0
+    test_setup:
+      - type: python
+        packages: viashpy
+  - type: nextflow
+    directives:
+      label: [midtime, midmem, midcpu]

src/common/check_dataset_schema/script.py

@@ -0,0 +1,60 @@
+import anndata as ad
+import yaml
+import json
+
+## VIASH START
+par = {
+  'input': 'work/d4/f4fabc8aa4f2308841d4ab57bcff62/_viash_par/input_1/dataset.h5ad',
+  'schema': 'work/d4/f4fabc8aa4f2308841d4ab57bcff62/_viash_par/schema_1/schema.yaml',
+  'stop_on_error': False,
+  'output': 'work/d4/f4fabc8aa4f2308841d4ab57bcff62/out.yaml',
+}
+## VIASH END
+
+def check_structure(slot, slot_info, adata_slot):
+  missing = []
+  if slot == "X":
+    slot_info["name"] = "X"
+    slot_info = [slot_info]
+  for obj in slot_info:
+    adata_data = adata_slot.get(obj['name']) if slot != 'X' else adata_slot
+    if obj.get('required') and adata_data is None:
+      missing.append(obj['name'])
+    # todo: check types
+  return missing
+
+print('Load data', flush=True)
+adata = ad.read_h5ad(par['input'])
+
+# create data structure
+out = {
+  "exit_code": 0,
+  "error": {},
+  "data_schema": "ok"
+}
+
+print("Check AnnData against schema", flush=True)
+with open(par["schema"], "r") as f:
+  data_struct = yaml.safe_load(f)
+
+def_slots = data_struct['info']['slots']
+
+out = {
+  "exit_code": 0,
+  "error": {},
+  "data_schema": "ok"
+}
+for slot in def_slots:
+  print("Checking slot", slot, flush=True)
+  missing = check_structure(slot, def_slots[slot], getattr(adata, slot))
+  if missing:
+    print(f"Dataset is missing {slot} {missing}", flush=True)
+    out['exit_code'] = 1
+    out['data_schema'] = 'not ok'
+    out['error'][slot] = missing
+
+with open(par["output"], "w") as f:
+  json.dump(out, f, indent=2)
+
+if par['stop_on_error']:
+  exit(out['exit_code'])