Pipeline Pilot module images utilized in LINCS and IDG work; NOT ACTUAL CODE
This contains images of the modular nature of Pipeline Pilot and some imaged samples of code, but not utilizable. Used primarily for ETL and curation tracking for key entities and experiments
Sharable LINCS Code/module use - LINCS primarily used a modular-based program (PipelinePilot) to obtain, edit, etc., various information. Some of it I can share, though out of context.
PLP enables use of global variables throughout the pipelines, referenced either with @ or $( .
Sample portion examining incoming files and sorting into proper locations for data downloads on LDP (i.e. http://lincsportal.ccs.miami.edu/datasets/view/LDS-1499)
What’s coming in? - Find out if incoming csv submissions have already been seen, are mapped to proper values internally, and, if new or updated, get it completed:
Updating data for cells (astrocytes) - in order to properly curate the mass amount of data, new information being brought in needed to be organized and indicated for provenance purposes.
Comma separated listing of id(s) to be updated, why, and what field(s)
Registration of New entities incoming to the LINCS system
Large scale preparation of incoming LINCS data from JSON files (from Harvard in this instance)