diff --git a/README.md b/README.md index cdfaa547..48b34b8d 100644 --- a/README.md +++ b/README.md @@ -13,10 +13,10 @@ New documentation for the new data loaders and module structure is under develop 3. [Module Features](#features) 4. [Loading Biosamples](#loading-biosamples) 5. [Loading Expression Data](#loading-expression-data) -6. [Protocols](#protocols) -7. [Viewing Data](#viewing-and-downloading-data) -9. [Administrative Pages](#administrative-pages) -10. [Example Files](#example-files) +6. [Viewing Data](#viewing-and-downloading-data) +7. [Administrative Pages](#administrative-pages) +8. [Protocols](#protocols) +9. [Example Files](#example-files) # Introduction Tripal Analysis: Expression is a [Drupal](https://www.drupal.org/) module built to extend the functionality of the [Tripal](http://tripal.info/) toolset. @@ -180,42 +180,14 @@ The biosample name will be taken as the name of the file minus the file extensio * **Regex for End of Data** - If the expression file has a footer, use this field to capture teh line that occurs after the end of expression data. This line of text and all text following will be ignored. #### Experimental Design Fields -The "Experimental Design" fields allow a complete description of the experimental design by implementing the [Chado MAGE design schema](http://gmod.org/wiki/Chado_Mage_Module). The Chado MAGE module uses the arraydesign, assay, quantification, and acquisition tables to describe an experiment. This is reflected in the following fields available to describe an experiment. +The "Experimental Design" fields allow a complete description of the experimental design by implementing the [Chado MAGE design schema](http://gmod.org/wiki/Chado_Mage_Module). The Chado MAGE module uses the arraydesign, assay, quantification, and acquisition tables to describe an experiment. The Tripal Analysis Expression creates generic instances of all these for you. -* **Biomaterial Provider** - The person or organization responsible for collecting the biosample. * **Array Design** - This is only applicable for microarray expression data. This may be left blank for experiments that do not utilize an array (ie next generation sequencing). - -#### Acquisition Details -This represents the quantification technique. In the case of a microarray, it is scanning, in the case of a sequencer, it is sequencing. The output of this process is a digitial image of an array for a microarray or a set of digital images or nucleotide base calls for a sequencer. - - * **Acquisition URI** - A web address to a page that describes the acquisition process. - * **Date Acquisition Run** - The date the acquisition was run. - * **Acquisition Protocol** - The acquisition protocol used in the experiment. (See protocol description below). - -#### Quantification Details - * **Units** - The units associated with the loaded values, such as FPKM. You may also update the units of your experiments using the **Quantification Units** admin page. - * **Date Quantification Run** - The date the quantification was run. - * **Quantification URI** - A web address to a page that describes the quantification process. - * **Quantification Operator** - The person or organization that ran the quantification. - * **Quantification Protocol** - The quantification protocol used in the experiment. (See protocol description below). - -![Experimental Design Fields](https://cloud.githubusercontent.com/assets/14822959/12991557/a4b0228e-d0dd-11e5-93de-2f206be6d5fe.png) -![Data Loader Fields](https://cloud.githubusercontent.com/assets/14822959/12991553/a4ade58c-d0dd-11e5-97d2-1096d78bb189.png) - - -# Protocols - -Acquisition, Quantification, Array Design, and Assays all utilize protocols to describe them. Think of protocols as the **experimental design**, and Acquisitions, Quantifications, Array Designs, and Assays as the experiment following that experimental design. -**Protocol Descripton** - The protocol content types can be created by navigating to **Add Tripal Content->Protocol**. A protocol can be used to add extra detail to an experimental design. A protocol can be used to describe the assay, acquisition, and quantification steps of the experiment design. A protocol can also be used to further describe the array design content type. The fields of a protocol are: -* **Protocol Name (must be unique - required)** -* **Protocol Link (Required)** - A web address to a page that describes the protocol. -* **Protocol Description** - A description of the protocol. -* **Hardware Description** - A description of the wardware used in the protocol. -* **Software Description** - A description of the software used in the protocol. -* **Protocol Type (required)** - The protocol type can acquisition, array design, assay, or quantification. The user can also create new protocol types by inserting new CVterms into the protocol type CV. -* **Publication** - A publication that describes the protocol. +>![file information portion of expression loader](example_files/doc_images/expression_loader_file_info.png) +>![experimental design portion of expression loader](example_files/doc_images/expression_loader_experimental_design.png) +> The expression loader accepts parameters describing your file, as well as some experimental design parameters. # Viewing and Downloading Data @@ -272,6 +244,24 @@ You can also assign quantification units to **all unitless quantifications** usi + +# Protocols + +Acquisition, Quantification, Array Design, and Assays all utilize protocols to describe them. Think of protocols as the **experimental design**, and Acquisitions, Quantifications, Array Designs, and Assays as the experiment following that experimental design. + +There is currently no support for inputting, or displaying, acquisitions, quantifications, or assays. The Expression module creates generic instances of these entities. + +**Protocol Descripton** - The protocol content types can be created by navigating to **Add Tripal Content->Protocol**. A protocol can be used to add extra detail to an experimental design. A protocol can be used to describe the assay, acquisition, and quantification steps of the experiment design. A protocol can also be used to further describe the array design content type. The fields of a protocol are: +* **Protocol Name (must be unique - required)** +* **Protocol Link (Required)** - A web address to a page that describes the protocol. +* **Protocol Description** - A description of the protocol. +* **Hardware Description** - A description of the wardware used in the protocol. +* **Software Description** - A description of the software used in the protocol. +* **Protocol Type (required)** - The protocol type can acquisition, array design, assay, or quantification. The user can also create new protocol types by inserting new CVterms into the protocol type CV. +* **Publication** - A publication that describes the protocol. + + + # Example Files ### Biomaterial Loader diff --git a/example_files/doc_images/expression_loader_experimental_design.png b/example_files/doc_images/expression_loader_experimental_design.png new file mode 100644 index 00000000..6748a2c9 Binary files /dev/null and b/example_files/doc_images/expression_loader_experimental_design.png differ diff --git a/example_files/doc_images/expression_loader_file_info.png b/example_files/doc_images/expression_loader_file_info.png new file mode 100644 index 00000000..486dbf97 Binary files /dev/null and b/example_files/doc_images/expression_loader_file_info.png differ diff --git a/tripal_analysis_expression/includes/TripalImporter/tripal_expression_data_loader.inc b/tripal_analysis_expression/includes/TripalImporter/tripal_expression_data_loader.inc index e2442111..d3253b0a 100644 --- a/tripal_analysis_expression/includes/TripalImporter/tripal_expression_data_loader.inc +++ b/tripal_analysis_expression/includes/TripalImporter/tripal_expression_data_loader.inc @@ -109,23 +109,6 @@ class tripal_expression_data_loader extends TripalImporter{ '#options' => $organisms, ]; - // At this time the analysis module has the extra fields. - // These fields do not appear to do anything for the analysis module. - // The following three form fields prevent an error form appearing - // when the expression biomaterial is submitted. - $form['removed'] = [ - '#type' => 'value', - '#default_value' => '', - ]; - $form['num_new'] = [ - '#type' => 'value', - '#default_value' => '', - ]; - $form['analysis_type'] = [ - '#type' => 'value', - '#default_value' => '', - ]; - //protocols, assays, etc go here $form['experimental_design'] = [ @@ -154,16 +137,18 @@ class tripal_expression_data_loader extends TripalImporter{ '#value' => $contacts, ]; - $lt_message = tripal_set_message(t('If the biomaterial provider is not present in the dropdown list, ') . l(t('add a new contact.'), 'bio_data/add') . t(' A biomaterial provider is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); + //TODO: WE DONT USE THIS (BUT WE SHOULD) - $form['experimental_design']['biosourceprovider_id'] = [ - '#title' => t('Biomaterial Provider'), - '#type' => 'select', - '#empty_value' => '', - '#description' => t('The contact who provided the biomaterial. If the biomaterial is already') . (' present in the database, the biomaterial provider selected here will') . (' overwrite the biomaterial provider set for the biomaterial in the database.'), - '#options' => $form['choose_contact']['#value'], - '#suffix' => $lt_message, - ]; + // $lt_message = tripal_set_message(t('If the biomaterial provider is not present in the dropdown list, ') . l(t('add a new contact.'), 'bio_data/add') . t(' A biomaterial provider is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); + +// $form['experimental_design']['biosourceprovider_id'] = [ +// '#title' => t('Biomaterial Provider'), +// '#type' => 'select', +// '#empty_value' => '', +// '#description' => t('The contact who provided the biomaterial. If the biomaterial is already') . (' present in the database, the biomaterial provider selected here will') . (' overwrite the biomaterial provider set for the biomaterial in the database.'), +// '#options' => $form['choose_contact']['#value'], +// '#suffix' => $lt_message, +// ]; $lt_message = tripal_set_message(t('If the array design does not appear in this list, ') . l(t('add a new arraydesign.'), 'bio_data/add') . t(' If an array design is not selected, then a generic array design will be created for the analysis.'), TRIPAL_INFO, ['return_html' => TRUE]); @@ -176,52 +161,60 @@ class tripal_expression_data_loader extends TripalImporter{ '#options' => $arraydesigns, '#suffix' => $lt_message, ]; - // - $form['experimental_design']['acquisition_details'] = [ - '#title' => t('Acquisition Details'), - '#type' => 'fieldset', - '#description' => t('This represents the quantification technique. In the case of a microarray, it is scanning, in the case of a sequencer, it is sequencing. The output of this process is a digitial image of an array for a microarray or a set of digital images or nucleotide base calls for a sequencer.'), - '#collapsible' => TRUE, - '#collapsed' => TRUE, - '#weight' => 1, - ]; - $form['experimental_design']['acquisition_details']['acquisitiondate'] = [ - '#type' => 'date', - '#title' => t('Date Acquisition Run'), - '#description' => t('If no date is provided, then the current date will be used.'), - '#required' => TRUE, - '#default_value' => [ - 'year' => $year, - 'month' => $month, - 'day' => $day, - ], - ]; + //TODO: WE DONT USE THIS (BUT WE SHOULD) - $form['experimental_design']['acquisition_details']['acquisition_uri'] = [ - '#type' => 'textfield', - '#title' => t('Acquisition URI'), - '#description' => t('URI location that describes the acquisition.'), - ]; + // +// $form['experimental_design']['acquisition_details'] = [ +// '#title' => t('Acquisition Details'), +// '#type' => 'fieldset', +// '#description' => t('This represents the quantification technique. In the case of a microarray, it is scanning, in the case of a sequencer, it is sequencing. The output of this process is a digitial image of an array for a microarray or a set of digital images or nucleotide base calls for a sequencer.'), +// '#collapsible' => TRUE, +// '#collapsed' => FALSE, +// '#weight' => 1, +// ]; + +// $form['experimental_design']['acquisition_details']['acquisitiondate'] = [ +// '#type' => 'date', +// '#title' => t('Date Acquisition Run'), +// '#description' => t('If no date is provided, then the current date will be used.'), +// '#required' => TRUE, +// '#default_value' => [ +// 'year' => $year, +// 'month' => $month, +// 'day' => $day, +// ], +// ]; + + //TODO: WE DONT USE THIS (BUT WE SHOULD) + + // $form['experimental_design']['acquisition_details']['acquisition_uri'] = [ +// '#type' => 'textfield', +// '#title' => t('Acquisition URI'), +// '#description' => t('URI location that describes the acquisition.'), +// ]; $lt_message = tripal_set_message(t('If the acquisition protocol is not present in the dropdown list, ') . l(t('add a new protocol.'), 'bio_data/add') . t(' An acquisition protocol is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); $acquisitions = tripal_get_protocol_select_options(FALSE, 'Acquisition Protocol'); - $form['experimental_design']['acquisition_details']['acquisition_protocol_id'] = [ - '#title' => t('Acquisition Protocol'), - '#type' => 'select', - '#description' => t('The protocol associated with this acquisition.'), - '#options' => $acquisitions, - '#suffix' => $lt_message, - ]; + //TODO: WE DONT USE THIS (BUT WE SHOULD) + + // +// $form['experimental_design']['acquisition_details']['acquisition_protocol_id'] = [ +// '#title' => t('Acquisition Protocol'), +// '#type' => 'select', +// '#description' => t('The protocol associated with this acquisition.'), +// '#options' => $acquisitions, +// '#suffix' => $lt_message, +// ]; $form['experimental_design']['quantification_details'] = [ '#title' => t('Quantification Details'), '#type' => 'fieldset', '#description' => t(' Quantification is the transformation of an image or set of sequences to numeric expression data. This typically involves statistical procedures.'), '#collapsible' => TRUE, - '#collapsed' => TRUE, + '#collapsed' => FALSE, '#weight' => 1, ]; $form['experimental_design']['quantification_details']['quantificationunits'] = [ @@ -229,26 +222,30 @@ class tripal_expression_data_loader extends TripalImporter{ '#title' => t('Units'), '#description' => t('The units associated with the loaded values (ie, FPKM, RPKM, raw counts).'), ]; - - $form['experimental_design']['quantification_details']['quantificationdate'] = [ - '#type' => 'date', - '#title' => t('Date Quantification Run'), - '#description' => t('If no date is provided, then the current date will be used.'), - '#required' => TRUE, - '#default_value' => [ - 'year' => $year, - 'month' => $month, - 'day' => $day, - ], - ]; - - $form['experimental_design']['quantification_details']['quantification_uri'] = [ - '#type' => 'textfield', - '#title' => t('Quantification URI'), - '#description' => t('URI location that describes the quantification.'), - ]; - - $lt_message = tripal_set_message(t('If the quantification operator is not present in the dropdown list, ') . l(t('add a new contact.'), 'bio_data/add') . t('A quantication operator is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); + //TODO: WE DONT USE THIS (BUT WE SHOULD) + +// $form['experimental_design']['quantification_details']['quantificationdate'] = [ +// '#type' => 'date', +// '#title' => t('Date Quantification Run'), +// '#description' => t('If no date is provided, then the current date will be used.'), +// '#required' => TRUE, +// '#default_value' => [ +// 'year' => $year, +// 'month' => $month, +// 'day' => $day, +// ], +// ]; + + + //TODO: WE DONT USE THIS (BUT WE SHOULD) + + // $form['experimental_design']['quantification_details']['quantification_uri'] = [ +// '#type' => 'textfield', +// '#title' => t('Quantification URI'), +// '#description' => t('URI location that describes the quantification.'), +// ]; +// +// $lt_message = tripal_set_message(t('If the quantification operator is not present in the dropdown list, ') . l(t('add a new contact.'), 'bio_data/add') . t('A quantication operator is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); // Obtain the list of contacts from the chado contact table. $sql = 'SELECT @@ -266,26 +263,30 @@ class tripal_expression_data_loader extends TripalImporter{ '#value' => $contacts, ]; - $form['experimental_design']['quantification_details']['quantification_operator_id'] = [ - '#title' => t('Quantification Operator'), - '#type' => 'select', - '#empty_value' => '', - '#description' => t('The quantification operator. If the operator does not appear in this list, ') . l(t('add a new contact.'), 'bio_data/add') . t(' An operator is not required.'), - '#options' => $form['choose_contact']['#value'], - '#suffix' => $lt_message, - ]; - - $lt_message = tripal_set_message(t('If the quantification protocol is not present in the dropdown list, ') . l(t('add a new protocol.'), 'bio_data/add') . t(' A quantification protocol is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); - - $quantifications = tripal_get_protocol_select_options(FALSE, 'Quantification Protocol'); - - $form['experimental_design']['quantification_details']['quantification_protocol_id'] = [ - '#title' => t('Quantification Protocol'), - '#type' => 'select', - '#description' => t('The protocol associated with this quantification.'), - '#options' => $quantifications, - '#suffix' => $lt_message, - ]; + //TODO: WE DONT USE THIS (BUT WE SHOULD) + // +// $form['experimental_design']['quantification_details']['quantification_operator_id'] = [ +// '#title' => t('Quantification Operator'), +// '#type' => 'select', +// '#empty_value' => '', +// '#description' => t('The quantification operator. If the operator does not appear in this list, ') . l(t('add a new contact.'), 'bio_data/add') . t(' An operator is not required.'), +// '#options' => $form['choose_contact']['#value'], +// '#suffix' => $lt_message, +// ]; + + + //TODO: WE DONT USE THIS (BUT WE SHOULD) +// $lt_message = tripal_set_message(t('If the quantification protocol is not present in the dropdown list, ') . l(t('add a new protocol.'), 'bio_data/add') . t(' A quantification protocol is not required.'), TRIPAL_INFO, ['return_html' => TRUE]); +// +// $quantifications = tripal_get_protocol_select_options(FALSE, 'Quantification Protocol'); + +// $form['experimental_design']['quantification_details']['quantification_protocol_id'] = [ +// '#title' => t('Quantification Protocol'), +// '#type' => 'select', +// '#description' => t('The protocol associated with this quantification.'), +// '#options' => $quantifications, +// '#suffix' => $lt_message, +// ]; $form['filetype'] = [ '#type' => 'radios', @@ -369,54 +370,18 @@ class tripal_expression_data_loader extends TripalImporter{ $analysis_id = $arguments['analysis_id']; $feature_uniquenames = $arguments['feature_uniquenames']; $arraydesign_id = $arguments['arraydesign_id']; - $acquisitiondate = $arguments['acquisitiondate']; - $create_biosamples = $arguments['create_biosamples']; - $quantificationdate = $arguments['quantificationdate']; + $quantificationunits = $arguments['quantificationunits']; - //missing - $assaydate = '';//$arguments['assaydate']; - $contact_id = '';//$arguments['contact_id']; - // - //$record = ['table' => 'analysis', 'id' => $node->analysis_id]; + //we dont actually use these variables so lets not collect the data. + //but, we do pass it around a million ways so lets declare the variables + $assaydate = null;//$arguments['assaydate']; + $contact_id = null;//$arguments['contact_id']; + $acquisitiondate = null;// $acquisitiondate = $arguments['acquisitiondate']; + $create_biosamples = null;// $create_biosamples = $arguments['create_biosamples']; + $quantificationdate = null;// $quantificationdate = $arguments['quantificationdate']; // - ////insert properties - ////note this is a subset of properties from the original, and im also not sure that we still need it - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_filetype', - // 'cv_name' => 'tripal', - // 'value' => trim($node->filetype), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_submit_job', - // 'cv_name' => 'tripal', - // 'value' => trim($node->submit_job), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_fileext', - // 'cv_name' => 'tripal', - // 'value' => trim($node->fileext), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_filepath', - // 'cv_name' => 'tripal', - // 'value' => trim($node->filepath), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_re_start', - // 'cv_name' => 'tripal', - // 'value' => trim($node->re_start), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_re_end', - // 'cv_name' => 'tripal', - // 'value' => trim($node->re_end), - //]); - //chado_insert_property($record, [ - // 'type_name' => 'analysis_expression_feature_uniquenames', - // 'cv_name' => 'tripal', - // 'value' => trim($node->feature_uniquenames), - //]); + tripal_set_message(t("Job Submitted! If new Biosamples will be created in this job, remember to") . l('publish them after the job runs.', diff --git a/tripal_biomaterial/api/tripal_biomaterial.api.inc b/tripal_biomaterial/api/tripal_biomaterial.api.inc index c11d06a2..d7f726fa 100644 --- a/tripal_biomaterial/api/tripal_biomaterial.api.inc +++ b/tripal_biomaterial/api/tripal_biomaterial.api.inc @@ -268,15 +268,26 @@ function expression_create_biomaterial_structure( //Attach the units to the quantification if ($quantificationunits) { - //insert cvterm. - //TODO move to install. - $cvterm_id = tripal_insert_cvterm([ - 'id' => 'sep:00056', + //get cvterm. + + + $cvterm_id = tripal_get_cvterm([ + 'name' => 'unit_of_measure', - 'cv_name' => 'sep', - 'definition' => 'A unit of measure is a quantity which is a standard of measurement for some dimension. For example, the Meter is a Unit O fMeasure for the dimension of length, as is the Inch. There is no intrinsic property of a UnitOfMeasure that makes it primitive or fundamental; rather, a system of units (e.g. Systeme International Unit) defines a set of orthogonal dimensions and assigns units for each. [ SUMO:unit of measure ]', + 'cv_id' => [ + 'name' => 'sep' + ] ])->cvterm_id; + if (!$cvterm_id) { + $cvterm_id = tripal_insert_cvterm([ + 'id' => 'sep:00056', + 'name' => 'unit_of_measure', + 'cv_name' => 'sep', + 'definition' => 'A unit of measure is a quantity which is a standard of measurement for some dimension. For example, the Meter is a Unit O fMeasure for the dimension of length, as is the Inch. There is no intrinsic property of a UnitOfMeasure that makes it primitive or fundamental; rather, a system of units (e.g. Systeme International Unit) defines a set of orthogonal dimensions and assigns units for each. [ SUMO:unit of measure ]', + ])->cvterm_id; + } + $values = [ 'quantification_id' => $quantification_id, 'type_id' => $cvterm_id,