Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure Chado Prepare works in automated testing #297

Merged
merged 42 commits into from
Nov 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
f6f1025
New prepareTestChado function added with theoretic files SQL files
risharde Oct 24, 2022
1c69ce9
Skeleton preparer test, currently fails.
laceysanderson Oct 24, 2022
8385d33
Fix: Trying to get property 'table_id' of non-object when the custom …
laceysanderson Oct 24, 2022
3a444ee
Merge branch 'tv4g2-issue217-tripaldbx' into tv4g1-issue291-testChado…
laceysanderson Oct 24, 2022
fdd3f58
Trying to ensure that the obo importer uses Tripal DBX.
laceysanderson Oct 24, 2022
135261d
Merge branch 'tv4g1-issue291-testChadoPrepare' of https://github.com/…
laceysanderson Oct 24, 2022
efdedba
First version of fill_chado_test_prepare.sql dump
risharde Oct 25, 2022
ceaa3a2
First version of fill_public_test_prepare.sql
risharde Oct 25, 2022
e2c0f44
Attempting to create a valid test
risharde Oct 25, 2022
e6e7c98
Attempting to create a valid test
risharde Oct 25, 2022
0738377
Semi working ChadoTestPrepare but using own custom function for readi…
risharde Oct 25, 2022
8d7712a
Minor adjustment to the prepareTestChadoTheWrongWay function
risharde Oct 25, 2022
9da66bf
TODO: CVTERM test
risharde Oct 25, 2022
0594c4c
Unworking fill_public_test_prepare.sql fixture
risharde Oct 29, 2022
e545a69
periodic commits
spficklin Oct 31, 2022
7c8119a
Add exception to catch if reflection class not available.
laceysanderson Oct 31, 2022
52fbe4b
Merge prepare chado approaches.
laceysanderson Oct 31, 2022
5e5d03b
Added a missing namespace.
laceysanderson Oct 31, 2022
d906b55
Don't throw exception when non-class-based APIs use Tripal DBX.
laceysanderson Oct 31, 2022
1445bab
Merge branch 'tv4g1-issue291-testChadoPrepare' into tv4g2-291-prepare…
laceysanderson Oct 31, 2022
017eeec
Fixed ChadoTestPrepare to use the prepareTestChado function
risharde Nov 1, 2022
898052e
Merge pull request #293 from tripal/tv4g2-291-prepare_test
laceysanderson Nov 3, 2022
03b099b
Make thrown exception more clear for debugging purposes.
laceysanderson Nov 3, 2022
e55e4f7
Fix table prefixing issue.
laceysanderson Nov 3, 2022
64c387b
Fix 'Object of class stdClass could not be converted to int'
laceysanderson Nov 3, 2022
cdc0fb1
Ensure schema def if returned + fix table name.
laceysanderson Nov 3, 2022
1566e6a
Ensure progress not printed to the screen as it breaks tests.
laceysanderson Nov 3, 2022
a5133b9
Merge branch '9.x-4.x' into tv4g1-issue291-testChadoPrepare
laceysanderson Nov 3, 2022
921c614
Remove incomplete OBO test for now.
laceysanderson Nov 3, 2022
964a5d1
Merge existing chado prepare step tests.
laceysanderson Nov 3, 2022
3978674
Merge branch 'tv4g1-issue291-testChadoPrepare' of https://github.com/…
laceysanderson Nov 3, 2022
9e8336f
Add PREPARE_TEST_CHADO constant.
laceysanderson Nov 3, 2022
2658e02
Ensure custom temp tables are created in simulated prepare.
laceysanderson Nov 3, 2022
e5d7a8c
Ensure mview tables are created in simulated prepare.
laceysanderson Nov 3, 2022
2c2c9b1
updates to OBOlaoder
spficklin Nov 4, 2022
cf0a6c4
Merge pull request #300 from tripal/tv4g2-291-prepare_test
laceysanderson Nov 4, 2022
3255602
Update sql testing scripts and add script to recreate the prepared ve…
laceysanderson Nov 7, 2022
fbd65bc
Ensure executeSqlFile supports passing in schema name + schema prefix…
laceysanderson Nov 7, 2022
c2315c5
Update creating a prepared test chado + remove the separate method.
laceysanderson Nov 7, 2022
68f7369
Expand testing and use differrent prepare method.
laceysanderson Nov 7, 2022
7141ba3
Merge branch 'tv4g1-issue291-testChadoPrepare' of https://github.com/…
laceysanderson Nov 7, 2022
0e6211a
Null pub is now included in the test environment by default.
laceysanderson Nov 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 44 additions & 12 deletions tripal/src/TripalDBX/TripalDbxConnection.php
Original file line number Diff line number Diff line change
Expand Up @@ -990,7 +990,7 @@ protected function setPrefix($prefix) {
* @return bool
* TRUE if default schema is not Drupal's but the Tripal DBX managed one.
*/
protected function shouldUseTripalDbxSchema() :bool {
public function shouldUseTripalDbxSchema() :bool {
$should = FALSE;

// Check the class/object who is using Tripal DBX:
Expand Down Expand Up @@ -1022,15 +1022,23 @@ protected function shouldUseTripalDbxSchema() :bool {
// Check all parents of the class who is using Tripal DBX:
// This allows for APIs to be added to the whitelist and all children class
// implementations to then automatically use the Tripal DBX managed schema.
$class = new \ReflectionClass($calling_class);
$inheritance_level = 0;
while ($parent = $class->getParentClass()) {
$inheritance_level++;
$parent_class = $parent->getName();
if (!empty($this->classesUsingTripalDbx[$parent_class])) {
$should = TRUE;
if (class_exists($calling_class)) {
$class = new \ReflectionClass($calling_class);
$inheritance_level = 0;
while ($parent = $class->getParentClass()) {
$inheritance_level++;
$parent_class = $parent->getName();
if (!empty($this->classesUsingTripalDbx[$parent_class])) {
$should = TRUE;
}
$class = $parent;
}
$class = $parent;
}
// If Tripal DBX was called from a stand-alone function (i.e. not within
// a class) then the calling class will be empty. We do not want to throw
// an exception in that case.
elseif (!empty($calling_class)) {
throw new \Exception("TripalDBX unable to find class for checking inheritance. This class must exist and be available in the current application space: $calling_class. Hint: make sure to 'use' all needed classes in your application.");
}

return $should;
Expand Down Expand Up @@ -1218,7 +1226,9 @@ public function executeSqlQueries(
?string $schema_name = NULL
) :bool {
// Get schema to use.
$schema_name = $this->getDefaultSchemaName($schema_name);
if (empty($schema_name)) {
$schema_name = $this->getDefaultSchemaName($schema_name);
}
// Set search_path.
if (!empty($schema_name)) {
$search_path = 'SET search_path = "' . $schema_name . '";';
Expand All @@ -1239,19 +1249,29 @@ public function executeSqlQueries(
elseif (is_array($search_path_mode)) {
$search = [];
$replace = [];

foreach ($search_path_mode as $old_name => $replacement) {

// Ensure the replacement pattern is sanitized.
// Secure replacement (we allow comas and spaces).
$replacement = preg_replace(
'/[^a-z_\\xA0-\\xFF0-9\s,]+/',
'',
$replacement
);

// Find/Replace any search path queries.
$search[] =
'/(SET\s*search_path\s*=(?:[^;]+,)?)\s*'
. preg_quote($old_name)
. '\s*((?:,[^;]+)?;)(?!\s*--\s*KEEP)/im'
;
$replace[] = '\1' . $replacement . '\2';

// Find/replace any in-query table prefixing.
$search[] = '/ '. preg_quote($old_name) . '\.(\w+) /';
$replace[] = ' ' . $replacement . '.\1 ';

}
$sql_queries = preg_replace(
$search,
Expand Down Expand Up @@ -1299,13 +1319,15 @@ public function executeSqlQueries(
*/
public function executeSqlFile(
string $sql_file_path,
$search_path_mode = FALSE
$search_path_mode = FALSE,
?string $schema_name = NULL
) :bool {
// Retrieve the SQL file.
$sql_queries = file_get_contents($sql_file_path);
return $this->executeSqlQueries(
$sql_queries,
$search_path_mode
$search_path_mode,
$schema_name
);
}

Expand Down Expand Up @@ -1341,6 +1363,16 @@ public function escapeTable($table) {
return parent::escapeTable($table);
}

/**
* Retrieve a list of classes which are using Tripal DBX byb default.
*
* @return array
* An array of class names including namespace.
*/
public function getListClassesUsingTripalDbx() {
return $this->classesUsingTripalDbx;
}

/**
* Implements the magic __toString method.
*/
Expand Down
14 changes: 6 additions & 8 deletions tripal/src/TripalImporter/TripalImporterBase.php
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,7 @@ protected function setItemsHandled($total_handled) {

if ($total_handled == 0) {
$memory = number_format(memory_get_usage());
print t("Percent complete: 0%. Memory: " . $memory . " bytes.") . "\r";
$this->logger->notice(t("Percent complete: 0%. Memory: " . $memory . " bytes.") . "\r");
return;
}

Expand All @@ -462,12 +462,10 @@ protected function setItemsHandled($total_handled) {
if ($ipercent > 0 and $ipercent != $this->reported and $ipercent % $this->interval == 0) {
$memory = number_format(memory_get_usage());
$spercent = sprintf("%.2f", $percent);
/*
print t("Percent complete: !percent %. Memory: !memory bytes.",
['!percent' => $spercent, '!memory' => $memory]) . "\r";
*/
print t("Percent complete: " . $spercent .
" %. Memory: " . $memory . " bytes.") . "\r";
$this->logger->notice(
t("Percent complete: " . $spercent . " %. Memory: " . $memory . " bytes.")
. "\r"
);

// If we have a job the update the job progress too.
if ($this->job) {
Expand Down Expand Up @@ -502,4 +500,4 @@ public function getArguments() {
return $this->arguments;
}

}
}
19 changes: 10 additions & 9 deletions tripal_chado/src/ChadoCustomTables/ChadoCustomTable.php
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,17 @@ public function __construct($table_name, string $chado_schema = NULL) {
// an empty record for it.
if (!$this->table_id) {
$public = \Drupal::database();
$chado = $this->getChado();
$insert = $public->insert('tripal_custom_tables');
$insert->fields([
'table_name' => $table_name,
'table_name' => $this->table_name,
'schema' => '',
'chado' => $chado->getSchemaName(),
'chado' => $this->chado_schema,
]);
$insert->execute();
$table_id = $insert->execute();
if (!$table_id) {
throw New \Exception('Could not add the custom table, "' . $this->table_name .
'" for the Chado schema "' . $this->chado_schema .'".');
}
$this->setTableId();
}
}
Expand Down Expand Up @@ -97,10 +100,8 @@ private function setTableId() {
$query->condition('ct.table_name', $this->table_name);
$query->condition('ct.chado', $this->chado_schema);
$results = $query->execute();
if ($results) {
$custom_table = $results->fetchObject();
$this->table_id = $custom_table->table_id;
}
$this->table_id = $results->fetchField();

}

/**
Expand Down Expand Up @@ -419,4 +420,4 @@ private function deleteCustomTable() {
}
return True;
}
}
}
8 changes: 4 additions & 4 deletions tripal_chado/src/ChadoCustomTables/ChadoMview.php
Original file line number Diff line number Diff line change
Expand Up @@ -239,10 +239,10 @@ public function populate() {
$transaction = $public->startTransaction();

try {
$chado->query("DELETE FROM {" . $this->getTableName() . "}");
$chado->query("DELETE FROM {1:" . $this->getTableName() . "}");
$sql_query = $this->getSqlQuery();
$chado->query("INSERT INTO {" . $this->getTableName() . "} ($sql_query)");
$results = $chado->query("SELECT COUNT(*) as num_rows FROM {" . $this->getTableName() . "}");
$chado->query("INSERT INTO {1:" . $this->getTableName() . "} ($sql_query)");
$results = $chado->query("SELECT COUNT(*) as num_rows FROM {1:" . $this->getTableName() . "}");
$num_rows = $results->fetchField();
$this->setStatus("Populated with " . $num_rows . " rows");
$this->setLastUpdate(time());
Expand Down Expand Up @@ -288,4 +288,4 @@ public function destroy() {

return parent::destroy();
}
}
}
Loading