Some fixes for MySQL and PostgreSQL files in TestLink 1.9.20 (branch testlink_1_9_20_fixed) #251

jmjjg · 2020-02-04T09:53:08Z

Hello,
following my previous pull request, here is some more.

Ubuntu 16.04.6 LTS
mysql Ver 15.1 Distrib 10.0.38-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
psql (PostgreSQL) 9.5.19

I wrote a little bash script (attached and explained below) to help me install, update and compare testlink database schemas from 1.9.19 updated to 1.9.20 versus a stock 19.20 installation (outputting the structure to an SQL file and using diff).

Proposed fixes in this pull request

I made a change in install/sql/alter_tables/1.9.20/mysql/DB.1.9.20/step1/db_schema_update.sql so that the diff would return no difference, although I believe the difference wouldn't break anything.

I also made changes to install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql and install/sql/postgres/testlink_create_tables.sql so the SQL script won't break at execution and the diff would return no difference either.

Explanations on the differences can be found below.

Homemade bash script (bonus)

Here is the bash script that helps me with installing, updating and comparing TestLink databases from the command-line. It is not perfect yet, but if you want, you can include it in the TestLink codebase (or I could do another pull request for that if you prefer).

tl_database.sh.txt

Sample usage

Get sources from github in directory /tmp/github.com for

the official TestLink account on branch testlink_1_9_20_fixed
a proposed solution from the Libriciel SCOP account on branch 1.9.20_tl-database_script_fix_sql_schemas

# Global variables
TL_DATABASE_SCRIPT="/home/cbuffin/Projets/www/gitlab.libriciel.fr/cbuffin/ls-testlink/tl_database.sh"
GITHUB_LOCAL_ROOT="/tmp/github.com"
TL_ORIGINAL_CODE="${GITHUB_LOCAL_ROOT}/TestLinkOpenSourceTRMS/testlink-code"
TL_ORIGINAL_BRANCH_1_9_20_FIXED="testlink_1_9_20_fixed"
LS_CORRECTION_CODE="${GITHUB_LOCAL_ROOT}/libriciel/testlink-code"
LS_CORRECTION_BRANCH="1.9.20_tl-database_script_fix_sql_schemas"
(
    set -o errexit
    set -o nounset
    set -o pipefail
    set -o xtrace

    # Get up-to-date with TL original code
    if [ ! -d "${TL_ORIGINAL_CODE}" ] ; then
        mkdir -p "`dirname "${TL_ORIGINAL_CODE}"`"
        cd "`dirname "${TL_ORIGINAL_CODE}"`"
        git clone https://github.com/TestLinkOpenSourceTRMS/testlink-code.git
    fi
    # Original code
    cd "${TL_ORIGINAL_CODE}"
    git checkout ${TL_ORIGINAL_BRANCH_1_9_20_FIXED}
    # Get up-to-date with TL proposed branch
    if [ ! -d "${LS_CORRECTION_CODE}" ] ; then
        mkdir -p "`dirname "${LS_CORRECTION_CODE}"`"
        cd "`dirname "${LS_CORRECTION_CODE}"`"
        git clone https://github.com/libriciel/testlink-code.git
        cd "${LS_CORRECTION_CODE}"
        git remote add upstream https://github.com/TestLinkOpenSourceTRMS/testlink-code.git
    fi
    cd "${LS_CORRECTION_CODE}"
    git fetch upstream
    git checkout ${LS_CORRECTION_BRANCH}
    git merge upstream/${TL_ORIGINAL_BRANCH_1_9_20_FIXED}
)

MySQL

Check the original code

# Try MySQL install at tag 1.9.19 and update in 1.9.20_fixed
( \
    cd ${TL_ORIGINAL_CODE} \
    && git checkout 1.9.19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_19 \
    && git checkout ${TL_ORIGINAL_BRANCH_1_9_20_FIXED} \
    && ${TL_DATABASE_SCRIPT} update --database testlink_from_1_9_19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_20 \
    && ${TL_DATABASE_SCRIPT} compare --database testlink_from_1_9_19 --reference testlink_from_1_9_20 \
) && echo "Success" || echo "Error"

Check the proposed solution

# Try MySQL install at tag 1.9.19 and update in 1.9.20_tl-database_script_fix_sql_schemas
( \
    cd ${LS_CORRECTION_CODE} \
    && git checkout 1.9.19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_19 \
    && git checkout ${LS_CORRECTION_BRANCH} \
    && ${TL_DATABASE_SCRIPT} update --database testlink_from_1_9_19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_20 \
    && ${TL_DATABASE_SCRIPT} compare --database testlink_from_1_9_19 --reference testlink_from_1_9_20 \
) && echo "Success" || echo "Error"

Explanations

1. Inconsistent `unsigned` usage

<   `enable_on_design` tinyint(1) unsigned NOT NULL DEFAULT '0',
<   `enable_on_execution` tinyint(1) unsigned NOT NULL DEFAULT '1',
---
>   `enable_on_design` tinyint(1) NOT NULL DEFAULT '0',
>   `enable_on_execution` tinyint(1) NOT NULL DEFAULT '1',

Solution

Update lines 25 and 26 from install/sql/alter_tables/1.9.20/mysql/DB.1.9.20/step1/db_schema_update.sql to use tinyint(1) unsigned

ALTER TABLE /*prefix*/platforms ADD COLUMN  enable_on_design tinyint(1) unsigned NOT NULL default '0';
ALTER TABLE /*prefix*/platforms ADD COLUMN  enable_on_execution tinyint(1) unsigned NOT NULL default '1';

Result

2020-02-03 15:11:55 INFO     OK, database structures are the same
2020-02-03 15:11:55 DEBUG    Deleting compare directory "/tmp/tmp.Apb0GtItg3"
2020-02-03 15:11:55 INFO     Process 23078 exited normally

PostgreSQL

Check the original code

# Try PostgreSQL install at tag 1.9.19 and update in 1.9.20_fixed
( \
    cd ${TL_ORIGINAL_CODE} \
    && git checkout 1.9.19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_19 --type postgres \
    && git checkout ${TL_ORIGINAL_BRANCH_1_9_20_FIXED} \
    && ${TL_DATABASE_SCRIPT} update --database testlink_from_1_9_19 --type postgres \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_20 --type postgres \
    && ${TL_DATABASE_SCRIPT} compare --database testlink_from_1_9_19 --reference testlink_from_1_9_20 --type postgres \
) && echo "Success" || echo "Error"

Check the proposed solution

# Try PostgreSQL install at tag 1.9.19 and update in 1.9.20_tl-database_script_fix_sql_schemas
( \
    cd ${LS_CORRECTION_CODE} \
    && git checkout 1.9.19 \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_19 --type postgres \
    && git checkout ${LS_CORRECTION_BRANCH} \
    && ${TL_DATABASE_SCRIPT} update --database testlink_from_1_9_19 --type postgres \
    && ${TL_DATABASE_SCRIPT} reinstall --database testlink_from_1_9_20 --type postgres \
    && ${TL_DATABASE_SCRIPT} compare --database testlink_from_1_9_19 --reference testlink_from_1_9_20 --type postgres \
) && echo "Success" || echo "Error"

Explanations

1. Unique name for indexes in `install/sql/postgres/testlink_create_tables.sql`

2020-02-03 15:51:14 INFO     \i ./install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql
ERROR:  relation "udx1" already exists

Solution

Update line 928 and [941,942] from install/sql/postgres/testlink_create_tables.sql to use unique index names

CREATE UNIQUE INDEX /*prefix*/udx1_baseline_l1l2_context ON /*prefix*/baseline_l1l2_context ("testplan_id","platform_id","creation_ts");

CREATE UNIQUE INDEX /*prefix*/udx1_baseline_l1l2_details
ON /*prefix*/baseline_l1l2_details ("context_id","top_tsuite_id","child_tsuite_id","status");

2. Unique name for indexes in `install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql`

2020-02-03 15:57:58 INFO     \i ./install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql
ERROR:  relation "udx1" already exists

Update line 45 and [58,59] from install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql to use unique index names

CREATE UNIQUE INDEX /*prefix*/udx1_baseline_l1l2_context ON /*prefix*/baseline_l1l2_context ("testplan_id","platform_id","creation_ts");

CREATE UNIQUE INDEX /*prefix*/udx1_baseline_l1l2_details
ON /*prefix*/baseline_l1l2_details ("context_id","top_tsuite_id","child_tsuite_id","status");

3. `INT unsigned` should be `INT`

2020-02-03 11:17:59 INFO     \i ./install/sql/postgres/testlink_create_tables.sql
ERROR:  syntax error at or near "unsigned"
LINE 938:   "qty" INT unsigned NOT NULL DEFAULT '0',
# Will be the same for next line: "total_tc" INT unsigned NOT NULL DEFAULT '0',

Solution

Update lines [937,938] from install/sql/postgres/testlink_create_tables.sql to not use unsigned, since they are
not supported by PostgreSQL.

  "qty" INT NOT NULL DEFAULT '0',
  "total_tc" INT NOT NULL DEFAULT '0',

4. Missing table `execution_tcsteps_wip` in `install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql`

2020-02-03 16:00:48 INFO     tl_database.sh: compare postgres database "testlink_from_1_9_19" with username "testlink"
2020-02-03 16:00:48 INFO     Comparing database structure of "testlink_from_1_9_19" with respect to "testlink_from_1_9_20" in directory "/tmp/tmp.xfmLRRwKWh"
567,602d566
< -- Name: execution_tcsteps_wip; Type: TABLE; Schema: public; Owner: -
< --
< 
< CREATE TABLE public.execution_tcsteps_wip (
...
< --
< -- Name: execution_tcsteps_wip_testplan_id_fkey; Type: FK CONSTRAINT; Schema: public; Owner: -
< --
< 
< ALTER TABLE ONLY public.execution_tcsteps_wip
<     ADD CONSTRAINT execution_tcsteps_wip_testplan_id_fkey FOREIGN KEY (testplan_id) REFERENCES public.testplans(id);
2020-02-03 16:00:49 CRITICAL Process 572 exited with error code 1

Solution

Add the table definition code in install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql around line 23

--
-- Table structure for table "execution_tcsteps_wip"
--
CREATE TABLE /*prefix*/execution_tcsteps_wip (
    "id" BIGSERIAL NOT NULL ,
    "tcstep_id" INTEGER NOT NULL DEFAULT '0' REFERENCES  /*prefix*/tcsteps (id),
    "testplan_id" INTEGER NOT NULL DEFAULT '0' REFERENCES  /*prefix*/testplans (id),
    "platform_id" INTEGER NOT NULL DEFAULT '0',
    "build_id" INTEGER NOT NULL DEFAULT '0',
    "tester_id" BIGINT NULL DEFAULT NULL,
    "creation_ts" TIMESTAMP NOT NULL DEFAULT now(),
    "notes" TEXT NULL DEFAULT NULL,
    "status" CHAR(1) NULL DEFAULT NULL,
    PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX /*prefix*/execution_tcsteps_wip_uidx1 ON  /*prefix*/execution_tcsteps_wip ("tcstep_id","testplan_id","platform_id","build_id");

…le_on_execution in install/sql/alter_tables/1.9.20/mysql/DB.1.9.20/step1/db_schema_update.sql with respect to install/sql/mysql/testlink_create_tables.sql

…baseline_l1l2_context and baseline_l1l2_details, fix unsigned type not available in PostgreSQL in file install/sql/postgres/testlink_create_tables.sql.

….9.20/postgres/DB.1.9.20/step1/db_schema_update.sql with respect to install/sql/postgres/testlink_create_tables.sql, fix unique index names (that should be unique database-wide in PostgreSQL) for tables baseline_l1l2_context and udx1_baseline_l1l2_details.

squash-labs · 2020-02-04T09:53:16Z

Manage this branch in Squash

Test this branch here: https://libriciel1920-tl-database-scri-f9iua.squash.io

kovacs-andras · 2024-08-02T11:48:54Z

Very many thanks!
I just upgraded an older install and without your postgresql fix I was not able to browse the tests tree.
THIS PR MUST BE MERGED!

Christian BUFFIN added 3 commits February 3, 2020 10:46

Fix MySQL data type for platforms.enable_on_design and platforms.enab…

c114ee2

…le_on_execution in install/sql/alter_tables/1.9.20/mysql/DB.1.9.20/step1/db_schema_update.sql with respect to install/sql/mysql/testlink_create_tables.sql

Fix unique index names (that must be unique database-wide) on tables …

493ccb4

…baseline_l1l2_context and baseline_l1l2_details, fix unsigned type not available in PostgreSQL in file install/sql/postgres/testlink_create_tables.sql.

jmjjg requested a review from fmancardi February 4, 2020 09:53

atisne mentioned this pull request Mar 11, 2020

TICKET 0008879: Failure on index creation #255

Closed

atisne mentioned this pull request Mar 30, 2020

TICKET 0008892: Install fails when using Postgresql #259

Merged

kenchan0130 mentioned this pull request Jun 10, 2020

Fix mysql and postgres migration script of 1.9.20 #272

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some fixes for MySQL and PostgreSQL files in TestLink 1.9.20 (branch testlink_1_9_20_fixed) #251

Some fixes for MySQL and PostgreSQL files in TestLink 1.9.20 (branch testlink_1_9_20_fixed) #251

jmjjg commented Feb 4, 2020

squash-labs bot commented Feb 4, 2020

kovacs-andras commented Aug 2, 2024

Some fixes for MySQL and PostgreSQL files in TestLink 1.9.20 (branch testlink_1_9_20_fixed) #251

Are you sure you want to change the base?

Some fixes for MySQL and PostgreSQL files in TestLink 1.9.20 (branch testlink_1_9_20_fixed) #251

Conversation

jmjjg commented Feb 4, 2020

Proposed fixes in this pull request

Homemade bash script (bonus)

Sample usage

MySQL

Check the original code

Check the proposed solution

Explanations

1. Inconsistent unsigned usage

Solution

Result

PostgreSQL

Check the original code

Check the proposed solution

Explanations

1. Unique name for indexes in install/sql/postgres/testlink_create_tables.sql

Solution

2. Unique name for indexes in install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql

3. INT unsigned should be INT

Solution

4. Missing table execution_tcsteps_wip in install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql

Solution

squash-labs bot commented Feb 4, 2020

Manage this branch in Squash

kovacs-andras commented Aug 2, 2024

1. Inconsistent `unsigned` usage

1. Unique name for indexes in `install/sql/postgres/testlink_create_tables.sql`

2. Unique name for indexes in `install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql`

3. `INT unsigned` should be `INT`

4. Missing table `execution_tcsteps_wip` in `install/sql/alter_tables/1.9.20/postgres/DB.1.9.20/step1/db_schema_update.sql`