Scripting access of django databases #276

deming7h777 · 2019-04-17T20:37:17Z

No description provided.

codecov · 2019-04-17T20:55:22Z

Codecov Report

Merging #276 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #276   +/-   ##
=======================================
  Coverage   51.01%   51.01%           
=======================================
  Files          64       64           
  Lines        2662     2662           
=======================================
  Hits         1358     1358           
  Misses       1304     1304

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fadd774...555ca93. Read the comment docs.

r06942072 · 2019-07-22T02:58:52Z

I am on my personal laptop.
I met a problem once I want to start running django

This is what I did:

Spin up the project by genomics-VM.
https://github.com/NAL-i5K/genomics-VM
Enter virtualenv
cd genomics-workspace
source py2.7/bin/activate
python manage.py runserver, and I got a message as below

r06942072 · 2019-07-23T14:58:03Z

postgresql not found

r06942072 · 2019-07-23T16:15:21Z

#280

python manage.py addorganism Aethina tumida

python manage.py addblast Aethina tumida -t peptide protein -f GCF_001937115.1_Atum_1.0_genomic.fna

python manage.py makeblast GCF_001937115.1_Atum_1.0_genomic.fna

python manage.py populatesequence GCF_001937115.1_Atum_1.0_genomic.fna

python manage.py delete -o Aethina tumida

python manage.py delete -b GCF_001937115.1_Atum_1.0_genomic.fna

python manage.py delete -h GCF_001937115.1_Atum_1.0_genomic.fna

python manage.py addhmmer GCF_001937115.1_Atum_1.0_genomic.fna

r06942072

The module is not found
This is the process I did

git clone --branch addorg2 https://github.com/NAL-i5K/genomics-workspace.git
source py2.7/bin/activate (use the py2.7 env created when running genomics-VM)
pip install -r requirements.txt
new added python manage.py command as below screenshot

hsiaoyi0504 · 2019-07-24T18:22:19Z

@r06942072 As far as I can tell, you should follow the step of setup guide. https://genomics-workspace.readthedocs.io/en/latest/setup_guide_centos.html

deming7h777 · 2019-07-25T14:43:03Z

The module is not found
This is the process I did

git clone --branch addorg2 https://github.com/NAL-i5K/genomics-workspace.git

virtualenv venv (I create virtualenv myself, but I do not know if there exist one in repo)

source venv/bin/activate

pip install -r requirements.txt

new added python manage.py command as below screenshot

try to add init.py under blast/management/commands

hsiaoyi0504 · 2019-07-25T14:58:10Z

@deming7h777
If this is the solution to @r06942072 's problem,

try to add init.py under blast/management/commands

maybe it should be added directly on that branch.

hsiaoyi0504 · 2019-07-25T15:00:24Z

misc/add_func.py

+    #organism = options["organism"][0].lower().capitalize() + " " + options["organism"][1].lower()
+    Organism.objects.filter(display_name = name).delete()
+    return ("remove %s in database"%name)
+'''


Don't include these commented out lines. It seems to me that these lines are for initial development, right ?

ok, good point

hsiaoyi0504 · 2019-07-25T15:01:17Z

misc/add_func.py

@@ -0,0 +1,160 @@
+from blast.models import BlastDb, SequenceType
+#from django.core.management.base import BaseCommand, CommandError


Remove unused import directly rather than commenting it out.

hsiaoyi0504 · 2019-07-25T15:02:54Z

misc/add_func.py

+            data2 = re2.json()
+            key = data1['query']['search'][0]['pageid']
+            key = str(key)
+            #print type(key)


remove debug purpose code will be great.

hsiaoyi0504 · 2019-07-25T15:03:32Z

misc/add_func.py

+        sys.exit(0)
+
+def delete_org(name):
+    #if options["organism"]:


Remove unused lines here.

r06942072 · 2019-07-25T17:09:11Z

@r06942072 As far as I can tell, you should follow the step of setup guide. https://genomics-workspace.readthedocs.io/en/latest/setup_guide_centos.html

@hsiaoyi0504 I use repo genomics-VM to set up a new VM, so I didn't go through the process manually in Setup Guide

r06942072 · 2019-07-25T20:06:16Z

@deming7h777
I tried to run those commands on #280
only addorganism work

Here is what I did (I use honeybee as an example organism)
- cd ~/Downloads
- wget ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/003/254/395/GCF_003254395.2_Amel_HAv3.1/GCF_003254395.2_Amel_HAv3.1_genomic.fna.gz
- gzip -d *.gz
- cd ~/genomics-workspace
- python manage.py addorganism Apis mellifera (This one work, I checked it through browser)
Down below three screenshot described the bug I met:
- 3 commands not found
- addblast
- addhmmer

deming7h777 · 2019-07-25T20:55:50Z

@r06942072 you get the error because you don't need to type the full path of fasta file , just the name and title of blast db or hmmer db

r06942072 · 2019-07-26T03:50:09Z

@r06942072 you get the error because you don't need to type the full path of fasta file , just the name and title of blast db or hmmer db

Do you mean like this? I got the same error(See pic1 and pic2)

pic1
pic2

@r06942072 you get the error because you don't need to type the full path of fasta file , just the name and title of blast db or hmmer db

hsiaoyi0504 · 2019-07-26T06:46:50Z

@deming7h777 In my opinion, it will be good to handle path no matter it‘s full path or not. That is, you should handle relative path and absolute path at the same time.

r06942072 · 2019-07-26T15:45:46Z

Friday, I pull the update of addorg2, and ran, same error messsage

pull
run

r06942072 · 2019-07-29T15:36:13Z

Monday, what I did
cd ~/genomics-workspace

*On personal laptop, make a folder called 'media'
mkdir -p media/blast/db media/hmmer/db
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/blast/db/
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/hmmer/db/

*command 1 works
python manage.py addblast Apis mellifera -t peptide protein -f GCF_003254395.2_Amel_HAv3.1_genomic.fna

*command2 not works
python manage.py blast_utility GCF_003254395.2_Amel_HAv3.1_genomic.fna -m
see screenshot

deming7h777

Monday, what I did
cd ~/genomics-workspace

*On personal laptop, make a folder called 'media'
mkdir -p media/blast/db media/hmmer/db
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/blast/db/
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/hmmer/db/

*command 1 works
python manage.py addblast Apis mellifera -t peptide protein -f GCF_003254395.2_Amel_HAv3.1_genomic.fna

*command2 not works
python manage.py blast_utility GCF_003254395.2_Amel_HAv3.1_genomic.fna -m
see screenshot

please run npm run build first

deming7h777 · 2019-07-25T15:50:28Z

misc/add_func.py

@@ -0,0 +1,160 @@
+from blast.models import BlastDb, SequenceType
+#from django.core.management.base import BaseCommand, CommandError


deming7h777 · 2019-07-25T15:50:42Z

misc/add_func.py

+            data2 = re2.json()
+            key = data1['query']['search'][0]['pageid']
+            key = str(key)
+            #print type(key)


deming7h777 · 2019-07-25T15:50:49Z

misc/add_func.py

+        sys.exit(0)
+
+def delete_org(name):
+    #if options["organism"]:


deming7h777 · 2019-07-25T15:51:04Z

misc/add_func.py

+    #organism = options["organism"][0].lower().capitalize() + " " + options["organism"][1].lower()
+    Organism.objects.filter(display_name = name).delete()
+    return ("remove %s in database"%name)
+'''


ok, good point

deming7h777 · 2019-07-29T17:56:15Z

Monday, what I did
cd ~/genomics-workspace
*On personal laptop, make a folder called 'media'
mkdir -p media/blast/db media/hmmer/db
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/blast/db/
cp ../Downloads/GCF_003254395.2_Amel_HAv3.1_genomic.fna ./media/hmmer/db/
*command 1 works
python manage.py addblast Apis mellifera -t peptide protein -f GCF_003254395.2_Amel_HAv3.1_genomic.fna
*command2 not works
python manage.py blast_utility GCF_003254395.2_Amel_HAv3.1_genomic.fna -m
see screenshot

please run npm run build first, finish the set up guide first

r06942072 · 2019-07-31T13:59:47Z

July.31, 1st post

create a checklist for this PR in All feature under manage.py command #280
sudo yum install npm
npm
npm run build not success, see error message

...
Try to makeblastdb on admin page through mouse clicking, but get error message same as running script, no such file
try addhmmer, but an error is too few argument

r06942072 · 2019-07-31T15:20:59Z

July31, 2nd post

see the checklist for this PR in All feature under manage.py command #280
update npm and node to a higher version, and run npm run build
successfully run npm run build
successfully run
python manage.py blast_utility GCF_003254395.2_Amel_HAv3.1_genomic.fna -m
python manage.py blast_utility GCF_003254395.2_Amel_HAv3.1_genomic.fna -p
check those through browser
successfully run (It was my typo in July31, 1st post)
python manage.py addhmmer Apis mellifera -f GCF_003254395.2_Amel_HAv3.1_genomic.fna
command python manage.py delete not found

vagrant added 5 commits April 17, 2019 16:27

addblast.py compoleted

f7b6cd3

addorganism.py completed

b3fe6df

split addblast to 3 steps and makeblastdb.pu completed

49f9768

populate sequence functionality completed and fix addorganism script

7729f38

addhmmer.py completed

cd55449

deming7h777 requested review from childers and r06942072 April 17, 2019 20:37

fix some blank space after else:

8051e72

NAL-i5K deleted a comment Apr 17, 2019

r06942072 reviewed Jul 24, 2019

View reviewed changes

hsiaoyi0504 requested changes Jul 25, 2019

View reviewed changes

vagrant added 3 commits July 25, 2019 11:41

add __init__.py

1750dd4

remove useless line in add_func.py

f92c8a7

update django 1.11.22

a70ff41

add more __init__.py

adac21e

comment unused module

555ca93

NAL-i5K deleted a comment Jul 25, 2019

deming7h777 commented Jul 29, 2019

View reviewed changes

r06942072 merged commit 555ca93 into master Aug 1, 2019

hsiaoyi0504 deleted the addorg2 branch September 23, 2019 06:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scripting access of django databases #276

Scripting access of django databases #276

deming7h777 commented Apr 17, 2019

codecov bot commented Apr 17, 2019 •

edited

Loading

r06942072 commented Jul 22, 2019 •

edited

Loading

r06942072 commented Jul 23, 2019

r06942072 commented Jul 23, 2019 •

edited

Loading

r06942072 left a comment •

edited

Loading

hsiaoyi0504 commented Jul 24, 2019

deming7h777 commented Jul 25, 2019

hsiaoyi0504 commented Jul 25, 2019

hsiaoyi0504 Jul 25, 2019

deming7h777 Jul 25, 2019

hsiaoyi0504 Jul 25, 2019

deming7h777 Jul 25, 2019

hsiaoyi0504 Jul 25, 2019

deming7h777 Jul 25, 2019

hsiaoyi0504 Jul 25, 2019

deming7h777 Jul 25, 2019

r06942072 commented Jul 25, 2019

r06942072 commented Jul 25, 2019

deming7h777 commented Jul 25, 2019

r06942072 commented Jul 26, 2019 •

edited

Loading

hsiaoyi0504 commented Jul 26, 2019

r06942072 commented Jul 26, 2019

r06942072 commented Jul 29, 2019

deming7h777 left a comment

deming7h777 Jul 25, 2019

deming7h777 Jul 25, 2019

deming7h777 Jul 25, 2019

deming7h777 Jul 25, 2019

deming7h777 commented Jul 29, 2019

r06942072 commented Jul 31, 2019 •

edited

Loading

r06942072 commented Jul 31, 2019 •

edited

Loading

		@@ -0,0 +1,160 @@
		from blast.models import BlastDb, SequenceType
		#from django.core.management.base import BaseCommand, CommandError

Scripting access of django databases #276

Scripting access of django databases #276

Conversation

deming7h777 commented Apr 17, 2019

codecov bot commented Apr 17, 2019 • edited Loading

Codecov Report

r06942072 commented Jul 22, 2019 • edited Loading

r06942072 commented Jul 23, 2019

r06942072 commented Jul 23, 2019 • edited Loading

r06942072 left a comment • edited Loading

Choose a reason for hiding this comment

hsiaoyi0504 commented Jul 24, 2019

deming7h777 commented Jul 25, 2019

hsiaoyi0504 commented Jul 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

r06942072 commented Jul 25, 2019

r06942072 commented Jul 25, 2019

deming7h777 commented Jul 25, 2019

r06942072 commented Jul 26, 2019 • edited Loading

hsiaoyi0504 commented Jul 26, 2019

r06942072 commented Jul 26, 2019

r06942072 commented Jul 29, 2019

deming7h777 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deming7h777 commented Jul 29, 2019

r06942072 commented Jul 31, 2019 • edited Loading

r06942072 commented Jul 31, 2019 • edited Loading

codecov bot commented Apr 17, 2019 •

edited

Loading

r06942072 commented Jul 22, 2019 •

edited

Loading

r06942072 commented Jul 23, 2019 •

edited

Loading

r06942072 left a comment •

edited

Loading

r06942072 commented Jul 26, 2019 •

edited

Loading

r06942072 commented Jul 31, 2019 •

edited

Loading

r06942072 commented Jul 31, 2019 •

edited

Loading