Parallel computing

[netbofia]

Hello this is a great tool but it takes a long time to run. Is it possible to parallelize it.
For example running it n time by dividing the query miRNAs and feeding them to the multiple running instances.

[MikeAxtell]

Thanks. Yes, it's slow. You can do a poor man's parallel processing by
splitting miRNA query files as you say and submitted to separate jobs on a
cluster.

Cheers,
Mike

On Sat, May 28, 2016 at 3:34 PM, Bruno Costa [email protected]
wrote:

Hello this is a great tool but it takes a long time to run. Is it possible
to parallelize it.
For example running it n time by dividing the query miRNAs and feeding
them to the multiple running instances.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#4, or mute the thread
https://github.com/notifications/unsubscribe/AGiXiU_sVb93z1aPMlnpVQjG-VEeX7frks5qGJirgaJpZM4IpJMH
.

Michael J. Axtell, Ph.D.
Associate Professor
Dept. of Biology
Penn State University
http://sites.psu.edu/axtell
Twitter: @MikeAxtell

[netbofia]

Thanks for the feedback. I've forked the project and created a wrapper in bash for Cleaveland4
It allows running in mode 1 & 2. (Which I think are the only ones that make sense parallelizing)
Grabs x Sequences from -u makes a new directory and runs.
Afterward all outputs are merged together.

Most code is commented should be easy to understand I hope.

Is this something you would be interested in merging with your work.
Check my fork: https://github.com/netbofia/CleaveLand4

Bruno

[MikeAxtell]

Cool, I'll check it out next week after I get out from under a grant
deadline and a trip.

Cheers,
Mike

On Tue, May 31, 2016 at 3:22 PM, Bruno Costa [email protected]
wrote:

Thanks for the feedback. I've forked the project and created a wrapper in
bash for Cleaveland4
It allows running in mode 1 & 2. (Which I think are the only ones that
make sense parallelizing)
Grabs x Sequences from -u makes a new directory and runs.
Afterward all outputs are merged together.

Most code is commented should be easy to understand I hope.

Is this something you would be interested in merging with your work.
Check my fork: https://github.com/netbofia/CleaveLand4

Bruno

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#4 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AGiXiaEr-iuG5gO7ak_ei2NzFjPffBXQks5qHIqBgaJpZM4IpJMH
.

Michael J. Axtell, Ph.D.
Associate Professor
Dept. of Biology
Penn State University
http://sites.psu.edu/axtell
Twitter: @MikeAxtell

[wangyugui]

@netbofia
the fork of netbofia is version 4.3 only.

@MikeAxtell
I tried to split miRNA query files with the command '/usr/bin/parallel', but it seem that we need something to do about temp file inside the process.

[netbofia]

Sorry only saw this today. I'll test it and give you feed back in the following days.

[netbofia]

@MikeAxtell @wangyugui
I'm sorry for this big delay.
@wangyugui The fork can work with any version of the input scheme is the same. Just change the cleaveLand var in line 169 to point to the cleaveland file of choice.

Let me know about any issues. I tested this version and it seams to hold up but strange issues might arise. Send me an email if necessary at: [email protected]