jobs silently fail

[bradfordcondon]

Hello,
I'm attempting to use your loader. I have created a simple db upload job from the db template file:

I create a job using the UI and run it using drush mcl-run. It runs with no feedback:

drush mcl-run
    Running uploading job in queue..

    Processing job : db_test [6].

    Processing job : test5 [5].

    DONE.

The job status page is updated to failed (see below).

Downloading the log file returns a zip file that is 0 bytes and cannot be unzipped:

$ unzip mcl-6-logs.zip
Archive:  mcl-6-logs.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of mcl-6-logs.zip or
        mcl-6-logs.zip.zip, and cannot find mcl-6-logs.zip.ZIP, period.

$ ls -l mcl-6-logs.zip
-rw-r--r--@ 1 chet  staff  0 Jun 21 10:51 mcl-6-logs.zip

to be clear: the db was not loaded:

select * from chado.db where name='mainlab';
drupal=>

Any thoughts on how to proceed?

Thank you

Bradford

[dsenalik]

I don't know if this is your problem with the zip file being empty, but there is a function mcl_check_setup() in sites/all/modules/mcl/mcl.module that should check that the zip extension is loaded, but as written it does nothing. I had to do this to install the missing zip extension:
sudo apt-get install php-zip

And for me the logs are saved to
sites/default/files/mcl/user/myusername/jobnumber/...
maybe you can find the logs there?

[bradfordcondon]

ah, right. The files, csv, and logs folder in that directory all empty.

I think i have permissions set ok? ( i just did chmod -R 777 for the mcl user under files/mcl...)

attemping to re-upload the file and rerun gives me this error:

Warning: scandir(/Users/chet/UTK/tripal/sites/default/files/mcl/user/admin/admin/6/logs/recent): failed to open dir: No such file or directory in _mcl_get_all_logs() (line 163 of /Users/chet/UTK/tripal/sites/all/modules/custom/mainlab_chado_loader/includes/form/mcl.job_view.form.inc).
Warning: scandir(): (errno 0): Undefined error: 0 in _mcl_get_all_logs() (line 163 of /Users/chet/UTK/tripal/sites/all/modules/custom/mainlab_chado_loader/includes/form/mcl.job_view.form.inc).
Warning: Invalid argument supplied for foreach() in _mcl_get_all_logs() (line 164 of /Users/chet/UTK/tripal/sites/all/modules/custom/mainlab_chado_loader/includes/form/mcl.job_view.form.inc).

same errors attempting to create a new job and upload. I think that it fails to put the file where it wants it. Manually moving the file there doesnt help...

[dsenalik]

I have these notes from when I was first debugging, I don't know if this is much different from what you are trying, but maybe you can try the same and re-run at the command line and see more information?:

Use this command to re-run a job so that I can see the output and error messages

job=1
php /usr/bin/drush mcl-rerun-job $job /var/www/html/cgd/sites/default/files/mcl/user/dsenalik/tmp/04.1992Alternaria.xlsx --move_input

And I also had this later note:

Fix this stupid stupid line 325 in
sites/all/modules/mcl/includes/form/mcl.upload_data.form.inc
$cmd = "drush mcl-rerun-job $job_id $dest_filepath --move_input $opt_no_transaction > /dev/null 2>/dev/null & ";
by instead redirecting to an actual file so that I can see errors!
… > /tmp/mcl-drush.stdout 2> /tmp/mcl-drush.stderr &";
and by doing this, I can now see this error and fix it
Directory /tmp/drush-/cache/default exists, but is not writable.

[bradfordcondon]

Hi @dsenalik , sorry I missed your response and spaced.

Running the job manually via drush does succeed (see below). Confirmed via SQL.

Thank you!

 job=2
drush mcl-rerun-job $job ~/mcl_db_test.xlsx      --move_input


	Process ID = 19482

    Re-running the job (2).

    Processing job : new_test_1 [2].

    ==================================
    Job [2] has been started.
    ----------------------------------

    ==================================
    Phase I : Copying CSV files.
    ----------------------------------
	Excel has 2 sheets

	copying headers

	copying db

        -- Log file (20-W) for COPY_CSV created.
    CSV file created.

    ==================================
    Phase II : Running syntax error check.
    ----------------------------------
    [25] db.csv.
        - Running the syntax error checking : [25] db.csv.
    Syntax error check completed without error.

    ==================================
    Phase III : Uploading data files.
    ----------------------------------
    [25] db.csv.
        - Running the data error checking.
        Data error check passed.

        - Uploading data file.
        Data have been uploaded.

        -- Log file (40-N) for db created.
    Data uploading completed without error.

    DONE : The uploading job was completed without an error..

[bradfordcondon]

So to summarize this issue:

running mcl run-job gives no errors etc, just the processing job notification. Visitng that job's file section shows no file was uploaded, no logs were written.

However, rerunning the job via mcl-rerun-job specifying the file does work.