missing resource config triggers only very late on stage out

[marksantcroos]

Error:
2014:02:01 11:52:05 MainThread troy.logger : [DEBUG ] copy file://localhost/Users/mark/proj/troy/troy/examples/gromacs/input/topol.tpr -> ssh://sierra.futuregrid.org / /N/u/marksant/troy_demo/tasks/2/topol.tpr
2014:02:01 11:52:05 MainThread troy.logger : [INFO ] TIMER stage_in_workload ((<troy.workload.workload_manager.WorkloadManager object at 0x110f74910>, 'wl.0002'), {}) : 4.47 sec
Traceback (most recent call last):
File "gromacs_demo.py", line 203, in
strategy = plugin_strategy)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/troy/strategy.py", line 43, in execute_workload
strategy.execute (workload_id, planner, overlay_mgr, workload_mgr)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/pilot/filemanagement/../../troy/plugins/strategy/plugin_strategy_basic_early_binding.py", line 106, in execute
overlay_mgr.provision_overlay (overlay_id)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/troy/utils/time_it.py", line 10, in timed
result = method (_args, *_kwargs)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/troy/overlay/overlay_manager.py", line 295, in provision_overlay
self._provisioner.provision (overlay)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/pilot/filemanagement/../../troy/plugins/overlay_provisioner/plugin_overlay_provisioner_bigjob_pilot.py", line 120, in provision
bj_pilot = self.cp_service.create_pilot (pilot_descr)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/pilot/impl/pilotcompute_manager.py", line 82, in create_pilot
bj = self.__start_bigjob(bj_dict)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/pilot/impl/pilotcompute_manager.py", line 188, in __start_bigjob
pilot_compute_description = bj_dict["pilot_compute_description"]
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/bigjob/bigjob_manager.py", line 388, in start_pilot_job
self.job.run()
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/radical/utils/signatures.py", line 446, in takes_invocation_proxy
return method(_pargs, *_pkwargs)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/radical/utils/signatures.py", line 477, in returns_invocation_proxy
result = method (_args, *_kwargs)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/job/job.py", line 366, in run
return self._adaptor.run (ttype=ttype)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/adaptors/cpi/decorators.py", line 51, in wrap_function
return sync_function (self, _args, *_kwargs)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/adaptors/pbs/pbsjob.py", line 1054, in run
self._id = self.js._job_run(self)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/adaptors/pbs/pbsjob.py", line 590, in _job_run
log_error_and_raise(message, saga.NoSuccess, self._logger)
File "/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/adaptors/pbs/pbsjob.py", line 97, in log_error_and_raise
raise exception(message)
saga.exceptions.NoSuccess: Couldn't create working directory - mkdir: cannot create directory `/Users': Permission denied
(/Users/mark/proj/troy/troy/examples/gromacs/troy_virtualenv/lib/python2.7/site-packages/saga/adaptors/pbs/pbsjob.py +97 (log_error_and_raise) : raise exception(message))
(

[andre-merzky]

Thanks Mark, I think I finally see (or remember, not sure) whats going on. This is not a staging problem, but a pilot submission problem: bigjob falls back to the local pwd when starting a pilot, if no other is specified. I opened a ticket in BigJob a couple of weeks ago (saga-project/BigJob#168), but I can add a value check in Troy for the time being...

Best, Andre.

[andre-merzky]

Decent error message is now raised in e6a4d3c