Update HTCondor API to remove deprecation warning #42
Conversation
|
@ikrommyd let me know if this works for you |
|
This didn't work, unless I'm doing something wrong. I copied Then I started a client In [1]: from distributed import Client
...: from lpcjobqueue import LPCCondorCluster
...: import logging
...:
...:
...: logging.basicConfig(level=logging.DEBUG)
...:
...: cluster = cluster = LPCCondorCluster(log_directory="/uscms/home/ikrommyd/dask_logs", ship_env=True)
...: cluster.adapt(minimum=1, maximum=2)
...: client = Client(cluster)And tried to run your In [4]: def baz():
...: import subprocess
...:
...: path = "/usr/local/lib/python3.11/site-packages/awkward_cpp/lib/libawkward.so"
...: return subprocess.getoutput(f"ldd -v {path}")
...:
...: print(baz())
...: print("="*30)
...: for k, v in client.run(baz).items():
...: print(k)
...: print(v)This ran into Same thing with this In [5]: from distributed import Client
...: from lpcjobqueue import LPCCondorCluster
...: import logging
...:
...:
...: logging.basicConfig(level=logging.DEBUG)
...:
...: cluster = cluster = LPCCondorCluster(log_directory="/uscms/home/ikrommyd/dask_logs", ship_env=True)
...: cluster.adapt(minimum=0, maximum=2)
...: client = Client(cluster)
...:
...: for future in client.map(lambda x: x * 5, range(2)):
...: print(future.result())
...: cluster.close()Any computation will give you this error with a pip-installed awkward due to |
|
General rule for #40 to be closed is from distributed import Client
from lpcjobqueue import LPCCondorCluster
import logging
logging.basicConfig(level=logging.DEBUG)
cluster = cluster = LPCCondorCluster(log_directory="/uscms/home/ikrommyd/dask_logs", ship_env=True)
cluster.adapt(minimum=0, maximum=2)
client = Client(cluster)
for future in client.map(lambda x: x * 5, range(2)):
print(future.result())
cluster.close()should work when |
|
Does it work if you use the image-provided |
The image provided awkward is a conda-forge one and yes that always worked. The conda-forge packages know to look for the libstd under /usr/local in the workers as well. It’s the pip installed ones that don’t. |
Also resolve symlink in image path
Closes #32 and #40