Dagit can't display the runs from python api.

[wangm23456]

Dagit can't display the runs from python api.
f.e. I run a python script to iterates through the pngs of a directory and then start a dagit to do this too (with same dagster.yaml), But history of runs that from script don't display in dagit, and runs' status from dagit was Queued.
So How should I do to display other runs and event that from python api?

[gibsondan]

Hi, would it be possible to post your dagster.yaml file? It sounds like you may have queuing enabled on your instance but the queued runs aren't getting started for some reason. If you're using queuing, have you started the dagster-daemon process that's used to launch the queued runs?

See https://docs.dagster.io/overview/pipeline-runs/limiting-run-concurrency#setup for how to set up run queueing in your Dagster deployment. (and if it is the daemon that needs to be set up, https://docs.dagster.io/deploying should have more information on how exactly to do that with your setup).

Let us know here or in Slack if that doesn't help.

[gibsondan]

Thanks! if you don't want to use run queueing, you can take out the run_coordinator entry from this. That should fix the problem where it's leaving runs in a queued state.

Alternately, if you do want to use the run queue to limit run concurrency, you could deploy the daemon as well using the deployment docs I mentioned earlier.

[wangm23456]

There is a error in dagster-daemon run:
NotImplementedError: The dagster daemon lives in a separate process. It cannot use in memory storage.

(.venv)  ✘ wangm@wangm  /my/path/to/   master ±  dagster-daemon run 
2021-01-20 10:46:48 - dagster-daemon - INFO - instance is configured with the following daemons: ['SensorDaemon']
2021-01-20 10:46:48 - dagster-daemon - ERROR - Caught error in DaemonType.SENSOR:
SerializableErrorInfo(message="AttributeError: 'NoneType' object has no attribute 'all_stored_job_state'\n", stack=['  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 97, in run_iteration\n    next(generator)\n', '  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/scheduler/sensor.py", line 109, in execute_sensor_iteration\n    for s in instance.all_stored_job_state(job_type=JobType.SENSOR)\n', '  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/core/instance/__init__.py", line 1339, in all_stored_job_state\n    return self._schedule_storage.all_stored_job_state(repository_origin_id, job_type)\n'], cls_name='AttributeError', cause=None)
Traceback (most recent call last):
  File "/my/path/to/.venv/bin/dagster-daemon", line 8, in <module>
    sys.exit(main())
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/cli/__init__.py", line 83, in main
    cli(obj={})  # pylint:disable=E1123
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/cli/__init__.py", line 25, in run_command
    controller.run_iteration(curr_time)
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 110, in run_iteration
    self._check_add_heartbeat(daemon, curr_time)
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 124, in _check_add_heartbeat
    last_stored_heartbeat = self._instance.get_daemon_heartbeats().get(daemon.daemon_type())
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/core/instance/__init__.py", line 1408, in get_daemon_heartbeats
    return self._run_storage.get_daemon_heartbeats()
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/core/storage/runs/in_memory.py", line 245, in get_daemon_heartbeats
    raise NotImplementedError(
NotImplementedError: The dagster daemon lives in a separate process. It cannot use in memory storage.

(.venv)  wangm@wangm  /my/path/to/   master ±  dagster-daemon health-check
Traceback (most recent call last):
  File "/my/path/to/.venv/bin/dagster-daemon", line 8, in <module>
    sys.exit(main())
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/cli/__init__.py", line 83, in main
    cli(obj={})  # pylint:disable=E1123
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/my/path/to/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/cli/__init__.py", line 34, in health_check_command
    if all_daemons_healthy(instance):
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 172, in all_daemons_healthy
    statuses = [
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 173, in <listcomp>
    get_daemon_status(instance, daemon_type, curr_time_seconds=curr_time_seconds)
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/daemon/controller.py", line 191, in get_daemon_status
    heartbeats = instance.get_daemon_heartbeats()
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/core/instance/__init__.py", line 1408, in get_daemon_heartbeats
    return self._run_storage.get_daemon_heartbeats()
  File "/my/path/to/.venv/lib/python3.8/site-packages/dagster/core/storage/runs/in_memory.py", line 245, in get_daemon_heartbeats
    raise NotImplementedError(
NotImplementedError: The dagster daemon lives in a separate process. It cannot use in memory storage.

[gibsondan]

Ah, got it - That "NotImplementedError: The dagster daemon lives in a separate process. It cannot use in memory storage." indicates that the daemon isn't able to load your dagster.yaml file. Have you set your DAGSTER_HOME environment variable? That's what the daemon and dagit use to find and load your instance.

[wangm23456]

!!! Oh! My bad! !!!
I don't set DAGSTER_HOME as global env, and forget to set it in the new shell.
!!! Thanks a million. @gibsondan !!!

[wangm23456]

I suggest that using env_file like docker-compose:

env_file: 
    - .env

[alangenfeld]

Also worth noting that you have to explicitly set instance=DagsterInstance.get() when using the python API since the default behavior is to use an ephemeral test instance.

[wangm23456]

Yes I set execute_pipeline(..., instance=DagsterInstance.get() ) , and I can see rus in postgresql.