How do I setup dbt Slim CI on Dagster hybrid deployment?

[lydialimlh]

What Problem Am I Trying To Solve?

Hi everyone. I'm trying to incorporate dbt runs into my CI/CD pipelines. Now, obviously I dont want to run every single dbt model for every PR and waste precious GH Actions minutes - hence Slim CI for dbt runs (i.e. I only want to dbt run the models that have changed VS production.)

How I Would Like to Solve It

My goal is to utilize the dagster-cloud ci dagster-dbt project manage-state command mentioned in the docs. Yet, I am still unable to get it to work. Looking for advice / help from others who have had success using this method. Thanks in advance.

My Setup

• Dagster Plus (aka Cloud) Hybrid Deployment on ECS
• Using GH Actions to run my CI

Issues encountered

FYI I've already raised a GH issue here, but yet to receive concrete feedback from the team, as of time of starting this discussion.

Footnotes

1. I've read several articles out there which recommend using an S3 bucket to store and retrieve production manifest, but I really want to avoid this whole shebang if I can... after all, it looks like the dagster-dbt integration is already headed in the right direction, just needs some refinement... (and I guess that it why I am writing this, so others can find a trail to follow?)

[garethbrickman]

Response in the GH Issue: #25724 (comment)