CI for CI should test idempotence

[rdebeasi]

It would be great if our "CI for CI" ran the playbook twice to test for idempotence. This could catch issues such as #228.

One trade-off: this would probably make builds take longer - and they already take a little while.

[sherl0cks]

@rdebeasi I'm hesitant to increase the complexity of the CI system given that it's already complex and only a couple people understand it well enough to maintain it. I could be persuaded, but that would need to come with enablement for a much broader set of team members (something that's been on my radar for some time) and a commitment to contribute to maintenance.

That said, I understand the frustration that seems to be derived from #228. Perhaps a more localized solution will do? Maybe we can pull this role into its own repo?

[rdebeasi]

Sure, I'd love to see this role moved into its own repo. A consumer of Labs CI/CD won't necessarily care about how Labs CI/CD itself is built. That change seems like it would fit in with the broader effort to pare down Labs CI/CD.

[sherl0cks]

ok. now to find the time and energy...

[rdebeasi]

You might want to compare notes with @pcarney8 @logandonley @oybed , who have been thinking about how to simplify Labs CI/CD and split out things into separate repos.

[sherl0cks]

Issues are linked now.

[sherl0cks]

also, for what its worth, this is a good place for an operator if someone wants to write one