Team practice for tracking more complex / medium-term work efforts

[choldgraf]

Description

We should develop a way for our team to track higher-level initiatives, as well as how these initiatives relate to a medium-term timeframe.

Currently, our team coordination and planning practices are based around a deliverables backlog and regular sprints. They have two important qualities:

• The deliverables backlog is relevant at the timescale of a few sprints
• The backlog corresponds to all work we are doing across the team, not focused on any one specific project.

This is a good way to run tactical planning, but it is missing the longer-term and more strategic parts of our planning.

Value / benefit

It's important that we have a higher-level / medium-term planning framework for a few reasons:

• Longer-term planning is important because strategy is relevant on a longer timescale than the next few sprints
• Higher-level planning is a way of grouping work together to make it easier to communicate what we wish to do to the outside world.
• Longer-term planning is a way for us to put issues that are more high-level or vague, but that we don't want to lose track of.
• Higher-level planning is a way to group natural issues together around focused topic areas, and help us decide "what should go on the deliverables backlog next

Implementation details

Thus far, we have played around with two ways to do this:

• Higher-level issues, like this one as tracking issues for other issues.
• Project boards, like this one to track collections of issues related to a broader initiative / theme

A proposal

After playing around with both approaches, my opinion is that we should stop using high-level tracking issues and instead should do higher level planning via GitHub projects. I think this would give us a clean separation of duties between issues and projects, and would allow us to leverage GitHub-native features more easily. In this approach:

• Enhancement issues are always treated like "deliverables". In agile terminology they are always "user stories" or "epics". Issues should always be actionable and concrete, or meant to be broken down into more bite-sized issues. We do not use issues for tracking lots of related work - that's what we use Projects for (and if we just wish to semantically tag issues for search etc we can use labels)
• GH Projects are treated as initiatives. These are collections of issues that fall under a particular theme or outcome. We can use columns to encode "major work areas" for a given project, just to make it easier to separate out different kinds of issues. The issues themselves should all be deliverables.

A few principles to follow

• Every enhancement issue worth working on should be attached to an initiative. Enhancements that do not have an initiative are OK but should be considered "maybe one day" issues. If an issue goes for long without an initiative, we should probably just close it.
• Initiatives should have an indication of when they are ready to be worked on. For example "actively working", "up next", "soon". Some initiatives might be scoped out before we have resources for them, and we can use this to try and get funding.
• Initiatives that are nearer to "active" should have issues that are more refined. This is a similar principle to our deliverables backlog - as things get closer in time, we should put more effort into scoping them out, breaking up into smaller issues, etc.
• Initiatives should have an indication of the team that is working on it, so it is clear who is responsible for the issues that are on the board.

what would we need to do to implement this now?

I think we'd need to these things if we wanted to implement this now:

1. Break apart our "meta" issues (like Major development Managed Hub Service v1 infrastructure#610) into more atomic issues, one per deliverable, and then close the "meta" issues.
2. Make sure that each deliverable issue is assigned to the proper board (right now, probably https://github.com/orgs/2i2c-org/projects/10)
3. Decide if there are other major initiatives we are undertaking now (maybe one for the Pangeo project, or one with the Executable Books project)

Tasks to complete

• Discuss the proposal here and decide if it is worth trying out as a team
• Implement the steps described above (or others as they come up)
• Test this out for a few months, revisit in November 2021
• Decide what to do from there

Updates

No response

[damianavila]

After playing around with both approaches, my opinion is that we should stop using high-level tracking issues and instead should do higher level planning via GitHub projects.

I agree with the proposal!
I certainly do not like issues collecting a lot of other issues.
Issues are not meant or designed for that purpose.
Boards instead are entities that naturally and by design can be used to group issues/ideas/enhancements, etc.

[yuvipanda]

I'm somewhat attached to this table 2i2c-org/infrastructure#610 :D

Here's the reasons I like it as a table:

1. it fits in one screen. I'm struggling with cognitive overload on most boards.
2. it shows clearly where some functionality is present in AWS / GCP but not Azure. I'm not sure how we can get this in a board
3. it shows clearly where functionality is cross cloud, and where it isn't but could be
4. it can be easily edited via text, in a way that leaves a change trail - unlike a board, which requires using a mouse.
5. We can put context around what is happening in the issue description, and use comments to have conversations.