Update event bus to use new producer configuration

[rgraber]

A/C

• All events produced on the edx.org event bus do so via configuration rather than explicit code
• No events are lost or duplicated in the upgrade
• Documentation reflects the new way of producing events

openedx/openedx-events#249 will update the event bus producer to produce via a Django configuration setting rather than explicit code. We will need to do a little bit of work to use the new settings without dropping or duplicating events. If we do the upgrade successfully for edx-platform, we should be able to write a guide for teams who produce events from other services.
The overview of the proposal below is that we update all the manual sends edx-platform to only send if the new configuration is not present, add the new configuration, then remove all the manual sends from edx-platform. We shouldn't need to make any changes in event-bus-kafka or the producers.

Tasks

Preview Give feedback

1. Update edx-platform to check for enabled events in EVENT_BUS_PRODUCER_CONFIG before explicitly sending events to the event bus. If the event is already configured to be sent to the event bus, don't manually call send_to_event_bus()
2. Update the version of openedx-events used in edx-platform
3. Add EVENT_BUS_PRODUCER_CONFIG to edx-platform on stage/prod
4. Remove all manual calls to send_to_event_bus() in edx-platform
5. Update the documentation on how to publish events

[robrap]

Thoughts:

1. I'm not entirely clear on the difference between your first two tasks. Maybe you can provide links to code to clarify?
2. In the existing hard-coded sends, could we add a set_custom_attribute for monitoring that shows whether we are sending via code or config? That would make it more clear that everything is doing exactly what we think it should be before removing.
3. Are there ways to confirm that we actually deployed the upgraded openedx-events before flipping config? That would be useful. I guess the fact that we can flip stage before prod also helps.
4. We should discuss whether it would be faster to replicate this process for other services, so others don't need to learn about this one-time transition.
5. Not sure if we actually need separate tickets, but I won't argue if you feel strongly.

[rgraber]

1. That was bad nesting on my part. The second one was supposed to be a clarification on the first. Will update.
2. Sure, good idea
3. We could add a log to openedx events on startup that just says "successfully processed event config"
4. I'm hesitant to take on any more responsibility than necessary but we can discuss after we do the process for edx-platform
5. I guess since the work can't really be done in parallel it should stay in one ticket

[rgraber]

Update: it seems like Navin's PR here should actually cover us since it updates the configuration in common.env, which is part of edx-platform. This should mean we don't lose events in a gap between upgrading edx-platform and adding the configs to edx-internal.
We can test on stage before going to prod. This will require pausing the build but not for terribly long.

[robrap]

[inform] @rgraber I added a task to openedx/openedx-events#210 (comment), after clicking the first issue link I saw in the PR description for openedx/openedx-events#272, but I guess you are really tracking that work in this ticket. I also moved openedx/openedx-events#210 to our board as blocked and assigned it to you, but hopefully we can close it when this ticket is complete.