discussion: go bindings for flux-core

[vsoch]

As we are planning on integrating flux better with Kubernetes, it would be nice to have flux-core expose Go bindings. As an example, the approach I'm taking now to expose a flux instance via a service -> ingress -> API that can be interacted with externally is via a sidecar in the main broker pod that is exposed via a port. This means having one physical node with one pod, and within the pod, two containers - one running an interactive flux cluster (it starts up and basically sleeps forever until we are done) and the second running the restful service (they share a common volume that has the flux socket so the second container can connect to it, see the queue, submit jobs, etc).

For the most basic need, having flux-core go bindings would generate a more robust service. FastAPI is great within Python space but if we are going to eventually have a heavier workload of requests, it can't really compare to gRPC. I think we would eventually want that - if not in the next few months, maybe end of year or next? And anything in Go is preferable to Python (aside from the speed / performance difference) but because it's much easier to grab a binary and run it vs. setting up a Python environment, installing stuff (that often breaks) and then running a python app. I think we might start thinking about what might be needed here for now. I also think having go bindings would open up the developer space better for more cloud native developers to use flux (that tend to prefer go).

For discussion - it's been the case that bindings are built within the same repository, so I suspect we can maintain that. I'd like to have discussion here about what others think (is this something we can do)? and if so, what would be a good way to go about it. I am wondering if we can somehow use the python generated headers as a shared set that would then be used for Go as well, or if we need/want a different approach? And then akin to flux-sched we would have a module in that subdirectory. Likely I can write the Go code for that, and if we are using common header files (and that's already done) it would just be a matter of figuring out a strategy to generate and share them between the two bindings.

Let me know your thoughts! I think I'd enjoy working on this. And I'm relieved we don't have cmake here yet... 😆

[vsoch]

Note that discussion started here: flux-framework/flux-sched#1134 (comment)

[grondo]

I believe flux-go has been created for this purpose, so ok to close this issue?

[vsoch]

Yes - thank you @grondo ! Closing.