Improve build infra for .NET Community Toolkit repository

[Nirmal4G]

Overview

As it stands now, in my opinion, both Windows and .NET Community Toolkit repos have very brittle setup of build infrastructure. There's no way to share the common build files and they are not composable. This makes them un-maintainable in the long term.

Case in point

The WindowsCommunityToolkit repo has not been updated in a long time. This is mainly because of the way the repo is structured and layered. I tried so many patches with the existing structure and somewhere, something breaks or doesn't build as expected. The release of v8 is delayed mainly because of developing the new infrastructure for it via Labs-Windows.

Problem

One of the solutions that's been actively pursued is the infrastructure in Labs-Windows repo. Sure, it does work like most of the time except it doesn't. It's not the implementor's fault, rather it's the implementation that's problematic. The whole setup is composed using so many build files that it makes it hard to edit. If we do edit something, it either breaks somewhere or we won't even be sure if it worked. You can only find a problem until something ships (bad!) and we get a report from the user saying it broke (ouch!).

As I also mentioned above, there's no way to share common build logic as they are not written with that use-case in mind. They are composed of so many redundant props/targets that can be greatly simplified. As such the repos become unmaintainable in the long term and becomes a roadblock when migrating to a new build infrastructure. Such as this case.

Requirements

We can build the basic infrastructure here and go from here. But the work should start from the simplest of all toolkit repos, which is the .NET Community Toolkit repo. After that we can expand from here to all other repos as and when they are ready. From what I saw, in separating the .NET libs into this new repo, we need these 3 things...

1. Custom packaging for source generators (needed for source gen in Windows repo for Design Tools)
2. A way to manage dependencies centrally (patch available but VS Tooling not yet supported)
3. To target multiple versions of the same library (also need in Windows repo).

I'm sure there will be more but for the sake of simplicity, let's just keep these as primary requirements or epics.

Solution

The solution I'm proposing is similar to the Arcade SDK that the .NET Foundation uses but, in my opinion, it's too bloated for simple projects like us. If there's an Arcade Lite SDK that's made specifically for our use-cases, then we could definitely go down on that but unfortunately, it's not. So, we had to do something similar for us.

The main goal is to have a system where a normal contributor doesn't need to fiddle with the build infra to add new features and even if they do, they should work seamlessly and be git blame, VSCode friendly.

This is where my proposal comes in. There are two ways to do our build infra unification.

1. Start with an Arcade Like CommunityToolkit.Arcade.Sdk or CommunityToolkit.Build.Sdk with most basic requirements like assembly/package defaults, central output paths, uniform resource embedding, external package files, text templating, implicit packages, package to project reference and vice versa, targeting multiple library versions and multi-targeting via target profiles. This will be quick but it'd be most churn and there's no way going back (at least for each repo).

2. We'll make sure everything is implemented in the most generic way possible (though not complicated) for each repository's requirements and when ready, we move the custom build scripts into the SDK package to be shared by all the repository. This will take a long time but it will be less impacting and we could revert back if we don't desire using the SDK pattern (for each repo).

Note: To have multi-targeting via target profiles, we need NuGet/Home#5154. Until .NET SDK or NuGet delivers that, we need our own custom targeting. This can be done either using multiple projects (if you need context switching in VS IDE) or using $(Platform) switching.

Notes

This is in no way a complete proposal or implementation path forward. This is just to start a discussion with the existing patches I already have and problems I'm facing when maintaining the forked repos while adding new features and contributing upstream.

[Nirmal4G]

Started from this #462 (comment) and my PR discussions. Refer to those for more context.

[Nirmal4G]

@Sergio0694 I would like to finish what I started but I need some guidance on this. The changes keep piling up as you add more code. It's hard to keep track of it all.