feat: Determine SCM and parse clone url in GitProvenance for more accurate path/origin/organiation/groups

[pstreef]

What's changed?

Determine the SCM from a list of known urls and any manually registered SCMs. The SCM is then used to parse a typed CloneUrl which in turn give information about a repository. Depending on the SCM there are some expectations about the clone url that must be met.

What's your motivation?

To determine the repository origin, path, organization, project and group path (for GitLab) is tricky, especially with self-hosted SCMs. Doing this in 1 place makes it more maintainable.

Anything in particular you'd like reviewers to focus on?

Any url patterns that are missed

Anyone you would like to review specifically?

@bryceatmoderne

Have you considered any alternatives or workarounds?

Implementing this in higher level code/places.

Any additional context

This is going to be used directly by Moderne CLI, but should be useful for other projects as well.

Checklist

• I've added unit tests to cover both positive and negative cases
• I've read and applied the recipe conventions and best practices
• I've used the IntelliJ IDEA auto-formatter on affected files

[timtebeek]

Resolved these issues with a quick recipe run from the IntelliJ IDEA plugin through

type: specs.openrewrite.org/v1beta/recipe
name: Scratch rewrite.yml
recipeList:
  - org.openrewrite.java.format.EmptyNewlineAtEndOfFile

The full list of checks on every pull request is maintained here:

rewrite/.github/workflows/receive-pr.yml

Lines 21 to 84 in 4568577

	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.OpenRewriteBestPracticesSubset
	displayName: OpenRewrite best practices
	description: Best practices for OpenRewrite recipe development.
	recipeList:
	- org.openrewrite.recipes.JavaRecipeBestPracticesSubset
	- org.openrewrite.recipes.RecipeTestingBestPracticesSubset
	- org.openrewrite.recipes.RecipeNullabilityBestPracticesSubset
	#- org.openrewrite.java.OrderImports
	- org.openrewrite.java.format.EmptyNewlineAtEndOfFile
	- org.openrewrite.staticanalysis.InlineVariable
	- org.openrewrite.staticanalysis.MissingOverrideAnnotation
	- org.openrewrite.staticanalysis.UseDiamondOperator
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.JavaRecipeBestPracticesSubset
	displayName: Java Recipe best practices
	description: Best practices for Java recipe development.
	preconditions:
	- org.openrewrite.java.search.FindTypes:
	fullyQualifiedTypeName: org.openrewrite.Recipe
	checkAssignability: true
	recipeList:
	- org.openrewrite.java.recipes.BlankLinesAroundFieldsWithAnnotations
	- org.openrewrite.java.recipes.ExecutionContextParameterName
	- org.openrewrite.java.recipes.MissingOptionExample
	- org.openrewrite.java.recipes.RecipeEqualsAndHashCodeCallSuper
	- org.openrewrite.java.recipes.UseTreeRandomId
	- org.openrewrite.staticanalysis.NeedBraces
	#- org.openrewrite.staticanalysis.RemoveSystemOutPrintln
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.RecipeTestingBestPracticesSubset
	displayName: Recipe testing best practices
	description: Best practices for testing recipes.
	preconditions:
	- org.openrewrite.java.search.FindTypes:
	fullyQualifiedTypeName: org.openrewrite.test.RewriteTest
	checkAssignability: true
	recipeList:
	- org.openrewrite.java.recipes.RewriteTestClassesShouldNotBePublic
	#- org.openrewrite.java.recipes.SelectRecipeExamples
	- org.openrewrite.java.recipes.SourceSpecTextBlockIndentation
	- org.openrewrite.java.testing.cleanup.RemoveTestPrefix
	- org.openrewrite.java.testing.cleanup.TestsShouldNotBePublic
	- org.openrewrite.staticanalysis.NeedBraces
	- org.openrewrite.staticanalysis.RemoveSystemOutPrintln
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.RecipeNullabilityBestPracticesSubset
	displayName: Recipe nullability best practices
	description: Use OpenRewrite internal nullability annotations; drop JetBrains annotations; use `package-info.java` instead.
	recipeList:
	- org.openrewrite.java.ChangeType:
	oldFullyQualifiedTypeName: org.jetbrains.annotations.Nullable
	newFullyQualifiedTypeName: org.openrewrite.internal.lang.Nullable
	- org.openrewrite.java.ChangeType:
	oldFullyQualifiedTypeName: jakarta.annotation.Nullable
	newFullyQualifiedTypeName: org.openrewrite.internal.lang.Nullable
	- org.openrewrite.java.RemoveAnnotation:
	annotationPattern: '@org.jetbrains.annotations.NotNull'
	- org.openrewrite.java.RemoveAnnotation:
	annotationPattern: '@jakarta.annotation.Nonnull'

I'm considering adding that to a root rewrite.yml to make it easier to run these recipes on your local machine. Thoughts?

[pstreef]

Resolved these issues with a quick recipe run from the IntelliJ IDEA plugin through

type: specs.openrewrite.org/v1beta/recipe
name: Scratch rewrite.yml
recipeList:
  - org.openrewrite.java.format.EmptyNewlineAtEndOfFile

The full list of checks on every pull request is maintained here:

rewrite/.github/workflows/receive-pr.yml

Lines 21 to 84 in 4568577

	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.OpenRewriteBestPracticesSubset
	displayName: OpenRewrite best practices
	description: Best practices for OpenRewrite recipe development.
	recipeList:
	- org.openrewrite.recipes.JavaRecipeBestPracticesSubset
	- org.openrewrite.recipes.RecipeTestingBestPracticesSubset
	- org.openrewrite.recipes.RecipeNullabilityBestPracticesSubset
	#- org.openrewrite.java.OrderImports
	- org.openrewrite.java.format.EmptyNewlineAtEndOfFile
	- org.openrewrite.staticanalysis.InlineVariable
	- org.openrewrite.staticanalysis.MissingOverrideAnnotation
	- org.openrewrite.staticanalysis.UseDiamondOperator
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.JavaRecipeBestPracticesSubset
	displayName: Java Recipe best practices
	description: Best practices for Java recipe development.
	preconditions:
	- org.openrewrite.java.search.FindTypes:
	fullyQualifiedTypeName: org.openrewrite.Recipe
	checkAssignability: true
	recipeList:
	- org.openrewrite.java.recipes.BlankLinesAroundFieldsWithAnnotations
	- org.openrewrite.java.recipes.ExecutionContextParameterName
	- org.openrewrite.java.recipes.MissingOptionExample
	- org.openrewrite.java.recipes.RecipeEqualsAndHashCodeCallSuper
	- org.openrewrite.java.recipes.UseTreeRandomId
	- org.openrewrite.staticanalysis.NeedBraces
	#- org.openrewrite.staticanalysis.RemoveSystemOutPrintln
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.RecipeTestingBestPracticesSubset
	displayName: Recipe testing best practices
	description: Best practices for testing recipes.
	preconditions:
	- org.openrewrite.java.search.FindTypes:
	fullyQualifiedTypeName: org.openrewrite.test.RewriteTest
	checkAssignability: true
	recipeList:
	- org.openrewrite.java.recipes.RewriteTestClassesShouldNotBePublic
	#- org.openrewrite.java.recipes.SelectRecipeExamples
	- org.openrewrite.java.recipes.SourceSpecTextBlockIndentation
	- org.openrewrite.java.testing.cleanup.RemoveTestPrefix
	- org.openrewrite.java.testing.cleanup.TestsShouldNotBePublic
	- org.openrewrite.staticanalysis.NeedBraces
	- org.openrewrite.staticanalysis.RemoveSystemOutPrintln
	---
	type: specs.openrewrite.org/v1beta/recipe
	name: org.openrewrite.recipes.RecipeNullabilityBestPracticesSubset
	displayName: Recipe nullability best practices
	description: Use OpenRewrite internal nullability annotations; drop JetBrains annotations; use `package-info.java` instead.
	recipeList:
	- org.openrewrite.java.ChangeType:
	oldFullyQualifiedTypeName: org.jetbrains.annotations.Nullable
	newFullyQualifiedTypeName: org.openrewrite.internal.lang.Nullable
	- org.openrewrite.java.ChangeType:
	oldFullyQualifiedTypeName: jakarta.annotation.Nullable
	newFullyQualifiedTypeName: org.openrewrite.internal.lang.Nullable
	- org.openrewrite.java.RemoveAnnotation:
	annotationPattern: '@org.jetbrains.annotations.NotNull'
	- org.openrewrite.java.RemoveAnnotation:
	annotationPattern: '@jakarta.annotation.Nonnull'

I'm considering adding that to a root rewrite.yml to make it easier to run these recipes on your local machine. Thoughts?

Ah that's what it was trying to do. The suggestions were not showing that and not applyable.. Yeah sound like a good plan!

[okundzich]

just talked to Bryce about azuredevops and this came up. We require configuration of baseurl in moderne CLI/agent. curious about this.

[jkschneider]

We should probably only calculate this when it is first used, otherwise this slows down deserialization.

[jkschneider]

I'm not sure it's necessary to deprecate since it is in widespread use. Fine to have a convenience method for organization on CloneUrl

[pstreef]

actually I think we should point back to the "old" method that does not take an argument, and deprecate this one which (again) takes the origin (with scheme).

[pstreef]

replaced by #4381