[#5731]feat(auth-ranger): RangerAuthorizationHDFSPlugin supports Fileset authorization

[theoryxu]

What changes were proposed in this pull request?

RangerAuthorizationHDFSPlugin supports Fileset authorization

Why are the changes needed?

Fix: #5731

Does this PR introduce any user-facing change?

Addition property keys in Fileset

How was this patch tested?

ITs

[xunliu]

@theoryxu Please rebase your PR to the latest main branch, Thanks.

[theoryxu]

@theoryxu Please rebase your PR to the latest main branch, Thanks.

done

[xunliu]

RangerFilesetIT > initializationError FAILED
    java.lang.IllegalArgumentException: Failed to operate catalog(s) [RangerFilesetE2EIT_catalog_1bb7992c] operation [CREATE] under metalake [metalake], reason [Invalid package path: /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs in [/Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs, /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/resources/main, /Users/xun/github/xunliu/gravitino/authorizations/authorization-ranger/build/libs]]
    java.lang.IllegalArgumentException: Invalid package path: /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs in [/Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs, /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/resources/main, /Users/xun/github/xunliu/gravitino/authorizations/authorization-ranger/build/libs]
        at org.apache.gravitino.utils.IsolatedClassLoader.buildClassLoader(IsolatedClassLoader.java:121)
        at org.apache.gravitino.catalog.CatalogManager.createClassLoader(CatalogManager.java:870)
        at org.apache.gravitino.catalog.CatalogManager.createCatalogWrapper(CatalogManager.java:800)
        at org.apache.gravitino.catalog.CatalogManager.lambda$createCatalog$5(CatalogManager.java:393)
        at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
        at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916)

I know why CI failed. Because we didn't add catalogs:catalog-hadoop:jar in the authorizaton-ranger-plugin depends.
So Gravitino server didn't success create hdfs catalog in the FilesetITs.

@theoryxu Please use next gradle config to update authorizations/authorization-ranger/build.gradle.kts

tasks.test {
  doFirst {
    environment("GRAVITINO_TEST", "true")
    environment("HADOOP_USER_NAME", "gravitino")
  }
  dependsOn(":catalogs:catalog-hive:jar", ":catalogs:catalog-hive:runtimeJars", ":catalogs:catalog-lakehouse-iceberg:jar", ":catalogs:catalog-lakehouse-iceberg:runtimeJars", ":catalogs:catalog-lakehouse-paimon:jar", ":catalogs:catalog-lakehouse-paimon:runtimeJars", ":catalogs:catalog-hadoop:jar", ":catalogs:catalog-hadoop:runtimeJars")

  val skipITs = project.hasProperty("skipITs")
  if (skipITs) {
    // Exclude integration tests
    exclude("**/integration/test/**")
  } else {
    dependsOn(tasks.jar)
  }
}

[theoryxu]

RangerFilesetIT > initializationError FAILED
    java.lang.IllegalArgumentException: Failed to operate catalog(s) [RangerFilesetE2EIT_catalog_1bb7992c] operation [CREATE] under metalake [metalake], reason [Invalid package path: /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs in [/Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs, /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/resources/main, /Users/xun/github/xunliu/gravitino/authorizations/authorization-ranger/build/libs]]
    java.lang.IllegalArgumentException: Invalid package path: /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs in [/Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/libs, /Users/xun/github/xunliu/gravitino/catalogs/catalog-hadoop/build/resources/main, /Users/xun/github/xunliu/gravitino/authorizations/authorization-ranger/build/libs]
        at org.apache.gravitino.utils.IsolatedClassLoader.buildClassLoader(IsolatedClassLoader.java:121)
        at org.apache.gravitino.catalog.CatalogManager.createClassLoader(CatalogManager.java:870)
        at org.apache.gravitino.catalog.CatalogManager.createCatalogWrapper(CatalogManager.java:800)
        at org.apache.gravitino.catalog.CatalogManager.lambda$createCatalog$5(CatalogManager.java:393)
        at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
        at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916)

I know why CI failed. Because we didn't add catalogs:catalog-hadoop:jar in the authorizaton-ranger-plugin depends. So Gravitino server didn't success create hdfs catalog in the FilesetITs.

@theoryxu Please use next gradle config to update authorizations/authorization-ranger/build.gradle.kts

tasks.test {
  doFirst {
    environment("GRAVITINO_TEST", "true")
    environment("HADOOP_USER_NAME", "gravitino")
  }
  dependsOn(":catalogs:catalog-hive:jar", ":catalogs:catalog-hive:runtimeJars", ":catalogs:catalog-lakehouse-iceberg:jar", ":catalogs:catalog-lakehouse-iceberg:runtimeJars", ":catalogs:catalog-lakehouse-paimon:jar", ":catalogs:catalog-lakehouse-paimon:runtimeJars", ":catalogs:catalog-hadoop:jar", ":catalogs:catalog-hadoop:runtimeJars")

  val skipITs = project.hasProperty("skipITs")
  if (skipITs) {
    // Exclude integration tests
    exclude("**/integration/test/**")
  } else {
    dependsOn(tasks.jar)
  }
}

I tried, but it didn't work. Using getInstanceForTest to set isTestEnv could pass now.

[xunliu]

hi @theoryxu Thank you for your contributions.
LGTM