Cannot find ScanCode license texts in Docker image

[mnonnenmacher]

ORT cannot find the ScanCode license texts in the Docker image:

$ docker run --rm ghcr.io/oss-review-toolkit/ort:13.0.0 requirements
[...]
ScanCode license texts not found.

The locations of the ScanCode binary and the license texts within the image are:
/opt/python/shims/scancode
/opt/python/versions/3.11.5/lib/python3.11/site-packages/licensedcode/data/licenses

The heuristic to find the license texts based on the path of the ScanCode binary fails for this:

ort/utils/spdx/src/main/kotlin/Utils.kt

Lines 47 to 54 in f88041a

	val scanCodeLicenseTextDir by lazy {
	val scanCodeExeDir = Os.getPathFromEnvironment("scancode")?.realFile()?.parentFile
	val pythonBinDir = listOf("bin", "Scripts")
	val scanCodeBaseDir = scanCodeExeDir?.takeUnless { it.name in pythonBinDir } ?: scanCodeExeDir?.parentFile
	scanCodeBaseDir?.walkTopDown()?.find { it.isDirectory && it.endsWith("licensedcode/data/licenses") }
	}

[sschuberth]

My proposal is to actually not fix this by adjusting the heuristic to find the license texts, at least not as a long-term solution. Instead, we should probably make use of the new scancode-license-data command. And I'd wish that before even doing that, we'd have the time to also extract license text providers to a proper interface / plugins.

[sschuberth]

extract license text providers to a proper interface / plugins.

I've created #9620 for this now.

[sschuberth]

Thinking about this again, can it be that the issue was a red herring? While it's correct that scanCodeLicenseTextDir (which is what ort requirements calls) returned null before #9622, that should not actually have mattered for the use-case of disclosure document generation, as

ort/utils/spdx/src/main/kotlin/Utils.kt

Lines 120 to 138 in 958c08c

	fun getLicenseTextReader(
	id: String,
	handleExceptions: Boolean = false,
	licenseTextDirectories: List<File> = emptyList()
	): (() -> String)? {
	return if (id.startsWith(LICENSE_REF_PREFIX)) {
	getLicenseTextResource(id)?.let { { it.readText() } }
	?: addScanCodeLicenseTextsDir(licenseTextDirectories).firstNotNullOfOrNull { dir ->
	getLicenseTextFile(id, dir)?.let { file ->
	{
	file.readText().removeYamlFrontMatter()
	}
	}
	}
	} else {
	SpdxLicense.forId(id.removeSuffix("+"))?.let { { it.text } }
	?: SpdxLicenseException.forId(id)?.takeIf { handleExceptions }?.let { { it.text } }
	}
	}

actually first tries to read license texts from ORT's built-in resources before falling back to scanCodeLicenseTextDir.

So @wkl3nk can you confirm that there actually was an issue with any disclosure documents? If so, for which license ID?

[mnonnenmacher]

So @wkl3nk can you confirm that there actually was an issue with any disclosure documents? If so, for which license ID?

To my knowledge the issue was not related to SPDX licenses but only licenses starting with "LicenseRef-scancode-".

[sschuberth]

To my knowledge the issue was not related to SPDX licenses but only licenses starting with "LicenseRef-scancode-".

I see. There are indeed not part of https://github.com/oss-review-toolkit/ort/tree/main/utils/spdx/src/main/resources/licenserefs. Something to reconsider, maybe.