PDF/A-3 support, validation fails

[hvbtup]

I am trying to generate a PDF/A-3 PDF with BIRT (as the first step of generating a ZUGFeRD invoice).
Unfortunately the VeraPDF validator complains about the generated PDF.

Validation error message:

Specification: ISO 19005-3:2012, Clause: 6.2.11.4.2, Test number: 2
If the FontDescriptor dictionary of an embedded CID font contains a CIDSet stream, then it shall identify all CIDs which are present in the font program, regardless of whether a CID in the font is referenced or used by the PDF or not

What I tested:

I used the example PDF pdf_enhancement_version_PDF_A1A_FBfont.pdf from the PDF/A enhancement.
I get the same result with a newly created trivial HelloWorld report with a TrueType font.

I used the latest VeraPDF Greenfield development installer 1.27.33 (one can download it from https://docs.verapdf.org/install/).

I manually set PDF flavour PDF/A-3A or PDF/A-3B in the VeraPDF GUI.

I know that BIRT with the PDF/A enhancement does not directly support PDF/A-3, only PDF/A-1.
But I think it is worth adding support for it.
As such, the VeraPDF validation report also shows other errors which are expected.

Please note that earlier versions of VeraPDF may contain a bug which is directly related to this topic, see veraPDF/veraPDF-library#1462. That issue was reported by Tilman Hausherr, the main committer of Apache PDFBox.

Please note that for PDF/A-3, the requirement for fonts is different in comparison to PDF/A-1.
If I understand this correctly, then PDF/A-3 requires that the "CIDSet" is either complete or not there at all, whereas for PDF/A-1 an incomplete set is OK.

There recently was a change in OpenPDF (see LibrePDF/OpenPDF#1041).

I am not quite sure what is to do here.

First question:
Obviously, we create an imcomplete CIDSet. And AFAIK BIRT uses OpenPDF 1.3.30. However, in the description of the mentioned issue, it is stated that this worked up to 1.3.30.

Much more important: How can we avoid the error?
I reckon that we would need to use a newer release of OpenPDF which includes the changes to allow disabling CIDSet generation, and this flag should be made acessible to BIRT developers as a UserProperty just like the other UserProperties from #1486.

[merks]

FYI, generally BIRT is always using the latest dependencies, here in tabular format:

https://download.eclipse.org/tools/orbit/simrel/orbit-aggregation/table.html

So currently this version:

Orbit updates to the latest version available from Maven Central semi-automatically...

To see what's definitely there now, you can use the Repository Explorer in advanced mode:

[hvbtup]

Hmm, that answers my first question. At least in theory, because I get the same error from a PDF generated with a BIRT release that still uses OpenPDF 1.3.0.

[speckyspooky]

The latest version of openPDF is version 2.0.2 but I don't know whether the release containce the fix.
Or if there an option to set the CIDSet then of course we can add a new user property.

[merks]

Orbit doesn’t generally update to new major versions because they are assumed to break API. But it could provide both.

[merks]

Here are reports that show for which things there are major updates:

• https://github.com/eclipse-orbit/orbit-simrel/blob/main/report/maven-bnd/merged-target/REPORT.md
• https://github.com/eclipse-orbit/orbit-simrel/blob/main/report/maven-osgi/merged-target/REPORT.md

I don't generally add them if there is no explicit request. Shall I add this one?

https://repo1.maven.org/maven2/com/github/librepdf/openpdf/2.0.2/

It's actually only a month newer. We won't know what if anything is broken until we try to use it...

[speckyspooky]

Yes, I request the change to the newer version.
Please let me know when I can support to open a ticket for Orbit.

[merks]

I’ll handle it tomorrow. Nothing else is required.

[hvbtup]

I didn't find a clear explanation about the differences between the 1.3.x, 1.4.x and 2.0.x releases of OpenPDF, except that the require Java 8, Java 11 and Java 17. Since BIRT itself now requires Java 17, we should use 2.0.x.

@speckyspooky Probably, if you are going to add support for PDF/A-3 in the PDF emitter, then for PDF/A-3 (opposed to PDF/A-1), you need to call the new API font.setIncludeCidSet(false); for each font. The documentation says

/**
 * Indicates if a CIDSet stream should be included in the document for {@link TrueTypeFontUnicode}.
 * When set to <CODE>true</CODE>, a CIDSet stream will be included in the document.
 * When set to <CODE>false</CODE>, and {@link PdfWriter#getPDFXConformance()} does not require it,
 * no CIDSet stream will be included.
 *
 * @param includeCidSet new value of {@link BaseFont#includeCidSet}
 */

Not sure about PDF/A-2, but I think it should work like PDF/A-3.

[merks]

If you re-resolve the target platform (Help -> Perform Setup Tasks...) today it will contain the 2.0.2 version. It does not cause any compile errors.

I'm currently investigating this issue which blocks publishing any new master builds:

#1819

[merks]

FYI, my local Tycho build with 2.0.2 was successful.

[merks]

The latest nightly build is using 2.0.2:

[wimjongman]

@hvbtup Henning, is this now soled? If not please reopen.

[hvbtup]

This is solved with Thomas' commits.