Coverage decreased (-0.005%) to 39.879% when pulling f427ad7 on feature/data-availability-statement into 54d1c29 on master.

Coverage decreased (-0.03%) to 39.928% when pulling 1925296 on feature/data-availability-statement into f9dc68f on master.

Coverage decreased (-0.006%) to 39.878% when pulling 40331eb on feature/data-availability-statement into 54d1c29 on master.

Hi @lfoppiano !

I reviewed everything and made some few changes - in particular put back the header labels into TaggingLabels.java to avoid a breaking change in all the current grobid modules (which uses labels from the class TaggingLabels - I actually wanted to put back the segmentation labels too in TaggingLabels to simplify a few years ago, but I didn't do it to avoid a breaking change).

Currently, when the availability statement is in the header, it is correctly labeled and stored in BiblioItem.java, but it is not serialized in the final TEI because the method used can only work with segmentation label:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/engines/FullTextParser.java#L2473

            // data availability statements
            StringBuilder dataAvailability = new StringBuilder();
            if (StringUtils.isNotBlank(resHeader.getDataAvailability())) {
                dataAvailability = getSectionAsTEI("availability", "\t\t\t", doc, TaggingLabels.HEADER_AVAILABILITY,
                    teiFormatter, resCitations, config);
            } else {
                dataAvailability = getSectionAsTEI("availability", "\t\t\t", doc, SegmentationLabels.AVAILABILITY,
                    teiFormatter, resCitations, config);
            }

getSectionAsTEI() then uses:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/engines/FullTextParser.java#L2517

SortedSet<DocumentPiece> sectionPart = doc.getDocumentPart(taggingLabel);

Only the SegmentationLabel work here with doc.getDocumentPart() (using TaggingLabels.HEADER_AVAILABILITY here will return nothing).

To retrieve the layout tokens relative to the availability statement from the header stored in BiblioItem.java, normally we use:

https://github.com/kermitt2/grobid/blob/feature/data-availability-statement/grobid-core/src/main/java/org/grobid/core/data/BiblioItem.java#L4238

List<LayoutToken> headerAvailabilityStatementTokens = biblio.getLayoutTokens(TaggingLabels.HEADER_AVAILABILITY);

Latest commits fixed the header availability statement TEI serialization.
createTraining for header and segmentation now also pre-labeled availability stmt and funding sections.