Merge branch 'RED-8933-4.0' into 'release/0.89.x'

RED-8933: Fixed bugs in DocumineClassificationService

See merge request fforesight/layout-parser!149

.gitlab-ci.yml

@@ -16,6 +16,6 @@ deploy:
     reports:
       dotenv: version.env
   rules:
-    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH 
     - if: $CI_COMMIT_BRANCH =~ /^release/
     - if: $CI_COMMIT_TAG

README.md

@@ -1,44 +1,39 @@
-# PDF Layout Parser Micro-Service: layout-parser
+# PDF Layout Parser Micro-Service: layout-parser  
 
 ## Introduction
-
-The layout-parser micro-service is a powerful tool designed to efficiently extract structured information from PDF documents. Written in Java and utilizing Spring Boot 3, Apache
-PDFBox, and RabbitMQ, this micro-service excels at parsing PDFs and organizing their content into a meaningful and coherent layout structure. Notably, the layout-parser
-micro-service distinguishes itself by relying solely on advanced algorithms, rather than machine learning techniques.
+The layout-parser micro-service is a powerful tool designed to efficiently extract structured information from PDF documents. Written in Java and utilizing Spring Boot 3, Apache PDFBox, and RabbitMQ, this micro-service excels at parsing PDFs and organizing their content into a meaningful and coherent layout structure. Notably, the layout-parser micro-service distinguishes itself by relying solely on advanced algorithms, rather than machine learning techniques.
 
 ### Key Steps in the PDF Layout Parsing Process:
 
 * **Text Position Extraction:**  
-  The micro-service leverages Apache PDFBox to extract precise text positions for each individual character within the PDF document.
+The micro-service leverages Apache PDFBox to extract precise text positions for each individual character within the PDF document.
 
 * **Word Segmentation and Text Block Formation:**  
-  Employing an array of diverse algorithms, the micro-service initially identifies and segments words, creating distinct text blocks.
+Employing an array of diverse algorithms, the micro-service initially identifies and segments words, creating distinct text blocks.
 
 * **Text Block Classification:**  
-  The segmented text blocks are then subjected to classification algorithms. These algorithms categorize the text blocks based on their content and visual properties,
-  distinguishing between sections, subsections, headlines, paragraphs, images, tables, table cells, headers, and footers.
+The segmented text blocks are then subjected to classification algorithms. These algorithms categorize the text blocks based on their content and visual properties, distinguishing between sections, subsections, headlines, paragraphs, images, tables, table cells, headers, and footers.
 
 * **Layout Coherence Establishment:**  
-  The classified text blocks are subsequently orchestrated into a cohesive layout structure. This process involves arranging sections, subsections, paragraphs, images, and other
-  elements in a logical and structured manner.
+The classified text blocks are subsequently orchestrated into a cohesive layout structure. This process involves arranging sections, subsections, paragraphs, images, and other elements in a logical and structured manner.
 
 * **Output Generation in Various Formats:**  
-  Once the layout structure is established, the micro-service generates output in multiple formats. These formats are designed for seamless integration with downstream
-  micro-services. The supported formats include JSON, XML, and others, ensuring flexibility in downstream data consumption.
+Once the layout structure is established, the micro-service generates output in multiple formats. These formats are designed for seamless integration with downstream micro-services. The supported formats include JSON, XML, and others, ensuring flexibility in downstream data consumption.
 
 ### Optional Enhancements:
 
 * **ML-Based Table Extraction:**  
-  For enhanced results, users have the option to incorporate machine learning-based table extraction. This feature can be activated by providing ML-generated results as a JSON
-  file, which are then integrated seamlessly into the layout structure.
+For enhanced results, users have the option to incorporate machine learning-based table extraction. This feature can be activated by providing ML-generated results as a JSON file, which are then integrated seamlessly into the layout structure.
 
 * **Image Classification using ML:**  
-  Additionally, for more accurate image classification, users can optionally feed ML-generated image classification results into the micro-service. Similar to the table extraction
-  option, the micro-service processes the pre-parsed results in JSON format, thus optimizing the accuracy of image content identification.
+Additionally, for more accurate image classification, users can optionally feed ML-generated image classification results into the micro-service. Similar to the table extraction option, the micro-service processes the pre-parsed results in JSON format, thus optimizing the accuracy of image content identification.
+
+In conclusion, the layout-parser micro-service is a versatile PDF layout parsing solution crafted entirely around advanced algorithms, without reliance on machine learning. It proficiently extracts text positions, segments content into meaningful blocks, classifies these blocks, arranges them coherently, and outputs structured data for downstream micro-services. Optional integration with ML-generated table extractions and image classifications further enhances its capabilities.
+
+
+
+
 
-In conclusion, the layout-parser micro-service is a versatile PDF layout parsing solution crafted entirely around advanced algorithms, without reliance on machine learning. It
-proficiently extracts text positions, segments content into meaningful blocks, classifies these blocks, arranges them coherently, and outputs structured data for downstream
-micro-services. Optional integration with ML-generated table extractions and image classifications further enhances its capabilities.
 
 ## Installation
 
@@ -54,55 +49,41 @@ To build and test the micro-service, follow these steps:
 ### Clone the Repository:
 
 bash
-
 ```
 git clone ssh://git@git.knecon.com:22222/fforesight/layout-parser.git
 cd layout-parser
 ```
-
 ### Build the Project:
-
 Use the following command to build the project using Gradle:
 
 ```
 gradle clean build
 ```
-
 ### Run Tests:
-
 Run the test suite using the following command:
-
 ```
 gradle test
 ```
-
 ## Building a Custom Docker Image
-
 To create a custom Docker image for the layout-parser micro-service, execute the provided script:
 
 ### Ensure Docker is Installed:
-
 Ensure that Docker is installed and running on your system.
 
 ### Run the Image Building Script:
-
 Execute the publish-custom-image script in the project directory:
 
 ```
 ./publish-custom-image
 ```
-
 ## Publishing to Internal Maven Repository
-
 To publish the layout-parser micro-service to your internal Maven repository, execute the following command:
 
 ```
 gradle -Pversion=buildVersion publish
 ```
-
 Replace buildVersion with the desired version number.
 
 ## Additional Notes
-
 Make sure to configure any necessary application properties before deploying the micro-service.
 For advanced usage and configurations, refer to Kilian or Dom or preferably the source code.

buildSrc/src/main/kotlin/com.knecon.fforesight.java-conventions.gradle.kts

@@ -24,6 +24,8 @@ tasks.named<Test>("test") {
     reports {
         junitXml.outputLocation.set(layout.buildDirectory.dir("reports/junit"))
     }
+    minHeapSize = "512m"
+    maxHeapSize = "2048m"
 }
 
 tasks.test {

config/pmd/pmd.xml

@@ -9,12 +9,13 @@
     </description>
 
     <rule ref="category/java/errorprone.xml">
-        <exclude name="DataflowAnomalyAnalysis"/>
         <exclude name="MissingSerialVersionUID"/>
-        <exclude name="NullAssignment"/>
         <exclude name="AvoidLiteralsInIfCondition"/>
         <exclude name="AvoidDuplicateLiterals"/>
-        <exclude name="AvoidFieldNameMatchingMethodName"/>
+        <exclude name="NullAssignment"/>
+        <exclude name="AssignmentInOperand"/>
+        <exclude name="BeanMembersShouldSerialize"/>
     </rule>
 
-</ruleset>
+</ruleset>
+

config/pmd/test_pmd.xml

@@ -10,14 +10,14 @@
 
 
     <rule ref="category/java/errorprone.xml">
-        <exclude name="DataflowAnomalyAnalysis"/>
         <exclude name="MissingSerialVersionUID"/>
-        <exclude name="NullAssignment"/>
         <exclude name="AvoidLiteralsInIfCondition"/>
         <exclude name="AvoidDuplicateLiterals"/>
         <exclude name="AvoidFieldNameMatchingMethodName"/>
-        <exclude name="AvoidFieldNameMatchingTypeName"/>
+        <exclude name="NullAssignment"/>
+        <exclude name="AssignmentInOperand"/>
         <exclude name="TestClassWithoutTestCases"/>
+        <exclude name="BeanMembersShouldSerialize"/>
     </rule>
 
-</ruleset>
+</ruleset>

gradle.properties.kts

@@ -1 +1 @@
-version = 0.1 - SNAPSHOT
+version = 0.1-SNAPSHOT

layoutparser-service/layoutparser-service-internal-api/build.gradle.kts

@@ -1,6 +1,6 @@
 plugins {
     id("com.knecon.fforesight.java-conventions")
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 }
 
 description = "layoutparser-service-internal-api"

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/DocumentData.java

@@ -1,5 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
+import java.io.Serializable;
+
 import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
@@ -12,7 +14,7 @@ import lombok.experimental.FieldDefaults;
 @AllArgsConstructor
 @FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
 @Schema(description = "Object containing the complete document layout parsing information. It is split into 4 categories, structure, text, positions and pages: " + "The document tree structure of SemanticNodes such as Section, Paragraph, Headline, etc. " + "The text, which is stored as separate blocks of data. " + "The text positions, which are also stored as separate blocks. The Blocks are equal to the text blocks in length and order. " + "The page information.")
-public class DocumentData {
+public class DocumentData implements Serializable {
 
     @Schema(description = "Contains information about the document's pages.")
     DocumentPage[] documentPages;

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/DocumentPage.java

@@ -1,5 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
+import java.io.Serializable;
+
 import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
@@ -14,7 +16,7 @@ import lombok.experimental.FieldDefaults;
 @AllArgsConstructor
 @FieldDefaults(level = AccessLevel.PRIVATE)
 @Schema(description = "Object containing information about the document's pages.")
-public class DocumentPage {
+public class DocumentPage implements Serializable {
 
     @Schema(description = "The page number, starting with 1.")
     int number;

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/DocumentPositionData.java

@@ -1,5 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
+import java.io.Serializable;
+
 import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
@@ -14,7 +16,7 @@ import lombok.experimental.FieldDefaults;
 @AllArgsConstructor
 @FieldDefaults(level = AccessLevel.PRIVATE)
 @Schema(description = "Object containing text positional information of a specific text block. A document is split into multiple text blocks, which are supposed to be read in order. Every text block can only occur on a single page.")
-public class DocumentPositionData {
+public class DocumentPositionData implements Serializable {
 
     @Schema(description = "Identifier of the text block.")
     Long id;

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/DocumentStructure.java

@@ -1,6 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
 import java.awt.geom.Rectangle2D;
+import java.io.Serializable;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Map;
@@ -20,13 +21,13 @@ import lombok.experimental.FieldDefaults;
 @AllArgsConstructor
 @FieldDefaults(level = AccessLevel.PRIVATE)
 @Schema(description = "Object containing information about the parsed tree structure of the SemanticNodes, such as Section, Paragraph, Headline etc inside of the document.")
-public class DocumentStructure {
+public class DocumentStructure implements Serializable {
 
     @Schema(description = "The root EntryData represents the Document.")
     EntryData root;
 
     @Schema(description = "Object containing the extra field names, a table has in its properties field.")
-    public static class TableProperties {
+    public static class TableProperties implements Serializable {
 
         public static final String NUMBER_OF_ROWS = "numberOfRows";
         public static final String NUMBER_OF_COLS = "numberOfCols";
@@ -34,7 +35,7 @@ public class DocumentStructure {
     }
 
     @Schema(description = "Object containing the extra field names, an Image has in its properties field.")
-    public static class ImageProperties {
+    public static class ImageProperties implements Serializable {
 
         public static final String TRANSPARENT = "transparent";
         public static final String IMAGE_TYPE = "imageType";
@@ -44,7 +45,7 @@ public class DocumentStructure {
     }
 
     @Schema(description = "Object containing the extra field names, a table cell has in its properties field.")
-    public static class TableCellProperties {
+    public static class TableCellProperties implements Serializable {
 
         public static final String B_BOX = "bBox";
         public static final String ROW = "row";
@@ -100,7 +101,7 @@ public class DocumentStructure {
     @AllArgsConstructor
     @FieldDefaults(level = AccessLevel.PRIVATE)
     @Schema(description = "Object containing information of a SemanticNode and also structuring the layout with children.")
-    public static class EntryData {
+    public static class EntryData implements Serializable {
 
         @Schema(description = "Type of the semantic node.", allowableValues = {"DOCUMENT", "SECTION", "PARAGRAPH", "HEADLINE", "TABLE", "TABLE_CELL", "HEADER", "FOOTER", "IMAGE"})
         NodeType type;

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/DocumentTextData.java

@@ -1,5 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
+import java.io.Serializable;
+
 import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
@@ -14,7 +16,7 @@ import lombok.experimental.FieldDefaults;
 @AllArgsConstructor
 @FieldDefaults(level = AccessLevel.PRIVATE)
 @Schema(description = "Object containing text information of a specific text block. A document is split into multiple text blocks, which are supposed to be read in order. Every text block can only occur on a single page.")
-public class DocumentTextData {
+public class DocumentTextData implements Serializable {
 
     @Schema(description = "Identifier of the text block.")
     Long id;

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/redaction/NodeType.java

@@ -1,8 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 
+import java.io.Serializable;
 import java.util.Locale;
 
-public enum NodeType {
+public enum NodeType implements Serializable {
     DOCUMENT,
     SECTION,
     HEADLINE,

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/data/taas/RowData.java

@@ -17,5 +17,4 @@ public class RowData {
     List<ParagraphData> cellText;
     @Schema(description = "The bounding box of this StructureObject. Is always exactly 4 values representing x, y, w, h, where x, y specify the lower left corner.")
     float[] bBox;
-
 }

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/queue/LayoutParsingFinishedEvent.java

@@ -8,9 +8,13 @@ import lombok.Builder;
 @Builder
 @Schema(description = "Object containing information about the layout parsing.")
 public record LayoutParsingFinishedEvent(
-        @Schema(description = "General purpose identifier. It is returned exactly the same way it is inserted with the LayoutParsingRequest.") Map<String, String> identifier,//
-        @Schema(description = "The duration of a single layout parsing in ms.") long duration,//
-        @Schema(description = "The number of pages of the parsed document.") int numberOfPages,//
-        @Schema(description = "A general message. It contains some information useful for a developer, like the paths where the files are stored. Not meant to be machine readable.") String message) {
+        @Schema(description = "General purpose identifier. It is returned exactly the same way it is inserted with the LayoutParsingRequest.")
+        Map<String, String> identifier,//
+        @Schema(description = "The duration of a single layout parsing in ms.")
+        long duration,//
+        @Schema(description = "The number of pages of the parsed document.")
+        int numberOfPages,//
+        @Schema(description = "A general message. It contains some information useful for a developer, like the paths where the files are stored. Not meant to be machine readable.")
+        String message) {
 
 }

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/queue/LayoutParsingQueueNames.java

@@ -5,5 +5,4 @@ public class LayoutParsingQueueNames {
     public static final String LAYOUT_PARSING_REQUEST_QUEUE = "layout_parsing_request_queue";
     public static final String LAYOUT_PARSING_DLQ = "layout_parsing_dead_letter_queue";
     public static final String LAYOUT_PARSING_FINISHED_EVENT_QUEUE = "layout_parsing_response_queue";
-
 }

layoutparser-service/layoutparser-service-internal-api/src/main/java/com/knecon/fforesight/service/layoutparser/internal/api/queue/LayoutParsingRequest.java

@@ -17,35 +17,24 @@ public record LayoutParsingRequest(
         Map<String, String> identifier,
 
         @Schema(description = "Path to the original PDF file.")//
-        @NonNull String originFileStorageId,
-//
-
-        @Schema(description = "Optional Path to the the visual layout parsing service file") Optional<String> visualLayoutParsingFileId,
+        @NonNull String originFileStorageId,//
         @Schema(description = "Optional Path to the table extraction file.")//
-        Optional<String> tablesFileStorageId,
-//
+        Optional<String> tablesFileStorageId,//
         @Schema(description = "Optional Path to the image classification file.")//
-        Optional<String> imagesFileStorageId,
-//
+        Optional<String> imagesFileStorageId,//
 
         @Schema(description = "Path where the Document Structure File will be stored.")//
-        @NonNull String structureFileStorageId,
-//
+        @NonNull String structureFileStorageId,//
         @Schema(description = "Path where the Research Data File will be stored.")//
-        String researchDocumentStorageId,
-//
+        String researchDocumentStorageId,//
         @Schema(description = "Path where the Document Text File will be stored.")//
-        @NonNull String textBlockFileStorageId,
-//
+        @NonNull String textBlockFileStorageId,//
         @Schema(description = "Path where the Document Positions File will be stored.")//
-        @NonNull String positionBlockFileStorageId,
-//
+        @NonNull String positionBlockFileStorageId,//
         @Schema(description = "Path where the Document Pages File will be stored.")//
-        @NonNull String pageFileStorageId,
-//
+        @NonNull String pageFileStorageId,//
         @Schema(description = "Path where the Simplified Text File will be stored.")//
-        @NonNull String simplifiedTextStorageId,
-//
+        @NonNull String simplifiedTextStorageId,//
         @Schema(description = "Path where the Viewer Document PDF will be stored.")//
         @NonNull String viewerDocumentStorageId) {
 

layoutparser-service/layoutparser-service-processor/build.gradle.kts

@@ -1,6 +1,6 @@
 plugins {
     id("com.knecon.fforesight.java-conventions")
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 }
 
 description = "layoutparser-service-processor"
@@ -10,13 +10,14 @@ val pdfBoxVersion = "3.0.0"
 
 dependencies {
     implementation(project(":layoutparser-service-internal-api"))
+    implementation(project(":viewer-doc-processor"))
 
     implementation("com.iqser.red.service:persistence-service-shared-api-v1:2.144.0") {
         exclude("org.springframework.boot", "spring-boot-starter-security")
         exclude("org.springframework.boot", "spring-boot-starter-validation")
     }
-    implementation("com.knecon.fforesight:tenant-commons:0.10.0")
-    implementation("com.iqser.red.commons:storage-commons:2.40.0")
+    implementation("com.knecon.fforesight:tenant-commons:0.21.0")
+    implementation("com.iqser.red.commons:storage-commons:2.45.0")
 
     implementation("org.apache.pdfbox:pdfbox:${pdfBoxVersion}")
     implementation("org.apache.pdfbox:pdfbox-tools:${pdfBoxVersion}")

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/LayoutParsingPipeline.java

@@ -3,13 +3,16 @@ package com.knecon.fforesight.service.layoutparser.processor;
 import static java.lang.String.format;
 
 import java.awt.geom.Rectangle2D;
-import java.io.ByteArrayOutputStream;
+import java.io.File;
 import java.io.IOException;
+import java.nio.file.Files;
 import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.concurrent.atomic.AtomicReference;
 
+import org.apache.pdfbox.Loader;
 import org.apache.pdfbox.pdmodel.PDDocument;
 import org.apache.pdfbox.pdmodel.PDPage;
 import org.apache.pdfbox.pdmodel.common.PDRectangle;
@@ -25,17 +28,15 @@ import com.knecon.fforesight.service.layoutparser.processor.model.Classification
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
 import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.CvTableParsingAdapter;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.ImageServiceResponseAdapter;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.VisualLayoutParsingAdapter;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.image.ImageServiceResponse;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResult;
 import com.knecon.fforesight.service.layoutparser.processor.services.BodyTextFrameService;
 import com.knecon.fforesight.service.layoutparser.processor.services.RulingCleaningService;
 import com.knecon.fforesight.service.layoutparser.processor.services.SectionsBuilderService;
@@ -51,134 +52,185 @@ import com.knecon.fforesight.service.layoutparser.processor.services.factory.Doc
 import com.knecon.fforesight.service.layoutparser.processor.services.mapper.DocumentDataMapper;
 import com.knecon.fforesight.service.layoutparser.processor.services.mapper.TaasDocumentDataMapper;
 import com.knecon.fforesight.service.layoutparser.processor.services.parsing.PDFLinesTextStripper;
-import com.knecon.fforesight.service.layoutparser.processor.services.visualization.ViewerDocumentService;
+import com.knecon.fforesight.service.layoutparser.processor.services.visualization.LayoutGridService;
 import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 
+import io.micrometer.observation.Observation;
+import io.micrometer.observation.ObservationRegistry;
+import io.micrometer.observation.annotation.Observed;
+import lombok.AccessLevel;
 import lombok.RequiredArgsConstructor;
 import lombok.SneakyThrows;
+import lombok.experimental.FieldDefaults;
 import lombok.extern.slf4j.Slf4j;
 
+@SuppressWarnings("PMD.CloseResource")
 @Slf4j
 @Service
 @RequiredArgsConstructor
+@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
 public class LayoutParsingPipeline {
 
-    private final ImageServiceResponseAdapter imageServiceResponseAdapter;
-    private final CvTableParsingAdapter cvTableParsingAdapter;
-    private final LayoutParsingStorageService layoutParsingStorageService;
-    private final SectionsBuilderService sectionsBuilderService;
-    private final TaasClassificationService taasClassificationService;
-    private final RedactManagerClassificationService redactManagerClassificationService;
-    private final DocuMineClassificationService docuMineClassificationService;
-    private final SimplifiedSectionTextService simplifiedSectionTextService;
-    private final BodyTextFrameService bodyTextFrameService;
-    private final RulingCleaningService rulingCleaningService;
-    private final TableExtractionService tableExtractionService;
-    private final VisualLayoutParsingAdapter visualLayoutParsingAdapter;
-    private final TaasBlockificationService taasBlockificationService;
-    private final DocuMineBlockificationService docuMineBlockificationService;
-    private final RedactManagerBlockificationService redactManagerBlockificationService;
-    private final ViewerDocumentService viewerDocumentService;
+    ImageServiceResponseAdapter imageServiceResponseAdapter;
+    CvTableParsingAdapter cvTableParsingAdapter;
+    LayoutParsingStorageService layoutParsingStorageService;
+    SectionsBuilderService sectionsBuilderService;
+    TaasClassificationService taasClassificationService;
+    RedactManagerClassificationService redactManagerClassificationService;
+    DocuMineClassificationService docuMineClassificationService;
+    SimplifiedSectionTextService simplifiedSectionTextService;
+    BodyTextFrameService bodyTextFrameService;
+    RulingCleaningService rulingCleaningService;
+    TableExtractionService tableExtractionService;
+    TaasBlockificationService taasBlockificationService;
+    DocuMineBlockificationService docuMineBlockificationService;
+    RedactManagerBlockificationService redactManagerBlockificationService;
+    LayoutGridService layoutGridService;
+    ObservationRegistry observationRegistry;
 
 
     public LayoutParsingFinishedEvent parseLayoutAndSaveFilesToStorage(LayoutParsingRequest layoutParsingRequest) throws IOException {
 
         long start = System.currentTimeMillis();
+        log.info("Starting layout parsing for {}", layoutParsingRequest.identifier());
 
-        try (PDDocument originDocument = layoutParsingStorageService.getOriginFile(layoutParsingRequest.originFileStorageId())) {
-            ImageServiceResponse imageServiceResponse = new ImageServiceResponse();
-            if (layoutParsingRequest.imagesFileStorageId().isPresent()) {
-                imageServiceResponse = layoutParsingStorageService.getImagesFile(layoutParsingRequest.imagesFileStorageId().get());
-            }
+        File originFile = layoutParsingStorageService.getOriginFile(layoutParsingRequest.originFileStorageId());
+//        File viewerDocumentFile = layoutParsingStorageService.getViewerDocFile(layoutParsingRequest.viewerDocumentStorageId())
+//                .orElse(originFile);
+        File viewerDocumentFile = originFile;
 
-            TableServiceResponse tableServiceResponse = new TableServiceResponse();
-            if (layoutParsingRequest.tablesFileStorageId().isPresent()) {
-                tableServiceResponse = layoutParsingStorageService.getTablesFile(layoutParsingRequest.tablesFileStorageId().get());
-            }
 
-            VisualLayoutParsingResponse visualLayoutParsingResponse = new VisualLayoutParsingResponse();
-            if (layoutParsingRequest.visualLayoutParsingFileId().isPresent()) {
-                visualLayoutParsingResponse = layoutParsingStorageService.getExtractedTablesFile(layoutParsingRequest.visualLayoutParsingFileId().get());
-            }
-
-            ClassificationDocument classificationDocument = parseLayout(layoutParsingRequest.layoutParsingType(), originDocument, imageServiceResponse, tableServiceResponse);
-            Document documentGraph = DocumentGraphFactory.buildDocumentGraph(classificationDocument);
-
-            int numberOfPages = originDocument.getNumberOfPages();
-
-            layoutParsingStorageService.storeDocumentData(layoutParsingRequest, DocumentDataMapper.toDocumentData(documentGraph));
-            layoutParsingStorageService.storeSimplifiedText(layoutParsingRequest, simplifiedSectionTextService.toSimplifiedText(documentGraph));
-
-            Map<Integer, List<VisualLayoutParsingResult>> extractedTableCells = visualLayoutParsingAdapter.buildExtractedTablesPerPage(visualLayoutParsingResponse);
-            try (var out = new ByteArrayOutputStream()) {
-                viewerDocumentService.createViewerDocument(originDocument, documentGraph, out, extractedTableCells, false);
-                layoutParsingStorageService.storeViewerDocument(layoutParsingRequest, out);
-            }
-
-            if (layoutParsingRequest.layoutParsingType().equals(LayoutParsingType.TAAS)) {
-                var researchDocumentData = TaasDocumentDataMapper.fromDocument(documentGraph);
-                layoutParsingStorageService.storeResearchDocumentData(layoutParsingRequest, researchDocumentData);
-            }
-
-            return LayoutParsingFinishedEvent.builder()
-                    .identifier(layoutParsingRequest.identifier())
-                    .numberOfPages(numberOfPages)
-                    .duration(System.currentTimeMillis() - start)
-                    .message(format("""
-                                    Layout parsing has finished in %.02f s.
-                                    identifiers: %s
-                                    %s
-                                    Files have been saved with Ids:
-                                    Structure: %s
-                                    Text: %s
-                                    Positions: %s
-                                    PageData: %s
-                                    Simplified Text: %s
-                                    Viewer Doc: %s""",
-                            ((float) (System.currentTimeMillis() - start)) / 1000,
-                            layoutParsingRequest.identifier(),
-                            buildSemanticNodeCountMessage(numberOfPages, documentGraph.buildSemanticNodeCounts()),
-                            layoutParsingRequest.structureFileStorageId(),
-                            layoutParsingRequest.textBlockFileStorageId(),
-                            layoutParsingRequest.positionBlockFileStorageId(),
-                            layoutParsingRequest.pageFileStorageId(),
-                            layoutParsingRequest.simplifiedTextStorageId(),
-                            layoutParsingRequest.viewerDocumentStorageId()))
-                    .build();
+        ImageServiceResponse imageServiceResponse = new ImageServiceResponse();
+        if (layoutParsingRequest.imagesFileStorageId()
+                .isPresent()) {
+            imageServiceResponse = layoutParsingStorageService.getImagesFile(layoutParsingRequest.imagesFileStorageId()
+                                                                                     .get());
         }
+
+        TableServiceResponse tableServiceResponse = new TableServiceResponse();
+        if (layoutParsingRequest.tablesFileStorageId()
+                .isPresent()) {
+            tableServiceResponse = layoutParsingStorageService.getTablesFile(layoutParsingRequest.tablesFileStorageId()
+                                                                                     .get());
+        }
+
+        ClassificationDocument classificationDocument = parseLayout(layoutParsingRequest.layoutParsingType(),
+                originFile,
+                imageServiceResponse,
+                tableServiceResponse,
+                layoutParsingRequest.identifier().toString());
+
+        log.info("Building document graph for {}", layoutParsingRequest.identifier());
+
+        Document documentGraph = observeBuildDocumentGraph(classificationDocument);
+
+        log.info("Creating viewer document for {}", layoutParsingRequest.identifier());
+
+        layoutGridService.addLayoutGrid(viewerDocumentFile, documentGraph, viewerDocumentFile, false);
+
+        log.info("Storing resulting files for {}", layoutParsingRequest.identifier());
+
+        layoutParsingStorageService.storeDocumentData(layoutParsingRequest, DocumentDataMapper.toDocumentData(documentGraph));
+        layoutParsingStorageService.storeSimplifiedText(layoutParsingRequest, simplifiedSectionTextService.toSimplifiedText(documentGraph));
+        layoutParsingStorageService.storeViewerDocument(layoutParsingRequest, viewerDocumentFile);
+
+        if (layoutParsingRequest.layoutParsingType().equals(LayoutParsingType.TAAS)) {
+            log.info("Building research document data for {}", layoutParsingRequest.identifier());
+            var researchDocumentData = TaasDocumentDataMapper.fromDocument(documentGraph);
+            layoutParsingStorageService.storeResearchDocumentData(layoutParsingRequest, researchDocumentData);
+        }
+
+        if (!viewerDocumentFile.equals(originFile)) {
+            viewerDocumentFile.delete();
+        }
+        originFile.delete();
+
+        return LayoutParsingFinishedEvent.builder()
+                .identifier(layoutParsingRequest.identifier())
+                .numberOfPages(documentGraph.getNumberOfPages())
+                .duration(System.currentTimeMillis() - start)
+                .message(format("""
+                                        Layout parsing has finished in %.02f s.
+                                        identifiers: %s
+                                        %s
+                                        Files have been saved with Ids:
+                                        Structure: %s
+                                        Text: %s
+                                        Positions: %s
+                                        PageData: %s
+                                        Simplified Text: %s
+                                        Viewer Doc: %s""",
+                                ((float) (System.currentTimeMillis() - start)) / 1000,
+                                layoutParsingRequest.identifier(),
+                                buildSemanticNodeCountMessage(documentGraph.getNumberOfPages(), documentGraph.buildSemanticNodeCounts()),
+                                layoutParsingRequest.structureFileStorageId(),
+                                layoutParsingRequest.textBlockFileStorageId(),
+                                layoutParsingRequest.positionBlockFileStorageId(),
+                                layoutParsingRequest.pageFileStorageId(),
+                                layoutParsingRequest.simplifiedTextStorageId(),
+                                layoutParsingRequest.viewerDocumentStorageId()))
+                .build();
+
+    }
+
+
+    private Document observeBuildDocumentGraph(ClassificationDocument classificationDocument) {
+
+        AtomicReference<Document> documentReference = new AtomicReference<>();
+
+        Observation.createNotStarted("LayoutParsingPipeline", observationRegistry)
+                .contextualName("build-document-graph")
+                .observe(() -> documentReference.set(DocumentGraphFactory.buildDocumentGraph(classificationDocument)));
+
+        return documentReference.get();
     }
 
 
     private String buildSemanticNodeCountMessage(int numberOfPages, Map<NodeType, Long> semanticNodeCounts) {
 
         return String.format("%d pages with %d sections, %d headlines, %d paragraphs, %d tables with %d cells, %d headers, and %d footers parsed",
-                numberOfPages,
-                semanticNodeCounts.get(NodeType.SECTION) == null ? 0 : semanticNodeCounts.get(NodeType.SECTION),
-                semanticNodeCounts.get(NodeType.HEADLINE) == null ? 0 : semanticNodeCounts.get(NodeType.HEADLINE),
-                semanticNodeCounts.get(NodeType.PARAGRAPH) == null ? 0 : semanticNodeCounts.get(NodeType.PARAGRAPH),
-                semanticNodeCounts.get(NodeType.TABLE) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE),
-                semanticNodeCounts.get(NodeType.TABLE_CELL) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE_CELL),
-                semanticNodeCounts.get(NodeType.HEADER) == null ? 0 : semanticNodeCounts.get(NodeType.HEADER),
-                semanticNodeCounts.get(NodeType.FOOTER) == null ? 0 : semanticNodeCounts.get(NodeType.FOOTER));
+                             numberOfPages,
+                             semanticNodeCounts.get(NodeType.SECTION) == null ? 0 : semanticNodeCounts.get(NodeType.SECTION),
+                             semanticNodeCounts.get(NodeType.HEADLINE) == null ? 0 : semanticNodeCounts.get(NodeType.HEADLINE),
+                             semanticNodeCounts.get(NodeType.PARAGRAPH) == null ? 0 : semanticNodeCounts.get(NodeType.PARAGRAPH),
+                             semanticNodeCounts.get(NodeType.TABLE) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE),
+                             semanticNodeCounts.get(NodeType.TABLE_CELL) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE_CELL),
+                             semanticNodeCounts.get(NodeType.HEADER) == null ? 0 : semanticNodeCounts.get(NodeType.HEADER),
+                             semanticNodeCounts.get(NodeType.FOOTER) == null ? 0 : semanticNodeCounts.get(NodeType.FOOTER));
     }
 
 
     @SneakyThrows
+    @Observed(name = "LayoutParsingPipeline", contextualName = "parse-layout")
     public ClassificationDocument parseLayout(LayoutParsingType layoutParsingType,
-                                              PDDocument originDocument,
+                                              File originFile,
                                               ImageServiceResponse imageServiceResponse,
-                                              TableServiceResponse tableServiceResponse) {
+                                              TableServiceResponse tableServiceResponse,
+                                              String identifier) {
 
+        PDDocument originDocument = openDocument(originFile);
+        addNumberOfPagesToTrace(originDocument.getNumberOfPages(), Files.size(originFile.toPath()));
         Map<Integer, List<TableCells>> pdfTableCells = cvTableParsingAdapter.buildCvParsedTablesPerPage(tableServiceResponse);
         Map<Integer, List<ClassifiedImage>> pdfImages = imageServiceResponseAdapter.buildClassifiedImagesPerPage(imageServiceResponse);
 
         ClassificationDocument classificationDocument = new ClassificationDocument();
         List<ClassificationPage> classificationPages = new ArrayList<>();
 
-        originDocument.setAllSecurityToBeRemoved(true);
         long pageCount = originDocument.getNumberOfPages();
 
         for (int pageNumber = 1; pageNumber <= pageCount; pageNumber++) {
+
+            if (pageNumber % 100 == 0) {
+                // re-open document every once in a while to save on RAM. This has no significant performance impact.
+                // This is due to PDFBox caching all images and some other stuff with Soft References. This dereferences them and forces the freeing of memory.
+                originDocument.close();
+                originDocument = openDocument(originFile);
+            }
+
+            if (pageNumber % 100 == 0 || pageNumber == pageCount || pageNumber == 1) {
+                log.info("Extracting text on Page {} for {}", pageNumber, identifier);
+            }
+
             classificationDocument.setPages(classificationPages);
             PDFLinesTextStripper stripper = new PDFLinesTextStripper();
             PDPage pdPage = originDocument.getPage(pageNumber - 1);
@@ -199,8 +251,10 @@ public class LayoutParsingPipeline {
             PDRectangle cropbox = pdPage.getCropBox();
             CleanRulings cleanRulings = rulingCleaningService.getCleanRulings(pdfTableCells.get(pageNumber), stripper.getRulings());
 
+            List<Cell> emptyTableCells = TableExtractionService.findCells(cleanRulings.getHorizontal(), cleanRulings.getVertical());
+
             ClassificationPage classificationPage = switch (layoutParsingType) {
-                case REDACT_MANAGER -> redactManagerBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
+                case REDACT_MANAGER -> redactManagerBlockificationService.blockify(stripper.getTextPositionSequences(), emptyTableCells);
                 case TAAS -> taasBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
                 case DOCUMINE -> docuMineBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
             };
@@ -228,21 +282,42 @@ public class LayoutParsingPipeline {
             classificationPages.add(classificationPage);
         }
 
-        bodyTextFrameService.setBodyTextFrames(classificationDocument, layoutParsingType);
+        originDocument.close();
 
+        log.info("Calculating BodyTextFrame for {}", identifier);
+        bodyTextFrameService.setBodyTextFrames(classificationDocument, layoutParsingType);
+        log.info("Classify TextBlocks for {}", identifier);
         switch (layoutParsingType) {
             case TAAS -> taasClassificationService.classifyDocument(classificationDocument);
             case DOCUMINE -> docuMineClassificationService.classifyDocument(classificationDocument);
             case REDACT_MANAGER -> redactManagerClassificationService.classifyDocument(classificationDocument);
         }
 
+        log.info("Building Sections for {}", identifier);
         sectionsBuilderService.buildSections(classificationDocument);
         sectionsBuilderService.addImagesToSections(classificationDocument);
-
         return classificationDocument;
     }
 
 
+    private void addNumberOfPagesToTrace(int numberOfPages, long size) {
+
+        if (observationRegistry.getCurrentObservation() != null) {
+            observationRegistry.getCurrentObservation().highCardinalityKeyValue("numberOfPages", String.valueOf(numberOfPages));
+            observationRegistry.getCurrentObservation().highCardinalityKeyValue("fileSize", String.valueOf(size));
+        }
+    }
+
+
+    @SneakyThrows
+    private PDDocument openDocument(File originFile) {
+
+        PDDocument document = Loader.loadPDF(originFile);
+        document.setAllSecurityToBeRemoved(true);
+        return document;
+    }
+
+
     private Map<String, List<Rectangle2D>> convertMarkedContents(List<PDMarkedContent> pdMarkedContents) {
 
         Map<String, List<Rectangle2D>> markedContentBboxes = new HashMap<>();
@@ -254,9 +329,7 @@ public class LayoutParsingPipeline {
 
     private void increaseDocumentStatistics(ClassificationPage classificationPage, ClassificationDocument document) {
 
-        if (!classificationPage.isLandscape()) {
-            document.getFontSizeCounter().addAll(classificationPage.getFontSizeCounter().getCountPerValue());
-        }
+        document.getFontSizeCounter().addAll(classificationPage.getFontSizeCounter().getCountPerValue());
         document.getFontCounter().addAll(classificationPage.getFontCounter().getCountPerValue());
         document.getTextHeightCounter().addAll(classificationPage.getTextHeightCounter().getCountPerValue());
         document.getFontStyleCounter().addAll(classificationPage.getFontStyleCounter().getCountPerValue());

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/LayoutParsingServiceProcessorConfiguration.java

@@ -1,10 +1,24 @@
 package com.knecon.fforesight.service.layoutparser.processor;
 
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.ComponentScan;
 import org.springframework.context.annotation.Configuration;
 
+import com.knecon.fforesight.service.viewerdoc.service.ViewerDocumentService;
+
+import io.micrometer.observation.ObservationRegistry;
+
 @Configuration
 @ComponentScan
 public class LayoutParsingServiceProcessorConfiguration {
 
+
+    @Bean
+    @Autowired
+    public ViewerDocumentService viewerDocumentService(ObservationRegistry registry) {
+
+        return new ViewerDocumentService(registry);
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/LayoutParsingStorageService.java

@@ -1,17 +1,15 @@
 package com.knecon.fforesight.service.layoutparser.processor;
 
-import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.File;
-import java.io.FileOutputStream;
+import java.io.FileInputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.nio.file.StandardOpenOption;
+import java.util.Optional;
 
-import org.apache.commons.io.IOUtils;
 import org.apache.pdfbox.Loader;
 import org.apache.pdfbox.pdmodel.PDDocument;
 import org.springframework.stereotype.Service;
@@ -24,9 +22,9 @@ import com.knecon.fforesight.service.layoutparser.internal.api.data.taas.Researc
 import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingRequest;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.image.ImageServiceResponse;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
 import com.knecon.fforesight.tenantcommons.TenantContext;
 
+import io.micrometer.observation.annotation.Observed;
 import lombok.RequiredArgsConstructor;
 import lombok.SneakyThrows;
 import lombok.extern.slf4j.Slf4j;
@@ -39,17 +37,24 @@ public class LayoutParsingStorageService {
     private final StorageService storageService;
     private final ObjectMapper objectMapper;
 
+    @Observed(name = "LayoutParsingStorageService", contextualName = "get-origin-file")
+    public File getOriginFile(String storageId) throws IOException {
 
-    public PDDocument getOriginFile(String storageId) throws IOException {
+        File tempFile = createTempFile("document", ".pdf");
+        storageService.downloadTo(TenantContext.getTenantId(), storageId, tempFile);
+        return tempFile;
+    }
 
-        try (var originDocumentInputStream = getObject(storageId)) {
-            File tempFile = createTempFile("document", ".pdf");
-            try (var tempFileOutputStream = new FileOutputStream(tempFile)) {
-                IOUtils.copy(originDocumentInputStream, tempFileOutputStream);
-                originDocumentInputStream.close();
-            }
-            return Loader.loadPDF(tempFile);
+
+    @Observed(name = "LayoutParsingStorageService", contextualName = "get-viewer-doc-file")
+    public Optional<File> getViewerDocFile(String storageId) throws IOException {
+
+        if (!storageService.objectExists(TenantContext.getTenantId(), storageId)) {
+            return Optional.empty();
         }
+        File tempFile = createTempFile("viewerDocument", ".pdf");
+        storageService.downloadTo(TenantContext.getTenantId(), storageId, tempFile);
+        return Optional.of(tempFile);
     }
 
 
@@ -75,16 +80,7 @@ public class LayoutParsingStorageService {
     }
 
 
-    public VisualLayoutParsingResponse getExtractedTablesFile(String storageId) throws IOException {
-
-        try (InputStream inputStream = getObject(storageId)) {
-            VisualLayoutParsingResponse visualLayoutParsingResponse = objectMapper.readValue(inputStream, VisualLayoutParsingResponse.class);
-            inputStream.close();
-            return visualLayoutParsingResponse;
-        }
-    }
-
-
+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-document-data")
     public void storeDocumentData(LayoutParsingRequest layoutParsingRequest, DocumentData documentData) {
 
         storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.structureFileStorageId(), documentData.getDocumentStructure());
@@ -125,6 +121,7 @@ public class LayoutParsingStorageService {
     }
 
 
+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-simplified-text")
     public void storeSimplifiedText(LayoutParsingRequest layoutParsingRequest, SimplifiedText simplifiedText) {
 
         storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.simplifiedTextStorageId(), simplifiedText);
@@ -142,10 +139,10 @@ public class LayoutParsingStorageService {
 
 
     @SneakyThrows
-    public void storeViewerDocument(LayoutParsingRequest layoutParsingRequest, ByteArrayOutputStream out) {
-
-        try (var in = new ByteArrayInputStream(out.toByteArray())) {
+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-viewer-document")
+    public void storeViewerDocument(LayoutParsingRequest layoutParsingRequest, File out) {
 
+        try (var in = new FileInputStream(out)) {
             storageService.storeObject(TenantContext.getTenantId(), layoutParsingRequest.viewerDocumentStorageId(), in);
         }
     }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/AbstractPageBlock.java

@@ -6,12 +6,14 @@ import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageB
 
 import lombok.AllArgsConstructor;
 import lombok.Data;
+import lombok.EqualsAndHashCode;
 import lombok.NoArgsConstructor;
 
 @Data
 @AllArgsConstructor
 @NoArgsConstructor
-public abstract class AbstractPageBlock {
+@EqualsAndHashCode(callSuper = true)
+public abstract class AbstractPageBlock extends Rectangle {
 
     @JsonIgnore
     protected float minX;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/ClassificationPage.java

@@ -14,6 +14,7 @@ import com.knecon.fforesight.service.layoutparser.processor.model.text.StringFre
 import lombok.Data;
 import lombok.NonNull;
 import lombok.RequiredArgsConstructor;
+import org.apache.pdfbox.pdmodel.documentinterchange.markedcontent.PDMarkedContent;
 
 @Data
 @RequiredArgsConstructor

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/PageContents.java

@@ -19,5 +19,4 @@ public class PageContents {
     Rectangle2D cropBox;
     Rectangle2D mediaBox;
     List<Ruling> rulings;
-
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/Boundary.java

@@ -12,6 +12,7 @@ import lombok.Setter;
 
 @Setter
 @EqualsAndHashCode
+@SuppressWarnings("PMD.AvoidFieldNameMatchingMethodName")
 public class Boundary implements Comparable<Boundary> {
 
     private int start;
@@ -108,13 +109,11 @@ public class Boundary implements Comparable<Boundary> {
         return splitBoundaries;
     }
 
-
     public IntStream intStream() {
 
         return IntStream.range(start, end);
     }
 
-
     public static Boundary merge(Collection<Boundary> boundaries) {
 
         int minStart = boundaries.stream().mapToInt(Boundary::start).min().orElseThrow(IllegalArgumentException::new);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Document.java

@@ -10,7 +10,6 @@ import java.util.Set;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
-import com.amazonaws.services.kms.model.NotFoundException;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.DocumentTree;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.entity.RedactionEntity;
@@ -84,7 +83,7 @@ public class Document implements GenericSemanticNode {
     @Override
     public Headline getHeadline() {
 
-        return streamAllSubNodesOfType(NodeType.HEADLINE).map(node -> (Headline) node).findFirst().orElseThrow(() -> new NotFoundException("No Headlines found in this document!"));
+        return streamAllSubNodesOfType(NodeType.HEADLINE).map(node -> (Headline) node).findFirst().orElse(Headline.builder().build());
     }
 
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Footer.java

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Footer implements GenericSemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+
 
     @Override
     public NodeType getType() {
@@ -62,4 +67,14 @@ public class Footer implements GenericSemanticNode {
         return treeId + ": " + NodeType.FOOTER + ": " + leafTextBlock.buildSummary();
     }
 
+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Header.java

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Header implements GenericSemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+
 
     @Override
     public boolean isLeaf() {
@@ -62,4 +67,14 @@ public class Header implements GenericSemanticNode {
         return treeId + ": " + NodeType.HEADER + ": " + leafTextBlock.buildSummary();
     }
 
+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Headline.java

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Headline implements GenericSemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+
 
     @Override
     public NodeType getType() {
@@ -69,4 +74,14 @@ public class Headline implements GenericSemanticNode {
         return this;
     }
 
+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Image.java

@@ -12,7 +12,6 @@ import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.No
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.DocumentTree;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.entity.RedactionEntity;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.TextBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.TextBlockCollector;
 
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
@@ -36,6 +35,8 @@ public class Image implements GenericSemanticNode {
     boolean transparent;
     Rectangle2D position;
 
+    TextBlock leafTextBlock;
+
     boolean redaction;
     boolean ignored;
     @Builder.Default
@@ -66,7 +67,7 @@ public class Image implements GenericSemanticNode {
     @Override
     public TextBlock getTextBlock() {
 
-        return streamAllSubNodes().filter(SemanticNode::isLeaf).map(SemanticNode::getLeafTextBlock).collect(new TextBlockCollector());
+        return leafTextBlock;
     }
 
 
@@ -92,4 +93,11 @@ public class Image implements GenericSemanticNode {
         return bBoxPerPage;
     }
 
+
+    @Override
+    public boolean isLeaf() {
+
+        return true;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Paragraph.java

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -32,6 +34,9 @@ public class Paragraph implements GenericSemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+
 
     @Override
     public NodeType getType() {
@@ -60,4 +65,14 @@ public class Paragraph implements GenericSemanticNode {
         return treeId + ": " + NodeType.PARAGRAPH + ": " + leafTextBlock.buildSummary();
     }
 
+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Section.java

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -35,6 +37,9 @@ public class Section implements GenericSemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+
 
     @Override
     public NodeType getType() {
@@ -74,4 +79,14 @@ public class Section implements GenericSemanticNode {
                 .orElseGet(() -> getParent().getHeadline());
     }
 
+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/SemanticNode.java

@@ -398,12 +398,11 @@ public interface SemanticNode {
      */
     default Map<Page, Rectangle2D> getBBox() {
 
-        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
         if (isLeaf()) {
-            return getBBoxFromLeafTextBlock(bBoxPerPage);
+            return getBBoxFromLeafTextBlock();
         }
 
-        return getBBoxFromChildren(bBoxPerPage);
+        return getBBoxFromChildren();
     }
 
 
@@ -426,25 +425,31 @@ public interface SemanticNode {
 
     /**
      * TODO: this produces unwanted results for sections spanning multiple columns.
-     *
-     * @param bBoxPerPage initial empty BoundingBox
+     * Computes the Union of the bounding boxes of all children recursively.
      * @return The union of the BoundingBoxes of all children
      */
-    private Map<Page, Rectangle2D> getBBoxFromChildren(Map<Page, Rectangle2D> bBoxPerPage) {
+    private Map<Page, Rectangle2D> getBBoxFromChildren() {
 
-        return streamChildren().map(SemanticNode::getBBox).reduce((map1, map2) -> {
-            map1.forEach((page, rectangle) -> map2.merge(page, rectangle, (rect1, rect2) -> rect1.createUnion(rect2).getBounds2D()));
-            return map2;
-        }).orElse(bBoxPerPage);
+        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
+        List<Map<Page, Rectangle2D>> childrenBBoxes = streamChildren().map(SemanticNode::getBBox).toList();
+        Set<Page> pages = childrenBBoxes.stream().flatMap(map -> map.keySet().stream()).collect(Collectors.toSet());
+        for (Page page : pages) {
+            Rectangle2D bBoxOnPage = childrenBBoxes.stream()
+                    .filter(childBboxPerPage -> childBboxPerPage.containsKey(page))
+                    .map(childBboxPerPage -> childBboxPerPage.get(page))
+                    .collect(RectangleTransformations.collectBBox());
+            bBoxPerPage.put(page, bBoxOnPage);
+        }
+        return bBoxPerPage;
     }
 
 
     /**
-     * @param bBoxPerPage initial empty BoundingBox
      * @return The union of all BoundingBoxes of the TextBlock of this node
      */
-    private Map<Page, Rectangle2D> getBBoxFromLeafTextBlock(Map<Page, Rectangle2D> bBoxPerPage) {
+    private Map<Page, Rectangle2D> getBBoxFromLeafTextBlock() {
 
+        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
         Map<Page, List<AtomicTextBlock>> atomicTextBlockPerPage = getTextBlock().getAtomicTextBlocks().stream().collect(Collectors.groupingBy(AtomicTextBlock::getPage));
         atomicTextBlockPerPage.forEach((page, atbs) -> bBoxPerPage.put(page, RectangleTransformations.bBoxUnionAtomicTextBlock(atbs)));
         return bBoxPerPage;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/Table.java

@@ -2,10 +2,12 @@ package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;
 
 import static java.lang.String.format;
 
+import java.awt.geom.Rectangle2D;
 import java.util.Collection;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Locale;
+import java.util.Map;
 import java.util.Set;
 import java.util.stream.IntStream;
 import java.util.stream.Stream;
@@ -40,6 +42,8 @@ public class Table implements SemanticNode {
     @EqualsAndHashCode.Exclude
     Set<RedactionEntity> entities = new HashSet<>();
 
+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
 
     /**
      * Streams all entities in this table, that appear in a row, which contains any of the provided strings.
@@ -207,7 +211,6 @@ public class Table implements SemanticNode {
         return IntStream.range(0, numberOfCols).boxed().map(col -> getCell(row, col));
     }
 
-
     /**
      * Streams all TableCells row-wise and filters them with header == true.
      *
@@ -312,5 +315,12 @@ public class Table implements SemanticNode {
 
         return treeId.toString() + ": " + NodeType.TABLE + ": #cols: " + numberOfCols + ", #rows: " + numberOfRows + ", " + this.getTextBlock().buildSummary();
     }
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
 
+        if (bBoxCache == null) {
+            bBoxCache = SemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/nodes/TableCell.java

@@ -84,14 +84,16 @@ public class TableCell implements GenericSemanticNode {
 
     private TextBlock buildTextBlock() {
 
-        return streamAllSubNodes().filter(SemanticNode::isLeaf).map(SemanticNode::getLeafTextBlock).collect(new TextBlockCollector());
+        return streamAllSubNodes().filter(SemanticNode::isLeaf)
+                .map(SemanticNode::getLeafTextBlock)
+                .collect(new TextBlockCollector());
     }
 
 
     @Override
     public String toString() {
 
-        return treeId + ": " + NodeType.TABLE_CELL + ": " + this.buildTextBlock().buildSummary();
+        return treeId + ": " + NodeType.TABLE_CELL + ": " + this.getTextBlock().buildSummary();
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/graph/textblock/AtomicTextBlock.java

@@ -109,7 +109,10 @@ public class AtomicTextBlock implements TextBlock {
     }
 
 
-    public static AtomicTextBlock fromAtomicTextBlockData(DocumentTextData documentTextData, DocumentPositionData documentPositionData, SemanticNode parent, Page page) {
+    public static AtomicTextBlock fromAtomicTextBlockData(DocumentTextData documentTextData,
+                                                          DocumentPositionData documentPositionData,
+                                                          SemanticNode parent,
+                                                          Page page) {
 
         return AtomicTextBlock.builder()
                 .id(documentTextData.getId())

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/table/Cell.java

@@ -1,6 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.table;
 
 import java.awt.geom.Point2D;
+import java.awt.geom.Rectangle2D;
 import java.util.ArrayList;
 import java.util.Iterator;
 import java.util.List;
@@ -36,6 +37,12 @@ public class Cell extends Rectangle {
     }
 
 
+    public Cell(Rectangle2D r) {
+
+        super((float) r.getY(), (float) r.getX(), (float) r.getWidth(), (float) r.getHeight());
+    }
+
+
     public void addTextBlock(TextPageBlock textBlock) {
 
         textBlocks.add(textBlock);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/table/Ruling.java

@@ -20,7 +20,8 @@ import lombok.extern.slf4j.Slf4j;
 @SuppressWarnings("all")
 public class Ruling extends Line2D.Float {
 
-    private static int PERPENDICULAR_PIXEL_EXPAND_AMOUNT = 2;
+    public static final int PERPENDICULAR_UNIT_EXPAND_AMOUNT = 2;
+    public static final int COLINEAR_OR_PARALLEL_UNIT_EXPAND_AMOUNT = 2;
 
 
     public Ruling(Point2D p1, Point2D p2) {
@@ -110,8 +111,8 @@ public class Ruling extends Line2D.Float {
         });
 
         for (Ruling h : horizontals) {
-            sos.add(new SortObject(SOType.HLEFT, h.getLeft() - PERPENDICULAR_PIXEL_EXPAND_AMOUNT, h));
-            sos.add(new SortObject(SOType.HRIGHT, h.getRight() + PERPENDICULAR_PIXEL_EXPAND_AMOUNT, h));
+            sos.add(new SortObject(SOType.HLEFT, h.getLeft() - PERPENDICULAR_UNIT_EXPAND_AMOUNT, h));
+            sos.add(new SortObject(SOType.HRIGHT, h.getRight() + PERPENDICULAR_UNIT_EXPAND_AMOUNT, h));
         }
 
         for (Ruling v : verticals) {
@@ -151,7 +152,7 @@ public class Ruling extends Line2D.Float {
                             if (i == null) {
                                 continue;
                             }
-                            rv.put(i, new Ruling[]{h.getKey().expand(PERPENDICULAR_PIXEL_EXPAND_AMOUNT), so.ruling.expand(PERPENDICULAR_PIXEL_EXPAND_AMOUNT)});
+                            rv.put(i, new Ruling[]{h.getKey().expand(PERPENDICULAR_UNIT_EXPAND_AMOUNT), so.ruling.expand(PERPENDICULAR_UNIT_EXPAND_AMOUNT)});
                         } catch (UnsupportedOperationException e) {
                             log.info("Some line are oblique, ignoring...");
                             continue;
@@ -267,7 +268,7 @@ public class Ruling extends Line2D.Float {
     }
 
 
-    public boolean nearlyIntersects(Ruling another, int colinearOrParallelExpandAmount) {
+    public boolean nearlyIntersects(Ruling another) {
 
         if (this.intersectsLine(another)) {
             return true;
@@ -276,9 +277,9 @@ public class Ruling extends Line2D.Float {
         boolean rv = false;
 
         if (this.perpendicularTo(another)) {
-            rv = this.expand(PERPENDICULAR_PIXEL_EXPAND_AMOUNT).intersectsLine(another);
+            rv = this.expand(PERPENDICULAR_UNIT_EXPAND_AMOUNT).intersectsLine(another);
         } else {
-            rv = this.expand(colinearOrParallelExpandAmount).intersectsLine(another.expand(colinearOrParallelExpandAmount));
+            rv = this.expand(COLINEAR_OR_PARALLEL_UNIT_EXPAND_AMOUNT).intersectsLine(another.expand(COLINEAR_OR_PARALLEL_UNIT_EXPAND_AMOUNT));
         }
 
         return rv;
@@ -319,8 +320,8 @@ public class Ruling extends Line2D.Float {
 
     public Point2D intersectionPoint(Ruling other) {
 
-        Ruling this_l = this.expand(PERPENDICULAR_PIXEL_EXPAND_AMOUNT);
-        Ruling other_l = other.expand(PERPENDICULAR_PIXEL_EXPAND_AMOUNT);
+        Ruling this_l = this.expand(PERPENDICULAR_UNIT_EXPAND_AMOUNT);
+        Ruling other_l = other.expand(PERPENDICULAR_UNIT_EXPAND_AMOUNT);
         Ruling horizontal, vertical;
 
         if (!this_l.intersectsLine(other_l)) {

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/table/TablePageBlock.java

@@ -3,6 +3,7 @@ package com.knecon.fforesight.service.layoutparser.processor.model.table;
 import java.awt.geom.Point2D;
 import java.util.ArrayList;
 import java.util.Collections;
+import java.util.Comparator;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
@@ -11,6 +12,7 @@ import java.util.TreeMap;
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.PageBlockType;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.utils.RectangleTransformations;
 
 import lombok.Getter;
 import lombok.Setter;
@@ -19,7 +21,8 @@ import lombok.extern.slf4j.Slf4j;
 @Slf4j
 public class TablePageBlock extends AbstractPageBlock {
 
-    private final TreeMap<CellPosition, Cell> cells = new TreeMap<>();
+    public static final double CELL_AREA_CONTAINED_THRESHOLD = 0.98;
+    private final TreeMap<CellPosition, Cell> cellTreeMap = new TreeMap<>();
 
     private final int rotation;
     @Getter
@@ -28,10 +31,14 @@ public class TablePageBlock extends AbstractPageBlock {
     private int unrotatedRowCount;
     private int unrotatedColCount;
     private List<List<Cell>> rows;
+    @Getter
+    @Setter
+    private List<Cell> cells;
 
 
     public TablePageBlock(List<Cell> cells, Rectangle area, int rotation) {
 
+        this.cells = cells;
         addCells(cells);
         minX = area.getLeft();
         minY = area.getBottom();
@@ -79,14 +86,17 @@ public class TablePageBlock extends AbstractPageBlock {
 
     public int getColCount() {
 
-        return getRows().stream().mapToInt(List::size).max().orElse(0);
+        return getRows().stream()
+                .mapToInt(List::size)
+                .max()
+                .orElse(0);
 
     }
 
 
     /**
      * Detect header cells (either first row or first column):
-     * Column is marked as header if cell text is bold and row cell text is not bold.
+     * Column is marked as header if originalCell text is bold and row originalCell text is not bold.
      * Defaults to row.
      */
     private void computeHeaders() {
@@ -94,7 +104,7 @@ public class TablePageBlock extends AbstractPageBlock {
         if (rows == null) {
             rows = computeRows();
         }
-        // A bold cell is a header cell as long as every cell to the left/top is bold, too
+        // A bold originalCell is a header originalCell as long as every originalCell to the left/top is bold, too
         // we move from left to right and top to bottom
         for (int rowIndex = 0; rowIndex < rows.size(); rowIndex++) {
             List<Cell> rowCells = rows.get(rowIndex);
@@ -119,7 +129,8 @@ public class TablePageBlock extends AbstractPageBlock {
                 List<Cell> cellsToTheTop = new ArrayList<>();
                 for (int i = 0; i < rowIndex; i++) {
                     try {
-                        cellsToTheTop.add(rows.get(i).get(colIndex));
+                        cellsToTheTop.add(rows.get(i)
+                                                  .get(colIndex));
                     } catch (IndexOutOfBoundsException e) {
                         log.debug("No cell {} in row {}, ignoring.", colIndex, rowIndex);
                     }
@@ -134,7 +145,8 @@ public class TablePageBlock extends AbstractPageBlock {
                 if (lastHeaderCell != null) {
                     cell.getHeaderCells().add(lastHeaderCell);
                 }
-                if (!cell.getTextBlocks().isEmpty() && cell.getTextBlocks().get(0).getMostPopularWordStyle().equals("bold")) {
+                if (!cell.getTextBlocks().isEmpty() && cell.getTextBlocks()
+                        .get(0).getMostPopularWordStyle().equals("bold")) {
                     cell.setHeaderCell(true);
                 }
             }
@@ -150,7 +162,7 @@ public class TablePageBlock extends AbstractPageBlock {
             for (int i = 0; i < unrotatedColCount; i++) { // rows
                 List<Cell> lastRow = new ArrayList<>();
                 for (int j = unrotatedRowCount - 1; j >= 0; j--) { // cols
-                    Cell cell = cells.get(new CellPosition(j, i));
+                    Cell cell = cellTreeMap.get(new CellPosition(j, i));
                     if (cell != null) {
                         lastRow.add(cell);
                     }
@@ -161,7 +173,7 @@ public class TablePageBlock extends AbstractPageBlock {
             for (int i = unrotatedColCount - 1; i >= 0; i--) { // rows
                 List<Cell> lastRow = new ArrayList<>();
                 for (int j = 0; j < unrotatedRowCount; j++) { // cols
-                    Cell cell = cells.get(new CellPosition(j, i));
+                    Cell cell = cellTreeMap.get(new CellPosition(j, i));
                     if (cell != null) {
                         lastRow.add(cell);
                     }
@@ -172,7 +184,7 @@ public class TablePageBlock extends AbstractPageBlock {
             for (int i = 0; i < unrotatedRowCount; i++) {
                 List<Cell> lastRow = new ArrayList<>();
                 for (int j = 0; j < unrotatedColCount; j++) {
-                    Cell cell = cells.get(new CellPosition(i, j)); // JAVA_8 use getOrDefault()
+                    Cell cell = cellTreeMap.get(new CellPosition(i, j)); // JAVA_8 use getOrDefault()
                     if (cell != null) {
                         lastRow.add(cell);
                     }
@@ -186,17 +198,6 @@ public class TablePageBlock extends AbstractPageBlock {
     }
 
 
-    private void add(Cell chunk, int row, int col) {
-
-        unrotatedRowCount = Math.max(unrotatedRowCount, row + 1);
-        unrotatedColCount = Math.max(unrotatedColCount, col + 1);
-
-        CellPosition cp = new CellPosition(row, col);
-        cells.put(cp, chunk);
-
-    }
-
-
     private void addCells(List<Cell> cells) {
 
         if (cells.isEmpty()) {
@@ -205,11 +206,12 @@ public class TablePageBlock extends AbstractPageBlock {
 
         cells.removeIf(cell -> cell.getWidth() < 1.1 || cell.getHeight() < 1.1);
 
-        List<List<Cell>> rowsOfCells = calculateStructure(cells);
+        List<List<Cell>> rowsOfCellsMatrix = calculateTableStructure(cells);
 
-        for (int i = 0; i < rowsOfCells.size(); i++) {
-            for (int j = 0; j < rowsOfCells.get(i).size(); j++) {
-                add(rowsOfCells.get(i).get(j), i, j);
+        for (int i = 0; i < rowsOfCellsMatrix.size(); i++) {
+            for (int j = 0; j < rowsOfCellsMatrix.get(i).size(); j++) {
+                addCellToRowAndCol(rowsOfCellsMatrix.get(i)
+                                           .get(j), i, j);
             }
         }
 
@@ -220,29 +222,36 @@ public class TablePageBlock extends AbstractPageBlock {
      * Calculates the structure of the table. For spanning rows and columns multiple cells with the same values will be inserted.
      *
      * @param cells The found cells
-     * @return TablePageBlock Structure
+     * @return TablePageBlock Structure as a rows of cells matrix
      */
-    private List<List<Cell>> calculateStructure(List<Cell> cells) {
-
-        List<List<Cell>> matrix = new ArrayList<>();
+    private List<List<Cell>> calculateTableStructure(List<Cell> cells) {
 
         if (cells.isEmpty()) {
-            return matrix;
+            return new ArrayList<>();
         }
 
         Set<Float> uniqueX = new HashSet<>();
         Set<Float> uniqueY = new HashSet<>();
-        cells.stream().filter(c -> !c.getTextBlocks().isEmpty() || c.getHeight() > 3 && c.getWidth() > 3).forEach(c -> {
-            uniqueX.add(c.getLeft());
-            uniqueX.add(c.getRight());
-            uniqueY.add(c.getBottom());
-            uniqueY.add(c.getTop());
-        });
+        cells.stream()
+                .filter(c -> !c.getTextBlocks().isEmpty() || c.getHeight() > 3 && c.getWidth() > 3)
+                .forEach(c -> {
+                    uniqueX.add(c.getLeft());
+                    uniqueX.add(c.getRight());
+                    uniqueY.add(c.getBottom());
+                    uniqueY.add(c.getTop());
+                });
 
-        var sortedUniqueX = uniqueX.stream().sorted().toList();
-        var sortedUniqueY = uniqueY.stream().sorted().toList();
+        var sortedUniqueX = uniqueX.stream()
+                .sorted()
+                .toList();
+        var sortedUniqueY = uniqueY.stream()
+                .sorted()
+                .toList();
+
+        List<List<Cell>> rowsOfCells = new ArrayList<>();
 
         Float prevY = null;
+
         for (Float y : sortedUniqueY) {
 
             List<Cell> row = new ArrayList<>();
@@ -251,38 +260,85 @@ public class TablePageBlock extends AbstractPageBlock {
             for (Float x : sortedUniqueX) {
 
                 if (prevY != null && prevX != null) {
-                    var cell = new Cell(new Point2D.Float(prevX, prevY), new Point2D.Float(x, y));
+                    var cellFromGridStructure = new Cell(new Point2D.Float(prevX, prevY), new Point2D.Float(x, y));
 
-                    var intersectionCell = cells.stream().filter(c -> intersects(cell, c)).findFirst();
+                    if (cellFromGridStructure.hasMinimumSize()) {
 
-                    intersectionCell.ifPresent(value -> cell.getTextBlocks().addAll(value.getTextBlocks()));
-                    if (cell.hasMinimumSize()) {
-                        row.add(cell);
+                        cells.stream()
+                                .map(originalCell -> new CellWithIntersection(originalCell, RectangleTransformations.calculateIntersectedArea(cellFromGridStructure, originalCell)))
+                                .filter(cellWithIntersection -> cellWithIntersection.intersectedArea > 0)
+                                .filter(cellWithIntersection -> cellWithIntersection.originalCell.getArea() > cellWithIntersection.intersectedArea * CELL_AREA_CONTAINED_THRESHOLD)
+                                .max(Comparator.comparing(CellWithIntersection::intersectedArea))
+                                .map(CellWithIntersection::originalCell)
+                                .ifPresent(matchingCell -> cellFromGridStructure.getTextBlocks().addAll(matchingCell.getTextBlocks()));
+
+                        row.add(cellFromGridStructure);
                     }
                 }
                 prevX = x;
             }
 
-            if (prevY != null && prevX != null && !row.isEmpty()) {
-                matrix.add(row);
+            // exclude empty rows and rows where all text blocks are empty
+            if (prevY != null && prevX != null && !row.isEmpty() && !row.stream()
+                    .allMatch(cell -> cell.getTextBlocks().isEmpty())) {
+
+                rowsOfCells.add(row);
             }
             prevY = y;
         }
 
-        Collections.reverse(matrix);
+        Collections.reverse(rowsOfCells);
 
-        return matrix;
+        // now cells are removed which are part of a column without any text blocks
+        // this is done by first computing the inverse matrix which contains call columns of cells
+        // then the column indices that have to be removed are determined
+        List<List<Cell>> columnsOfCells = new ArrayList<>();
+        int maxRowLength = rowsOfCells.stream()
+                .map(List::size)
+                .max(java.util.Comparator.naturalOrder())
+                .orElse(0);
+        for (int i = 0; i < maxRowLength; i++) {
+            columnsOfCells.add(new ArrayList<>());
+        }
+
+        for (List<Cell> row : rowsOfCells) {
+            for (int j = 0; j < row.size(); j++) {
+                columnsOfCells.get(j).add(row.get(j));
+            }
+        }
+
+        List<Integer> columnIndicesToRemove = new ArrayList<>();
+        int columnIndex = 0;
+        for (List<Cell> col : columnsOfCells) {
+            if (col.stream()
+                    .allMatch(cell -> cell.getTextBlocks().isEmpty())) {
+                columnIndicesToRemove.add(columnIndex);
+            }
+            columnIndex++;
+        }
+        columnIndicesToRemove.sort(Collections.reverseOrder());
+
+        // update all rows so that the values of the empty columns get removed
+        var rowsOfCellsBefore = new ArrayList<>(rowsOfCells);
+        rowsOfCells = new ArrayList<>();
+        for (List<Cell> row : rowsOfCellsBefore) {
+            var updatedRow = new ArrayList<>(row);
+            columnIndicesToRemove.forEach(idxToRemove -> updatedRow.remove(updatedRow.get(idxToRemove)));
+            rowsOfCells.add(updatedRow);
+        }
+
+        return rowsOfCells;
     }
 
 
-    public boolean intersects(Cell cell1, Cell cell2) {
+    private void addCellToRowAndCol(Cell cell, int row, int col) {
+
+        unrotatedRowCount = Math.max(unrotatedRowCount, row + 1);
+        unrotatedColCount = Math.max(unrotatedColCount, col + 1);
+
+        CellPosition cp = new CellPosition(row, col);
+        cellTreeMap.put(cp, cell);
 
-        if (cell1.getHeight() <= 0 || cell2.getHeight() <= 0) {
-            return false;
-        }
-        double x0 = cell1.getX() + 2;
-        double y0 = cell1.getY() + 2;
-        return (cell2.x + cell2.width > x0 && cell2.y + cell2.height > y0 && cell2.x < x0 + cell1.getWidth() - 2 && cell2.y < y0 + cell1.getHeight() - 2);
     }
 
 
@@ -309,7 +365,7 @@ public class TablePageBlock extends AbstractPageBlock {
                             if (!first) {
                                 sb.append("\n");
                             }
-                            sb.append('\"').append(textBlock.getText().replaceAll("\"", "\\\"")).append('\"');
+                            sb.append('\"').append(textBlock.getText().replaceAll("\"", "\"")).append('\"');
                             first = false;
                         }
                     }
@@ -356,4 +412,9 @@ public class TablePageBlock extends AbstractPageBlock {
         return sb.toString();
     }
 
+
+    record CellWithIntersection(Cell originalCell, double intersectedArea) {
+
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/text/RedTextPosition.java

@@ -1,7 +1,6 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.text;
 
 import org.apache.pdfbox.text.TextPosition;
-import org.springframework.beans.BeanUtils;
 
 import com.fasterxml.jackson.annotation.JsonIgnore;
 
@@ -50,7 +49,13 @@ public class RedTextPosition {
     public static RedTextPosition fromTextPosition(TextPosition textPosition) {
 
         var pos = new RedTextPosition();
-        BeanUtils.copyProperties(textPosition, pos);
+        pos.setRotation(textPosition.getRotation());
+        pos.setPageHeight(textPosition.getPageHeight());
+        pos.setPageWidth(textPosition.getPageWidth());
+        pos.setUnicode(textPosition.getUnicode());
+        pos.setDir(textPosition.getDir());
+        pos.setWidthOfSpace(textPosition.getWidthOfSpace());
+        pos.setFontSizeInPt(textPosition.getFontSizeInPt());
         pos.setFontName(textPosition.getFont().getName());
 
         pos.setFontSizeInPt(textPosition.getFontSizeInPt());

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/text/SearchableText.java

@@ -2,7 +2,6 @@ package com.knecon.fforesight.service.layoutparser.processor.model.text;
 
 import java.util.ArrayList;
 import java.util.List;
-
 import com.knecon.fforesight.service.layoutparser.processor.utils.TextNormalizationUtilities;
 
 import lombok.Getter;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/text/TextPageBlock.java

@@ -73,7 +73,7 @@ public class TextPageBlock extends AbstractPageBlock {
 
         return sequences.get(0).getPageWidth();
     }
-
+    
 
     public static TextPageBlock merge(List<TextPageBlock> textBlocksToMerge) {
 
@@ -82,7 +82,6 @@ public class TextPageBlock extends AbstractPageBlock {
         return fromTextPositionSequences(sequences);
     }
 
-
     public static TextPageBlock fromTextPositionSequences(List<TextPositionSequence> wordBlockList) {
 
         TextPageBlock textBlock = null;
@@ -134,6 +133,7 @@ public class TextPageBlock extends AbstractPageBlock {
     }
 
 
+
     /**
      * Returns the minX value in pdf coordinate system.
      * Note: This needs to use Pdf Coordinate System where {0,0} rotated with the page rotation.

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/text/TextPositionSequence.java

@@ -9,8 +9,6 @@ import java.util.stream.Collectors;
 
 import org.apache.pdfbox.text.TextPosition;
 
-import com.dslplatform.json.JsonAttribute;
-import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Point;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Rectangle;
 
@@ -142,8 +140,7 @@ public class TextPositionSequence implements CharSequence {
      *
      * @return the text direction adjusted minX value
      */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
     public float getMinXDirAdj() {
 
         return textPositions.get(0).getXDirAdj();
@@ -157,8 +154,7 @@ public class TextPositionSequence implements CharSequence {
      *
      * @return the text direction adjusted maxX value
      */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
     public float getMaxXDirAdj() {
 
         return textPositions.get(textPositions.size() - 1).getXDirAdj() + textPositions.get(textPositions.size() - 1).getWidthDirAdj() + HEIGHT_PADDING;
@@ -172,8 +168,7 @@ public class TextPositionSequence implements CharSequence {
      *
      * @return the text direction adjusted minY value. The upper border of the bounding box of the word.
      */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
     public float getMinYDirAdj() {
 
         return textPositions.get(0).getYDirAdj() - getTextHeight();
@@ -187,8 +182,7 @@ public class TextPositionSequence implements CharSequence {
      *
      * @return the text direction adjusted maxY value. The lower border of the bounding box of the word.
      */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
     public float getMaxYDirAdj() {
 
         return textPositions.get(0).getYDirAdj();
@@ -196,32 +190,30 @@ public class TextPositionSequence implements CharSequence {
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+    public float getTextHeightNoPadding() {
+
+        return textPositions.get(0).getHeightDir();
+    }
+
+
     public float getTextHeight() {
 
         return textPositions.get(0).getHeightDir() + HEIGHT_PADDING;
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public float getHeight() {
 
         return getMaxYDirAdj() - getMinYDirAdj();
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public float getWidth() {
 
         return getMaxXDirAdj() - getMinXDirAdj();
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public String getFont() {
 
         if (textPositions.get(0).getFontName() == null) {
@@ -231,8 +223,6 @@ public class TextPositionSequence implements CharSequence {
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public String getFontStyle() {
 
         if (textPositions.get(0).getFontName() == null) {
@@ -252,16 +242,12 @@ public class TextPositionSequence implements CharSequence {
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public float getFontSize() {
 
         return textPositions.get(0).getFontSizeInPt();
     }
 
 
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
     public float getSpaceWidth() {
 
         return textPositions.get(0).getWidthOfSpace();
@@ -277,8 +263,7 @@ public class TextPositionSequence implements CharSequence {
      *
      * @return bounding box of the word in Pdf Coordinate System
      */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
     @SneakyThrows
     public Rectangle getRectangle() {
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/visualization/ColoredLine.java

@@ -1,8 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.model.visualization;
-
-import java.awt.Color;
-import java.awt.geom.Line2D;
-
-public record ColoredLine(Line2D line, Color color) {
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/model/visualization/PlacedText.java

@@ -1,7 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.model.visualization;
-
-import java.awt.geom.Point2D;
-
-public record PlacedText(String text, Point2D lineStart) {
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/adapter/ImageServiceResponseAdapter.java

@@ -9,10 +9,10 @@ import java.util.Map;
 
 import org.springframework.stereotype.Service;
 
-import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.ImageType;
-import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.image.ImageServiceResponse;
+import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
+import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.ImageType;
 
 import lombok.RequiredArgsConstructor;
 
@@ -20,7 +20,8 @@ import lombok.RequiredArgsConstructor;
 @RequiredArgsConstructor
 public class ImageServiceResponseAdapter {
 
-    public Map<Integer, List<ClassifiedImage>> buildClassifiedImagesPerPage(ImageServiceResponse imageServiceResponse) {
+
+    public Map<Integer, List<ClassifiedImage>> buildClassifiedImagesPerPage(ImageServiceResponse imageServiceResponse ) {
 
         Map<Integer, List<ClassifiedImage>> images = new HashMap<>();
         imageServiceResponse.getData().forEach(imageMetadata -> {

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/adapter/VisualLayoutParsingAdapter.java

@@ -1,52 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.adapter;
-
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-import org.springframework.stereotype.Service;
-
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingBox;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResult;
-
-import lombok.RequiredArgsConstructor;
-
-@Service
-@RequiredArgsConstructor
-public class VisualLayoutParsingAdapter {
-
-    public Map<Integer, List<VisualLayoutParsingResult>> buildExtractedTablesPerPage(VisualLayoutParsingResponse visualLayoutParsingResponse) {
-
-        Map<Integer, List<VisualLayoutParsingResult>> tableCells = new HashMap<>();
-        visualLayoutParsingResponse.getData()
-                .forEach(tableData -> tableCells.computeIfAbsent(tableData.getPage_idx(), tableCell -> new ArrayList<>()).addAll(convertTableCells(tableData.getBoxes())));
-
-        return tableCells;
-
-    }
-
-
-    public List<VisualLayoutParsingResult> convertTableCells(List<VisualLayoutParsingBox> tableObjects) {
-
-        List<VisualLayoutParsingResult> parsedTableCells = new ArrayList<>();
-
-        tableObjects.stream().forEach(t -> {
-            VisualLayoutParsingResult result = new VisualLayoutParsingResult();
-            result.setX0(t.getBox().getX1());
-            result.setX1(t.getBox().getX2());
-            result.setY0(t.getBox().getY1());
-            result.setY1(t.getBox().getY2());
-            result.setWidth(result.getX1() - result.getX0());
-            result.setHeight(result.getY1() - result.getY0());
-            result.setLabel(t.getLabel());
-            parsedTableCells.add(result);
-        });
-
-        return parsedTableCells;
-
-    }
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/ExtractedTable.java

@@ -1,5 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-public class ExtractedTable {
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/VisualLayoutParsingBox.java

@@ -1,20 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-import java.util.List;
-
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-@Data
-@Builder
-@NoArgsConstructor
-@AllArgsConstructor
-public class VisualLayoutParsingBox {
-
-    private VisualLayoutParsingBoxValue box;
-    private String label;
-    private float probability;
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/VisualLayoutParsingBoxValue.java

@@ -1,19 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-@Data
-@Builder
-@NoArgsConstructor
-@AllArgsConstructor
-public class VisualLayoutParsingBoxValue {
-
-    private float x1;
-    private float y1;
-    private float x2;
-    private float y2;
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/VisualLayoutParsingData.java

@@ -1,20 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-import java.util.List;
-
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-@Data
-@Builder
-@NoArgsConstructor
-@AllArgsConstructor
-public class VisualLayoutParsingData {
-
-    private int page_idx;
-
-    private List<VisualLayoutParsingBox> boxes;
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/VisualLayoutParsingResponse.java

@@ -1,23 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-import java.util.List;
-
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-@Data
-@Builder
-@NoArgsConstructor
-@AllArgsConstructor
-public class VisualLayoutParsingResponse {
-
-    private String dossierId;
-    private String fileId;
-    private String targetFileExtension;
-    private String responseFileExtension;
-    private String X_TENANT_ID;
-    private List<VisualLayoutParsingData> data;
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/python_api/model/table/VisualLayoutParsingResult.java

@@ -1,22 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
-
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Data;
-import lombok.NoArgsConstructor;
-
-@Data
-@Builder
-@NoArgsConstructor
-@AllArgsConstructor
-public class VisualLayoutParsingResult {
-
-    private float x0;
-    private float y0;
-    private float x1;
-    private float y1;
-    private float width;
-    private float height;
-    private String label;
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/BodyTextFrameService.java

@@ -25,7 +25,6 @@ public class BodyTextFrameService {
     private static final float RULING_HEIGHT_THRESHOLD = 0.15f; // multiplied with page height. Header/Footer Rulings must be within that border of the page.
     private static final float RULING_WIDTH_THRESHOLD = 0.75f; // multiplied with page width. Header/Footer Rulings must be at least that wide.
 
-
     public void setBodyTextFrames(ClassificationDocument classificationDocument, LayoutParsingType layoutParsingType) {
 
         Rectangle bodyTextFrame = calculateBodyTextFrame(classificationDocument.getPages(), classificationDocument.getFontSizeCounter(), false, layoutParsingType);
@@ -156,9 +155,8 @@ public class BodyTextFrameService {
                         continue;
                     }
 
-                    if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER) || MarkedContentUtils.intersects(textBlock,
-                            page.getMarkedContentBboxPerType(),
-                            MarkedContentUtils.FOOTER)) {
+                    if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER)
+                            || MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER)) {
                         continue;
                     }
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/DividingColumnDetectionService.java

@@ -22,6 +22,7 @@ public class DividingColumnDetectionService {
 
     public List<Rectangle2D> detectColumns(PageContents pageContents) {
 
+
         if (pageContents.getSortedTextPositionSequences().size() < 2) {
             return List.of(pageContents.getCropBox());
         }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/GapDetectionService.java

@@ -72,13 +72,11 @@ public class GapDetectionService {
         return mirrorY(RectangleTransformations.toRectangle2D(textPosition.getRectangle()));
     }
 
-
     private static Rectangle2D mirrorY(Rectangle2D rectangle2D) {
 
         return new Rectangle2D.Double(rectangle2D.getX(), Math.min(rectangle2D.getMinY(), rectangle2D.getMaxY()), rectangle2D.getWidth(), Math.abs(rectangle2D.getHeight()));
     }
 
-
     private static void addGapToLine(Rectangle2D currentTextPosition, Rectangle2D previousTextPosition, XGapsContext context) {
 
         context.gapsInCurrentLine.add(new Rectangle2D.Double(previousTextPosition.getMaxX(),

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/GapsAcrossLinesService.java

@@ -6,6 +6,7 @@ import java.util.LinkedList;
 import java.util.List;
 import java.util.Queue;
 import java.util.stream.Stream;
+import com.iqser.red.commons.jackson.ObjectMapperFactory;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.GapInformation;
 
@@ -50,9 +51,7 @@ public class GapsAcrossLinesService {
         }
 
         return columnFactory.outputGaps.stream()
-                .filter(gapAcrossLines -> columnFactory.outputGaps.stream()
-                        .filter(gapAcrossLines::intersectsX)
-                        .noneMatch(gapAcrossLines1 -> gapAcrossLines1.lineCount > gapAcrossLines.lineCount))
+                .filter(gapAcrossLines -> columnFactory.outputGaps.stream().filter(gapAcrossLines::intersectsX).noneMatch(gapAcrossLines1 -> gapAcrossLines1.lineCount > gapAcrossLines.lineCount))
                 .filter(gapAcrossLines -> Math.abs(gapAcrossLines.rectangle2D.getMinX() - mainBodyTextFrame.getMinX()) > DISTANCE_TO_BORDER_THRESHOLD)
                 .filter(gapAcrossLines -> Math.abs(gapAcrossLines.rectangle2D.getMaxX() - mainBodyTextFrame.getMaxX()) > DISTANCE_TO_BORDER_THRESHOLD)
                 .map(GapAcrossLines::getRectangle2D)

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/LineDetectionService.java

@@ -6,8 +6,8 @@ import java.util.List;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.GapInformation;
 import com.knecon.fforesight.service.layoutparser.processor.model.LineInformation;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RectangleTransformations;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 
 import lombok.AllArgsConstructor;
 import lombok.Getter;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/MainBodyTextFrameExtractionService.java

@@ -16,7 +16,8 @@ public class MainBodyTextFrameExtractionService {
 
     public Rectangle2D calculateMainBodyTextFrame(LineInformation lineInformation) {
 
-        Rectangle2D mainBodyTextFrame = lineInformation.getLineBBox().stream().collect(RectangleTransformations.collectBBox());
+        Rectangle2D mainBodyTextFrame = lineInformation.getLineBBox().stream()
+                .collect(RectangleTransformations.collectBBox());
 
         return RectangleTransformations.pad(mainBodyTextFrame, mainBodyTextFrame.getWidth() * TEXT_FRAME_PAD_WIDTH, mainBodyTextFrame.getHeight() * TEXT_FRAME_PAD_HEIGHT);
     }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/PageContentExtractor.java

@@ -52,7 +52,7 @@ public class PageContentExtractor {
                         stripper.getRulings()));
             }
         }
-
+    
         return textPositionSequencesPerPage;
     }
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/RulingCleaningService.java

@@ -1,21 +1,21 @@
 package com.knecon.fforesight.service.layoutparser.processor.services;
 
-import java.awt.geom.Line2D;
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.X_FIRST_RULING_COMPARATOR;
+
 import java.awt.geom.Point2D;
 import java.util.ArrayList;
-import java.util.Collection;
-import java.util.Collections;
-import java.util.Comparator;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.stream.Collectors;
 
 import org.springframework.stereotype.Service;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Rectangle;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
-import com.knecon.fforesight.service.layoutparser.processor.utils.DoubleComparisons;
+import com.knecon.fforesight.service.layoutparser.processor.utils.UnionFind;
 
 import lombok.RequiredArgsConstructor;
 import lombok.extern.slf4j.Slf4j;
@@ -25,25 +25,145 @@ import lombok.extern.slf4j.Slf4j;
 @RequiredArgsConstructor
 public class RulingCleaningService {
 
-    private static final float THRESHOLD = 6;
+    private static final float THRESHOLD_X_VERTICAL = 1;
+    private static final float THRESHOLD_Y_VERTICAL = 2;
+    private static final float THRESHOLD_X_HORIZONTAL = 2;
+    private static final float THRESHOLD_Y_HORIZONTAL = 3;
 
 
     public CleanRulings getCleanRulings(List<TableCells> tableCells, List<Ruling> rulings) {
 
+        Rulings verticalAndHorizontalRulingLines;
+
         if (!rulings.isEmpty()) {
-            snapPoints(rulings);
+            verticalAndHorizontalRulingLines = extractVerticalAndHorizontalRulingLines(rulings);
+        } else {
+            verticalAndHorizontalRulingLines = getRulingsFromParsedCells(tableCells);
         }
 
+        verticalAndHorizontalRulingLines.verticalLines.sort(X_FIRST_RULING_COMPARATOR);
+        verticalAndHorizontalRulingLines.horizontalLines.sort(X_FIRST_RULING_COMPARATOR);
+        verticalAndHorizontalRulingLines = cleanRulings(verticalAndHorizontalRulingLines);
+
+        return CleanRulings.builder().vertical(verticalAndHorizontalRulingLines.verticalLines()).horizontal(verticalAndHorizontalRulingLines.horizontalLines()).build();
+    }
+
+
+    private Rulings cleanRulings(Rulings rulings) {
+
+        List<List<Rectangle>> groupedOverlappingVerticalRectangles = groupOverlappingRectangles(rulings.verticalLines.stream()
+                                                                                                        .map(RulingCleaningService::getOverlapRectangle)
+                                                                                                        .distinct()
+                                                                                                        .toList());
+        List<Ruling> cleanedVerticalRulings = groupedOverlappingVerticalRectangles.stream()
+                .map(rectList -> getXCenteredRuling(Rectangle.boundingBoxOf(rectList)))
+                .toList();
+
+        List<List<Rectangle>> groupedOverlappingHorizontalRectangles = groupOverlappingRectangles(rulings.horizontalLines.stream()
+                                                                                                          .map(RulingCleaningService::getOverlapRectangle)
+                                                                                                          .distinct()
+                                                                                                          .toList());
+
+        List<Ruling> cleanedHorizontalRulings = groupedOverlappingHorizontalRectangles.stream()
+                .map(rectList -> getYCenteredRuling(Rectangle.boundingBoxOf(rectList)))
+                .collect(Collectors.toList());
+
+        return new Rulings(cleanedVerticalRulings, cleanedHorizontalRulings);
+    }
+
+
+    private List<List<Rectangle>> groupOverlappingRectangles(List<Rectangle> rectangles) {
+
+        UnionFind<Rectangle> unionFind = new UnionFind<>();
+        for (int i = 0; i < rectangles.size(); i++) {
+            for (int j = i + 1; j < rectangles.size(); j++) {
+                Rectangle rectangle1 = rectangles.get(i);
+                Rectangle rectangle2 = rectangles.get(j);
+
+                // we can stop early when we are too far off because of x-y-sorting
+                if(rectangle1.getRight() < rectangle2.getLeft() && rectangle1.getBottom() < rectangle2.getTop()) {
+                    break;
+                }
+
+                if (rectangle1.intersects(rectangle2)) {
+                    unionFind.union(rectangle1, rectangle2);
+                }
+            }
+        }
+
+        Map<Rectangle, List<Rectangle>> groups = new HashMap<>();
+        for (Rectangle rectangle : rectangles) {
+            Rectangle root = unionFind.find(rectangle);
+            groups.computeIfAbsent(root, k -> new ArrayList<>()).add(rectangle);
+        }
+        return new ArrayList<>(groups.values());
+    }
+
+
+    private static Rectangle getOverlapRectangle(Ruling ruling) {
+
+        float top;
+        float left;
+        float w;
+        float h;
+
+        if (ruling.x1 < ruling.x2) {
+            left = ruling.x1;
+            w = ruling.x2 - ruling.x1;
+        } else {
+            left = ruling.x2;
+            w = ruling.x1 - ruling.x2;
+        }
+        if (ruling.y1 < ruling.y2) {
+            top = ruling.y1;
+            h = ruling.y2 - ruling.y1;
+        } else {
+            top = ruling.y2;
+            h = ruling.y1 - ruling.y2;
+        }
+
+        if (ruling.horizontal()) {
+            return new Rectangle(top - THRESHOLD_Y_HORIZONTAL, left - THRESHOLD_X_HORIZONTAL, w + 2 * THRESHOLD_X_HORIZONTAL, h + 2 * THRESHOLD_Y_HORIZONTAL);
+        } else {
+            return new Rectangle(top - THRESHOLD_Y_VERTICAL, left - THRESHOLD_X_VERTICAL, w + 2 * THRESHOLD_X_VERTICAL, h + 2 * THRESHOLD_Y_VERTICAL);
+        }
+    }
+
+
+    public static Ruling getXCenteredRuling(Rectangle rectangle) {
+
+        float x = (float) rectangle.getCenterX();
+        float y1 = rectangle.getTop();
+        float y2 = rectangle.getBottom();
+
+        Point2D point1 = new Point2D.Float(x, y1 + THRESHOLD_Y_VERTICAL);
+        Point2D point2 = new Point2D.Float(x, y2 - THRESHOLD_Y_VERTICAL);
+
+        return new Ruling(point1, point2);
+    }
+
+
+    public static Ruling getYCenteredRuling(Rectangle rectangle) {
+
+        float x1 = rectangle.getLeft();
+        float x2 = rectangle.getRight();
+        float y = (float) rectangle.getCenterY();
+
+        Point2D point1 = new Point2D.Float(x1 + THRESHOLD_X_HORIZONTAL, y);
+        Point2D point2 = new Point2D.Float(x2 - THRESHOLD_X_HORIZONTAL, y);
+
+        return new Ruling(point1, point2);
+    }
+
+
+    private Rulings extractVerticalAndHorizontalRulingLines(List<Ruling> rulings) {
+
         List<Ruling> vrs = new ArrayList<>();
         for (Ruling vr : rulings) {
             if (vr.vertical()) {
                 vrs.add(vr);
             }
         }
-        if (vrs.isEmpty()) {
-            vrs.addAll(extractVerticalRulings(tableCells));
-        }
-        List<Ruling> verticalRulingLines = collapseOrientedRulings(vrs);
 
         List<Ruling> hrs = new ArrayList<>();
         for (Ruling hr : rulings) {
@@ -51,98 +171,26 @@ public class RulingCleaningService {
                 hrs.add(hr);
             }
         }
-        if (hrs.isEmpty()) {
-            hrs.addAll(extractHorizontalRulings(tableCells));
-        }
-        List<Ruling> horizontalRulingLines = collapseOrientedRulings(hrs);
-
-        return CleanRulings.builder().vertical(verticalRulingLines).horizontal(horizontalRulingLines).build();
+        return new Rulings(vrs, hrs);
     }
 
 
-    public void snapPoints(List<? extends Line2D.Float> rulings) {
+    private Rulings getRulingsFromParsedCells(List<TableCells> tableCells) {
 
-        // collect points and keep a Line -> p1,p2 map
-        Map<Line2D.Float, Point2D[]> linesToPoints = new HashMap<>();
-        List<Point2D> points = new ArrayList<>();
-        for (Line2D.Float r : rulings) {
-            Point2D p1 = r.getP1();
-            Point2D p2 = r.getP2();
-            linesToPoints.put(r, new Point2D[]{p1, p2});
-            points.add(p1);
-            points.add(p2);
-        }
-
-        // snap by X
-        points.sort(Comparator.comparingDouble(Point2D::getX));
-
-        List<List<Point2D>> groupedPoints = new ArrayList<>();
-        groupedPoints.add(new ArrayList<>(Collections.singletonList(points.get(0))));
-
-        for (Point2D p : points.subList(1, points.size() - 1)) {
-            List<Point2D> last = groupedPoints.get(groupedPoints.size() - 1);
-            if (Math.abs(p.getX() - last.get(0).getX()) < THRESHOLD) {
-                groupedPoints.get(groupedPoints.size() - 1).add(p);
-            } else {
-                groupedPoints.add(new ArrayList<>(Collections.singletonList(p)));
-            }
-        }
-
-        for (List<Point2D> group : groupedPoints) {
-            float avgLoc = 0;
-            for (Point2D p : group) {
-                avgLoc += p.getX();
-            }
-            avgLoc /= group.size();
-            for (Point2D p : group) {
-                p.setLocation(avgLoc, p.getY());
-            }
-        }
-        // ---
-
-        // snap by Y
-        points.sort(Comparator.comparingDouble(Point2D::getY));
-
-        groupedPoints = new ArrayList<>();
-        groupedPoints.add(new ArrayList<>(Collections.singletonList(points.get(0))));
-
-        for (Point2D p : points.subList(1, points.size() - 1)) {
-            List<Point2D> last = groupedPoints.get(groupedPoints.size() - 1);
-            if (Math.abs(p.getY() - last.get(0).getY()) < THRESHOLD) {
-                groupedPoints.get(groupedPoints.size() - 1).add(p);
-            } else {
-                groupedPoints.add(new ArrayList<>(Collections.singletonList(p)));
-            }
-        }
-
-        for (List<Point2D> group : groupedPoints) {
-            float avgLoc = 0;
-            for (Point2D p : group) {
-                avgLoc += p.getY();
-            }
-            avgLoc /= group.size();
-            for (Point2D p : group) {
-                p.setLocation(p.getX(), avgLoc);
-            }
-        }
-        // ---
-
-        // finally, modify lines
-        for (Map.Entry<Line2D.Float, Point2D[]> ltp : linesToPoints.entrySet()) {
-            Point2D[] p = ltp.getValue();
-            ltp.getKey().setLine(p[0], p[1]);
-        }
+        List<Ruling> vrs = extractVerticalRulingsFromParsedCells(tableCells);
+        List<Ruling> hrs = extractHorizontalRulingsFromParsedCells(tableCells);
+        return new Rulings(vrs, hrs);
     }
 
 
-    private Collection<? extends Ruling> extractVerticalRulings(List<TableCells> cvParsedTableCells) {
+    private List<Ruling> extractVerticalRulingsFromParsedCells(List<TableCells> tableCells) {
 
         List<Ruling> vrs = new ArrayList<>();
 
-        if (cvParsedTableCells != null) {
-            for (TableCells cvParsedTableCell : cvParsedTableCells) {
-                Ruling leftLine = createRuling(cvParsedTableCell.getX0(), cvParsedTableCell.getX0(), cvParsedTableCell.getY0(), cvParsedTableCell.getY1());
-                Ruling rightLine = createRuling(cvParsedTableCell.getX1(), cvParsedTableCell.getX1(), cvParsedTableCell.getY0(), cvParsedTableCell.getY1());
+        if (tableCells != null) {
+            for (TableCells tableCell : tableCells) {
+                Ruling leftLine = createRuling(tableCell.getX0(), tableCell.getX0(), tableCell.getY0(), tableCell.getY1());
+                Ruling rightLine = createRuling(tableCell.getX1(), tableCell.getX1(), tableCell.getY0(), tableCell.getY1());
                 vrs.add(leftLine);
                 vrs.add(rightLine);
             }
@@ -151,19 +199,18 @@ public class RulingCleaningService {
     }
 
 
-    private Collection<? extends Ruling> extractHorizontalRulings(List<TableCells> cvParsedTableCells) {
+    private List<Ruling> extractHorizontalRulingsFromParsedCells(List<TableCells> tableCells) {
 
         List<Ruling> hrs = new ArrayList<>();
 
-        if (cvParsedTableCells != null) {
-            for (TableCells cvParsedTableCell : cvParsedTableCells) {
-                Ruling topLine = createRuling(cvParsedTableCell.getX0(), cvParsedTableCell.getX1(), cvParsedTableCell.getY1(), cvParsedTableCell.getY1());
-                Ruling baseLine = createRuling(cvParsedTableCell.getX0(), cvParsedTableCell.getX1(), cvParsedTableCell.getY0(), cvParsedTableCell.getY0());
+        if (tableCells != null) {
+            for (TableCells tableCell : tableCells) {
+                Ruling topLine = createRuling(tableCell.getX0(), tableCell.getX1(), tableCell.getY1(), tableCell.getY1());
+                Ruling baseLine = createRuling(tableCell.getX0(), tableCell.getX1(), tableCell.getY0(), tableCell.getY0());
                 hrs.add(topLine);
                 hrs.add(baseLine);
             }
         }
-
         return hrs;
     }
 
@@ -189,46 +236,8 @@ public class RulingCleaningService {
     }
 
 
-    private List<Ruling> collapseOrientedRulings(List<Ruling> lines) {
+    private record Rulings(List<Ruling> verticalLines, List<Ruling> horizontalLines) {
 
-        int COLINEAR_OR_PARALLEL_PIXEL_EXPAND_AMOUNT = 1;
-        return collapseOrientedRulings(lines, COLINEAR_OR_PARALLEL_PIXEL_EXPAND_AMOUNT);
-    }
-
-
-    private List<Ruling> collapseOrientedRulings(List<Ruling> lines, int expandAmount) {
-
-        ArrayList<Ruling> rv = new ArrayList<>();
-        lines.sort((a, b) -> {
-            final float diff = a.getPosition() - b.getPosition();
-            return Float.compare(diff == 0 ? a.getStart() - b.getStart() : diff, 0f);
-        });
-
-        for (Ruling next_line : lines) {
-            Ruling last = rv.isEmpty() ? null : rv.get(rv.size() - 1);
-            // if current line colinear with next, and are "close enough": expand current line
-            if (last != null && DoubleComparisons.feq(next_line.getPosition(), last.getPosition()) && last.nearlyIntersects(next_line, expandAmount)) {
-                final float lastStart = last.getStart();
-                final float lastEnd = last.getEnd();
-
-                final boolean lastFlipped = lastStart > lastEnd;
-                final boolean nextFlipped = next_line.getStart() > next_line.getEnd();
-
-                boolean differentDirections = nextFlipped != lastFlipped;
-                float nextS = differentDirections ? next_line.getEnd() : next_line.getStart();
-                float nextE = differentDirections ? next_line.getStart() : next_line.getEnd();
-
-                final float newStart = lastFlipped ? Math.max(nextS, lastStart) : Math.min(nextS, lastStart);
-                final float newEnd = lastFlipped ? Math.min(nextE, lastEnd) : Math.max(nextE, lastEnd);
-                last.setStartEnd(newStart, newEnd);
-                assert !last.oblique();
-            } else if (next_line.length() == 0) {
-                continue;
-            } else {
-                rv.add(next_line);
-            }
-        }
-        return rv;
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/SectionsBuilderService.java

@@ -43,7 +43,6 @@ public class SectionsBuilderService {
         for (ClassificationPage page : document.getPages()) {
             List<TextPageBlock> header = new ArrayList<>();
             List<TextPageBlock> footer = new ArrayList<>();
-            List<TextPageBlock> unclassifiedText = new ArrayList<>();
             for (AbstractPageBlock current : page.getTextBlocks()) {
 
                 if (current.getClassification() == null) {
@@ -62,11 +61,6 @@ public class SectionsBuilderService {
                     continue;
                 }
 
-                if (current.getClassification().equals(PageBlockType.OTHER)) {
-                    unclassifiedText.add((TextPageBlock) current);
-                    continue;
-                }
-
                 if (prev != null && current.getClassification().isHeadline() && !prev.getClassification().isHeadline() || !document.isHeadlines()) {
                     ClassificationSection chunkBlock = buildTextBlock(chunkWords, lastHeadline);
                     chunkBlock.setHeadline(lastHeadline);
@@ -94,9 +88,6 @@ public class SectionsBuilderService {
             if (!footer.isEmpty()) {
                 footers.add(new ClassificationFooter(footer));
             }
-            if (!unclassifiedText.isEmpty()) {
-                unclassifiedTexts.add(new UnclassifiedText(unclassifiedText));
-            }
         }
 
         ClassificationSection chunkBlock = buildTextBlock(chunkWords, lastHeadline);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/SimplifiedSectionTextService.java

@@ -5,9 +5,9 @@ import java.util.List;
 import org.springframework.stereotype.Service;
 
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.SimplifiedSectionText;
-import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.SimplifiedText;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Section;
+import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.SimplifiedText;
 
 @Service
 public class SimplifiedSectionTextService {
@@ -23,5 +23,4 @@ public class SimplifiedSectionTextService {
 
         return SimplifiedSectionText.builder().sectionNumber(section.getTreeId().get(0)).text(section.getTextBlock().getSearchText()).build();
     }
-
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/TableExtractionService.java

@@ -1,14 +1,13 @@
 package com.knecon.fforesight.service.layoutparser.processor.services;
 
-import java.awt.geom.Point2D;
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.CELL_SIZE_COMPARATOR;
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.RECTANGLE_SIZE_COMPARATOR;
+
 import java.util.ArrayList;
-import java.util.Comparator;
-import java.util.HashMap;
 import java.util.HashSet;
-import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
-import java.util.Set;
+import java.util.stream.Collectors;
 
 import org.springframework.stereotype.Service;
 
@@ -21,59 +20,15 @@ import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.utils.DoubleComparisons;
+import com.knecon.fforesight.service.layoutparser.processor.utils.RectangularIntersectionFinder;
+import com.knecon.fforesight.service.layoutparser.processor.utils.SpreadsheetFinder;
 
 @Service
 public class TableExtractionService {
 
-    private static final Comparator<Point2D> X_FIRST_POINT_COMPARATOR = (arg0, arg1) -> {
-
-        int rv = 0;
-        float arg0X = DoubleComparisons.round(arg0.getX(), 2);
-        float arg0Y = DoubleComparisons.round(arg0.getY(), 2);
-        float arg1X = DoubleComparisons.round(arg1.getX(), 2);
-        float arg1Y = DoubleComparisons.round(arg1.getY(), 2);
-
-        if (arg0X > arg1X) {
-            rv = 1;
-        } else if (arg0X < arg1X) {
-            rv = -1;
-        } else if (arg0Y > arg1Y) {
-            rv = 1;
-        } else if (arg0Y < arg1Y) {
-            rv = -1;
-        }
-        return rv;
-    };
-    private static final Comparator<Point2D> POINT_COMPARATOR = (arg0, arg1) -> {
-
-        int rv = 0;
-        float arg0X = DoubleComparisons.round(arg0.getX(), 2);
-        float arg0Y = DoubleComparisons.round(arg0.getY(), 2);
-        float arg1X = DoubleComparisons.round(arg1.getX(), 2);
-        float arg1Y = DoubleComparisons.round(arg1.getY(), 2);
-
-        if (arg0Y > arg1Y) {
-            rv = 1;
-        } else if (arg0Y < arg1Y) {
-            rv = -1;
-        } else if (arg0X > arg1X) {
-            rv = 1;
-        } else if (arg0X < arg1X) {
-            rv = -1;
-        }
-        return rv;
-    };
-
-
-    public boolean contains(Cell cell, double x, double y, double w, double h) {
-
-        if (cell.isEmpty() || w <= 0 || h <= 0) {
-            return false;
-        }
-        double x0 = cell.getX();
-        double y0 = cell.getY();
-        return (x >= x0 - 2 && y >= y0 - 2 && (x + w) <= x0 + cell.getWidth() + 2 && (y + h) <= y0 + cell.getHeight() + 2);
-    }
+    private static final int MAX_TABLE_CONTAINED_CELLS_WITH_TEXT = 1;
+    private static final int TEXT_BLOCK_CONTAINMENT_TOLERANCE = 2;
+    private static final double TABLE_UNIFORMITY_THRESHOLD = 0.7;
 
 
     /**
@@ -89,22 +44,18 @@ public class TableExtractionService {
      * @param cleanRulings The lines used to build the table.
      * @param page         Page object that contains textblocks and statistics.
      */
+
     public void extractTables(CleanRulings cleanRulings, ClassificationPage page) {
 
         List<Cell> cells = findCells(cleanRulings.getHorizontal(), cleanRulings.getVertical());
-
-        List<TextPageBlock> toBeRemoved = new ArrayList<>();
+        // sort cells by size (height * width) ascending so that textBlocks are always assigned to the smallest cells that contain them
+        cells.sort(CELL_SIZE_COMPARATOR);
 
         for (AbstractPageBlock abstractPageBlock : page.getTextBlocks()) {
             TextPageBlock textBlock = (TextPageBlock) abstractPageBlock;
             for (Cell cell : cells) {
-                if (cell.hasMinimumSize() && contains(cell,
-                        textBlock.getPdfMinX(),
-                        textBlock.getPdfMinY(),
-                        textBlock.getPdfMaxX() - textBlock.getPdfMinX(),
-                        textBlock.getPdfMaxY() - textBlock.getPdfMinY())) {
+                if (cell.hasMinimumSize() && doesCellContainTextBlock(cell, textBlock)) {
                     cell.addTextBlock(textBlock);
-                    toBeRemoved.add(textBlock);
                     break;
                 }
             }
@@ -113,245 +64,94 @@ public class TableExtractionService {
         cells = new ArrayList<>(new HashSet<>(cells));
         DoubleComparisons.sort(cells, Rectangle.ILL_DEFINED_ORDER);
 
-        List<Rectangle> spreadsheetAreas = findSpreadsheetsFromCells(cells);
+        List<Rectangle> spreadsheetAreas = SpreadsheetFinder.findSpreadsheetsFromCells(cells);
+        // sort spreadsheetAreas by size (height * width) ascending so that cells are placed in the smallest tables first
+        // this way no cell duplication occurs when tables are contained in other tables and only the most inner table contains the cells
+        spreadsheetAreas.sort(RECTANGLE_SIZE_COMPARATOR);
 
         List<TablePageBlock> tables = new ArrayList<>();
         for (Rectangle area : spreadsheetAreas) {
 
-            List<Cell> overlappingCells = new ArrayList<>();
+            List<Cell> containedCells = new ArrayList<>();
             for (Cell c : cells) {
-                if (c.hasMinimumSize() && c.intersects(area)) {
-                    overlappingCells.add(c);
+                if (c.hasMinimumSize() && area.contains(c)) {
+                    containedCells.add(c);
                 }
             }
-            tables.add(new TablePageBlock(overlappingCells, area, page.getRotation()));
+
+            var containedCellsWithText = containedCells.stream()
+                    .filter(cell -> !cell.getTextBlocks().isEmpty())
+                    .toList();
+
+            // verify if table would contain fewer cells with text than the threshold allows
+            if (containedCellsWithText.size() >= MAX_TABLE_CONTAINED_CELLS_WITH_TEXT && checkIfTableCellsAreUniform(containedCells)) {
+                tables.add(new TablePageBlock(containedCells, area, page.getRotation()));
+                cells.removeAll(containedCells);
+            }
         }
 
         for (TablePageBlock table : tables) {
             int position = -1;
 
-            Iterator<AbstractPageBlock> itty = page.getTextBlocks().iterator();
-            while (itty.hasNext()) {
-                AbstractPageBlock textBlock = itty.next();
-                if (textBlock instanceof TextPageBlock ? table.containsBlock((TextPageBlock) textBlock) : table.contains(textBlock) && position == -1) {
-                    position = page.getTextBlocks().indexOf(textBlock);
+            for (AbstractPageBlock pageBlock : page.getTextBlocks()) {
+                if (pageBlock instanceof TextPageBlock ? table.containsBlock((TextPageBlock) pageBlock) : table.contains(pageBlock) && position == -1) {
+                    position = page.getTextBlocks().indexOf(pageBlock);
                 }
             }
             if (position != -1) {
                 page.getTextBlocks().add(position, table);
+
+                var toBeRemoved = table.getCells()
+                        .stream()
+                        .map(Cell::getTextBlocks)
+                        .flatMap(List::stream)
+                        .toList();
+                // remove text blocks from the page that were also added with the table (from its contained cells)
+                page.getTextBlocks().removeAll(toBeRemoved);
             }
         }
-
-        page.getTextBlocks().removeAll(toBeRemoved);
     }
 
 
-    public List<Cell> findCells(List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
+    private boolean checkIfTableCellsAreUniform(List<Cell> containedCells) {
 
-        // Fix for 211.pdf
-        for (Ruling r : horizontalRulingLines) {
-            if (r.getX2() < r.getX1()) {
-                double a = r.getX2();
-                r.x2 = (float) r.getX1();
-                r.x1 = (float) a;
-            }
+        if(containedCells.size() <= 2) {
+            return true;
         }
 
-        List<Cell> cellsFound = new ArrayList<>();
-        Map<Point2D, Ruling[]> intersectionPoints = Ruling.findIntersections(horizontalRulingLines, verticalRulingLines);
-        List<Point2D> intersectionPointsList = new ArrayList<>(intersectionPoints.keySet());
-        intersectionPointsList.sort(POINT_COMPARATOR);
+        Map<Long, List<Long>> cellsGroupedByRoundedWidth = containedCells.stream()
+                .map(Rectangle::getWidth)
+                .map(size -> Math.round(size / 10.0) * 10)
+                .collect(Collectors.groupingBy(Long::longValue));
 
-        for (int i = 0; i < intersectionPointsList.size(); i++) {
-            Point2D topLeft = intersectionPointsList.get(i);
-            Ruling[] hv = intersectionPoints.get(topLeft);
-
-            // CrossingPointsDirectlyBelow( topLeft );
-            List<Point2D> xPoints = new ArrayList<>();
-            // CrossingPointsDirectlyToTheRight( topLeft );
-            List<Point2D> yPoints = new ArrayList<>();
-
-            for (Point2D p : intersectionPointsList.subList(i, intersectionPointsList.size())) {
-                if (p.getX() == topLeft.getX() && p.getY() > topLeft.getY()) {
-                    xPoints.add(p);
-                }
-                if (p.getY() == topLeft.getY() && p.getX() > topLeft.getX()) {
-                    yPoints.add(p);
-                }
-            }
-            outer:
-            for (Point2D xPoint : xPoints) {
-                // is there a vertical edge b/w topLeft and xPoint?
-                if (!hv[1].equals(intersectionPoints.get(xPoint)[1])) {
-                    continue;
-                }
-                for (Point2D yPoint : yPoints) {
-                    // is there an horizontal edge b/w topLeft and yPoint ?
-                    if (!hv[0].equals(intersectionPoints.get(yPoint)[0])) {
-                        continue;
-                    }
-                    Point2D btmRight = new Point2D.Float((float) yPoint.getX(), (float) xPoint.getY());
-                    if (intersectionPoints.containsKey(btmRight) && intersectionPoints.get(btmRight)[0].equals(intersectionPoints.get(xPoint)[0]) && intersectionPoints.get(btmRight)[1].equals(
-                            intersectionPoints.get(yPoint)[1])) {
-                        cellsFound.add(new Cell(topLeft, btmRight));
-                        break outer;
-                    }
-                }
-            }
-        }
-
-        // TODO create cells for vertical ruling lines with aligned endpoints at the top/bottom of a grid
-        // that aren't connected with an horizontal ruler?
-        // see: https://github.com/jazzido/tabula-extractor/issues/78#issuecomment-41481207
-
-        return cellsFound;
+        return (double) cellsGroupedByRoundedWidth.size() / containedCells.size() <= TABLE_UNIFORMITY_THRESHOLD;
     }
 
 
-    private List<Rectangle> findSpreadsheetsFromCells(List<? extends Rectangle> cells) {
-        // via: http://stackoverflow.com/questions/13746284/merging-multiple-adjacent-rectangles-into-one-polygon
-        List<Rectangle> rectangles = new ArrayList<>();
-        Set<Point2D> pointSet = new HashSet<>();
-        Map<Point2D, Point2D> edgesH = new HashMap<>();
-        Map<Point2D, Point2D> edgesV = new HashMap<>();
-        int i = 0;
+    private boolean doesCellContainTextBlock(Cell cell, TextPageBlock textBlock) {
 
-        for (Rectangle cell : cells) {
-            for (Point2D pt : cell.getPoints()) {
-                if (pointSet.contains(pt)) { // shared vertex, remove it
-                    pointSet.remove(pt);
-                } else {
-                    pointSet.add(pt);
-                }
-            }
+        double x = textBlock.getPdfMinX();
+        double y = textBlock.getPdfMinY();
+        double w = textBlock.getPdfMaxX() - textBlock.getPdfMinX();
+        double h = textBlock.getPdfMaxY() - textBlock.getPdfMinY();
+        if (cell.isEmpty() || w <= 0 || h <= 0) {
+            return false;
         }
-
-        // X first sort
-        List<Point2D> pointsSortX = new ArrayList<>(pointSet);
-        pointsSortX.sort(X_FIRST_POINT_COMPARATOR);
-        // Y first sort
-        List<Point2D> pointsSortY = new ArrayList<>(pointSet);
-        pointsSortY.sort(POINT_COMPARATOR);
-
-        while (i < pointSet.size()) {
-            float currY = (float) pointsSortY.get(i).getY();
-            while (i < pointSet.size() && DoubleComparisons.feq(pointsSortY.get(i).getY(), currY)) {
-                edgesH.put(pointsSortY.get(i), pointsSortY.get(i + 1));
-                edgesH.put(pointsSortY.get(i + 1), pointsSortY.get(i));
-                i += 2;
-            }
-        }
-
-        i = 0;
-        while (i < pointSet.size()) {
-            float currX = (float) pointsSortX.get(i).getX();
-            while (i < pointSet.size() && DoubleComparisons.feq(pointsSortX.get(i).getX(), currX)) {
-                edgesV.put(pointsSortX.get(i), pointsSortX.get(i + 1));
-                edgesV.put(pointsSortX.get(i + 1), pointsSortX.get(i));
-                i += 2;
-            }
-        }
-
-        // Get all the polygons
-        List<List<PolygonVertex>> polygons = new ArrayList<>();
-        Point2D nextVertex;
-        while (!edgesH.isEmpty()) {
-            ArrayList<PolygonVertex> polygon = new ArrayList<>();
-            Point2D first = edgesH.keySet().iterator().next();
-            polygon.add(new PolygonVertex(first, Direction.HORIZONTAL));
-            edgesH.remove(first);
-
-            while (true) {
-                PolygonVertex curr = polygon.get(polygon.size() - 1);
-                PolygonVertex lastAddedVertex;
-                if (curr.direction == Direction.HORIZONTAL) {
-                    nextVertex = edgesV.get(curr.point);
-                    edgesV.remove(curr.point);
-                    lastAddedVertex = new PolygonVertex(nextVertex, Direction.VERTICAL);
-                } else {
-                    nextVertex = edgesH.get(curr.point);
-                    edgesH.remove(curr.point);
-                    lastAddedVertex = new PolygonVertex(nextVertex, Direction.HORIZONTAL);
-                }
-                polygon.add(lastAddedVertex);
-
-                if (lastAddedVertex.equals(polygon.get(0))) {
-                    // closed polygon
-                    polygon.remove(polygon.size() - 1);
-                    break;
-                }
-            }
-
-            for (PolygonVertex vertex : polygon) {
-                edgesH.remove(vertex.point);
-                edgesV.remove(vertex.point);
-            }
-            polygons.add(polygon);
-        }
-
-        // calculate grid-aligned minimum area rectangles for each found polygon
-        for (List<PolygonVertex> poly : polygons) {
-            float top = Float.MAX_VALUE;
-            float left = Float.MAX_VALUE;
-            float bottom = Float.MIN_VALUE;
-            float right = Float.MIN_VALUE;
-            for (PolygonVertex pt : poly) {
-                top = (float) Math.min(top, pt.point.getY());
-                left = (float) Math.min(left, pt.point.getX());
-                bottom = (float) Math.max(bottom, pt.point.getY());
-                right = (float) Math.max(right, pt.point.getX());
-            }
-            rectangles.add(new Rectangle(top, left, right - left, bottom - top));
-        }
-
-        return rectangles;
+        double x0 = cell.getX();
+        double y0 = cell.getY();
+        return (x >= x0 - TEXT_BLOCK_CONTAINMENT_TOLERANCE
+                && y >= y0 - TEXT_BLOCK_CONTAINMENT_TOLERANCE
+                && (x + w) <= x0 + cell.getWidth() + 2 * TEXT_BLOCK_CONTAINMENT_TOLERANCE
+                && (y + h) <= y0 + cell.getHeight() + 2 * TEXT_BLOCK_CONTAINMENT_TOLERANCE);
     }
 
 
-    private enum Direction {
-        HORIZONTAL,
-        VERTICAL
-    }
-
-    static class PolygonVertex {
-
-        Point2D point;
-        Direction direction;
-
-
-        PolygonVertex(Point2D point, Direction direction) {
-
-            this.direction = direction;
-            this.point = point;
-        }
-
-
-        @Override
-        public boolean equals(Object other) {
-
-            if (this == other) {
-                return true;
-            }
-            if (!(other instanceof PolygonVertex)) {
-                return false;
-            }
-            return this.point.equals(((PolygonVertex) other).point);
-        }
-
-
-        @Override
-        public int hashCode() {
-
-            return this.point.hashCode();
-        }
-
-
-        @Override
-        public String toString() {
-
-            return String.format("%s[point=%s,direction=%s]", this.getClass().getName(), this.point.toString(), this.direction.toString());
-        }
+    public static List<Cell> findCells(List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
 
+        return RectangularIntersectionFinder.find(horizontalRulingLines, verticalRulingLines)
+                .stream()
+                .map(Cell::new)
+                .collect(Collectors.toList());
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/blockification/DocuMineBlockificationService.java

@@ -61,8 +61,8 @@ public class DocuMineBlockificationService {
             boolean newLineAfterSplit = prev != null && word.getMinYDirAdj() != prev.getMinYDirAdj() && wasSplitted && splitX1 != word.getMinXDirAdj();
             boolean isSplitByRuling = isSplitByRuling(minX, minY, maxX, maxY, word, horizontalRulingLines, verticalRulingLines);
             boolean splitByDir = prev != null && !prev.getDir().equals(word.getDir());
-            boolean splitByOtherFontAndOtherY = prev != null && prev.getMaxYDirAdj() != word.getMaxYDirAdj() && (word.getFontStyle().contains("bold") && !prev.getFontStyle()
-                    .contains("bold") || prev.getFontStyle().contains("bold") && !word.getFontStyle().contains("bold"));
+            boolean splitByOtherFontAndOtherY = prev != null && Math.abs(prev.getMaxYDirAdj() - word.getMaxYDirAdj()) > word.getTextHeight() * 0.2 && (word.getFontStyle()
+                    .contains("bold") && !prev.getFontStyle().contains("bold") || prev.getFontStyle().contains("bold") && !word.getFontStyle().contains("bold"));
 
             Matcher matcher = pattern.matcher(chunkWords.stream().collect(Collectors.joining(" ")).toString());
             boolean startsOnSameX = Math.abs(minX - word.getMinXDirAdj()) < 5 && matcher.matches();

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/blockification/RedactManagerBlockificationService.java

@@ -2,6 +2,7 @@ package com.knecon.fforesight.service.layoutparser.processor.services.blockifica
 
 import static java.util.stream.Collectors.toSet;
 
+import java.awt.geom.Point2D;
 import java.util.ArrayList;
 import java.util.Comparator;
 import java.util.Iterator;
@@ -13,10 +14,13 @@ import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBl
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.FloatFrequencyCounter;
 import com.knecon.fforesight.service.layoutparser.processor.model.Orientation;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.StringFrequencyCounter;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+import com.knecon.fforesight.service.layoutparser.processor.utils.RectangleTransformations;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RulingTextDirAdjustUtil;
 
 @SuppressWarnings("all")
@@ -31,12 +35,12 @@ public class RedactManagerBlockificationService {
      * This method must use text direction adjusted postions (DirAdj). Where {0,0} is on the upper left. Never try to change this!
      * Rulings (TablePageBlock lines) must be adjusted to the text directions as well, when checking if a block is split by a ruling.
      *
-     * @param textPositions         The words of a page.
-     * @param horizontalRulingLines Horizontal table lines.
-     * @param verticalRulingLines   Vertical table lines.
+     * @param textPositions The words of a page.
      * @return Page object that contains the Textblock and text statistics.
      */
-    public ClassificationPage blockify(List<TextPositionSequence> textPositions, List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
+    public ClassificationPage blockify(List<TextPositionSequence> textPositions, List<Cell> emptyCells) {
+
+        CleanRulings usedRulings = RectangleTransformations.extractRulings(emptyCells);
 
         int indexOnPage = 0;
         List<TextPositionSequence> chunkWords = new ArrayList<>();
@@ -54,7 +58,7 @@ public class RedactManagerBlockificationService {
             boolean splitByX = prev != null && maxX + 50 < word.getMinXDirAdj() && prev.getMinYDirAdj() == word.getMinYDirAdj();
             boolean xIsBeforeFirstX = prev != null && word.getMinXDirAdj() < minX;
             boolean newLineAfterSplit = prev != null && word.getMinYDirAdj() != prev.getMinYDirAdj() && wasSplitted && splitX1 != word.getMinXDirAdj();
-            boolean isSplitByRuling = isSplitByRuling(minX, minY, maxX, maxY, word, horizontalRulingLines, verticalRulingLines);
+            boolean isSplitByRuling = isSplitByRuling(minX, minY, maxX, maxY, word, usedRulings.getHorizontal(), usedRulings.getVertical());
             boolean splitByDir = prev != null && !prev.getDir().equals(word.getDir());
 
             if (prev != null && (lineSeparation || startFromTop || splitByX || splitByDir || isSplitByRuling)) {

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/blockification/TaasBlockificationService.java

@@ -1,20 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.blockification;
 
+
 // TODO: figure out, why this fails the build
 // import static com.knecon.fforesight.service.layoutparser.processor.services.factory.SearchTextWithTextPositionFactory.HEIGHT_PADDING;
 
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Set;
-import java.util.regex.Matcher;
-import java.util.regex.Pattern;
-import java.util.stream.Stream;
-
-import org.springframework.stereotype.Service;
-
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.Orientation;
@@ -22,6 +11,12 @@ import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RulingTextDirAdjustUtil;
+import org.springframework.stereotype.Service;
+
+import java.util.*;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;
 
 @Service
 @SuppressWarnings("all")
@@ -88,13 +83,13 @@ public class TaasBlockificationService {
                 continue;
             }
 
+
             Matcher listIdentifierPattern = listIdentifier.matcher(currentTextBlock.getText());
             boolean isListIdentifier = listIdentifierPattern.find();
 
             boolean yGap = Math.abs(currentTextBlock.getPdfMaxY() - previousTextBlock.getPdfMinY()) < previousTextBlock.getMostPopularWordHeight() * Y_GAP_SPLIT_HEIGHT_MODIFIER;
 
-            boolean sameFont = previousTextBlock.getMostPopularWordFont()
-                    .equals(currentTextBlock.getMostPopularWordFont()) && previousTextBlock.getMostPopularWordFontSize() == currentTextBlock.getMostPopularWordFontSize();
+            boolean sameFont = previousTextBlock.getMostPopularWordFont().equals(currentTextBlock.getMostPopularWordFont()) && previousTextBlock.getMostPopularWordFontSize() == currentTextBlock.getMostPopularWordFontSize();
 //            boolean yGap = previousTextBlock != null && currentTextBlock.getMinYDirAdj() - maxY > Math.min(word.getHeight(), prev.getHeight()) * Y_GAP_SPLIT_HEIGHT_MODIFIER;
 
             boolean alignsXRight = Math.abs(currentTextBlock.getPdfMaxX() - previousTextBlock.getPdfMaxX()) < X_ALIGNMENT_THRESHOLD;
@@ -124,9 +119,8 @@ public class TaasBlockificationService {
             }
             alreadyMerged.add(textPageBlock);
             textBlocksToMerge.add(Stream.concat(Stream.of(textPageBlock),
-                    textPageBlocks.stream()
-                            .filter(textPageBlock2 -> textPageBlock.almostIntersects(textPageBlock2, INTERSECTS_Y_THRESHOLD, 0) && !alreadyMerged.contains(textPageBlock2))
-                            .peek(alreadyMerged::add)).toList());
+                            textPageBlocks.stream().filter(textPageBlock2 -> textPageBlock.almostIntersects(textPageBlock2, INTERSECTS_Y_THRESHOLD, 0) && !alreadyMerged.contains(textPageBlock2)).peek(alreadyMerged::add))
+                    .toList());
         }
         return textBlocksToMerge.stream().map(TextPageBlock::merge).toList();
     }
@@ -169,7 +163,8 @@ public class TaasBlockificationService {
         while (itty.hasNext()) {
             TextPageBlock block = (TextPageBlock) itty.next();
 
-            if (previous != null && previous.getOrientation().equals(Orientation.LEFT) && block.getOrientation().equals(Orientation.LEFT) && equalsWithThreshold(block.getMaxY(),
+            if (previous != null && previous.getOrientation().equals(Orientation.LEFT) && block.getOrientation().equals(Orientation.LEFT) && equalsWithThreshold(
+                    block.getMaxY(),
                     previous.getMaxY()) || previous != null && previous.getOrientation().equals(Orientation.LEFT) && block.getOrientation()
                     .equals(Orientation.RIGHT) && equalsWithThreshold(block.getMaxY(), previous.getMaxY())) {
                 previous.add(block);
@@ -194,6 +189,7 @@ public class TaasBlockificationService {
         TextPositionSequence prev = null;
         // TODO: make static final constant
 
+
         boolean wasSplitted = false;
         Float splitX1 = null;
         for (TextPositionSequence word : textPositions) {

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/classification/DocuMineClassificationService.java

@@ -23,7 +23,7 @@ import lombok.extern.slf4j.Slf4j;
 @RequiredArgsConstructor
 public class DocuMineClassificationService {
 
-    private static final Pattern pattern = Pattern.compile("^(\\d{1,2}\\.){1,3}\\d{1,2}\\.?\\s[0-9A-Za-z \\[\\]]{2,50}", Pattern.CASE_INSENSITIVE);
+    private static final Pattern pattern = Pattern.compile("^([1-9]\\d?\\.){1,3}\\d{1,2}\\.?\\s[0-9A-Za-z \\[\\]]{2,50}", Pattern.CASE_INSENSITIVE);
     private static final Pattern pattern2 = Pattern.compile("\\p{L}{3,}", Pattern.CASE_INSENSITIVE);
     private static final Pattern pattern3 = Pattern.compile("^(\\d{1,1}\\.){1,3}\\d{1,2}\\.?\\s[a-z]{1,2}\\/[a-z]{1,2}.*");
 
@@ -63,47 +63,55 @@ public class DocuMineClassificationService {
             textBlock.setClassification(PageBlockType.OTHER);
             return;
         }
-        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER) || PositionUtils.isOverBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
+        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER)
+            || PositionUtils.isOverBodyTextFrame(bodyTextFrame, textBlock, page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null
+                                                                                                   || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
                 .getMostPopular())) {
             textBlock.setClassification(PageBlockType.HEADER);
 
-        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER) || PositionUtils.isUnderBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
+        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER)
+                   || PositionUtils.isUnderBodyTextFrame(bodyTextFrame, textBlock, page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null
+                                                                                                           || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
                 .getMostPopular())) {
             textBlock.setClassification(PageBlockType.FOOTER);
-        } else if (page.getPageNumber() == 1 && (PositionUtils.getHeightDifferenceBetweenChunkWordAndDocumentWord(textBlock,
-                document.getTextHeightCounter().getMostPopular()) > 2.5 && textBlock.getHighestFontSize() > document.getFontSizeCounter().getMostPopular() || page.getTextBlocks()
-                .size() == 1)) {
+        } else if (page.getPageNumber() == 1 && (PositionUtils.getHeightDifferenceBetweenChunkWordAndDocumentWord(textBlock, document.getTextHeightCounter().getMostPopular()) > 2.5
+                                                 && textBlock.getHighestFontSize() > document.getFontSizeCounter().getMostPopular() || page.getTextBlocks().size() == 1)) {
             if (!Pattern.matches("[0-9]+", textBlock.toString())) {
                 textBlock.setClassification(PageBlockType.TITLE);
             }
-        } else if (textBlock.getText().length() > 5 && (textBlock.getMostPopularWordHeight() > document.getTextHeightCounter()
-                .getMostPopular() || textBlock.getMostPopularWordFontSize() > document.getFontSizeCounter().getMostPopular()) && PositionUtils.getApproxLineCount(textBlock) < 5.9
-
-                && (textBlock.getMostPopularWordStyle().contains("bold") && Character.isDigit(textBlock.toString().charAt(0)) && !matcher2.matches() && !textBlock.toString()
-                .contains(":") || textBlock.toString().equals(textBlock.toString().toUpperCase(Locale.ROOT)) && !matcher2.matches() && !textBlock.toString()
-                .contains(":") || textBlock.toString().startsWith("APPENDIX") || textBlock.toString().startsWith("FIGURE") || textBlock.toString()
-                .startsWith("TABLE")) && !textBlock.toString().endsWith(":") && matcher2.find()) {
+        } else if (textBlock.getText().length() > 5
+                   && (textBlock.getMostPopularWordHeight() > document.getTextHeightCounter().getMostPopular()
+                       || textBlock.getMostPopularWordFontSize() > document.getFontSizeCounter().getMostPopular())
+                   && PositionUtils.getApproxLineCount(textBlock) < 5.9
+                   && (textBlock.getMostPopularWordStyle().contains("bold") && Character.isDigit(textBlock.toString().charAt(0)) && matcher2.reset().find() && !textBlock.toString()
+                .contains(":")
+                       || textBlock.toString().equals(textBlock.toString().toUpperCase(Locale.ROOT)) && matcher2.reset().find() && !textBlock.toString().contains(":")
+                       || textBlock.toString().startsWith("APPENDIX")
+                       || textBlock.toString().startsWith("FIGURE")
+                       || textBlock.toString().startsWith("TABLE"))
+                   && !textBlock.toString().endsWith(":")
+                   && matcher2.reset().find()) {
             textBlock.setClassification(PageBlockType.getHeadlineType(1));
             document.setHeadlines(true);
 
-        } else if (matcher.find() && PositionUtils.getApproxLineCount(textBlock) < 2.9 && matcher2.find() && !matcher3.matches()) {
+        } else if (matcher.reset().find() && PositionUtils.getApproxLineCount(textBlock) < 2.9 && matcher2.reset().find() && !matcher3.reset().matches()) {
             textBlock.setClassification(PageBlockType.getHeadlineType(2));
             document.setHeadlines(true);
-        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock) && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter()
-                .getMostPopular() && textBlock.getMostPopularWordStyle().equals("bold") && !document.getFontStyleCounter().getMostPopular().equals("bold")) {
+        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock)
+                   && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter().getMostPopular()
+                   && textBlock.getMostPopularWordStyle().equals("bold")
+                   && !document.getFontStyleCounter().getMostPopular().equals("bold")) {
             textBlock.setClassification(PageBlockType.PARAGRAPH_BOLD);
-        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock) && textBlock.getMostPopularWordFont()
-                .equals(document.getFontCounter().getMostPopular()) && textBlock.getMostPopularWordStyle()
-                .equals(document.getFontStyleCounter().getMostPopular()) && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter().getMostPopular()) {
+        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock)
+                   && textBlock.getMostPopularWordFont().equals(document.getFontCounter().getMostPopular())
+                   && textBlock.getMostPopularWordStyle().equals(document.getFontStyleCounter().getMostPopular())
+                   && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter().getMostPopular()) {
             textBlock.setClassification(PageBlockType.PARAGRAPH);
-        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock) && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter()
-                .getMostPopular() && textBlock.getMostPopularWordStyle().equals("italic") && !document.getFontStyleCounter()
-                .getMostPopular()
-                .equals("italic") && PositionUtils.getApproxLineCount(textBlock) < 2.9) {
+        } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock)
+                   && textBlock.getMostPopularWordFontSize() == document.getFontSizeCounter().getMostPopular()
+                   && textBlock.getMostPopularWordStyle().equals("italic")
+                   && !document.getFontStyleCounter().getMostPopular().equals("italic")
+                   && PositionUtils.getApproxLineCount(textBlock) < 2.9) {
             textBlock.setClassification(PageBlockType.PARAGRAPH_ITALIC);
         } else if (PositionUtils.isWithinBodyTextFrame(bodyTextFrame, textBlock)) {
             textBlock.setClassification(PageBlockType.PARAGRAPH_UNKNOWN);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/classification/RedactManagerClassificationService.java

@@ -3,6 +3,7 @@ package com.knecon.fforesight.service.layoutparser.processor.services.classifica
 import java.util.List;
 import java.util.regex.Pattern;
 
+import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import org.springframework.stereotype.Service;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
@@ -10,7 +11,6 @@ import com.knecon.fforesight.service.layoutparser.processor.model.Classification
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.PageBlockType;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
-import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import com.knecon.fforesight.service.layoutparser.processor.utils.PositionUtils;
 
 import lombok.RequiredArgsConstructor;
@@ -21,6 +21,7 @@ import lombok.extern.slf4j.Slf4j;
 @RequiredArgsConstructor
 public class RedactManagerClassificationService {
 
+
     public void classifyDocument(ClassificationDocument document) {
 
         List<Float> headlineFontSizes = document.getFontSizeCounter().getHighterThanMostPopular();
@@ -51,16 +52,14 @@ public class RedactManagerClassificationService {
             textBlock.setClassification(PageBlockType.OTHER);
             return;
         }
-        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER) || PositionUtils.isOverBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
-                .getMostPopular())) {
+        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER)
+                || PositionUtils.isOverBodyTextFrame(bodyTextFrame, textBlock, page.getRotation()) && (document.getFontSizeCounter()
+                .getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter().getMostPopular())) {
             textBlock.setClassification(PageBlockType.HEADER);
 
-        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER) || PositionUtils.isUnderBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation()) && (document.getFontSizeCounter().getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter()
-                .getMostPopular())) {
+        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER)
+                || PositionUtils.isUnderBodyTextFrame(bodyTextFrame, textBlock, page.getRotation()) && (document.getFontSizeCounter()
+                .getMostPopular() == null || textBlock.getHighestFontSize() <= document.getFontSizeCounter().getMostPopular())) {
             textBlock.setClassification(PageBlockType.FOOTER);
         } else if (page.getPageNumber() == 1 && (PositionUtils.getHeightDifferenceBetweenChunkWordAndDocumentWord(textBlock,
                 document.getTextHeightCounter().getMostPopular()) > 2.5 && textBlock.getHighestFontSize() > document.getFontSizeCounter().getMostPopular() || page.getTextBlocks()

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/classification/TaasClassificationService.java

@@ -3,6 +3,7 @@ package com.knecon.fforesight.service.layoutparser.processor.services.classifica
 import java.util.List;
 import java.util.regex.Pattern;
 
+import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import org.springframework.stereotype.Service;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
@@ -11,7 +12,6 @@ import com.knecon.fforesight.service.layoutparser.processor.model.Classification
 import com.knecon.fforesight.service.layoutparser.processor.model.PageBlockType;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.services.BodyTextFrameService;
-import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import com.knecon.fforesight.service.layoutparser.processor.utils.PositionUtils;
 
 import lombok.RequiredArgsConstructor;
@@ -27,6 +27,7 @@ public class TaasClassificationService {
 
     public void classifyDocument(ClassificationDocument document) {
 
+
         List<Float> headlineFontSizes = document.getFontSizeCounter().getHighterThanMostPopular();
 
         log.debug("Document FontSize counters are: {}", document.getFontSizeCounter().getCountPerValue());
@@ -56,13 +57,11 @@ public class TaasClassificationService {
             textBlock.setClassification(PageBlockType.OTHER);
             return;
         }
-        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER) || PositionUtils.isOverBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation())) {
+        if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.HEADER)
+                || PositionUtils.isOverBodyTextFrame(bodyTextFrame, textBlock, page.getRotation())) {
             textBlock.setClassification(PageBlockType.HEADER);
-        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER) || PositionUtils.isUnderBodyTextFrame(bodyTextFrame,
-                textBlock,
-                page.getRotation())) {
+        } else if (MarkedContentUtils.intersects(textBlock, page.getMarkedContentBboxPerType(), MarkedContentUtils.FOOTER)
+                || PositionUtils.isUnderBodyTextFrame(bodyTextFrame, textBlock, page.getRotation())) {
             textBlock.setClassification(PageBlockType.FOOTER);
         } else if (page.getPageNumber() == 1 && (PositionUtils.getHeightDifferenceBetweenChunkWordAndDocumentWord(textBlock,
                 document.getTextHeightCounter().getMostPopular()) > 2.5 && textBlock.getHighestFontSize() > document.getFontSizeCounter().getMostPopular() || page.getTextBlocks()

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/factory/DocumentGraphFactory.java

@@ -49,9 +49,14 @@ public class DocumentGraphFactory {
         Document documentGraph = new Document();
         Context context = new Context(documentGraph);
 
-        document.getPages().forEach(context::buildAndAddPageWithCounter);
-        document.getSections().stream().flatMap(section -> section.getImages().stream()).forEach(image -> context.getImages().add(image));
-        addSections(document, context);
+        document.getPages()
+                .forEach(context::buildAndAddPageWithCounter);
+        document.getSections()
+                .stream()
+                .flatMap(section -> section.getImages()
+                        .stream())
+                .forEach(image -> context.getImages().add(image));
+        addSections(document, context, documentGraph);
         addHeaderAndFooterToEachPage(document, context);
 
         documentGraph.setNumberOfPages(context.pages.size());
@@ -62,9 +67,10 @@ public class DocumentGraphFactory {
     }
 
 
-    private void addSections(ClassificationDocument document, Context context) {
+    private void addSections(ClassificationDocument classificationDocument, Context context, Document document) {
 
-        document.getSections().forEach(section -> SectionNodeFactory.addSection(null, section.getNonEmptyPageBlocks(), section.getImages(), context));
+        classificationDocument.getSections()
+                .forEach(section -> SectionNodeFactory.addSection(null, section.getNonEmptyPageBlocks(), section.getImages(), context, document));
     }
 
 
@@ -74,9 +80,11 @@ public class DocumentGraphFactory {
 
         GenericSemanticNode node;
         if (originalTextBlock.isHeadline()) {
-            node = Headline.builder().documentTree(context.getDocumentTree()).build();
+            node = Headline.builder().documentTree(context.getDocumentTree())
+                    .build();
         } else {
-            node = Paragraph.builder().documentTree(context.getDocumentTree()).build();
+            node = Paragraph.builder().documentTree(context.getDocumentTree())
+                    .build();
         }
 
         page.getMainBody().add(node);
@@ -91,7 +99,16 @@ public class DocumentGraphFactory {
     }
 
 
-    public void addImage(Section section, ClassifiedImage image, Context context) {
+    public void addImage(GenericSemanticNode parent, ClassifiedImage image, Context context) {
+
+        Image imageNode = createImage(image, context);
+        List<Integer> treeId = context.getDocumentTree().createNewChildEntryAndReturnId(parent, imageNode);
+        imageNode.setTreeId(treeId);
+        imageNode.setLeafTextBlock(context.textBlockFactory.emptyTextBlock(parent, context, context.getPage(image.getPage())));
+    }
+
+
+    private Image createImage(ClassifiedImage image, Context context) {
 
         Rectangle2D position = image.getPosition();
         Page page = context.getPage(image.getPage());
@@ -104,9 +121,7 @@ public class DocumentGraphFactory {
                 .documentTree(context.getDocumentTree())
                 .build();
         page.getMainBody().add(imageNode);
-
-        List<Integer> tocId = context.getDocumentTree().createNewChildEntryAndReturnId(section, imageNode);
-        imageNode.setTreeId(tocId);
+        return imageNode;
     }
 
 
@@ -146,10 +161,7 @@ public class DocumentGraphFactory {
 
         Page page = context.getPage(textBlocks.get(0).getPage());
         Footer footer = Footer.builder().documentTree(context.getDocumentTree()).build();
-        AtomicTextBlock textBlock = context.textBlockFactory.buildAtomicTextBlock(TextPositionOperations.mergeAndSortTextPositionSequenceByYThenX(textBlocks),
-                footer,
-                context,
-                page);
+        AtomicTextBlock textBlock = context.textBlockFactory.buildAtomicTextBlock(TextPositionOperations.mergeTextPositionSequence(textBlocks), footer, context, page);
         List<Integer> tocId = context.getDocumentTree().createNewMainEntryAndReturnId(footer);
         footer.setTreeId(tocId);
         footer.setLeafTextBlock(textBlock);
@@ -161,7 +173,7 @@ public class DocumentGraphFactory {
 
         Page page = context.getPage(textBlocks.get(0).getPage());
         Header header = Header.builder().documentTree(context.getDocumentTree()).build();
-        AtomicTextBlock textBlock = context.textBlockFactory.buildAtomicTextBlock(TextPositionOperations.mergeAndSortTextPositionSequenceByYThenX(textBlocks), header, 0, page);
+        AtomicTextBlock textBlock = context.textBlockFactory.buildAtomicTextBlock(TextPositionOperations.mergeTextPositionSequence(textBlocks), header, 0, page);
         List<Integer> tocId = context.getDocumentTree().createNewMainEntryAndReturnId(header);
         header.setTreeId(tocId);
         header.setLeafTextBlock(textBlock);
@@ -172,7 +184,8 @@ public class DocumentGraphFactory {
     private void addEmptyFooter(int pageIndex, Context context) {
 
         Page page = context.getPage(pageIndex);
-        Footer footer = Footer.builder().documentTree(context.getDocumentTree()).build();
+        Footer footer = Footer.builder().documentTree(context.getDocumentTree())
+                .build();
         AtomicTextBlock textBlock = context.textBlockFactory.emptyTextBlock(footer, context, page);
         List<Integer> tocId = context.getDocumentTree().createNewMainEntryAndReturnId(footer);
         footer.setTreeId(tocId);
@@ -184,7 +197,8 @@ public class DocumentGraphFactory {
     private void addEmptyHeader(int pageIndex, Context context) {
 
         Page page = context.getPage(pageIndex);
-        Header header = Header.builder().documentTree(context.getDocumentTree()).build();
+        Header header = Header.builder().documentTree(context.getDocumentTree())
+                .build();
         AtomicTextBlock textBlock = context.textBlockFactory.emptyTextBlock(header, 0, page);
         List<Integer> tocId = context.getDocumentTree().createNewMainEntryAndReturnId(header);
         header.setTreeId(tocId);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/factory/SearchTextWithTextPositionFactory.java

@@ -8,10 +8,10 @@ import java.util.List;
 import java.util.Locale;
 import java.util.Objects;
 
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.Boundary;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.RedTextPosition;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextDirection;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.Boundary;
 
 import lombok.experimental.UtilityClass;
 
@@ -110,7 +110,6 @@ public class SearchTextWithTextPositionFactory {
         return context.stringIdx - context.lastHyphenIdx < MAX_HYPHEN_LINEBREAK_DISTANCE;
     }
 
-
     private static List<Boundary> mergeToBoundaries(List<Integer> integers) {
 
         if (integers.isEmpty()) {
@@ -126,9 +125,8 @@ public class SearchTextWithTextPositionFactory {
             }
             end = current + 1;
         }
-        if (boundaries.isEmpty()) {
+        if (boundaries.isEmpty())
             boundaries.add(new Boundary(start, end));
-        }
         return boundaries;
     }
 
@@ -140,7 +138,6 @@ public class SearchTextWithTextPositionFactory {
         }
     }
 
-
     private boolean isLineBreak(RedTextPosition currentTextPosition, RedTextPosition previousTextPosition) {
 
         return Objects.equals(currentTextPosition.getUnicode(), "\n") || isDeltaYLargerThanTextHeight(currentTextPosition, previousTextPosition);

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/factory/SectionNodeFactory.java

@@ -11,12 +11,13 @@ import java.util.Map;
 import java.util.Set;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.GenericSemanticNode;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Section;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
 import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.GenericSemanticNode;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Section;
 import com.knecon.fforesight.service.layoutparser.processor.utils.TableMergingUtility;
 
 import lombok.experimental.UtilityClass;
@@ -24,27 +25,46 @@ import lombok.experimental.UtilityClass;
 @UtilityClass
 public class SectionNodeFactory {
 
-    public void addSection(GenericSemanticNode parentNode, List<AbstractPageBlock> pageBlocks, List<ClassifiedImage> images, DocumentGraphFactory.Context context) {
+    public void addSection(GenericSemanticNode parentNode,
+                           List<AbstractPageBlock> pageBlocks,
+                           List<ClassifiedImage> images,
+                           DocumentGraphFactory.Context context,
+                           Document document) {
+
+        // This is for the case where we have images on a page without any text/footer/header.
+        // The pageBlocks list is empty, but we still need to add those images to the document.
+        if (!images.isEmpty() && pageBlocks.isEmpty()) {
+            images.stream()
+                    .distinct()
+                    .forEach(image -> DocumentGraphFactory.addImage(document, image, context));
+            return;
+        }
 
         if (pageBlocks.isEmpty()) {
             return;
         }
-        Map<Integer, List<AbstractPageBlock>> blocksPerPage = pageBlocks.stream().collect(groupingBy(AbstractPageBlock::getPage));
-        Section section = Section.builder().documentTree(context.getDocumentTree()).build();
+
+        Map<Integer, List<AbstractPageBlock>> blocksPerPage = pageBlocks.stream()
+                .collect(groupingBy(AbstractPageBlock::getPage));
+        Section section = Section.builder().documentTree(context.getDocumentTree())
+                .build();
 
         context.getSections().add(section);
-        blocksPerPage.keySet().forEach(pageNumber -> addSectionNodeToPageNode(context, section, pageNumber));
+        blocksPerPage.keySet()
+                .forEach(pageNumber -> addSectionNodeToPageNode(context, section, pageNumber));
 
         section.setTreeId(getTreeId(parentNode, context, section));
 
-        addFirstHeadlineDirectlyToSection(pageBlocks, context, section);
+        addFirstHeadlineDirectlyToSection(pageBlocks, context, section, document);
         if (containsTablesAndTextBlocks(pageBlocks)) {
-            splitPageBlocksIntoSubSections(pageBlocks).forEach(subSectionPageBlocks -> addSection(section, subSectionPageBlocks, emptyList(), context));
+            splitPageBlocksIntoSubSections(pageBlocks).forEach(subSectionPageBlocks -> addSection(section, subSectionPageBlocks, emptyList(), context, document));
         } else {
-            addTablesAndParagraphsAndHeadlinesToSection(pageBlocks, context, section);
+            addTablesAndParagraphsAndHeadlinesToSection(pageBlocks, context, section, document);
         }
 
-        images.stream().distinct().forEach(image -> DocumentGraphFactory.addImage(section, image, context));
+        images.stream()
+                .distinct()
+                .forEach(image -> DocumentGraphFactory.addImage(section, image, context));
     }
 
 
@@ -58,16 +78,16 @@ public class SectionNodeFactory {
     }
 
 
-    private void addFirstHeadlineDirectlyToSection(List<AbstractPageBlock> pageBlocks, DocumentGraphFactory.Context context, Section section) {
+    private void addFirstHeadlineDirectlyToSection(List<AbstractPageBlock> pageBlocks, DocumentGraphFactory.Context context, Section section, Document document) {
 
         if (pageBlocks.get(0).isHeadline()) {
-            addTablesAndParagraphsAndHeadlinesToSection(List.of(pageBlocks.get(0)), context, section);
+            addTablesAndParagraphsAndHeadlinesToSection(List.of(pageBlocks.get(0)), context, section, document);
             pageBlocks.remove(0);
         }
     }
 
 
-    private void addTablesAndParagraphsAndHeadlinesToSection(List<AbstractPageBlock> pageBlocks, DocumentGraphFactory.Context context, Section section) {
+    private void addTablesAndParagraphsAndHeadlinesToSection(List<AbstractPageBlock> pageBlocks, DocumentGraphFactory.Context context, Section section, Document document) {
 
         Set<AbstractPageBlock> alreadyMerged = new HashSet<>();
         List<AbstractPageBlock> remainingBlocks = new LinkedList<>(pageBlocks);
@@ -86,7 +106,7 @@ public class SectionNodeFactory {
             } else if (abstractPageBlock instanceof TablePageBlock tablePageBlock) {
                 List<TablePageBlock> tablesToMerge = TableMergingUtility.findConsecutiveTablesWithSameColCountAndSameHeaders(tablePageBlock, remainingBlocks);
                 alreadyMerged.addAll(tablesToMerge);
-                TableNodeFactory.addTable(section, tablesToMerge, context);
+                TableNodeFactory.addTable(section, tablesToMerge, context, document);
             } else {
                 throw new RuntimeException(format("Unhandled AbstractPageBlockType %s!", abstractPageBlock.getClass()));
             }
@@ -96,7 +116,9 @@ public class SectionNodeFactory {
 
     private boolean containsTablesAndTextBlocks(List<AbstractPageBlock> pageBlocks) {
 
-        return pageBlocks.stream().anyMatch(pageBlock -> pageBlock instanceof TablePageBlock) && pageBlocks.stream().anyMatch(pageBlock -> pageBlock instanceof TextPageBlock);
+        return pageBlocks.stream()
+                       .anyMatch(pageBlock -> pageBlock instanceof TablePageBlock) && pageBlocks.stream()
+                       .anyMatch(pageBlock -> pageBlock instanceof TextPageBlock);
     }
 
 
@@ -112,7 +134,9 @@ public class SectionNodeFactory {
 
         List<List<AbstractPageBlock>> splitList = splitIntoCoherentList(pageBlocks);
         movePrecedingHeadlineToTableList(splitList);
-        return splitList.stream().filter(list -> !list.isEmpty()).toList();
+        return splitList.stream()
+                .filter(list -> !list.isEmpty())
+                .toList();
     }
 
 
@@ -133,7 +157,8 @@ public class SectionNodeFactory {
 
     private boolean listIsTablesOnly(List<AbstractPageBlock> abstractPageBlocks) {
 
-        return abstractPageBlocks.stream().allMatch(abstractPageBlock -> abstractPageBlock instanceof TablePageBlock);
+        return abstractPageBlocks.stream()
+                .allMatch(abstractPageBlock -> abstractPageBlock instanceof TablePageBlock);
     }
 
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/factory/TableNodeFactory.java

@@ -8,15 +8,16 @@ import java.util.Set;
 import java.util.stream.Collectors;
 
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.GenericSemanticNode;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.SemanticNode;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.TextBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
-import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.utils.TextPositionOperations;
 
 import lombok.experimental.UtilityClass;
@@ -27,23 +28,26 @@ public class TableNodeFactory {
     public final double TABLE_CELL_MERGE_CONTENTS_SIZE_THRESHOLD = 0.05;
 
 
-    public void addTable(GenericSemanticNode parentNode, List<TablePageBlock> tablesToMerge, DocumentGraphFactory.Context context) {
+    public void addTable(GenericSemanticNode parentNode, List<TablePageBlock> tablesToMerge, DocumentGraphFactory.Context context, Document document) {
 
         setPageNumberInCells(tablesToMerge);
-        Set<Page> pages = tablesToMerge.stream().map(AbstractPageBlock::getPage).map(context::getPage).collect(Collectors.toSet());
-        List<List<Cell>> mergedRows = tablesToMerge.stream().map(TablePageBlock::getRows).flatMap(Collection::stream).toList();
+        Set<Page> pages = tablesToMerge.stream()
+                .map(AbstractPageBlock::getPage)
+                .map(context::getPage)
+                .collect(Collectors.toSet());
+        List<List<Cell>> mergedRows = tablesToMerge.stream()
+                .map(TablePageBlock::getRows)
+                .flatMap(Collection::stream)
+                .toList();
 
-        Table table = Table.builder()
-                .documentTree(context.getDocumentTree())
-                .numberOfCols(mergedRows.isEmpty() ? 0 : mergedRows.get(0).size())
-                .numberOfRows(mergedRows.size())
+        Table table = Table.builder().documentTree(context.getDocumentTree()).numberOfCols(mergedRows.isEmpty() ? 0 : mergedRows.get(0).size()).numberOfRows(mergedRows.size())
                 .build();
 
         pages.forEach(page -> addTableToPage(page, parentNode, table));
 
         List<Integer> treeId = context.getDocumentTree().createNewChildEntryAndReturnId(parentNode, table);
         table.setTreeId(treeId);
-        addTableCells(mergedRows, table, context);
+        addTableCells(mergedRows, table, context, document);
 
         ifTableHasNoHeadersSetFirstRowAsHeaders(table);
     }
@@ -63,7 +67,8 @@ public class TableNodeFactory {
 
     private void setPageNumberInTextBlocksWithPageNumberSetTo0(TablePageBlock table, Cell cell) {
 
-        cell.getTextBlocks().stream()//
+        cell.getTextBlocks()
+                .stream()//
                 .filter(tb -> tb.getPage() == 0)//
                 .forEach(tb -> tb.setPage(table.getPage()));
     }
@@ -82,28 +87,32 @@ public class TableNodeFactory {
 
     private void ifTableHasNoHeadersSetFirstRowAsHeaders(Table table) {
 
-        if (table.streamHeaders().findAny().isEmpty()) {
-            table.streamRow(0).forEach(tableCellNode -> tableCellNode.setHeader(true));
+        if (table.streamHeaders()
+                .findAny().isEmpty()) {
+            table.streamRow(0)
+                    .forEach(tableCellNode -> tableCellNode.setHeader(true));
         }
     }
 
 
-    private void addTableCells(List<List<Cell>> rows, Table table, DocumentGraphFactory.Context context) {
+    private void addTableCells(List<List<Cell>> rows, Table table, DocumentGraphFactory.Context context, Document document) {
 
         for (int rowIndex = 0; rowIndex < rows.size(); rowIndex++) {
             for (int colIndex = 0; colIndex < rows.get(rowIndex).size(); colIndex++) {
-                addTableCell(rows.get(rowIndex).get(colIndex), rowIndex, colIndex, table, context);
+                addTableCell(rows.get(rowIndex)
+                                     .get(colIndex), rowIndex, colIndex, table, context, document);
             }
         }
     }
 
 
     @SuppressWarnings("PMD.UnusedPrivateMethod") // PMD actually flags this wrong
-    private void addTableCell(Cell cell, int rowIndex, int colIndex, Table tableNode, DocumentGraphFactory.Context context) {
+    private void addTableCell(Cell cell, int rowIndex, int colIndex, Table tableNode, DocumentGraphFactory.Context context, Document document) {
 
         Page page = context.getPage(cell.getPageNumber());
 
-        TableCell tableCell = TableCell.builder().documentTree(context.getDocumentTree()).row(rowIndex).col(colIndex).header(cell.isHeaderCell()).bBox(cell.getBounds2D()).build();
+        TableCell tableCell = TableCell.builder().documentTree(context.getDocumentTree()).row(rowIndex).col(colIndex).header(cell.isHeaderCell()).bBox(cell.getBounds2D())
+                .build();
         page.getMainBody().add(tableCell);
 
         List<Integer> treeId = context.getDocumentTree().createNewTableChildEntryAndReturnId(tableNode, tableCell);
@@ -113,16 +122,26 @@ public class TableNodeFactory {
         if (cell.getTextBlocks().isEmpty()) {
             tableCell.setLeafTextBlock(context.getTextBlockFactory().emptyTextBlock(tableNode, context, page));
         } else if (cell.getTextBlocks().size() == 1) {
-            textBlock = context.getTextBlockFactory().buildAtomicTextBlock(cell.getTextBlocks().get(0).getSequences(), tableCell, context, page);
+            textBlock = context.getTextBlockFactory()
+                    .buildAtomicTextBlock(cell.getTextBlocks()
+                                                  .get(0).getSequences(), tableCell, context, page);
             tableCell.setLeafTextBlock(textBlock);
         } else if (firstTextBlockIsHeadline(cell)) {
-            SectionNodeFactory.addSection(tableCell, cell.getTextBlocks().stream().map(tb -> (AbstractPageBlock) tb).toList(), emptyList(), context);
+            SectionNodeFactory.addSection(tableCell,
+                                          cell.getTextBlocks()
+                                                  .stream()
+                                                  .map(tb -> (AbstractPageBlock) tb)
+                                                  .toList(),
+                                          emptyList(),
+                                          context,
+                                          document);
         } else if (cellAreaIsSmallerThanPageAreaTimesThreshold(cell, page)) {
             List<TextPositionSequence> sequences = TextPositionOperations.mergeAndSortTextPositionSequenceByYThenX(cell.getTextBlocks());
             textBlock = context.getTextBlockFactory().buildAtomicTextBlock(sequences, tableCell, context, page);
             tableCell.setLeafTextBlock(textBlock);
         } else {
-            cell.getTextBlocks().forEach(tb -> DocumentGraphFactory.addParagraphOrHeadline(tableCell, tb, context, emptyList()));
+            cell.getTextBlocks()
+                    .forEach(tb -> DocumentGraphFactory.addParagraphOrHeadline(tableCell, tb, context, emptyList()));
         }
     }
 
@@ -135,7 +154,8 @@ public class TableNodeFactory {
 
     private boolean firstTextBlockIsHeadline(Cell cell) {
 
-        return cell.getTextBlocks().get(0).isHeadline();
+        return cell.getTextBlocks()
+                .get(0).isHeadline();
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/factory/TextBlockFactory.java

@@ -2,10 +2,10 @@ package com.knecon.fforesight.service.layoutparser.processor.services.factory;
 
 import java.util.List;
 
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.SemanticNode;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.AtomicTextBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 
 import lombok.AccessLevel;
 import lombok.experimental.FieldDefaults;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/mapper/DocumentGraphMapper.java

@@ -7,11 +7,11 @@ import java.util.List;
 import java.util.Map;
 import java.util.NoSuchElementException;
 
-import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentData;
-import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentPage;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentPositionData;
-import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentStructure;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentTextData;
+import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentData;
+import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentStructure;
+import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.DocumentTree;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Footer;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/mapper/PropertiesMapper.java

@@ -1,6 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.mapper;
 
 import java.awt.geom.Rectangle2D;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.Locale;
 import java.util.Map;
@@ -8,6 +9,7 @@ import java.util.Map;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentStructure;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Image;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.ImageType;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/parsing/PDFLinesTextStripper.java

@@ -329,7 +329,6 @@ public class PDFLinesTextStripper extends PDFTextStripper {
                 .getXDirAdj() - (previous.getXDirAdj() + previous.getWidthDirAdj()) < maximumGapSize;
     }
 
-
     @Override
     public String getText(PDDocument doc) throws IOException {
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/parsing/PDFTextStripper.java

@@ -25,23 +25,10 @@ import java.io.StringWriter;
 import java.io.Writer;
 import java.text.Bidi;
 import java.text.Normalizer;
-import java.util.ArrayDeque;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.Comparator;
-import java.util.Deque;
-import java.util.HashMap;
-import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Map;
-import java.util.SortedMap;
-import java.util.SortedSet;
-import java.util.StringTokenizer;
-import java.util.TreeMap;
-import java.util.TreeSet;
+import java.util.*;
 import java.util.regex.Pattern;
 
+import lombok.Getter;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
 import org.apache.pdfbox.cos.COSDictionary;
@@ -59,8 +46,6 @@ import org.apache.pdfbox.text.TextPositionComparator;
 
 import com.knecon.fforesight.service.layoutparser.processor.utils.QuickSort;
 
-import lombok.Getter;
-
 /**
  * This is just a copy except i only adjusted lines 594-607 cause this is a bug in Pdfbox.
  * see S416.pdf
@@ -209,33 +194,40 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
     }
 
 
-    public void beginMarkedContentSequence(COSName tag, COSDictionary properties) {
 
+    public void beginMarkedContentSequence(COSName tag, COSDictionary properties)
+    {
         PDMarkedContent markedContent = PDMarkedContent.create(tag, properties);
-        if (this.currentMarkedContents.isEmpty()) {
+        if (this.currentMarkedContents.isEmpty())
+        {
             this.markedContents.add(markedContent);
-        } else {
-            PDMarkedContent currentMarkedContent = this.currentMarkedContents.peek();
-            if (currentMarkedContent != null) {
+        }
+        else
+        {
+            PDMarkedContent currentMarkedContent =
+                    this.currentMarkedContents.peek();
+            if (currentMarkedContent != null)
+            {
                 currentMarkedContent.addMarkedContent(markedContent);
             }
         }
         this.currentMarkedContents.push(markedContent);
     }
 
-
     @Override
-    public void endMarkedContentSequence() {
-
-        if (!this.currentMarkedContents.isEmpty()) {
+    public void endMarkedContentSequence()
+    {
+        if (!this.currentMarkedContents.isEmpty())
+        {
             this.currentMarkedContents.pop();
         }
     }
 
 
-    public void xobject(PDXObject xobject) {
-
-        if (!this.currentMarkedContents.isEmpty()) {
+    public void xobject(PDXObject xobject)
+    {
+        if (!this.currentMarkedContents.isEmpty())
+        {
             this.currentMarkedContents.peek().addXObject(xobject);
         }
     }
@@ -643,6 +635,7 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
                         var normalized = normalize(line);
 //                        normalized.stream().filter(l -> System.out.println(l.getText().contains("Plenarprotokoll 20/24")).findFirst().isPresent()
 
+
                         lastLineStartPosition = handleLineSeparation(current, lastPosition, lastLineStartPosition, maxHeightForLine);
                         writeLine(normalized, current.isParagraphStart);
                         line.clear();
@@ -921,7 +914,8 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
                     textList.add(text);
                 }
             }
-            if (!this.currentMarkedContents.isEmpty()) {
+            if (!this.currentMarkedContents.isEmpty())
+            {
                 this.currentMarkedContents.peek().addText(text);
             }
         }
@@ -2108,9 +2102,7 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
             return endParagraphWritten;
         }
 
-
-        public void setEndParagraphWritten() {
-
+        public void setEndParagraphWritten(){
             endParagraphWritten = true;
         }
 
@@ -2153,6 +2145,7 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
             this.isHangingIndent = true;
         }
 
+
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/visualization/LayoutGridService.java

@@ -1,12 +1,11 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.visualization;
 
-import static com.knecon.fforesight.service.layoutparser.processor.services.visualization.ViewerDocumentService.LINE_WIDTH;
-
 import java.awt.Color;
 import java.awt.geom.Line2D;
 import java.awt.geom.Point2D;
 import java.awt.geom.Rectangle2D;
 import java.awt.geom.RectangularShape;
+import java.io.File;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.Comparator;
@@ -24,25 +23,59 @@ import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Pa
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.SemanticNode;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredLine;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.LayoutGrid;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.PlacedText;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RectangleTransformations;
+import com.knecon.fforesight.service.viewerdoc.ContentStreams;
+import com.knecon.fforesight.service.viewerdoc.model.ColoredLine;
+import com.knecon.fforesight.service.viewerdoc.model.ColoredRectangle;
+import com.knecon.fforesight.service.viewerdoc.model.LayoutGrid;
+import com.knecon.fforesight.service.viewerdoc.model.PlacedText;
+import com.knecon.fforesight.service.viewerdoc.model.Standard14EmbeddableFont;
+import com.knecon.fforesight.service.viewerdoc.model.Visualizations;
+import com.knecon.fforesight.service.viewerdoc.service.ViewerDocumentService;
+
+import io.micrometer.observation.annotation.Observed;
+import lombok.AccessLevel;
+import lombok.RequiredArgsConstructor;
+import lombok.SneakyThrows;
+import lombok.experimental.FieldDefaults;
 
 @Service
+@RequiredArgsConstructor
+@FieldDefaults(level = AccessLevel.PRIVATE, makeFinal = true)
 public class LayoutGridService {
 
-    private static final Color INNER_LINES_COLOR = new Color(255, 175, 175);
-    private static final Color PARAGRAPH_COLOR = new Color(70, 130, 180);
-    public static final Color TABLE_COLOR = new Color(102, 205, 170);
-    public static final Color SECTION_COLOR = new Color(50, 50, 50);
-    public static final Color HEADLINE_COLOR = new Color(162, 56, 56);
-    public static final Color HEADER_COLOR = new Color(171, 131, 6);
-    public static final Color IMAGE_COLOR = new Color(253, 63, 146);
+    ViewerDocumentService viewerDocumentService;
+
+    static float FONT_SIZE = 10f;
+    static float LINE_WIDTH = 1f;
+    static Standard14EmbeddableFont FONT = Standard14EmbeddableFont.helvetica();
+
+    static Color INNER_LINES_COLOR = new Color(255, 175, 175);
+    static Color PARAGRAPH_COLOR = new Color(70, 130, 180);
+    static Color TABLE_COLOR = new Color(102, 205, 170);
+    static Color SECTION_COLOR = new Color(50, 50, 50);
+    static Color HEADLINE_COLOR = new Color(162, 56, 56);
+    static Color HEADER_COLOR = new Color(171, 131, 6);
+    static Color IMAGE_COLOR = new Color(253, 63, 146);
 
 
-    public LayoutGrid createLayoutGrid(Document document) {
+    @SneakyThrows
+    @Observed(name = "ViewerDocumentService", contextualName = "create-viewer-document")
+    public void addLayoutGrid(File originFile, Document document, File destinationFile, boolean layerVisibilityDefaultValue) {
+
+        LayoutGrid layoutGrid = createLayoutGrid(document);
+
+        viewerDocumentService.addVisualizationsOnPage(originFile,
+                destinationFile,
+                Visualizations.builder()
+                        .layer(ContentStreams.KNECON_LAYOUT)
+                        .visualizationsOnPages(layoutGrid.getVisualizationsPerPages())
+                        .layerVisibilityDefaultValue(layerVisibilityDefaultValue)
+                        .build());
+    }
+
+
+    private LayoutGrid createLayoutGrid(Document document) {
 
         LayoutGrid layoutGrid = new LayoutGrid(document.getNumberOfPages());
         document.streamAllSubNodes().forEach(semanticNode -> {
@@ -77,11 +110,13 @@ public class LayoutGridService {
             return;
         }
         for (Page page : table.getPages()) {
+
             Optional<Integer> optionalFirstRowOnPage = table.streamCol(0).filter(tableCell -> tableCell.isOnPage(page.getNumber())).map(TableCell::getRow).findFirst();
             if (optionalFirstRowOnPage.isEmpty()) {
                 continue;
             }
             int firstRowOnPage = optionalFirstRowOnPage.get();
+
             Stream<Double> xStream = switch (page.getRotation()) {
                 case 90 -> streamBBoxOfCellsOnPage(table.streamCol(0), page).map(RectangularShape::getMinX);
                 case 180 -> streamBBoxOfCellsOnPage(table.streamRow(firstRowOnPage), page).map(RectangularShape::getMaxX);
@@ -90,6 +125,7 @@ public class LayoutGridService {
             };
             List<Double> xs = xStream.collect(Collectors.toList());
             xs.remove(0);
+
             Stream<Double> yStream = switch (page.getRotation()) {
                 case 90 -> streamBBoxOfCellsOnPage(table.streamRow(firstRowOnPage), page).map(RectangularShape::getMinY);
                 case 180 -> streamBBoxOfCellsOnPage(table.streamCol(0), page).map(RectangularShape::getMinY);
@@ -99,15 +135,15 @@ public class LayoutGridService {
             List<Double> ys = yStream.collect(Collectors.toList());
             ys.remove(0);
 
-            Rectangle2D tableBBox = table.getBBox().get(table.getFirstPage());
+            Rectangle2D tableBBox = table.getBBox().get(page);
             List<ColoredLine> coloredLines = layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getColoredLines();
             xs.forEach(x -> {
                 Line2D line = new Line2D.Double(new Point2D.Double(x, tableBBox.getMaxY()), new Point2D.Double(x, tableBBox.getMinY()));
-                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR));
+                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR, LINE_WIDTH));
             });
             ys.forEach(y -> {
                 Line2D line = new Line2D.Double(new Point2D.Double(tableBBox.getMinX(), y), new Point2D.Double(tableBBox.getMaxX(), y));
-                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR));
+                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR, LINE_WIDTH));
             });
         }
     }
@@ -135,9 +171,9 @@ public class LayoutGridService {
             List<Line2D> lines = createLinesFromRectangle(r, firstPage.getRotation());
             // add string to top line
             var firstLine = lines.remove(0);
-            coloredLines.add(new ColoredLine(firstLine, color));
+            coloredLines.add(new ColoredLine(firstLine, color, LINE_WIDTH));
             for (Line2D line : lines) {
-                coloredLines.add(new ColoredLine(line, color));
+                coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
             }
             return;
         }
@@ -152,16 +188,37 @@ public class LayoutGridService {
     }
 
 
+    @SneakyThrows
     private void addPlacedText(Page page, Rectangle2D textBBox, String s, LayoutGrid layoutGrid) {
 
-        Point2D.Float upperLeftCorner = switch (page.getRotation()) {
-            case 90 -> new Point2D.Float((float) (textBBox.getMinX()), (float) textBBox.getMinY());
-            case 180 -> new Point2D.Float((float) (textBBox.getMaxX()), (float) textBBox.getMinY());
-            case 270 -> new Point2D.Float((float) (textBBox.getMaxX()), (float) textBBox.getMaxY());
-            default -> new Point2D.Float((float) (textBBox.getMinX()), (float) textBBox.getMaxY());
-        };
+        // translates text, such that its right edge is a bit to the left of the drawn box
+        float translationAmount = ((FONT.getStringWidth(s) / 1000) * FONT_SIZE + (2 * LINE_WIDTH) + 4);
+
+        Point2D upperLeftCorner;
+        Point2D translationVector;
+        switch (page.getRotation()) {
+            case 90 -> {
+                upperLeftCorner = new Point2D.Double(textBBox.getMinX(), textBBox.getMinY());
+                translationVector = new Point2D.Double(FONT_SIZE, -translationAmount);
+            }
+            case 180 -> {
+                upperLeftCorner = new Point2D.Double(textBBox.getMaxX(), textBBox.getMinY());
+                translationVector = new Point2D.Double(translationAmount, FONT_SIZE);
+            }
+            case 270 -> {
+                upperLeftCorner = new Point2D.Double(textBBox.getMaxX(), textBBox.getMaxY());
+                translationVector = new Point2D.Double(-FONT_SIZE, translationAmount);
+            }
+            default -> {
+                upperLeftCorner = new Point2D.Double(textBBox.getMinX(), textBBox.getMaxY());
+                translationVector = new Point2D.Double(-translationAmount, -FONT_SIZE);
+            }
+        }
+
+        upperLeftCorner = add(upperLeftCorner, translationVector);
+
         var placedTexts = layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getPlacedTexts();
-        placedTexts.add(new PlacedText(s, upperLeftCorner));
+        placedTexts.add(PlacedText.textFacingUp(s, upperLeftCorner, FONT_SIZE, Color.BLACK, FONT));
     }
 
 
@@ -176,9 +233,9 @@ public class LayoutGridService {
         midPageLines.remove(1);
         // add string to left line
         var leftLine = midPageLines.remove(1);
-        coloredLines.add(new ColoredLine(leftLine, color));
+        coloredLines.add(new ColoredLine(leftLine, color, LINE_WIDTH));
         for (Line2D line : midPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
         }
     }
 
@@ -192,9 +249,9 @@ public class LayoutGridService {
         lastPageLines.remove(0);
         // add string to left line
         var leftLine = lastPageLines.remove(2);
-        coloredLines.add(new ColoredLine(leftLine, color));
+        coloredLines.add(new ColoredLine(leftLine, color, LINE_WIDTH));
         for (Line2D line : lastPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
         }
     }
 
@@ -208,9 +265,9 @@ public class LayoutGridService {
         firstPageLines.remove(2);
         // add string to top line
         var firstLine = firstPageLines.remove(0);
-        coloredLines.add(new ColoredLine(firstLine, color));
+        coloredLines.add(new ColoredLine(firstLine, color, LINE_WIDTH));
         for (Line2D line : firstPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
         }
     }
 
@@ -276,7 +333,16 @@ public class LayoutGridService {
     private void addAsRectangle(SemanticNode semanticNode, LayoutGrid layoutGrid, Color color) {
 
         semanticNode.getBBox()
-                .forEach((page, textBBox) -> layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getColoredRectangles().add(new ColoredRectangle(textBBox, color)));
+                .forEach((page, textBBox) -> layoutGrid.getVisualizationsPerPages()
+                        .get(page.getNumber() - 1)
+                        .getColoredRectangles()
+                        .add(new ColoredRectangle(textBBox, color, LINE_WIDTH)));
+    }
+
+
+    private Point2D add(Point2D a, Point2D b) {
+
+        return new Point2D.Double(a.getX() + b.getX(), a.getY() + b.getY());
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/services/visualization/ViewerDocumentService.java

@@ -1,212 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.services.visualization;
-
-import java.awt.Color;
-import java.awt.geom.AffineTransform;
-import java.awt.geom.Rectangle2D;
-import java.io.IOException;
-import java.io.OutputStream;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-import org.apache.pdfbox.cos.COSDictionary;
-import org.apache.pdfbox.cos.COSName;
-import org.apache.pdfbox.pdmodel.PDDocument;
-import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
-import org.apache.pdfbox.pdmodel.PDPage;
-import org.apache.pdfbox.pdmodel.PDPageContentStream;
-import org.apache.pdfbox.pdmodel.PDResources;
-import org.apache.pdfbox.pdmodel.font.PDFont;
-import org.apache.pdfbox.pdmodel.font.PDType1Font;
-import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
-import org.apache.pdfbox.pdmodel.graphics.optionalcontent.PDOptionalContentGroup;
-import org.apache.pdfbox.pdmodel.graphics.optionalcontent.PDOptionalContentProperties;
-import org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState;
-import org.apache.pdfbox.util.Matrix;
-import org.springframework.stereotype.Service;
-
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredLine;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.FilledRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.LayoutGrid;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.PlacedText;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.VisualizationsOnPage;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResult;
-
-import lombok.RequiredArgsConstructor;
-import lombok.SneakyThrows;
-import lombok.extern.slf4j.Slf4j;
-
-@Slf4j
-@Service
-@RequiredArgsConstructor
-public class ViewerDocumentService {
-
-    private static final String LAYER_NAME = "Layout grid";
-    private static final int FONT_SIZE = 10;
-    public static final float LINE_WIDTH = 1f;
-
-    private final LayoutGridService layoutGridService;
-
-
-    @SneakyThrows
-    public void createViewerDocument(PDDocument pdDocument,
-                                     Document document,
-                                     OutputStream outputStream,
-                                     Map<Integer, List<VisualLayoutParsingResult>> extractedTableCells,
-                                     boolean layerVisibilityDefaultValue) {
-
-        LayoutGrid layoutGrid = layoutGridService.createLayoutGrid(document);
-        // PDDocument.save() is very slow, since it actually traverses the entire pdf and writes a new one.
-        // If we collect all COSDictionaries we changed and tell it explicitly to only add the changed ones by using saveIncremental it's very fast.
-        Set<COSDictionary> dictionariesToUpdate = new HashSet<>();
-        PDOptionalContentGroup layer = addLayerToDocument(pdDocument, dictionariesToUpdate, layerVisibilityDefaultValue);
-        PDOptionalContentGroup visualLayoutParsingLayer = addLayerToDocument(pdDocument, dictionariesToUpdate, true);
-        PDFont font = new PDType1Font(Standard14Fonts.FontName.HELVETICA);
-
-        for (int pageNumber = 0; pageNumber < pdDocument.getNumberOfPages(); pageNumber++) {
-            PDPage pdPage = pdDocument.getPage(pageNumber);
-
-            AffineTransform textDeRotationMatrix = getTextDeRotationTransform(pdPage);
-            addLayerToPageRessources(pdPage);
-
-            // We need to save the graphics state before, such that our appended content cannot be affected by previous content streams with side effects,
-            // e.g. not escaped matrix transformations.
-            escapePreviousContents(pdDocument, pdPage);
-
-            VisualizationsOnPage visualizationsOnPage = layoutGrid.getVisualizationsPerPages().get(pageNumber);
-            assert pageNumber == visualizationsOnPage.getPageNumber();
-            // We need to append to the content stream, otherwise the content could be overlapped by following content.
-            try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.APPEND, true)) {
-
-                contentStream.beginMarkedContent(COSName.OC, layer);
-                contentStream.saveGraphicsState();
-
-                contentStream.setLineWidth(LINE_WIDTH);
-                for (ColoredLine coloredLine : visualizationsOnPage.getColoredLines()) {
-                    contentStream.setStrokingColor(coloredLine.color());
-                    contentStream.moveTo((float) coloredLine.line().getX1(), (float) coloredLine.line().getY1());
-                    contentStream.lineTo((float) coloredLine.line().getX2(), (float) coloredLine.line().getY2());
-                    contentStream.stroke();
-                }
-                for (ColoredRectangle coloredRectangle : visualizationsOnPage.getColoredRectangles()) {
-                    contentStream.setStrokingColor(coloredRectangle.color());
-                    Rectangle2D r = coloredRectangle.rectangle2D();
-                    contentStream.addRect((float) r.getX(), (float) r.getY(), (float) r.getWidth(), (float) r.getHeight());
-                    contentStream.stroke();
-                }
-                for (FilledRectangle filledRectangle : visualizationsOnPage.getFilledRectangles()) {
-                    contentStream.setNonStrokingColor(filledRectangle.color());
-                    PDExtendedGraphicsState graphicsState = new PDExtendedGraphicsState();
-                    graphicsState.setNonStrokingAlphaConstant(filledRectangle.alpha());
-                    contentStream.setGraphicsStateParameters(graphicsState);
-                    Rectangle2D r = filledRectangle.rectangle2D();
-                    contentStream.addRect((float) r.getX(), (float) r.getY(), (float) r.getWidth(), (float) r.getHeight());
-                    contentStream.fill();
-                }
-                for (PlacedText placedText : visualizationsOnPage.getPlacedTexts()) {
-                    contentStream.setFont(font, FONT_SIZE);
-                    contentStream.beginText();
-                    Matrix textMatrix = new Matrix((float) textDeRotationMatrix.getScaleX(),
-                            (float) textDeRotationMatrix.getShearX(),
-                            (float) textDeRotationMatrix.getShearY(),
-                            (float) textDeRotationMatrix.getScaleY(),
-                            (float) placedText.lineStart().getX(),
-                            (float) placedText.lineStart().getY());
-                    textMatrix.translate(-((font.getStringWidth(placedText.text()) / 1000) * FONT_SIZE + (2 * LINE_WIDTH) + 4), -FONT_SIZE);
-                    contentStream.setTextMatrix(textMatrix);
-                    contentStream.showText(placedText.text());
-                    contentStream.endText();
-                }
-                contentStream.restoreGraphicsState();
-                contentStream.endMarkedContent();
-
-                contentStream.beginMarkedContent(COSName.OC, visualLayoutParsingLayer);
-                contentStream.saveGraphicsState();
-
-                contentStream.setLineWidth(LINE_WIDTH);
-                for (VisualLayoutParsingResult tableCells : extractedTableCells.get(pageNumber)) {
-                    contentStream.setStrokingColor(new Color(0xFF0000));
-                    contentStream.addRect((float) tableCells.getX0(), (float) tableCells.getY0(), (float) tableCells.getWidth(), (float) tableCells.getHeight());
-                    contentStream.stroke();
-                    contentStream.setFont(font, FONT_SIZE);
-                    contentStream.beginText();
-                    Matrix textMatrix = new Matrix((float) textDeRotationMatrix.getScaleX(),
-                            (float) textDeRotationMatrix.getShearX(),
-                            (float) textDeRotationMatrix.getShearY(),
-                            (float) textDeRotationMatrix.getScaleY(),
-                            tableCells.getX0() ,
-                            tableCells.getY0());
-                    textMatrix.translate(-((font.getStringWidth(tableCells.getLabel()) / 1000) * FONT_SIZE + (2 * LINE_WIDTH) + 4), -FONT_SIZE);
-                    contentStream.setTextMatrix(textMatrix);
-                    contentStream.showText(tableCells.getLabel());
-                    contentStream.endText();
-                }
-                contentStream.restoreGraphicsState();
-                contentStream.endMarkedContent();
-            }
-            dictionariesToUpdate.add(pdPage.getCOSObject());
-            dictionariesToUpdate.add(pdPage.getResources().getCOSObject());
-        }
-        dictionariesToUpdate.add(pdDocument.getDocumentInformation().getCOSObject());
-//        dictionariesToUpdate.add(pdDocument.getDocument().getTrailer());
-        pdDocument.saveIncremental(outputStream, dictionariesToUpdate);
-    }
-
-
-    private static void addLayerToPageRessources(PDPage pdPage) {
-
-        PDResources resources = pdPage.getResources();
-        if (resources == null) {
-            resources = new PDResources();
-            pdPage.setResources(resources);
-        }
-    }
-
-
-    private static void escapePreviousContents(PDDocument pdDocument, PDPage pdPage) throws IOException {
-
-        try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.PREPEND, false)) {
-            contentStream.saveGraphicsState();
-        }
-        try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.APPEND, false)) {
-            contentStream.restoreGraphicsState();
-        }
-    }
-
-
-    private static PDOptionalContentGroup addLayerToDocument(PDDocument pdDocument, Set<COSDictionary> dictionariesToUpdate, boolean layerVisibilityDefaultValue) {
-
-        PDDocumentCatalog catalog = pdDocument.getDocumentCatalog();
-        PDOptionalContentProperties ocprops = catalog.getOCProperties();
-        if (ocprops == null) {
-            ocprops = new PDOptionalContentProperties();
-            catalog.setOCProperties(ocprops);
-        }
-        PDOptionalContentGroup layer = null;
-        if (ocprops.hasGroup(LAYER_NAME)) {
-            layer = ocprops.getGroup(LAYER_NAME);
-        } else {
-            layer = new PDOptionalContentGroup(LAYER_NAME);
-            ocprops.addGroup(layer);
-        }
-        ocprops.setGroupEnabled(layer, layerVisibilityDefaultValue);
-        dictionariesToUpdate.add(catalog.getCOSObject());
-        return layer;
-    }
-
-
-    private static AffineTransform getTextDeRotationTransform(PDPage page) {
-
-        return AffineTransform.getQuadrantRotateInstance(switch (page.getRotation()) {
-            case 90 -> 3;
-            case 180 -> 2;
-            case 270 -> 1;
-            default -> 0;
-        });
-    }
-
-}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/DrawingOptions.java

@@ -0,0 +1,28 @@
+package com.knecon.fforesight.service.layoutparser.processor.utils;
+
+import java.awt.Color;
+
+import lombok.AccessLevel;
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Getter;
+import lombok.experimental.FieldDefaults;
+
+@Builder
+@AllArgsConstructor
+@Getter
+@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
+public class DrawingOptions {
+
+    boolean stroke;
+    @Builder.Default
+    Color strokeColor = Color.BLACK;
+    @Builder.Default
+    float strokeWidth = 1f;
+
+    boolean fill;
+    @Builder.Default
+    Color fillColor = Color.BLACK;
+
+}
+

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/GeometricComparators.java

@@ -0,0 +1,88 @@
+package com.knecon.fforesight.service.layoutparser.processor.utils;
+
+import java.awt.geom.Point2D;
+import java.util.Comparator;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Rectangle;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
+
+public class GeometricComparators {
+
+    private static final int COMPARATOR_ROUNDING = 2;
+
+    public static final Comparator<Point2D> X_FIRST_POINT_COMPARATOR = (point1, point2) -> {
+
+        int rv = 0;
+        float point1X = DoubleComparisons.round(point1.getX(), COMPARATOR_ROUNDING);
+        float point1Y = DoubleComparisons.round(point1.getY(), COMPARATOR_ROUNDING);
+        float point2X = DoubleComparisons.round(point2.getX(), COMPARATOR_ROUNDING);
+        float point2Y = DoubleComparisons.round(point2.getY(), COMPARATOR_ROUNDING);
+
+        if (point1X > point2X) {
+            rv = 1;
+        } else if (point1X < point2X) {
+            rv = -1;
+        } else if (point1Y > point2Y) {
+            rv = 1;
+        } else if (point1Y < point2Y) {
+            rv = -1;
+        }
+        return rv;
+    };
+
+    public static final Comparator<Point2D> Y_FIRST_POINT_COMPARATOR = (point1, point2) -> {
+
+        int rv = 0;
+        float point1X = DoubleComparisons.round(point1.getX(), COMPARATOR_ROUNDING);
+        float point1Y = DoubleComparisons.round(point1.getY(), COMPARATOR_ROUNDING);
+        float point2X = DoubleComparisons.round(point2.getX(), COMPARATOR_ROUNDING);
+        float point2Y = DoubleComparisons.round(point2.getY(), COMPARATOR_ROUNDING);
+
+        if (point1Y > point2Y) {
+            rv = 1;
+        } else if (point1Y < point2Y) {
+            rv = -1;
+        } else if (point1X > point2X) {
+            rv = 1;
+        } else if (point1X < point2X) {
+            rv = -1;
+        }
+        return rv;
+    };
+
+    public static final Comparator<Cell> CELL_SIZE_COMPARATOR = (cell1, cell2) -> {
+
+        Double cell1Size = cell1.getHeight() * cell1.getWidth();
+        Double cell2Size = cell2.getHeight() * cell2.getWidth();
+        return cell1Size.compareTo(cell2Size);
+    };
+
+    public static final Comparator<Rectangle> RECTANGLE_SIZE_COMPARATOR = (rect1, rect2) -> {
+
+        Double rect1Size = rect1.getHeight() * rect1.getWidth();
+        Double rect2Size = rect2.getHeight() * rect2.getWidth();
+        return rect1Size.compareTo(rect2Size);
+    };
+
+    public static final Comparator<Ruling> X_FIRST_RULING_COMPARATOR = (ruling1, ruling2) -> {
+
+        int rv = 0;
+        float point1X = DoubleComparisons.round(Math.min(ruling1.getLeft(), ruling1.getRight()), COMPARATOR_ROUNDING);
+        float point1Y = DoubleComparisons.round(Math.min(ruling1.getTop(), ruling1.getBottom()), COMPARATOR_ROUNDING);
+        float point2X = DoubleComparisons.round(Math.min(ruling2.getLeft(), ruling2.getRight()), COMPARATOR_ROUNDING);
+        float point2Y = DoubleComparisons.round(Math.min(ruling2.getTop(), ruling2.getBottom()), COMPARATOR_ROUNDING);
+
+        if (point1X > point2X) {
+            rv = 1;
+        } else if (point1X < point2X) {
+            rv = -1;
+        } else if (point1Y > point2Y) {
+            rv = 1;
+        } else if (point1Y < point2Y) {
+            rv = -1;
+        }
+        return rv;
+    };
+
+}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/MarkedContentUtils.java

@@ -1,5 +1,12 @@
 package com.knecon.fforesight.service.layoutparser.processor.utils;
 
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+import lombok.experimental.UtilityClass;
+import org.apache.pdfbox.cos.COSName;
+import org.apache.pdfbox.pdmodel.documentinterchange.markedcontent.PDMarkedContent;
+import org.apache.pdfbox.text.TextPosition;
+
 import java.awt.geom.Rectangle2D;
 import java.util.Collection;
 import java.util.Collections;
@@ -7,22 +14,12 @@ import java.util.List;
 import java.util.Map;
 import java.util.stream.Collectors;
 
-import org.apache.pdfbox.cos.COSName;
-import org.apache.pdfbox.pdmodel.documentinterchange.markedcontent.PDMarkedContent;
-import org.apache.pdfbox.text.TextPosition;
-
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
-
-import lombok.experimental.UtilityClass;
-
 @UtilityClass
 public class MarkedContentUtils {
 
     public static final String HEADER = "Header";
     public static final String FOOTER = "Footer";
 
-
     public List<Rectangle2D> getMarkedContentBboxPerLine(List<PDMarkedContent> markedContents, String subtype) {
 
         if (markedContents == null) {
@@ -34,8 +31,7 @@ public class MarkedContentUtils {
                 .filter(m -> m.getProperties() != null)
                 .filter(m -> m.getProperties().getItem("Subtype") != null)
                 .filter(m -> ((COSName) m.getProperties().getItem("Subtype")).getName().equals(subtype))
-                .map(PDMarkedContent::getContents)
-                .flatMap(Collection::stream)
+                .map(PDMarkedContent::getContents).flatMap(Collection::stream)
                 .filter(t -> t instanceof TextPosition)
                 .map(t -> (TextPosition) t)
                 .filter(t -> !t.getUnicode().equals(" "))
@@ -45,19 +41,16 @@ public class MarkedContentUtils {
             return Collections.emptyList();
         }
 
-        return markedContentByYPosition.values()
-                .stream()
-                .map(textPositions -> new TextPositionSequence(textPositions.stream().toList(), 0, true).getRectangle())
-                .map(t -> new Rectangle2D.Float(t.getTopLeft().getX(), t.getTopLeft().getY() - Math.abs(t.getHeight()), t.getWidth(), Math.abs(t.getHeight())))
-                .collect(Collectors.toList());
+        return markedContentByYPosition.values().stream()
+                .map(textPositions -> new TextPositionSequence(textPositions.stream()
+                        .toList(), 0, true)
+                        .getRectangle())
+                .map(t -> new Rectangle2D.Float(t.getTopLeft().getX(), t.getTopLeft().getY() - Math.abs(t.getHeight()), t.getWidth(), Math.abs(t.getHeight()))).collect(Collectors.toList());
     }
 
 
     public boolean intersects(TextPageBlock textBlock, Map<String, List<Rectangle2D>> markedContentBboxPerType, String type) {
-
-        return markedContentBboxPerType.get(type) != null && markedContentBboxPerType.get(type)
-                .stream()
-                .anyMatch(rectangle -> rectangle.intersects(textBlock.getPdfMinX(), textBlock.getPdfMinY(), textBlock.getWidth(), textBlock.getHeight()));
+        return markedContentBboxPerType.get(type) != null && markedContentBboxPerType.get(type).stream().anyMatch(rectangle -> rectangle.intersects(textBlock.getPdfMinX(), textBlock.getPdfMinY(), textBlock.getWidth(), textBlock.getHeight()));
     }
 
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/PdfVisualisationUtility.java

@@ -21,11 +21,7 @@ import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Pa
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.AtomicTextBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.TextBlock;
 
-import lombok.AccessLevel;
-import lombok.Builder;
-import lombok.Getter;
 import lombok.SneakyThrows;
-import lombok.experimental.FieldDefaults;
 import lombok.experimental.UtilityClass;
 import lombok.extern.slf4j.Slf4j;
 
@@ -41,20 +37,20 @@ public class PdfVisualisationUtility {
 
     public void drawNode(PDDocument document, DocumentTree.Entry entry) {
 
-        Options options = buildStandardOptionsForNodes(entry);
+        DrawingOptions options = buildStandardOptionsForNodes(entry);
 
         drawBBoxAndLabelAndNumberOnPage(document, entry, options);
 
     }
 
 
-    public void drawTextBlock(PDDocument document, TextBlock textBlock, Options options) {
+    public void drawTextBlock(PDDocument document, TextBlock textBlock, DrawingOptions options) {
 
         textBlock.getAtomicTextBlocks().forEach(atb -> drawAtomicTextBlock(document, atb, options));
     }
 
 
-    public void drawAtomicTextBlock(PDDocument document, AtomicTextBlock atomicTextBlock, Options options) {
+    public void drawAtomicTextBlock(PDDocument document, AtomicTextBlock atomicTextBlock, DrawingOptions options) {
 
         drawRectangle2DList(document, atomicTextBlock.getPage().getNumber(), atomicTextBlock.getPositions().stream().toList(), options);
 
@@ -62,7 +58,7 @@ public class PdfVisualisationUtility {
 
 
     @SneakyThrows
-    public void drawText(String string, PDDocument document, Point2D location, Integer pageNumber, Options options) {
+    public void drawText(String string, PDDocument document, Point2D location, Integer pageNumber, DrawingOptions options) {
 
         var pdPage = document.getPage(pageNumber - 1);
         var contentStream = new PDPageContentStream(document, pdPage, PDPageContentStream.AppendMode.APPEND, true);
@@ -80,14 +76,14 @@ public class PdfVisualisationUtility {
 
 
     @SneakyThrows
-    public void drawRectangle2DList(PDDocument document, int pageNumber, List<Rectangle2D> rectCollection, Options options) {
+    public void drawRectangle2DList(PDDocument document, int pageNumber, List<Rectangle2D> rectCollection, DrawingOptions options) {
 
         var pdPage = document.getPage(pageNumber - 1);
         drawRectangle2DList(document, rectCollection, options, pdPage);
     }
 
 
-    private void drawRectangle2DList(PDDocument document, List<Rectangle2D> rectCollection, Options options, PDPage pdPage) throws IOException {
+    private void drawRectangle2DList(PDDocument document, List<Rectangle2D> rectCollection, DrawingOptions options, PDPage pdPage) throws IOException {
 
         var contentStream = new PDPageContentStream(document, pdPage, PDPageContentStream.AppendMode.APPEND, true);
 
@@ -110,9 +106,9 @@ public class PdfVisualisationUtility {
     }
 
 
-    private Options buildStandardOptionsForNodes(DocumentTree.Entry entry) {
+    private DrawingOptions buildStandardOptionsForNodes(DocumentTree.Entry entry) {
 
-        return Options.builder().stroke(true).strokeColor(switch (entry.getType()) {
+        return DrawingOptions.builder().stroke(true).strokeColor(switch (entry.getType()) {
             case DOCUMENT -> Color.LIGHT_GRAY;
             case HEADER, FOOTER -> Color.GREEN;
             case PARAGRAPH -> Color.BLUE;
@@ -125,7 +121,7 @@ public class PdfVisualisationUtility {
     }
 
 
-    private void drawBBoxAndLabelAndNumberOnPage(PDDocument document, DocumentTree.Entry entry, Options options) {
+    private void drawBBoxAndLabelAndNumberOnPage(PDDocument document, DocumentTree.Entry entry, DrawingOptions options) {
 
         Map<Page, Rectangle2D> rectanglesPerPage = entry.getNode().getBBox();
         rectanglesPerPage.forEach((page, rectangle2D) -> {
@@ -152,7 +148,7 @@ public class PdfVisualisationUtility {
 
 
     @SneakyThrows
-    public static void drawLine2DList(PDDocument pdDocument, int pageNumber, List<? extends Line2D> line2DS, Options options) {
+    public static void drawLine2DList(PDDocument pdDocument, int pageNumber, List<? extends Line2D> line2DS, DrawingOptions options) {
 
         var pdPage = pdDocument.getPage(pageNumber - 1);
         var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.APPEND, true);
@@ -176,21 +172,4 @@ public class PdfVisualisationUtility {
         contentStream.close();
     }
 
-
-    @Builder
-    @Getter
-    @FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
-    public static class Options {
-
-        boolean fill;
-        boolean stroke;
-        @Builder.Default
-        Color strokeColor = Color.BLACK;
-        @Builder.Default
-        float strokeWidth = 1f;
-        @Builder.Default
-        Color fillColor = Color.BLACK;
-
-    }
-
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/PositionUtils.java

@@ -19,9 +19,10 @@ public final class PositionUtils {
 
         double threshold = textBlock.getMostPopularWordHeight() * 3;
 
-        if (textBlock.getPdfMinX() + threshold > btf.getTopLeft().getX() && textBlock.getPdfMaxX() - threshold < btf.getTopLeft()
-                .getX() + btf.getWidth() && textBlock.getPdfMinY() + threshold > btf.getTopLeft().getY() && textBlock.getPdfMaxY() - threshold < btf.getTopLeft()
-                .getY() + btf.getHeight()) {
+        if (textBlock.getPdfMinX() + threshold > btf.getTopLeft().getX()
+                && textBlock.getPdfMaxX() - threshold < btf.getTopLeft().getX() + btf.getWidth()
+                && textBlock.getPdfMinY() + threshold > btf.getTopLeft().getY()
+                && textBlock.getPdfMaxY() - threshold < btf.getTopLeft().getY() + btf.getHeight()) {
             return true;
         } else {
             return false;

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/RectangleTransformations.java

@@ -2,8 +2,11 @@ package com.knecon.fforesight.service.layoutparser.processor.utils;
 
 import static java.lang.String.format;
 
+import java.awt.geom.Area;
+import java.awt.geom.Point2D;
 import java.awt.geom.Rectangle2D;
 import java.awt.geom.RectangularShape;
+import java.util.ArrayList;
 import java.util.Collections;
 import java.util.LinkedList;
 import java.util.List;
@@ -19,6 +22,8 @@ import org.apache.pdfbox.pdmodel.common.PDRectangle;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Point;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Rectangle;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.textblock.AtomicTextBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 
 import lombok.AllArgsConstructor;
 import lombok.NoArgsConstructor;
@@ -37,6 +42,16 @@ public class RectangleTransformations {
     }
 
 
+    public static double calculateIntersectedArea(Rectangle2D r1, Rectangle2D r2) {
+
+        Area a1 = new Area(r1);
+        Area a2 = new Area(r2);
+        a1.intersect(a2);
+        Rectangle2D intersection = a1.getBounds2D();
+        return intersection.getWidth() * intersection.getHeight();
+    }
+
+
     public static Rectangle2D bBoxUnionAtomicTextBlock(List<AtomicTextBlock> atomicTextBlocks) {
 
         return atomicTextBlocks.stream().flatMap(atomicTextBlock -> atomicTextBlock.getPositions().stream()).collect(new Rectangle2DBBoxCollector());
@@ -138,7 +153,27 @@ public class RectangleTransformations {
                 previousRectangle = currentRectangle;
             }
         }
-        return rectangleListsWithGaps.stream().map(RectangleTransformations::rectangle2DBBox).toList();
+        return rectangleListsWithGaps.stream()
+                .map(RectangleTransformations::rectangle2DBBox)
+                .toList();
+    }
+
+
+    public static CleanRulings extractRulings(List<? extends Rectangle2D.Float> rectangles) {
+
+        // Underlined or strikethrough are also in rulings but we dont want to split blocks with them so we use cells.
+        List<Ruling> horizontalRulings = new ArrayList<>();
+        List<Ruling> verticalRulings = new ArrayList<>();
+
+        rectangles.forEach(rectangle -> {
+            horizontalRulings.add(new Ruling(new Point2D.Float(rectangle.x, rectangle.y), new Point2D.Float(rectangle.x + rectangle.width, rectangle.y)));
+            horizontalRulings.add(new Ruling(new Point2D.Float(rectangle.x, rectangle.y + rectangle.height),
+                                             new Point2D.Float(rectangle.x + rectangle.width, rectangle.y + rectangle.height)));
+            verticalRulings.add(new Ruling(new Point2D.Float(rectangle.x, rectangle.y), new Point2D.Float(rectangle.x, rectangle.y + rectangle.height)));
+            verticalRulings.add(new Ruling(new Point2D.Float(rectangle.x + rectangle.width, rectangle.y),
+                                           new Point2D.Float(rectangle.x + rectangle.width, rectangle.y + rectangle.height)));
+        });
+        return CleanRulings.builder().vertical(verticalRulings).horizontal(horizontalRulings).build();
     }
 
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/RectangularIntersectionFinder.java

@@ -0,0 +1,77 @@
+package com.knecon.fforesight.service.layoutparser.processor.utils;
+
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.Y_FIRST_POINT_COMPARATOR;
+
+import java.awt.geom.Point2D;
+import java.awt.geom.Rectangle2D;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
+
+public class RectangularIntersectionFinder {
+
+    public static List<Rectangle2D> find(List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
+
+        // Fix for 211.pdf
+        for (Ruling r : horizontalRulingLines) {
+            if (r.getX2() < r.getX1()) {
+                double a = r.getX2();
+                r.x2 = (float) r.getX1();
+                r.x1 = (float) a;
+            }
+        }
+
+        List<Rectangle2D> foundRectangles = new ArrayList<>();
+        Map<Point2D, Ruling[]> intersectionPoints = Ruling.findIntersections(horizontalRulingLines, verticalRulingLines);
+        List<Point2D> intersectionPointsList = new ArrayList<>(intersectionPoints.keySet());
+        intersectionPointsList.sort(Y_FIRST_POINT_COMPARATOR);
+
+        for (int i = 0; i < intersectionPointsList.size(); i++) {
+            Point2D topLeft = intersectionPointsList.get(i);
+            Ruling[] hv = intersectionPoints.get(topLeft);
+
+            // CrossingPointsDirectlyBelow( topLeft );
+            List<Point2D> xPoints = new ArrayList<>();
+            // CrossingPointsDirectlyToTheRight( topLeft );
+            List<Point2D> yPoints = new ArrayList<>();
+
+            for (Point2D p : intersectionPointsList.subList(i, intersectionPointsList.size())) {
+                if (p.getX() == topLeft.getX() && p.getY() > topLeft.getY()) {
+                    xPoints.add(p);
+                }
+                if (p.getY() == topLeft.getY() && p.getX() > topLeft.getX()) {
+                    yPoints.add(p);
+                }
+            }
+            outer:
+            for (Point2D xPoint : xPoints) {
+                // is there a vertical edge b/w topLeft and xPoint?
+                if (!hv[1].equals(intersectionPoints.get(xPoint)[1])) {
+                    continue;
+                }
+                for (Point2D yPoint : yPoints) {
+                    // is there a horizontal edge b/w topLeft and yPoint ?
+                    if (!hv[0].equals(intersectionPoints.get(yPoint)[0])) {
+                        continue;
+                    }
+                    Point2D btmRight = new Point2D.Float((float) yPoint.getX(), (float) xPoint.getY());
+                    if (intersectionPoints.containsKey(btmRight)
+                        && intersectionPoints.get(btmRight)[0].equals(intersectionPoints.get(xPoint)[0])
+                        && intersectionPoints.get(btmRight)[1].equals(intersectionPoints.get(yPoint)[1])) {
+                        foundRectangles.add(new Rectangle2D.Double(topLeft.getX(), topLeft.getY(), btmRight.getX() - topLeft.getX(), btmRight.getY() - topLeft.getY()));
+                        break outer;
+                    }
+                }
+            }
+        }
+
+        // TODO create cells for vertical ruling lines with aligned endpoints at the top/bottom of a grid
+        // that aren't connected with an horizontal ruler?
+        // see: https://github.com/jazzido/tabula-extractor/issues/78#issuecomment-41481207
+
+        return foundRectangles;
+    }
+
+}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/SpreadsheetFinder.java

@@ -0,0 +1,172 @@
+package com.knecon.fforesight.service.layoutparser.processor.utils;
+
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.X_FIRST_POINT_COMPARATOR;
+import static com.knecon.fforesight.service.layoutparser.processor.utils.GeometricComparators.Y_FIRST_POINT_COMPARATOR;
+
+import java.awt.geom.Point2D;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Rectangle;
+
+public class SpreadsheetFinder {
+
+    private static final int MAX_OUTER_POINT_TOLERANCE = 10;
+    private static final float AREA_TOLERANCE = 0.001f;
+
+
+    public static List<Rectangle> findSpreadsheetsFromCells(List<? extends Rectangle> cells) {
+        // via: http://stackoverflow.com/questions/13746284/merging-multiple-adjacent-rectangles-into-one-polygon
+        List<Rectangle> rectangles = new ArrayList<>();
+        Set<Point2D> pointSet = new HashSet<>();
+        Map<Point2D, Point2D> edgesH = new HashMap<>();
+        Map<Point2D, Point2D> edgesV = new HashMap<>();
+
+        for (Rectangle cell : cells) {
+            for (Point2D pt : cell.getPoints()) {
+                if (pointSet.contains(pt)) { // shared vertex, remove it
+                    pointSet.remove(pt);
+                } else {
+                    pointSet.add(pt);
+                }
+            }
+        }
+
+        // X first sort
+        List<Point2D> pointsSortX = new ArrayList<>(pointSet);
+        pointsSortX.sort(X_FIRST_POINT_COMPARATOR);
+        // Y first sort
+        List<Point2D> pointsSortY = new ArrayList<>(pointSet);
+        pointsSortY.sort(Y_FIRST_POINT_COMPARATOR);
+
+        int i = 0;
+        while (i < pointSet.size()) {
+            float currY = (float) pointsSortY.get(i).getY();
+            while (i < pointSet.size() && DoubleComparisons.feq(pointsSortY.get(i).getY(), currY)) {
+                edgesH.put(pointsSortY.get(i), pointsSortY.get(i + 1));
+                edgesH.put(pointsSortY.get(i + 1), pointsSortY.get(i));
+                i += 2;
+            }
+        }
+
+        i = 0;
+        while (i < pointSet.size()) {
+            float currX = (float) pointsSortX.get(i).getX();
+            while (i < pointSet.size() && DoubleComparisons.feq(pointsSortX.get(i).getX(), currX)) {
+                edgesV.put(pointsSortX.get(i), pointsSortX.get(i + 1));
+                edgesV.put(pointsSortX.get(i + 1), pointsSortX.get(i));
+                i += 2;
+            }
+        }
+
+        // Get all the polygons
+        List<List<PolygonVertex>> polygons = new ArrayList<>();
+        Point2D nextVertex;
+        while (!edgesH.isEmpty()) {
+            ArrayList<PolygonVertex> polygon = new ArrayList<>();
+            Point2D first = edgesH.keySet()
+                    .iterator().next();
+            polygon.add(new PolygonVertex(first, Direction.HORIZONTAL));
+            edgesH.remove(first);
+
+            while (true) {
+                PolygonVertex curr = polygon.get(polygon.size() - 1);
+                PolygonVertex lastAddedVertex;
+                if (curr.direction == Direction.HORIZONTAL) {
+                    nextVertex = edgesV.get(curr.point);
+                    edgesV.remove(curr.point);
+                    lastAddedVertex = new PolygonVertex(nextVertex, Direction.VERTICAL);
+                } else {
+                    nextVertex = edgesH.get(curr.point);
+                    edgesH.remove(curr.point);
+                    lastAddedVertex = new PolygonVertex(nextVertex, Direction.HORIZONTAL);
+                }
+                polygon.add(lastAddedVertex);
+
+                if (lastAddedVertex.equals(polygon.get(0))) {
+                    // closed polygon
+                    polygon.remove(polygon.size() - 1);
+                    break;
+                }
+            }
+
+            for (PolygonVertex vertex : polygon) {
+                edgesH.remove(vertex.point);
+                edgesV.remove(vertex.point);
+            }
+            polygons.add(polygon);
+        }
+
+        // calculate grid-aligned minimum area rectangles for each found polygon
+        for (List<PolygonVertex> poly : polygons) {
+            float top = Float.MAX_VALUE;
+            float left = Float.MAX_VALUE;
+            float bottom = Float.MIN_VALUE;
+            float right = Float.MIN_VALUE;
+            for (PolygonVertex pt : poly) {
+                top = (float) Math.min(top, pt.point.getY());
+                left = (float) Math.min(left, pt.point.getX());
+                bottom = (float) Math.max(bottom, pt.point.getY());
+                right = (float) Math.max(right, pt.point.getX());
+            }
+
+            // do not add polygons with too many outer points as they are unlikely to be tables
+            if (poly.size() <= MAX_OUTER_POINT_TOLERANCE) {
+                rectangles.add(new Rectangle(top - AREA_TOLERANCE, left - AREA_TOLERANCE, right - left + 2 * AREA_TOLERANCE, bottom - top + 2 * AREA_TOLERANCE));
+            }
+        }
+        return rectangles;
+    }
+
+
+    private enum Direction {
+        HORIZONTAL,
+        VERTICAL
+    }
+
+    static class PolygonVertex {
+
+        Point2D point;
+        Direction direction;
+
+
+        PolygonVertex(Point2D point, Direction direction) {
+
+            this.direction = direction;
+            this.point = point;
+        }
+
+
+        @Override
+        public boolean equals(Object other) {
+
+            if (this == other) {
+                return true;
+            }
+            if (!(other instanceof PolygonVertex)) {
+                return false;
+            }
+            return this.point.equals(((PolygonVertex) other).point);
+        }
+
+
+        @Override
+        public int hashCode() {
+
+            return this.point.hashCode();
+        }
+
+
+        @Override
+        public String toString() {
+
+            return String.format("%s[point=%s,direction=%s]", this.getClass().getName(), this.point.toString(), this.direction.toString());
+        }
+
+    }
+
+}

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/TableMergingUtility.java

@@ -30,8 +30,6 @@ public class TableMergingUtility {
             if (consecutiveTable.getColCount() == originalTablePageBlock.getColCount() && !hasTableHeader(consecutiveTable) && outerBoundaryAlignsX(originalTablePageBlock,
                     consecutiveTable)) {
                 consecutiveTablesWithSameColCountAndHeaders.add(consecutiveTable);
-            } else {
-                break;
             }
         }
         return Stream.concat(Stream.of(originalTablePageBlock), consecutiveTablesWithSameColCountAndHeaders.stream()).toList();

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/TextPositionOperations.java

@@ -3,6 +3,7 @@ package com.knecon.fforesight.service.layoutparser.processor.utils;
 import java.util.List;
 import java.util.stream.Collectors;
 
+
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 
@@ -22,4 +23,10 @@ public class TextPositionOperations {
         return sequence;
     }
 
+
+    public static List<TextPositionSequence> mergeTextPositionSequence(List<TextPageBlock> textBlocks) {
+
+        return textBlocks.stream().flatMap(tb -> tb.getSequences().stream()).collect(Collectors.toList());
+    }
+
 }

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/TextPositionSequenceComparator.java

@@ -46,8 +46,8 @@ public class TextPositionSequenceComparator implements Comparator<TextPositionSe
         float pos2YBottom = pos2.getMaxYDirAdj();
 
         // note that the coordinates have been adjusted so 0,0 is in upper left
-        float pos1YTop = pos1YBottom - pos1.getTextHeight();
-        float pos2YTop = pos2YBottom - pos2.getTextHeight();
+        float pos1YTop = pos1YBottom - pos1.getTextHeightNoPadding();
+        float pos2YTop = pos2YBottom - pos2.getTextHeightNoPadding();
 
         float yDifference = Math.abs(pos1YBottom - pos2YBottom);
 

layoutparser-service/layoutparser-service-processor/src/main/java/com/knecon/fforesight/service/layoutparser/processor/utils/UnionFind.java

@@ -0,0 +1,44 @@
+package com.knecon.fforesight.service.layoutparser.processor.utils;
+
+import java.util.HashMap;
+import java.util.Map;
+
+// simple implementation of a disjoint-set data structure
+// https://en.wikipedia.org/wiki/Disjoint-set_data_structure
+public class UnionFind<T> {
+
+    Map<T, T> parents = new HashMap<>();
+    Map<T, Integer> numberOfObjects = new HashMap<>();
+
+
+    public T find(T node) {
+
+        if (!parents.containsKey(node)) {
+            parents.put(node, node);
+            numberOfObjects.put(node, 1);
+        }
+        if (!node.equals(parents.get(node))) {
+            parents.put(node, find(parents.get(node)));
+        }
+        return parents.get(node);
+    }
+
+
+    public void union(T node1, T node2) {
+
+        T root1 = find(node1);
+        T root2 = find(node2);
+
+        if (!root1.equals(root2)) {
+            if (numberOfObjects.getOrDefault(root1, 1) < numberOfObjects.getOrDefault(root2, 1)) {
+                parents.put(root1, root2);
+                numberOfObjects.put(root2, numberOfObjects.get(root2) + numberOfObjects.get(root1));
+            } else {
+                parents.put(root2, root1);
+                numberOfObjects.put(root1, numberOfObjects.get(root1) + numberOfObjects.get(root2));
+            }
+        }
+    }
+
+}
+

layoutparser-service/layoutparser-service-server/build.gradle.kts

@@ -3,40 +3,53 @@ import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
 plugins {
     application
     id("com.knecon.fforesight.java-conventions")
-    id("org.springframework.boot") version "3.1.3"
+    id("org.springframework.boot") version "3.1.5"
     id("io.spring.dependency-management") version "1.1.3"
     id("org.sonarqube") version "4.3.0.3225"
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 //    id("org.graalvm.buildtools.native") version "0.9.23"
 }
 
 description = "layoutparser-service-server"
 
+configurations {
+    all {
+        exclude(group = "commons-logging", module = "commons-logging")
+        exclude(group = "org.springframework.boot", module = "spring-boot-starter-log4j2")
+        exclude(group = "com.iqser.red.commons", module = "logging-commons")
+    }
+}
+
+
 val jacksonVersion = "2.15.2"
 val pdfBoxVersion = "3.0.0"
+val springBootStarterVersion = "3.1.5"
 
 dependencies {
     implementation(project(":layoutparser-service-processor"))
     implementation(project(":layoutparser-service-internal-api"))
 
     implementation("com.iqser.red.commons:storage-commons:2.45.0")
-    implementation("com.knecon.fforesight:tenant-commons:0.14.0")
-    implementation("org.springframework.boot:spring-boot-starter-actuator:3.1.3")
+    implementation("com.knecon.fforesight:tenant-commons:0.21.0")
+    implementation("com.knecon.fforesight:tracing-commons:0.5.0")
+    implementation("org.springframework.boot:spring-boot-starter-actuator:${springBootStarterVersion}")
+    implementation("org.springframework.boot:spring-boot-starter-amqp:${springBootStarterVersion}")
     implementation("com.amazonaws:aws-java-sdk-s3:1.12.536")
-    implementation("org.springframework.boot:spring-boot-starter-amqp:3.1.3")
     implementation("org.springframework.cloud:spring-cloud-starter-openfeign:4.0.4")
+    implementation("net.logstash.logback:logstash-logback-encoder:7.4")
 
     // for integration testing only
+    testImplementation(project(":viewer-doc-processor"))
     testImplementation(project(":layoutparser-service-internal-api"))
 
-    testImplementation("org.springframework.boot:spring-boot-starter-amqp:3.1.3")
+    testImplementation("org.springframework.boot:spring-boot-starter-amqp:${springBootStarterVersion}")
+    testImplementation("org.springframework.boot:spring-boot-starter-test:${springBootStarterVersion}")
     testImplementation("com.iqser.red.service:persistence-service-shared-api-v1:2.144.0")
     testImplementation("com.iqser.red.commons:jackson-commons:1.3.0")
     testImplementation("com.fasterxml.jackson.module:jackson-module-afterburner:${jacksonVersion}")
     testImplementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${jacksonVersion}")
     testImplementation("org.apache.pdfbox:pdfbox:${pdfBoxVersion}")
     testImplementation("org.apache.pdfbox:pdfbox-tools:${pdfBoxVersion}")
-    testImplementation("org.springframework.boot:spring-boot-starter-test:3.1.3")
 }
 
 /*

layoutparser-service/layoutparser-service-server/src/main/java/com/knecon/fforesight/service/layoutparser/server/Application.java

@@ -5,6 +5,7 @@ import org.springframework.boot.actuate.autoconfigure.security.servlet.Managemen
 import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;
+import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Import;
 
 import com.amazonaws.services.s3.model.metrics.MetricsConfiguration;
@@ -13,6 +14,9 @@ import com.knecon.fforesight.service.layoutparser.processor.LayoutParsingService
 import com.knecon.fforesight.service.layoutparser.server.queue.MessagingConfiguration;
 import com.knecon.fforesight.tenantcommons.MultiTenancyAutoConfiguration;
 
+import io.micrometer.observation.ObservationRegistry;
+import io.micrometer.observation.aop.ObservedAspect;
+
 @ImportAutoConfiguration({MultiTenancyAutoConfiguration.class})
 @Import({MetricsConfiguration.class, StorageAutoConfiguration.class, LayoutParsingServiceProcessorConfiguration.class, MessagingConfiguration.class})
 @SpringBootApplication(exclude = {SecurityAutoConfiguration.class, ManagementWebSecurityAutoConfiguration.class})
@@ -23,4 +27,11 @@ public class Application {
         SpringApplication.run(Application.class, args);
     }
 
+
+    @Bean
+    public ObservedAspect observedAspect(ObservationRegistry observationRegistry) {
+
+        return new ObservedAspect(observationRegistry);
+    }
+
 }

layoutparser-service/layoutparser-service-server/src/main/java/com/knecon/fforesight/service/layoutparser/server/queue/MessagingConfiguration.java

@@ -24,8 +24,7 @@ public class MessagingConfiguration {
     @Bean
     public Queue layoutparsingResponseQueue() {
 
-        return QueueBuilder.durable(LayoutParsingQueueNames.LAYOUT_PARSING_FINISHED_EVENT_QUEUE)//
-                .withArgument("x-dead-letter-exchange", "").withArgument("x-dead-letter-routing-key", LayoutParsingQueueNames.LAYOUT_PARSING_DLQ).build();
+        return QueueBuilder.durable(LayoutParsingQueueNames.LAYOUT_PARSING_FINISHED_EVENT_QUEUE).build();
     }
 
 

layoutparser-service/layoutparser-service-server/src/main/resources/application.yml

@@ -5,10 +5,19 @@ tenant-user-management-service.url: "http://tenant-user-management-service:8080/
 
 fforesight.tenants.remote: true
 
+logging.pattern.level: "%5p [${spring.application.name},%X{traceId:-},%X{spanId:-}]"
+
+logging.type: ${LOGGING_TYPE:CONSOLE}
+kubernetes.namespace: ${NAMESPACE:default}
+project.version: 1.0-SNAPSHOT
+
+
 server:
   port: 8080
 
 spring:
+  application:
+    name: layoutparser-service
   main:
     allow-circular-references: true # FIXME
   rabbitmq:
@@ -32,6 +41,13 @@ management:
     prometheus.enabled: ${monitoring.enabled:false}
     health.enabled: true
   endpoints.web.exposure.include: prometheus, health
+  tracing:
+    enabled: ${TRACING_ENABLED:false}
+    sampling:
+      probability: ${TRACING_PROBABILITY:1.0}
+  otlp:
+    tracing:
+      endpoint: ${OTLP_ENDPOINT:http://otel-collector-opentelemetry-collector.otel-collector:4318/v1/traces}
 
 
 storage:

layoutparser-service/layoutparser-service-server/src/main/resources/logback-spring.xml

@@ -0,0 +1,17 @@
+<configuration>
+
+    <springProperty scope="configuration" name="logType" source="logging.type"/>
+    <springProperty scope="context" name="application.name" source="spring.application.name"/>
+    <springProperty scope="context" name="version" source="project.version"/>
+    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
+    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>
+
+    <appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
+        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
+    </appender>
+
+    <root level="INFO">
+        <appender-ref ref="${logType}"/>
+    </root>
+
+</configuration>

layoutparser-service/layoutparser-service-server/src/test/java/com/knecon/fforesight/service/layoutparser/server/BdrJsonBuildTest.java

@@ -29,6 +29,7 @@ import com.knecon.fforesight.service.layoutparser.processor.python_api.model.ima
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;
 import com.knecon.fforesight.service.layoutparser.processor.services.factory.DocumentGraphFactory;
 import com.knecon.fforesight.service.layoutparser.processor.services.mapper.TaasDocumentDataMapper;
+import com.knecon.fforesight.service.layoutparser.processor.utils.DrawingOptions;
 import com.knecon.fforesight.service.layoutparser.server.utils.AbstractTest;
 import com.knecon.fforesight.service.layoutparser.server.utils.visualizations.PdfDraw;
 
@@ -46,12 +47,11 @@ public class BdrJsonBuildTest extends AbstractTest {
     @SneakyThrows
     protected Document buildGraph(File file) {
 
-        try (PDDocument pdDocument = Loader.loadPDF(file)) {
-            return DocumentGraphFactory.buildDocumentGraph(layoutParsingPipeline.parseLayout(LayoutParsingType.TAAS,
-                    pdDocument,
-                    new ImageServiceResponse(),
-                    new TableServiceResponse()));
-        }
+        return DocumentGraphFactory.buildDocumentGraph(layoutParsingPipeline.parseLayout(LayoutParsingType.TAAS,
+                file,
+                new ImageServiceResponse(),
+                new TableServiceResponse(),
+                file.toString()));
     }
 
 
@@ -112,7 +112,7 @@ public class BdrJsonBuildTest extends AbstractTest {
 
         try (PDDocument pdDocument = Loader.loadPDF(file); var outputStream = new FileOutputStream(resultingFileName)) {
             PdfDraw.drawDocumentGraph(pdDocument, document);
-            PdfDraw.drawTextBlock(pdDocument, textBlock, PdfDraw.Options.builder().stroke(true).strokeWidth(0.1f).strokeColor(Color.YELLOW).build());
+            PdfDraw.drawTextBlock(pdDocument, textBlock, DrawingOptions.builder().stroke(true).strokeWidth(0.1f).strokeColor(Color.YELLOW).build());
             pdDocument.save(outputStream);
         }
     }

layoutparser-service/layoutparser-service-server/src/test/java/com/knecon/fforesight/service/layoutparser/server/HeadlinesGoldStandardIntegrationTest.java

@@ -6,7 +6,6 @@ import java.util.List;
 import java.util.Set;
 import java.util.stream.Collectors;
 
-import org.apache.pdfbox.Loader;
 import org.assertj.core.api.Assertions;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
@@ -96,9 +95,10 @@ public class HeadlinesGoldStandardIntegrationTest {
         goldStandardLog.getRedactionLogEntry().forEach(e -> goldStandardHeadlines.add(new Headline(e.getPositions().get(0).getPage(), e.getValue())));
 
         Document documentGraph = DocumentGraphFactory.buildDocumentGraph(layoutParsingPipeline.parseLayout(LayoutParsingType.REDACT_MANAGER,
-                Loader.loadPDF(pdfFileResource.getFile()),
+                pdfFileResource.getFile(),
                 new ImageServiceResponse(),
-                new TableServiceResponse()));
+                new TableServiceResponse(),
+                filePath));
 
         var foundHeadlines = documentGraph.streamAllSubNodes()
                 .map(SemanticNode::getHeadline)
@@ -148,6 +148,7 @@ public class HeadlinesGoldStandardIntegrationTest {
     @EqualsAndHashCode
     @AllArgsConstructor
     @ToString
+    @SuppressWarnings("PMD.AvoidFieldNameMatchingTypeName")
     private class Headline {
 
         private int page;