RED-8666

2024-03-04 16:31:54 +01:00 · 2024-03-04 09:23:30 +01:00 · 2024-02-29 18:30:22 +01:00 · 2024-02-29 13:01:36 +01:00 · 2024-02-28 17:49:23 +01:00 · 2024-02-28 15:28:13 +01:00
168 changed files with 5593 additions and 1338 deletions
@@ -10,7 +10,7 @@ deploy:
  script:
    - echo "Building with gradle version ${BUILDVERSION}"
    - gradle -Pversion=${BUILDVERSION} publish
-    - gradle bootBuildImage --cleanCache --publishImage -PbuildbootDockerHostNetwork=true -Pversion=${BUILDVERSION}
+    - gradle bootBuildImage --publishImage -PbuildbootDockerHostNetwork=true -Pversion=${BUILDVERSION}
    - echo "BUILDVERSION=$BUILDVERSION" >> version.env
  artifacts:
    reports:
@@ -18,4 +18,4 @@ deploy:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH 
    - if: $CI_COMMIT_BRANCH =~ /^release/
-    - if: $CI_COMMIT_TAG
+    - if: $CI_COMMIT_TAG
@@ -1 +1,89 @@
+# PDF Layout Parser Micro-Service: layout-parser  

+## Introduction
+The layout-parser micro-service is a powerful tool designed to efficiently extract structured information from PDF documents. Written in Java and utilizing Spring Boot 3, Apache PDFBox, and RabbitMQ, this micro-service excels at parsing PDFs and organizing their content into a meaningful and coherent layout structure. Notably, the layout-parser micro-service distinguishes itself by relying solely on advanced algorithms, rather than machine learning techniques.
+
+### Key Steps in the PDF Layout Parsing Process:
+
+* **Text Position Extraction:**  
+The micro-service leverages Apache PDFBox to extract precise text positions for each individual character within the PDF document.
+
+* **Word Segmentation and Text Block Formation:**  
+Employing an array of diverse algorithms, the micro-service initially identifies and segments words, creating distinct text blocks.
+
+* **Text Block Classification:**  
+The segmented text blocks are then subjected to classification algorithms. These algorithms categorize the text blocks based on their content and visual properties, distinguishing between sections, subsections, headlines, paragraphs, images, tables, table cells, headers, and footers.
+
+* **Layout Coherence Establishment:**  
+The classified text blocks are subsequently orchestrated into a cohesive layout structure. This process involves arranging sections, subsections, paragraphs, images, and other elements in a logical and structured manner.
+
+* **Output Generation in Various Formats:**  
+Once the layout structure is established, the micro-service generates output in multiple formats. These formats are designed for seamless integration with downstream micro-services. The supported formats include JSON, XML, and others, ensuring flexibility in downstream data consumption.
+
+### Optional Enhancements:
+
+* **ML-Based Table Extraction:**  
+For enhanced results, users have the option to incorporate machine learning-based table extraction. This feature can be activated by providing ML-generated results as a JSON file, which are then integrated seamlessly into the layout structure.
+
+* **Image Classification using ML:**  
+Additionally, for more accurate image classification, users can optionally feed ML-generated image classification results into the micro-service. Similar to the table extraction option, the micro-service processes the pre-parsed results in JSON format, thus optimizing the accuracy of image content identification.
+
+In conclusion, the layout-parser micro-service is a versatile PDF layout parsing solution crafted entirely around advanced algorithms, without reliance on machine learning. It proficiently extracts text positions, segments content into meaningful blocks, classifies these blocks, arranges them coherently, and outputs structured data for downstream micro-services. Optional integration with ML-generated table extractions and image classifications further enhances its capabilities.
+
+
+
+
+
+
+## Installation
+
+### Prerequisites
+
+Before building and using the layout-parser micro-service, please ensure you have the following software and tools installed:
+
+Java Development Kit (JDK) 17 or later
+Gradle build tool (preinstalled)
+Build and Test
+To build and test the micro-service, follow these steps:
+
+### Clone the Repository:
+
+bash
+```
+git clone ssh://git@git.knecon.com:22222/fforesight/layout-parser.git
+cd layout-parser
+```
+### Build the Project:
+Use the following command to build the project using Gradle:
+
+```
+gradle clean build
+```
+### Run Tests:
+Run the test suite using the following command:
+```
+gradle test
+```
+## Building a Custom Docker Image
+To create a custom Docker image for the layout-parser micro-service, execute the provided script:
+
+### Ensure Docker is Installed:
+Ensure that Docker is installed and running on your system.
+
+### Run the Image Building Script:
+Execute the publish-custom-image script in the project directory:
+
+```
+./publish-custom-image
+```
+## Publishing to Internal Maven Repository
+To publish the layout-parser micro-service to your internal Maven repository, execute the following command:
+
+```
+gradle -Pversion=buildVersion publish
+```
+Replace buildVersion with the desired version number.
+
+## Additional Notes
+Make sure to configure any necessary application properties before deploying the micro-service.
+For advanced usage and configurations, refer to Kilian or Dom or preferably the source code.
@@ -24,6 +24,8 @@ tasks.named<Test>("test") {
    reports {
        junitXml.outputLocation.set(layout.buildDirectory.dir("reports/junit"))
    }
+    minHeapSize = "512m"
+    maxHeapSize = "2048m"
 }

 tasks.test {
@@ -9,12 +9,13 @@
    </description>

    <rule ref="category/java/errorprone.xml">
-        <exclude name="DataflowAnomalyAnalysis"/>
        <exclude name="MissingSerialVersionUID"/>
-        <exclude name="NullAssignment"/>
        <exclude name="AvoidLiteralsInIfCondition"/>
        <exclude name="AvoidDuplicateLiterals"/>
-        <exclude name="AvoidFieldNameMatchingMethodName"/>
+        <exclude name="NullAssignment"/>
+        <exclude name="AssignmentInOperand"/>
+        <exclude name="BeanMembersShouldSerialize"/>
    </rule>

-</ruleset>
+</ruleset>
+
@@ -10,14 +10,14 @@


    <rule ref="category/java/errorprone.xml">
-        <exclude name="DataflowAnomalyAnalysis"/>
        <exclude name="MissingSerialVersionUID"/>
-        <exclude name="NullAssignment"/>
        <exclude name="AvoidLiteralsInIfCondition"/>
        <exclude name="AvoidDuplicateLiterals"/>
        <exclude name="AvoidFieldNameMatchingMethodName"/>
-        <exclude name="AvoidFieldNameMatchingTypeName"/>
+        <exclude name="NullAssignment"/>
+        <exclude name="AssignmentInOperand"/>
        <exclude name="TestClassWithoutTestCases"/>
+        <exclude name="BeanMembersShouldSerialize"/>
    </rule>

-</ruleset>
+</ruleset>
@@ -1,6 +1,10 @@
 plugins {
    id("com.knecon.fforesight.java-conventions")
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 }

 description = "layoutparser-service-internal-api"
+
+dependencies {
+    implementation("io.swagger.core.v3:swagger-annotations:2.2.15")
+}
@@ -1,5 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import java.io.Serializable;
+
+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
@@ -10,11 +13,16 @@ import lombok.experimental.FieldDefaults;
@Builder
@AllArgsConstructor
@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
-public class DocumentData {
+@Schema(description = "Object containing the complete document layout parsing information. It is split into 4 categories, structure, text, positions and pages: " + "The document tree structure of SemanticNodes such as Section, Paragraph, Headline, etc. " + "The text, which is stored as separate blocks of data. " + "The text positions, which are also stored as separate blocks. The Blocks are equal to the text blocks in length and order. " + "The page information.")
+public class DocumentData implements Serializable {

+    @Schema(description = "Contains information about the document's pages.")
    DocumentPage[] documentPages;
+    @Schema(description = "Contains information about the document's text.")
    DocumentTextData[] documentTextData;
+    @Schema(description = "Contains information about the document's text positions.")
    DocumentPositionData[] documentPositions;
+    @Schema(description = "Contains information about the document's semantic structure.")
    DocumentStructure documentStructure;

 }
@@ -1,5 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import java.io.Serializable;
+
+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
@@ -12,11 +15,16 @@ import lombok.experimental.FieldDefaults;
@NoArgsConstructor
@AllArgsConstructor
@FieldDefaults(level = AccessLevel.PRIVATE)
-public class DocumentPage {
+@Schema(description = "Object containing information about the document's pages.")
+public class DocumentPage implements Serializable {

+    @Schema(description = "The page number, starting with 1.")
    int number;
+    @Schema(description = "The page height in PDF user units.", example = "792")
    int height;
+    @Schema(description = "The page width in PDF user units.", example = "694")
    int width;
+    @Schema(description = "The page rotation as specified by the PDF.", example = "90", allowableValues = {"0", "90", "180", "270"})
    int rotation;

 }
@@ -1,5 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import java.io.Serializable;
+
+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
@@ -12,10 +15,14 @@ import lombok.experimental.FieldDefaults;
@NoArgsConstructor
@AllArgsConstructor
@FieldDefaults(level = AccessLevel.PRIVATE)
-public class DocumentPositionData {
+@Schema(description = "Object containing text positional information of a specific text block. A document is split into multiple text blocks, which are supposed to be read in order. Every text block can only occur on a single page.")
+public class DocumentPositionData implements Serializable {

+    @Schema(description = "Identifier of the text block.")
    Long id;
+    @Schema(description = "For each string coordinate in the search text of the text block, the array contains an entry relating the string coordinate to the position coordinate. This is required due to the text and position coordinates not being equal.")
    int[] stringIdxToPositionIdx;
+    @Schema(description = "The bounding box for each glyph as a rectangle. This matrix is of size (n,4), where n is the number of glyphs in the text block. The second dimension specifies the rectangle with the value x, y, width, height, with x, y specifying the lower left corner. In order to access this information, the stringIdxToPositionIdx array must be used to transform the coordinates.")
    float[][] positions;

 }
@@ -1,11 +1,13 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

 import java.awt.geom.Rectangle2D;
+import java.io.Serializable;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Map;
 import java.util.stream.Stream;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
@@ -18,19 +20,22 @@ import lombok.experimental.FieldDefaults;
@NoArgsConstructor
@AllArgsConstructor
@FieldDefaults(level = AccessLevel.PRIVATE)
-public class DocumentStructure {
+@Schema(description = "Object containing information about the parsed tree structure of the SemanticNodes, such as Section, Paragraph, Headline etc inside of the document.")
+public class DocumentStructure implements Serializable {

+    @Schema(description = "The root EntryData represents the Document.")
    EntryData root;

-
-    public static class TableProperties {
+    @Schema(description = "Object containing the extra field names, a table has in its properties field.")
+    public static class TableProperties implements Serializable {

        public static final String NUMBER_OF_ROWS = "numberOfRows";
        public static final String NUMBER_OF_COLS = "numberOfCols";

    }

-    public static class ImageProperties {
+    @Schema(description = "Object containing the extra field names, an Image has in its properties field.")
+    public static class ImageProperties implements Serializable {

        public static final String TRANSPARENT = "transparent";
        public static final String IMAGE_TYPE = "imageType";
@@ -39,7 +44,8 @@ public class DocumentStructure {

    }

-    public static class TableCellProperties {
+    @Schema(description = "Object containing the extra field names, a table cell has in its properties field.")
+    public static class TableCellProperties implements Serializable {

        public static final String B_BOX = "bBox";
        public static final String ROW = "row";
@@ -50,6 +56,7 @@ public class DocumentStructure {

    public static final String RECTANGLE_DELIMITER = ";";

+
    public static Rectangle2D parseRectangle2D(String bBox) {

        List<Float> floats = Arrays.stream(bBox.split(RECTANGLE_DELIMITER)).map(Float::parseFloat).toList();
@@ -93,13 +100,20 @@ public class DocumentStructure {
    @NoArgsConstructor
    @AllArgsConstructor
    @FieldDefaults(level = AccessLevel.PRIVATE)
-    public static class EntryData {
+    @Schema(description = "Object containing information of a SemanticNode and also structuring the layout with children.")
+    public static class EntryData implements Serializable {

+        @Schema(description = "Type of the semantic node.", allowableValues = {"DOCUMENT", "SECTION", "PARAGRAPH", "HEADLINE", "TABLE", "TABLE_CELL", "HEADER", "FOOTER", "IMAGE"})
        NodeType type;
+        @Schema(description = "Specifies the position in the parsed tree structure.", example = "[1, 0, 2]")
        int[] treeId;
+        @Schema(description = "Specifies the text block IDs associated with this semantic node. The value should be joined with the DocumentTextData/DocumentPositionData. Is empty, if no text block is directly associated with this semantic node. Only Paragraph, Headline, Header or Footer is directly associated with a text block.", example = "[1]")
        Long[] atomicBlockIds;
+        @Schema(description = "Specifies the pages this semantic node appears on. The value should be joined with the PageData.", example = "[1, 2, 3]")
        Long[] pageNumbers;
+        @Schema(description = "Some semantic nodes have additional information, this information is stored in this Map. The extra fields are specified by the Properties subclasses.", example = "For a Table: {\"numberOfRows\": 3, \"numberOfCols\": 4}")
        Map<String, String> properties;
+        @Schema(description = "All child Entries of this Entry.", example = "[1, 2, 3]")
        List<EntryData> children;


@@ -1,7 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import java.io.Serializable;

-
+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AccessLevel;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
@@ -14,14 +15,22 @@ import lombok.experimental.FieldDefaults;
@NoArgsConstructor
@AllArgsConstructor
@FieldDefaults(level = AccessLevel.PRIVATE)
-public class DocumentTextData {
+@Schema(description = "Object containing text information of a specific text block. A document is split into multiple text blocks, which are supposed to be read in order. Every text block can only occur on a single page.")
+public class DocumentTextData implements Serializable {

+    @Schema(description = "Identifier of the text block.")
    Long id;
+    @Schema(description = "The page the text block occurs on.")
    Long page;
+    @Schema(description = "The text the text block.")
    String searchText;
+    @Schema(description = "Each text block is assigned a number on a page, starting from 0.")
    int numberOnPage;
+    @Schema(description = "The text blocks are ordered, this number represents the start of the text block as a string offset.")
    int start;
+    @Schema(description = "The text blocks are ordered, this number represents the end of the text block as a string offset.")
    int end;
+    @Schema(description = "The line breaks in the text of this semantic node in string offsets. They are exclusive end. At the end of each semantic node there is an implicit linebreak.", example = "[5, 10]")
    int[] lineBreaks;

 }
@@ -1,8 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import java.io.Serializable;
 import java.util.Locale;

-public enum NodeType {
+public enum NodeType implements Serializable {
    DOCUMENT,
    SECTION,
    HEADLINE,
@@ -1,5 +1,6 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
 import lombok.Data;
@@ -9,9 +10,12 @@ import lombok.NoArgsConstructor;
@Builder
@NoArgsConstructor
@AllArgsConstructor
+@Schema(description = "Object containing a simplified version, which contains almost exclusively text, of the document structure Section class.")
 public class SimplifiedSectionText {

+    @Schema(description = "The number of this Section. This is used to map the simplified section text back to the original Section.")
    private int sectionNumber;
+    @Schema(description = "The text in this Section.")
    private String text;

 }
@@ -3,6 +3,7 @@ package com.knecon.fforesight.service.layoutparser.internal.api.data.redaction;
 import java.util.ArrayList;
 import java.util.List;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
 import lombok.Data;
@@ -12,9 +13,12 @@ import lombok.NoArgsConstructor;
@Builder
@NoArgsConstructor
@AllArgsConstructor
+@Schema(description = "Object containing a simplified version, which contains almost exclusively text, of the document structure.")
 public class SimplifiedText {

+    @Schema(description = "Number of pages in the entire document.")
    private int numberOfPages;
+    @Schema(description = "A List of simplified Sections, which contains almost exclusively text.")
    private List<SimplifiedSectionText> sectionTexts = new ArrayList<>();

 }
@@ -2,20 +2,29 @@ package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

 import java.util.List;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.Builder;
 import lombok.Data;

@Data
@Builder
+@Schema(description = "Object containing information about a Paragraph/Headline/Header/Footer.")
 public class ParagraphData {

+    @Schema(description = "The text of this Semantic Node, without any linebreaks.", example = "This is some text.")
    private String text;
+    @Schema(description = "A list of text ranges in string offsets. Every character in any of the ranges is bold.", example = "[0, 15]")
    List<Range> boldTextBoundaries;
+    @Schema(description = "A list of text ranges in string offsets. Every character in any of the ranges is italic.", example = "[0, 15]")
    List<Range> italicTextBoundaries;
+    @Schema(description = "The line breaks in the text of this semantic node in string offsets. They are exclusive end. At the end of each semantic node there is an implicit linebreak.", example = "[5, 10]")
    List<Integer> linebreaks;
+    @Schema(description = "The classification of this Paragraph.", allowableValues = "{paragraph, headline, header, footer}")
    private String classification;

+    @Schema(description = "Describes the text orientation of this semantic node. Any semantic node only has a single text orientation.", allowableValues = "{ZERO, QUARTER_CIRCLE, HALF_CIRCLE, THREE_QUARTER_CIRCLE}")
    private String orientation;
+    @Schema(description = "Describes the text direction in degrees of this semantic node. Any semantic node only has a single text direction.", minimum = "0", maximum = "359")
    private int textDirection;

 }
@@ -1,5 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

+import io.swagger.v3.oas.annotations.media.Schema;
+
+@Schema(description = "Object specifying the start and end offsets of a text range in string offsets.")
 public record Range(int start, int end) {

 }
@@ -2,6 +2,7 @@ package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

 import java.util.List;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
 import lombok.Data;
@@ -9,8 +10,12 @@ import lombok.Data;
@Builder
@Data
@AllArgsConstructor
+@Schema(description = "Object containing a simplified version of the document structure. This simplified form only knows Paragraphs and Tables. The Paragraph Objects might be a Paragraph, Headline, Header or Footer.")
 public class ResearchDocumentData {

+    @Schema(description = "File name of the original uploaded file.")
    String originalFile;
+    @Schema(description = "A List of all paragraphs/headline or table objects, that have been parsed in this document.")
    List<StructureObject> structureObjects;
+
 }
@@ -2,14 +2,19 @@ package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

 import java.util.List;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Data;

@Data
@AllArgsConstructor
+@Schema(description = "Object containing information about a Table Row.")
 public class RowData {

+    @Schema(description = "Boolean indicating whether this table row is classified as a header row.")
    boolean header;
+    @Schema(description = "A list of Objects containing information about the text in each cell of this row.")
    List<ParagraphData> cellText;
+    @Schema(description = "The bounding box of this StructureObject. Is always exactly 4 values representing x, y, w, h, where x, y specify the lower left corner.")
    float[] bBox;
 }
@@ -1,5 +1,6 @@
 package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Builder;
 import lombok.Data;
@@ -7,13 +8,20 @@ import lombok.Data;
@Data
@Builder
@AllArgsConstructor
+@Schema(description = "Object containing information about either a Paragraph/Headline/Header/Footer or a Table.")
 public class StructureObject {

+    @Schema(description = "The ID of this StructureObject.")
    Integer structureObjectNumber;
+    @Schema(description = "This value indicates the start of the string offsets in this Object, with respect to the reading order.")
    int page;
+    @Schema(description = "This stringOffset indicates the start of the string offsets in this Object, with respect to the reading order of the entire document. It is equal to the previous' StructureObject stringOffset + its length.")
    int stringOffset;
+    @Schema(description = "The bounding box of this StructureObject. Is always exactly 4 values representing x, y, w, h, where x, y specify the lower left corner.", example = "[100, 100, 50, 50]")
    float[] boundingBox;
+    @Schema(description = "Object containing information about a Paragraph/Headline/Header/Footer. Either this or table is null.")
    ParagraphData paragraph;
+    @Schema(description = "Object containing information about a Table. Either this or paragraph is null.")
    TableData table;

 }
@@ -2,15 +2,20 @@ package com.knecon.fforesight.service.layoutparser.internal.api.data.taas;

 import java.util.List;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.AllArgsConstructor;
 import lombok.Data;

@Data
@AllArgsConstructor
+@Schema(description = "Object containing information about a Table.")
 public class TableData {

+    @Schema(description = "A list of Objects containing information about all rows in this table.")
    List<RowData> rowData;
+    @Schema(description = "Number of columns in this table.")
    Integer numberOfCols;
+    @Schema(description = "Number of rows in this table.")
    Integer numberOfRows;

 }
@@ -2,9 +2,19 @@ package com.knecon.fforesight.service.layoutparser.internal.api.queue;

 import java.util.Map;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.Builder;

@Builder
-public record LayoutParsingFinishedEvent(Map<String, String> identifier, long duration, int numberOfPages, String message) {
+@Schema(description = "Object containing information about the layout parsing.")
+public record LayoutParsingFinishedEvent(
+        @Schema(description = "General purpose identifier. It is returned exactly the same way it is inserted with the LayoutParsingRequest.")
+        Map<String, String> identifier,//
+        @Schema(description = "The duration of a single layout parsing in ms.")
+        long duration,//
+        @Schema(description = "The number of pages of the parsed document.")
+        int numberOfPages,//
+        @Schema(description = "A general message. It contains some information useful for a developer, like the paths where the files are stored. Not meant to be machine readable.")
+        String message) {

 }
@@ -3,23 +3,43 @@ package com.knecon.fforesight.service.layoutparser.internal.api.queue;
 import java.util.Map;
 import java.util.Optional;

+import io.swagger.v3.oas.annotations.media.Schema;
 import lombok.Builder;
 import lombok.NonNull;

@Builder
+@Schema(description = "Object containing all storage paths the service needs to know.")
 public record LayoutParsingRequest(
+        @Schema(description = "Enum specifying the type of layout parsing to be performed.", allowableValues = "{RedactManager, DocuMine, TAAS}")//
        @NonNull LayoutParsingType layoutParsingType,
+
+        @Schema(description = "General purpose identifiers. They are not changed by the service at all and are returned as is in the response queue.")//
        Map<String, String> identifier,
-        @NonNull String originFileStorageId,
-        Optional<String> tablesFileStorageId,
-        Optional<String> imagesFileStorageId,
-        @NonNull String structureFileStorageId,
-        String researchDocumentStorageId,
-        @NonNull String textBlockFileStorageId,
-        @NonNull String positionBlockFileStorageId,
-        @NonNull String pageFileStorageId,
-        @NonNull String simplifiedTextStorageId,
-        @NonNull String viewerDocumentStorageId,
-        @NonNull String sectionGridStorageId) {
+
+        @Schema(description = "Path to the original PDF file.")//
+        @NonNull String originFileStorageId,//
+
+
+        @Schema(description = "Optional Path to the table extraction file.")//
+        Optional<String> tablesFileStorageId,//
+        @Schema(description = "Optional Path to the image classification file.")//
+        Optional<String> imagesFileStorageId,//
+
+        @Schema(description = "Optional Path to the the visual layout parsing service file") Optional<String> visualLayoutParsingFileId,//
+
+        @Schema(description = "Path where the Document Structure File will be stored.")//
+        @NonNull String structureFileStorageId,//
+        @Schema(description = "Path where the Research Data File will be stored.")//
+        String researchDocumentStorageId,//
+        @Schema(description = "Path where the Document Text File will be stored.")//
+        @NonNull String textBlockFileStorageId,//
+        @Schema(description = "Path where the Document Positions File will be stored.")//
+        @NonNull String positionBlockFileStorageId,//
+        @Schema(description = "Path where the Document Pages File will be stored.")//
+        @NonNull String pageFileStorageId,//
+        @Schema(description = "Path where the Simplified Text File will be stored.")//
+        @NonNull String simplifiedTextStorageId,//
+        @Schema(description = "Path where the Viewer Document PDF will be stored.")//
+        @NonNull String viewerDocumentStorageId) {

 }
@@ -3,5 +3,8 @@ package com.knecon.fforesight.service.layoutparser.internal.api.queue;
 public enum LayoutParsingType {
    REDACT_MANAGER,
    TAAS,
-    DOCUMINE
+    DOCUMINE,
+
+    DOCSTRUM,
+    DOCSTRUM_ROW_WISE
 }
@@ -1,26 +1,27 @@
 plugins {
    id("com.knecon.fforesight.java-conventions")
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 }

 description = "layoutparser-service-processor"

 val jacksonVersion = "2.15.2"
-val pdfBoxVersion = "3.0.0-RC1"
+val pdfBoxVersion = "3.0.0"

 dependencies {
    implementation(project(":layoutparser-service-internal-api"))
+    implementation(project(":viewer-doc-processor"))

-    implementation("com.iqser.red.service:persistence-service-shared-api-v1:2.138.0") {
+    implementation("com.iqser.red.service:persistence-service-shared-api-v1:2.144.0") {
        exclude("org.springframework.boot", "spring-boot-starter-security")
        exclude("org.springframework.boot", "spring-boot-starter-validation")
    }
-    implementation("com.knecon.fforesight:tenant-commons:0.10.0")
-    implementation("com.iqser.red.commons:storage-commons:2.36.0")
+    implementation("com.knecon.fforesight:tenant-commons:0.21.0")
+    implementation("com.iqser.red.commons:storage-commons:2.45.0")

    implementation("org.apache.pdfbox:pdfbox:${pdfBoxVersion}")
    implementation("org.apache.pdfbox:pdfbox-tools:${pdfBoxVersion}")
    implementation("com.fasterxml.jackson.module:jackson-module-afterburner:${jacksonVersion}")
    implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${jacksonVersion}")
-    implementation("org.springframework.boot:spring-boot-starter-web:3.1.2")
+    implementation("org.springframework.boot:spring-boot-starter-web:3.1.3")
 }
@@ -1,28 +1,34 @@
 package com.knecon.fforesight.service.layoutparser.processor;

+import static com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingType.DOCSTRUM;
+import static com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingType.DOCSTRUM_ROW_WISE;
 import static java.lang.String.format;

 import java.awt.geom.Rectangle2D;
-import java.io.ByteArrayOutputStream;
+import java.io.File;
 import java.io.IOException;
+import java.nio.file.Files;
 import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.concurrent.atomic.AtomicReference;

-import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
+import org.apache.pdfbox.Loader;
 import org.apache.pdfbox.pdmodel.PDDocument;
 import org.apache.pdfbox.pdmodel.PDPage;
 import org.apache.pdfbox.pdmodel.common.PDRectangle;
 import org.apache.pdfbox.pdmodel.documentinterchange.markedcontent.PDMarkedContent;
 import org.springframework.stereotype.Service;

+import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
 import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingFinishedEvent;
 import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingRequest;
 import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingType;
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationDocument;
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
+import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationSection;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
 import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
@@ -30,15 +36,17 @@ import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageB
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.CvTableParsingAdapter;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.ImageServiceResponseAdapter;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.adapter.VisualLayoutParsingAdapter;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.image.ImageServiceResponse;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
 import com.knecon.fforesight.service.layoutparser.processor.services.BodyTextFrameService;
 import com.knecon.fforesight.service.layoutparser.processor.services.RulingCleaningService;
-import com.knecon.fforesight.service.layoutparser.processor.services.SectionGridCreatorService;
 import com.knecon.fforesight.service.layoutparser.processor.services.SectionsBuilderService;
 import com.knecon.fforesight.service.layoutparser.processor.services.SimplifiedSectionTextService;
 import com.knecon.fforesight.service.layoutparser.processor.services.TableExtractionService;
+import com.knecon.fforesight.service.layoutparser.processor.services.blockification.DocstrumBlockificationService;
 import com.knecon.fforesight.service.layoutparser.processor.services.blockification.DocuMineBlockificationService;
 import com.knecon.fforesight.service.layoutparser.processor.services.blockification.RedactManagerBlockificationService;
 import com.knecon.fforesight.service.layoutparser.processor.services.blockification.TaasBlockificationService;
@@ -49,99 +57,191 @@ import com.knecon.fforesight.service.layoutparser.processor.services.factory.Doc
 import com.knecon.fforesight.service.layoutparser.processor.services.mapper.DocumentDataMapper;
 import com.knecon.fforesight.service.layoutparser.processor.services.mapper.TaasDocumentDataMapper;
 import com.knecon.fforesight.service.layoutparser.processor.services.parsing.PDFLinesTextStripper;
-import com.knecon.fforesight.service.layoutparser.processor.services.visualization.ViewerDocumentService;
+import com.knecon.fforesight.service.layoutparser.processor.services.visualization.LayoutGridService;
+import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;

+import io.micrometer.observation.Observation;
+import io.micrometer.observation.ObservationRegistry;
+import io.micrometer.observation.annotation.Observed;
+import lombok.AccessLevel;
 import lombok.RequiredArgsConstructor;
 import lombok.SneakyThrows;
+import lombok.experimental.FieldDefaults;
 import lombok.extern.slf4j.Slf4j;

+@SuppressWarnings("PMD.CloseResource")
@Slf4j
@Service
@RequiredArgsConstructor
+@FieldDefaults(makeFinal = true, level = AccessLevel.PRIVATE)
 public class LayoutParsingPipeline {

-    private final ImageServiceResponseAdapter imageServiceResponseAdapter;
-    private final CvTableParsingAdapter cvTableParsingAdapter;
-    private final LayoutParsingStorageService layoutParsingStorageService;
-    private final SectionsBuilderService sectionsBuilderService;
-    private final SectionGridCreatorService sectionGridCreatorService;
-    private final TaasClassificationService taasClassificationService;
-    private final RedactManagerClassificationService redactManagerClassificationService;
-    private final DocuMineClassificationService docuMineClassificationService;
-    private final SimplifiedSectionTextService simplifiedSectionTextService;
-    private final BodyTextFrameService bodyTextFrameService;
-    private final RulingCleaningService rulingCleaningService;
-    private final TableExtractionService tableExtractionService;
-    private final TaasBlockificationService taasBlockificationService;
-    private final DocuMineBlockificationService docuMineBlockificationService;
-    private final RedactManagerBlockificationService redactManagerBlockificationService;
-    private final ViewerDocumentService viewerDocumentService;
+    ImageServiceResponseAdapter imageServiceResponseAdapter;
+    CvTableParsingAdapter cvTableParsingAdapter;
+    LayoutParsingStorageService layoutParsingStorageService;
+    SectionsBuilderService sectionsBuilderService;
+    TaasClassificationService taasClassificationService;
+    RedactManagerClassificationService redactManagerClassificationService;
+    DocuMineClassificationService docuMineClassificationService;
+    SimplifiedSectionTextService simplifiedSectionTextService;
+    BodyTextFrameService bodyTextFrameService;
+    RulingCleaningService rulingCleaningService;
+    TableExtractionService tableExtractionService;
+    TaasBlockificationService taasBlockificationService;
+    DocuMineBlockificationService docuMineBlockificationService;
+    RedactManagerBlockificationService redactManagerBlockificationService;
+    DocstrumBlockificationService docstrumBlockificationService;
+    LayoutGridService layoutGridService;
+    ObservationRegistry observationRegistry;
+    VisualLayoutParsingAdapter visualLayoutParsingAdapter;


    public LayoutParsingFinishedEvent parseLayoutAndSaveFilesToStorage(LayoutParsingRequest layoutParsingRequest) throws IOException {

        long start = System.currentTimeMillis();
+        log.info("Starting layout parsing for {}", layoutParsingRequest.identifier());

-        try (PDDocument originDocument = layoutParsingStorageService.getOriginFile(layoutParsingRequest.originFileStorageId())) {
-            ImageServiceResponse imageServiceResponse = new ImageServiceResponse();
-            if (layoutParsingRequest.imagesFileStorageId().isPresent()) {
-                imageServiceResponse = layoutParsingStorageService.getImagesFile(layoutParsingRequest.imagesFileStorageId().get());
-            }
+        File originFile = layoutParsingStorageService.getOriginFile(layoutParsingRequest.originFileStorageId());
+        File viewerDocumentFile = layoutParsingStorageService.getViewerDocFile(layoutParsingRequest.viewerDocumentStorageId()).orElse(originFile);

-            TableServiceResponse tableServiceResponse = new TableServiceResponse();
-            if (layoutParsingRequest.tablesFileStorageId().isPresent()) {
-                tableServiceResponse = layoutParsingStorageService.getTablesFile(layoutParsingRequest.tablesFileStorageId().get());
-            }
-
-            ClassificationDocument classificationDocument = parseLayout(layoutParsingRequest.layoutParsingType(), originDocument, imageServiceResponse, tableServiceResponse);
-            Document documentGraph = DocumentGraphFactory.buildDocumentGraph(classificationDocument);
-
-            int numberOfPages = originDocument.getNumberOfPages();
-
-            layoutParsingStorageService.storeSectionGrid(layoutParsingRequest, sectionGridCreatorService.createSectionGrid(documentGraph));
-            layoutParsingStorageService.storeDocumentData(layoutParsingRequest, DocumentDataMapper.toDocumentData(documentGraph));
-            layoutParsingStorageService.storeSimplifiedText(layoutParsingRequest, simplifiedSectionTextService.toSimplifiedText(documentGraph));
-
-            try (var out = new ByteArrayOutputStream()) {
-                viewerDocumentService.createViewerDocument(originDocument, documentGraph, out);
-                layoutParsingStorageService.storeViewerDocument(layoutParsingRequest, out);
-            }
-
-            if (layoutParsingRequest.layoutParsingType().equals(LayoutParsingType.TAAS)) {
-                var researchDocumentData = TaasDocumentDataMapper.fromDocument(documentGraph);
-                layoutParsingStorageService.storeResearchDocumentData(layoutParsingRequest, researchDocumentData);
-            }
-
-            return LayoutParsingFinishedEvent.builder()
-                    .identifier(layoutParsingRequest.identifier())
-                    .numberOfPages(numberOfPages)
-                    .duration(System.currentTimeMillis() - start)
-                    .message(format("Layout parsing is finished and files have been saved with Ids:\n Structure: %s\nText: %s\nPositions: %s\nPageData: %s",
-                            layoutParsingRequest.structureFileStorageId(),
-                            layoutParsingRequest.textBlockFileStorageId(),
-                            layoutParsingRequest.positionBlockFileStorageId(),
-                            layoutParsingRequest.pageFileStorageId()))
-                    .build();
+        VisualLayoutParsingResponse visualLayoutParsingResponse = new VisualLayoutParsingResponse();
+        if (layoutParsingRequest.visualLayoutParsingFileId().isPresent()) {
+            visualLayoutParsingResponse = layoutParsingStorageService.getVisualLayoutParsingFile(layoutParsingRequest.visualLayoutParsingFileId().get());
        }
+
+        ImageServiceResponse imageServiceResponse = new ImageServiceResponse();
+        if (layoutParsingRequest.imagesFileStorageId().isPresent()) {
+            imageServiceResponse = layoutParsingStorageService.getImagesFile(layoutParsingRequest.imagesFileStorageId().get());
+        }
+
+        TableServiceResponse tableServiceResponse = new TableServiceResponse();
+        if (layoutParsingRequest.tablesFileStorageId().isPresent()) {
+            tableServiceResponse = layoutParsingStorageService.getTablesFile(layoutParsingRequest.tablesFileStorageId().get());
+        }
+
+        ClassificationDocument classificationDocument = parseLayout(layoutParsingRequest.layoutParsingType(),
+                originFile,
+                imageServiceResponse,
+                tableServiceResponse,
+                visualLayoutParsingResponse,
+                layoutParsingRequest.identifier().toString());
+
+        log.info("Building document graph for {}", layoutParsingRequest.identifier());
+
+        Document documentGraph = observeBuildDocumentGraph(classificationDocument);
+
+        log.info("Creating viewer document for {}", layoutParsingRequest.identifier());
+
+        layoutGridService.addLayoutGrid(viewerDocumentFile, documentGraph, viewerDocumentFile, false);
+
+        log.info("Storing resulting files for {}", layoutParsingRequest.identifier());
+
+        layoutParsingStorageService.storeDocumentData(layoutParsingRequest, DocumentDataMapper.toDocumentData(documentGraph));
+        layoutParsingStorageService.storeSimplifiedText(layoutParsingRequest, simplifiedSectionTextService.toSimplifiedText(documentGraph));
+        layoutParsingStorageService.storeViewerDocument(layoutParsingRequest, viewerDocumentFile);
+
+        if (layoutParsingRequest.layoutParsingType().equals(LayoutParsingType.TAAS)) {
+            log.info("Building research document data for {}", layoutParsingRequest.identifier());
+            var researchDocumentData = TaasDocumentDataMapper.fromDocument(documentGraph);
+            layoutParsingStorageService.storeResearchDocumentData(layoutParsingRequest, researchDocumentData);
+        }
+
+        if (!viewerDocumentFile.equals(originFile)) {
+            viewerDocumentFile.delete();
+        }
+        originFile.delete();
+
+        return LayoutParsingFinishedEvent.builder()
+                .identifier(layoutParsingRequest.identifier())
+                .numberOfPages(documentGraph.getNumberOfPages())
+                .duration(System.currentTimeMillis() - start)
+                .message(format("""
+                                Layout parsing has finished in %.02f s.
+                                identifiers: %s
+                                %s
+                                Files have been saved with Ids:
+                                Structure: %s
+                                Text: %s
+                                Positions: %s
+                                PageData: %s
+                                Simplified Text: %s
+                                Viewer Doc: %s""",
+                        ((float) (System.currentTimeMillis() - start)) / 1000,
+                        layoutParsingRequest.identifier(),
+                        buildSemanticNodeCountMessage(documentGraph.getNumberOfPages(), documentGraph.buildSemanticNodeCounts()),
+                        layoutParsingRequest.structureFileStorageId(),
+                        layoutParsingRequest.textBlockFileStorageId(),
+                        layoutParsingRequest.positionBlockFileStorageId(),
+                        layoutParsingRequest.pageFileStorageId(),
+                        layoutParsingRequest.simplifiedTextStorageId(),
+                        layoutParsingRequest.viewerDocumentStorageId()))
+                .build();
+
+    }
+
+
+    private Document observeBuildDocumentGraph(ClassificationDocument classificationDocument) {
+
+        AtomicReference<Document> documentReference = new AtomicReference<>();
+
+        Observation.createNotStarted("LayoutParsingPipeline", observationRegistry)
+                .contextualName("build-document-graph")
+                .observe(() -> documentReference.set(DocumentGraphFactory.buildDocumentGraph(classificationDocument)));
+
+        return documentReference.get();
+    }
+
+
+    private String buildSemanticNodeCountMessage(int numberOfPages, Map<NodeType, Long> semanticNodeCounts) {
+
+        return String.format("%d pages with %d sections, %d headlines, %d paragraphs, %d tables with %d cells, %d headers, and %d footers parsed",
+                numberOfPages,
+                semanticNodeCounts.get(NodeType.SECTION) == null ? 0 : semanticNodeCounts.get(NodeType.SECTION),
+                semanticNodeCounts.get(NodeType.HEADLINE) == null ? 0 : semanticNodeCounts.get(NodeType.HEADLINE),
+                semanticNodeCounts.get(NodeType.PARAGRAPH) == null ? 0 : semanticNodeCounts.get(NodeType.PARAGRAPH),
+                semanticNodeCounts.get(NodeType.TABLE) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE),
+                semanticNodeCounts.get(NodeType.TABLE_CELL) == null ? 0 : semanticNodeCounts.get(NodeType.TABLE_CELL),
+                semanticNodeCounts.get(NodeType.HEADER) == null ? 0 : semanticNodeCounts.get(NodeType.HEADER),
+                semanticNodeCounts.get(NodeType.FOOTER) == null ? 0 : semanticNodeCounts.get(NodeType.FOOTER));
    }


    @SneakyThrows
+    @Observed(name = "LayoutParsingPipeline", contextualName = "parse-layout")
    public ClassificationDocument parseLayout(LayoutParsingType layoutParsingType,
-                                              PDDocument originDocument,
+                                              File originFile,
                                              ImageServiceResponse imageServiceResponse,
-                                              TableServiceResponse tableServiceResponse) {
+                                              TableServiceResponse tableServiceResponse,
+                                              VisualLayoutParsingResponse visualLayoutParsingResponse,
+                                              String identifier) {

+        PDDocument originDocument = openDocument(originFile);
+        addNumberOfPagesToTrace(originDocument.getNumberOfPages(), Files.size(originFile.toPath()));
        Map<Integer, List<TableCells>> pdfTableCells = cvTableParsingAdapter.buildCvParsedTablesPerPage(tableServiceResponse);
        Map<Integer, List<ClassifiedImage>> pdfImages = imageServiceResponseAdapter.buildClassifiedImagesPerPage(imageServiceResponse);
+        Map<Integer, List<ClassifiedImage>> signatures = new HashMap<>();
+        if (signatures.size() > 0) {
+            visualLayoutParsingAdapter.buildExtractedSignaturesPerPage(visualLayoutParsingResponse);
+        }

        ClassificationDocument classificationDocument = new ClassificationDocument();
        List<ClassificationPage> classificationPages = new ArrayList<>();

-        originDocument.setAllSecurityToBeRemoved(true);
        long pageCount = originDocument.getNumberOfPages();

        for (int pageNumber = 1; pageNumber <= pageCount; pageNumber++) {
+
+            if (pageNumber % 100 == 0) {
+                // re-open document every once in a while to save on RAM. This has no significant performance impact.
+                // This is due to PDFBox caching all images and some other stuff with Soft References. This dereferences them and forces the freeing of memory.
+                originDocument.close();
+                originDocument = openDocument(originFile);
+            }
+
+            if (pageNumber % 100 == 0 || pageNumber == pageCount || pageNumber == 1) {
+                log.info("Extracting text on Page {} for {}", pageNumber, identifier);
+            }
+
            classificationDocument.setPages(classificationPages);
            PDFLinesTextStripper stripper = new PDFLinesTextStripper();
            PDPage pdPage = originDocument.getPage(pageNumber - 1);
@@ -160,18 +260,14 @@ public class LayoutParsingPipeline {
            boolean isLandscape = pdr.getWidth() > pdr.getHeight() && (rotation == 0 || rotation == 180) || pdr.getHeight() > pdr.getWidth() && (rotation == 90 || rotation == 270);

            PDRectangle cropbox = pdPage.getCropBox();
-            CleanRulings cleanRulings = rulingCleaningService.getCleanRulings(pdfTableCells.get(pageNumber),
-                    stripper.getRulings(),
-                    stripper.getMinCharWidth(),
-                    stripper.getMaxCharHeight());
+            CleanRulings cleanRulings = rulingCleaningService.getCleanRulings(pdfTableCells.get(pageNumber), stripper.getRulings());

            ClassificationPage classificationPage = switch (layoutParsingType) {
-                case REDACT_MANAGER ->
-                        redactManagerBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
-                case TAAS ->
-                        taasBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
-                case DOCUMINE ->
-                        docuMineBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
+                case REDACT_MANAGER -> redactManagerBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
+                case TAAS -> taasBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
+                case DOCUMINE -> docuMineBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical());
+                case DOCSTRUM -> docstrumBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical(), true);
+                case DOCSTRUM_ROW_WISE -> docstrumBlockificationService.blockify(stripper.getTextPositionSequences(), cleanRulings.getHorizontal(), cleanRulings.getVertical(), false);
            };
            classificationPage.setCleanRulings(cleanRulings);
            classificationPage.setRotation(rotation);
@@ -189,40 +285,85 @@ public class LayoutParsingPipeline {
                imageServiceResponseAdapter.findOcr(classificationPage);
            }

-            tableExtractionService.extractTables(cleanRulings, classificationPage, layoutParsingType);
+            if (signatures.containsKey(pageNumber)) {
+                classificationPage.setImages(signatures.get(pageNumber));
+            }
+
+            tableExtractionService.extractTables(cleanRulings, classificationPage);
+
+            if (layoutParsingType == DOCSTRUM || layoutParsingType == DOCSTRUM_ROW_WISE) {
+                // docstrumBlockificationService.combineBlocks(classificationPage); //todo 8666
+            }
+
            buildPageStatistics(classificationPage);
            increaseDocumentStatistics(classificationPage, classificationDocument);

            classificationPages.add(classificationPage);
        }

-        bodyTextFrameService.setBodyTextFrames(classificationDocument, layoutParsingType);
+        originDocument.close();

+        log.info("Calculating BodyTextFrame for {}", identifier);
+        bodyTextFrameService.setBodyTextFrames(classificationDocument, layoutParsingType);
+        log.info("Classify TextBlocks for {}", identifier);
        switch (layoutParsingType) {
            case TAAS -> taasClassificationService.classifyDocument(classificationDocument);
            case DOCUMINE -> docuMineClassificationService.classifyDocument(classificationDocument);
            case REDACT_MANAGER -> redactManagerClassificationService.classifyDocument(classificationDocument);
+            case DOCSTRUM_ROW_WISE -> redactManagerClassificationService.classifyDocument(classificationDocument);
        }

-        sectionsBuilderService.buildSections(classificationDocument);
-        sectionsBuilderService.addImagesToSections(classificationDocument);
+        log.info("Building Sections for {}", identifier);

+        if (layoutParsingType == DOCSTRUM || layoutParsingType == DOCSTRUM_ROW_WISE) {
+            // Currently for debugging return paragraphs as sections, because there is a merging logic in sectionBuilder
+            List<ClassificationSection> sections = new ArrayList<>();
+            for (var page : classificationPages) {
+                page.getTextBlocks().forEach(block -> {
+                    block.setPage(page.getPageNumber());
+                    var section = sectionsBuilderService.buildTextBlock(List.of(block), "a");
+                    sections.add(section);
+                });
+            }
+            classificationDocument.setSections(sections);
+        } else {
+            sectionsBuilderService.buildSections(classificationDocument);
+            sectionsBuilderService.addImagesToSections(classificationDocument);
+        }
        return classificationDocument;
    }


-    private Map<String, Rectangle2D> convertMarkedContents(List<PDMarkedContent> pdMarkedContents) {
-        Map<String, Rectangle2D> markedContentBboxes = new HashMap<>();
+    private void addNumberOfPagesToTrace(int numberOfPages, long size) {
+
+        if (observationRegistry.getCurrentObservation() != null) {
+            observationRegistry.getCurrentObservation().highCardinalityKeyValue("numberOfPages", String.valueOf(numberOfPages));
+            observationRegistry.getCurrentObservation().highCardinalityKeyValue("fileSize", String.valueOf(size));
+        }
+    }
+
+
+    @SneakyThrows
+    private PDDocument openDocument(File originFile) {
+
+        PDDocument document = Loader.loadPDF(originFile);
+        document.setAllSecurityToBeRemoved(true);
+        return document;
+    }
+
+
+    private Map<String, List<Rectangle2D>> convertMarkedContents(List<PDMarkedContent> pdMarkedContents) {
+
+        Map<String, List<Rectangle2D>> markedContentBboxes = new HashMap<>();
        markedContentBboxes.put(MarkedContentUtils.HEADER, MarkedContentUtils.getMarkedContentBboxPerLine(pdMarkedContents, MarkedContentUtils.HEADER));
        markedContentBboxes.put(MarkedContentUtils.FOOTER, MarkedContentUtils.getMarkedContentBboxPerLine(pdMarkedContents, MarkedContentUtils.FOOTER));
        return markedContentBboxes;
    }

+
    private void increaseDocumentStatistics(ClassificationPage classificationPage, ClassificationDocument document) {

-        if (!classificationPage.isLandscape()) {
-            document.getFontSizeCounter().addAll(classificationPage.getFontSizeCounter().getCountPerValue());
-        }
+        document.getFontSizeCounter().addAll(classificationPage.getFontSizeCounter().getCountPerValue());
        document.getFontCounter().addAll(classificationPage.getFontCounter().getCountPerValue());
        document.getTextHeightCounter().addAll(classificationPage.getTextHeightCounter().getCountPerValue());
        document.getFontStyleCounter().addAll(classificationPage.getFontStyleCounter().getCountPerValue());
@@ -1,10 +1,24 @@
 package com.knecon.fforesight.service.layoutparser.processor;

+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.ComponentScan;
 import org.springframework.context.annotation.Configuration;

+import com.knecon.fforesight.service.viewerdoc.service.ViewerDocumentService;
+
+import io.micrometer.observation.ObservationRegistry;
+
@Configuration
@ComponentScan
 public class LayoutParsingServiceProcessorConfiguration {

+
+    @Bean
+    @Autowired
+    public ViewerDocumentService viewerDocumentService(ObservationRegistry registry) {
+
+        return new ViewerDocumentService(registry);
+    }
+
 }
@@ -1,24 +1,18 @@
 package com.knecon.fforesight.service.layoutparser.processor;

-import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.File;
-import java.io.FileOutputStream;
+import java.io.FileInputStream;
 import java.io.IOException;
 import java.io.InputStream;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.nio.file.StandardOpenOption;
+import java.util.Optional;

-import org.apache.commons.io.IOUtils;
-import org.apache.pdfbox.Loader;
-import org.apache.pdfbox.io.MemoryUsageSetting;
-import org.apache.pdfbox.pdmodel.PDDocument;
 import org.springframework.stereotype.Service;

 import com.fasterxml.jackson.databind.ObjectMapper;
-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.section.SectionGrid;
 import com.iqser.red.storage.commons.service.StorageService;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentData;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.SimplifiedText;
@@ -26,8 +20,10 @@ import com.knecon.fforesight.service.layoutparser.internal.api.data.taas.Researc
 import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingRequest;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.image.ImageServiceResponse;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
 import com.knecon.fforesight.tenantcommons.TenantContext;

+import io.micrometer.observation.annotation.Observed;
 import lombok.RequiredArgsConstructor;
 import lombok.SneakyThrows;
 import lombok.extern.slf4j.Slf4j;
@@ -40,17 +36,24 @@ public class LayoutParsingStorageService {
    private final StorageService storageService;
    private final ObjectMapper objectMapper;

+    @Observed(name = "LayoutParsingStorageService", contextualName = "get-origin-file")
+    public File getOriginFile(String storageId) throws IOException {

-    public PDDocument getOriginFile(String storageId) throws IOException {
+        File tempFile = createTempFile("document", ".pdf");
+        storageService.downloadTo(TenantContext.getTenantId(), storageId, tempFile);
+        return tempFile;
+    }

-        try (var originDocumentInputStream = getObject(storageId)) {
-            File tempFile = createTempFile("document", ".pdf");
-            try (var tempFileOutputStream = new FileOutputStream(tempFile)) {
-                IOUtils.copy(originDocumentInputStream, tempFileOutputStream);
-                originDocumentInputStream.close();
-            }
-            return Loader.loadPDF(tempFile, MemoryUsageSetting.setupMixed(67108864L));
+
+    @Observed(name = "LayoutParsingStorageService", contextualName = "get-viewer-doc-file")
+    public Optional<File> getViewerDocFile(String storageId) throws IOException {
+
+        if (!storageService.objectExists(TenantContext.getTenantId(), storageId)) {
+            return Optional.empty();
        }
+        File tempFile = createTempFile("viewerDocument", ".pdf");
+        storageService.downloadTo(TenantContext.getTenantId(), storageId, tempFile);
+        return Optional.of(tempFile);
    }


@@ -75,7 +78,16 @@ public class LayoutParsingStorageService {
        }
    }

+    public VisualLayoutParsingResponse getVisualLayoutParsingFile(String storageId) throws IOException {

+        try (InputStream inputStream = getObject(storageId)) {
+            VisualLayoutParsingResponse visualLayoutParsingResponse = objectMapper.readValue(inputStream, VisualLayoutParsingResponse.class);
+            return visualLayoutParsingResponse;
+        }
+    }
+
+
+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-document-data")
    public void storeDocumentData(LayoutParsingRequest layoutParsingRequest, DocumentData documentData) {

        storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.structureFileStorageId(), documentData.getDocumentStructure());
@@ -85,12 +97,6 @@ public class LayoutParsingStorageService {
    }


-    public void storeSectionGrid(LayoutParsingRequest layoutParsingRequest, SectionGrid sectionGrid) {
-
-        storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.sectionGridStorageId(), sectionGrid);
-    }
-
-
    public void storeResearchDocumentData(LayoutParsingRequest layoutParsingRequest, ResearchDocumentData researchDocumentData) {

        storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.researchDocumentStorageId(), researchDocumentData);
@@ -122,6 +128,7 @@ public class LayoutParsingStorageService {
    }


+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-simplified-text")
    public void storeSimplifiedText(LayoutParsingRequest layoutParsingRequest, SimplifiedText simplifiedText) {

        storageService.storeJSONObject(TenantContext.getTenantId(), layoutParsingRequest.simplifiedTextStorageId(), simplifiedText);
@@ -139,10 +146,10 @@ public class LayoutParsingStorageService {


    @SneakyThrows
-    public void storeViewerDocument(LayoutParsingRequest layoutParsingRequest, ByteArrayOutputStream out) {
-
-        try (var in = new ByteArrayInputStream(out.toByteArray())) {
+    @Observed(name = "LayoutParsingStorageService", contextualName = "store-viewer-document")
+    public void storeViewerDocument(LayoutParsingRequest layoutParsingRequest, File out) {

+        try (var in = new FileInputStream(out)) {
            storageService.storeObject(TenantContext.getTenantId(), layoutParsingRequest.viewerDocumentStorageId(), in);
        }
    }
@@ -45,6 +45,12 @@ public abstract class AbstractPageBlock {
    }


+    public boolean containsBlock(TextPageBlock other, float threshold) {
+
+        return this.minX <= other.getMinX() + threshold && this.maxX >= other.getMaxX() - threshold && this.minY <= other.getMinY() + threshold && this.maxY >= other.getMaxY() - threshold;
+    }
+
+
    public boolean contains(AbstractPageBlock other) {

        return this.minX <= other.minX && this.maxX >= other.maxX && this.minY >= other.minY && this.maxY <= other.maxY;
@@ -72,9 +78,36 @@ public abstract class AbstractPageBlock {
    }


-    public boolean intersectsY(AbstractPageBlock atc) {
+    public boolean intersectsY(AbstractPageBlock apb) {

-        return this.minY <= atc.getMaxY() && this.maxY >= atc.getMinY();
+        return this.minY <= apb.getMaxY() && this.maxY >= apb.getMinY();
    }

+
+    public boolean almostIntersects(AbstractPageBlock apb, float yThreshold, float xThreshold) {
+
+        return this.almostIntersectsX(apb, xThreshold) && this.almostIntersectsY(apb, yThreshold);
+    }
+
+
+    private boolean almostIntersectsY(AbstractPageBlock apb, float threshold) {
+
+        return this.minY - threshold <= apb.getMaxY() && this.maxY + threshold >= apb.getMinY();
+    }
+
+
+    private boolean almostIntersectsX(AbstractPageBlock apb, float threshold) {
+
+        return this.minX - threshold <= apb.getMaxX() && this.maxX + threshold >= apb.getMinX();
+    }
+
+
+    public boolean intersects(AbstractPageBlock apb) {
+
+        return this.minY < apb.getMaxY() && this.maxY >= apb.getMinY() && this.minX < apb.getMaxX() && this.maxX > apb.getMinX();
+    }
+
+
+    public abstract boolean isEmpty();
+
 }
@@ -3,7 +3,6 @@ package com.knecon.fforesight.service.layoutparser.processor.model;
 import java.util.ArrayList;
 import java.util.List;

-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.section.SectionGrid;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.StringFrequencyCounter;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.UnclassifiedText;

@@ -25,7 +24,6 @@ public class ClassificationDocument {
    private StringFrequencyCounter fontStyleCounter = new StringFrequencyCounter();
    private boolean headlines;

-    private SectionGrid sectionGrid = new SectionGrid();
    private long rulesVersion;

 }
@@ -42,6 +42,6 @@ public class ClassificationPage {

    CleanRulings cleanRulings;

-    private Map<String, Rectangle2D> markedContentBboxPerType = new HashMap<>();
+    private Map<String, List<Rectangle2D>> markedContentBboxPerType = new HashMap<>();

 }
@@ -2,6 +2,7 @@ package com.knecon.fforesight.service.layoutparser.processor.model;

 import java.util.ArrayList;
 import java.util.List;
+import java.util.stream.Collectors;

 import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
@@ -29,4 +30,10 @@ public class ClassificationSection {
        return tables;
    }

+
+    public List<AbstractPageBlock> getNonEmptyPageBlocks() {
+
+        return pageBlocks.stream().filter(pageBlock -> !pageBlock.isEmpty()).collect(Collectors.toList());
+    }
+
 }
@@ -12,6 +12,7 @@ import lombok.Setter;

@Setter
@EqualsAndHashCode
+@SuppressWarnings("PMD.AvoidFieldNameMatchingMethodName")
 public class Boundary implements Comparable<Boundary> {

    private int start;
@@ -10,7 +10,6 @@ import java.util.Set;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;

-import com.amazonaws.services.kms.model.NotFoundException;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.DocumentTree;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.entity.RedactionEntity;
@@ -84,7 +83,7 @@ public class Document implements GenericSemanticNode {
    @Override
    public Headline getHeadline() {

-        return streamAllSubNodesOfType(NodeType.HEADLINE).map(node -> (Headline) node).findFirst().orElseThrow(() -> new NotFoundException("No Headlines found in this document!"));
+        return streamAllSubNodesOfType(NodeType.HEADLINE).map(node -> (Headline) node).findFirst().orElse(Headline.builder().build());
    }


@@ -100,6 +99,12 @@ public class Document implements GenericSemanticNode {
    }


+    public Map<NodeType, Long> buildSemanticNodeCounts() {
+
+        return streamAllSubNodes().collect(Collectors.groupingBy(SemanticNode::getType, Collectors.counting()));
+    }
+
+
    @Override
    public String toString() {

@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Footer implements GenericSemanticNode {
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+

    @Override
    public NodeType getType() {
@@ -62,4 +67,14 @@ public class Footer implements GenericSemanticNode {
        return treeId + ": " + NodeType.FOOTER + ": " + leafTextBlock.buildSummary();
    }

+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }
@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Header implements GenericSemanticNode {
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+

    @Override
    public boolean isLeaf() {
@@ -62,4 +67,14 @@ public class Header implements GenericSemanticNode {
        return treeId + ": " + NodeType.HEADER + ": " + leafTextBlock.buildSummary();
    }

+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }
@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -34,6 +36,9 @@ public class Headline implements GenericSemanticNode {
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+

    @Override
    public NodeType getType() {
@@ -69,4 +74,14 @@ public class Headline implements GenericSemanticNode {
        return this;
    }

+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }
@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -32,6 +34,9 @@ public class Paragraph implements GenericSemanticNode {
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+

    @Override
    public NodeType getType() {
@@ -60,4 +65,14 @@ public class Paragraph implements GenericSemanticNode {
        return treeId + ": " + NodeType.PARAGRAPH + ": " + leafTextBlock.buildSummary();
    }

+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }
@@ -1,7 +1,9 @@
 package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

+import java.awt.geom.Rectangle2D;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
@@ -35,6 +37,9 @@ public class Section implements GenericSemanticNode {
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;
+

    @Override
    public NodeType getType() {
@@ -74,4 +79,14 @@ public class Section implements GenericSemanticNode {
                .orElseGet(() -> getParent().getHeadline());
    }

+
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {
+
+        if (bBoxCache == null) {
+            bBoxCache = GenericSemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
+
 }
@@ -398,12 +398,11 @@ public interface SemanticNode {
     */
    default Map<Page, Rectangle2D> getBBox() {

-        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
        if (isLeaf()) {
-            return getBBoxFromLeafTextBlock(bBoxPerPage);
+            return getBBoxFromLeafTextBlock();
        }

-        return getBBoxFromChildren(bBoxPerPage);
+        return getBBoxFromChildren();
    }


@@ -426,25 +425,31 @@ public interface SemanticNode {

    /**
     * TODO: this produces unwanted results for sections spanning multiple columns.
-     *
-     * @param bBoxPerPage initial empty BoundingBox
+     * Computes the Union of the bounding boxes of all children recursively.
     * @return The union of the BoundingBoxes of all children
     */
-    private Map<Page, Rectangle2D> getBBoxFromChildren(Map<Page, Rectangle2D> bBoxPerPage) {
+    private Map<Page, Rectangle2D> getBBoxFromChildren() {

-        return streamChildren().map(SemanticNode::getBBox).reduce((map1, map2) -> {
-            map1.forEach((page, rectangle) -> map2.merge(page, rectangle, (rect1, rect2) -> rect1.createUnion(rect2).getBounds2D()));
-            return map2;
-        }).orElse(bBoxPerPage);
+        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
+        List<Map<Page, Rectangle2D>> childrenBBoxes = streamChildren().map(SemanticNode::getBBox).toList();
+        Set<Page> pages = childrenBBoxes.stream().flatMap(map -> map.keySet().stream()).collect(Collectors.toSet());
+        for (Page page : pages) {
+            Rectangle2D bBoxOnPage = childrenBBoxes.stream()
+                    .filter(childBboxPerPage -> childBboxPerPage.containsKey(page))
+                    .map(childBboxPerPage -> childBboxPerPage.get(page))
+                    .collect(RectangleTransformations.collectBBox());
+            bBoxPerPage.put(page, bBoxOnPage);
+        }
+        return bBoxPerPage;
    }


    /**
-     * @param bBoxPerPage initial empty BoundingBox
     * @return The union of all BoundingBoxes of the TextBlock of this node
     */
-    private Map<Page, Rectangle2D> getBBoxFromLeafTextBlock(Map<Page, Rectangle2D> bBoxPerPage) {
+    private Map<Page, Rectangle2D> getBBoxFromLeafTextBlock() {

+        Map<Page, Rectangle2D> bBoxPerPage = new HashMap<>();
        Map<Page, List<AtomicTextBlock>> atomicTextBlockPerPage = getTextBlock().getAtomicTextBlocks().stream().collect(Collectors.groupingBy(AtomicTextBlock::getPage));
        atomicTextBlockPerPage.forEach((page, atbs) -> bBoxPerPage.put(page, RectangleTransformations.bBoxUnionAtomicTextBlock(atbs)));
        return bBoxPerPage;
@@ -2,10 +2,12 @@ package com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes;

 import static java.lang.String.format;

+import java.awt.geom.Rectangle2D;
 import java.util.Collection;
 import java.util.HashSet;
 import java.util.List;
 import java.util.Locale;
+import java.util.Map;
 import java.util.Set;
 import java.util.stream.IntStream;
 import java.util.stream.Stream;
@@ -34,13 +36,14 @@ public class Table implements SemanticNode {

    int numberOfRows;
    int numberOfCols;
-
    TextBlock textBlock;

    @Builder.Default
    @EqualsAndHashCode.Exclude
    Set<RedactionEntity> entities = new HashSet<>();

+    @EqualsAndHashCode.Exclude
+    Map<Page, Rectangle2D> bBoxCache;

    /**
     * Streams all entities in this table, that appear in a row, which contains any of the provided strings.
@@ -208,7 +211,6 @@ public class Table implements SemanticNode {
        return IntStream.range(0, numberOfCols).boxed().map(col -> getCell(row, col));
    }

-
    /**
     * Streams all TableCells row-wise and filters them with header == true.
     *
@@ -313,5 +315,12 @@ public class Table implements SemanticNode {

        return treeId.toString() + ": " + NodeType.TABLE + ": #cols: " + numberOfCols + ", #rows: " + numberOfRows + ", " + this.getTextBlock().buildSummary();
    }
+    @Override
+    public Map<Page, Rectangle2D> getBBox() {

+        if (bBoxCache == null) {
+            bBoxCache = SemanticNode.super.getBBox();
+        }
+        return bBoxCache;
+    }
 }
@@ -76,16 +76,22 @@ public class TableCell implements GenericSemanticNode {
        }

        if (textBlock == null) {
-            textBlock = streamAllSubNodes().filter(SemanticNode::isLeaf).map(SemanticNode::getLeafTextBlock).collect(new TextBlockCollector());
+            textBlock = buildTextBlock();
        }
        return textBlock;
    }


+    private TextBlock buildTextBlock() {
+
+        return streamAllSubNodes().filter(SemanticNode::isLeaf).map(SemanticNode::getLeafTextBlock).collect(new TextBlockCollector());
+    }
+
+
    @Override
    public String toString() {

-        return treeId + ": " + NodeType.TABLE_CELL + ": " + this.getTextBlock().buildSummary();
+        return treeId + ": " + NodeType.TABLE_CELL + ": " + this.buildTextBlock().buildSummary();
    }

 }
@@ -76,4 +76,14 @@ public class Cell extends Rectangle {
        return this.getHeight() >= MIN_SIZE && this.getWidth() >= MIN_SIZE;
    }

+    public boolean nearlyIntersects(Cell other) {
+
+        if (this.getHeight() <= 0 || other.getHeight() <= 0) {
+            return false;
+        }
+        double x0 = this.getX() + 2;
+        double y0 = this.getY() + 2;
+        return (other.x + other.width > x0 && other.y + other.height > y0 && other.x < x0 + this.getWidth() - 2 && other.y < y0 + this.getHeight() - 2);
+    }
+
 }
@@ -19,7 +19,7 @@ import lombok.extern.slf4j.Slf4j;
@Slf4j
 public class TablePageBlock extends AbstractPageBlock {

-    private final TreeMap<CellPosition, Cell> cells = new TreeMap<>();
+    private final TreeMap<CellPosition, Cell> cellTreeMap = new TreeMap<>();

    private final int rotation;
    @Getter
@@ -28,10 +28,14 @@ public class TablePageBlock extends AbstractPageBlock {
    private int unrotatedRowCount;
    private int unrotatedColCount;
    private List<List<Cell>> rows;
+    @Getter
+    @Setter
+    private List<Cell> cells;


    public TablePageBlock(List<Cell> cells, Rectangle area, int rotation) {

+        this.cells = cells;
        addCells(cells);
        minX = area.getLeft();
        minY = area.getBottom();
@@ -42,6 +46,13 @@ public class TablePageBlock extends AbstractPageBlock {
    }


+    @Override
+    public boolean isEmpty() {
+
+        return getColCount() == 0 || getRowCount() == 0;
+    }
+
+
    public List<List<Cell>> getRows() {

        if (rows == null) {
@@ -72,7 +83,10 @@ public class TablePageBlock extends AbstractPageBlock {

    public int getColCount() {

-        return getRows().stream().mapToInt(List::size).max().orElse(0);
+        return getRows().stream()
+                .mapToInt(List::size)
+                .max()
+                .orElse(0);

    }

@@ -112,7 +126,8 @@ public class TablePageBlock extends AbstractPageBlock {
                List<Cell> cellsToTheTop = new ArrayList<>();
                for (int i = 0; i < rowIndex; i++) {
                    try {
-                        cellsToTheTop.add(rows.get(i).get(colIndex));
+                        cellsToTheTop.add(rows.get(i)
+                                                  .get(colIndex));
                    } catch (IndexOutOfBoundsException e) {
                        log.debug("No cell {} in row {}, ignoring.", colIndex, rowIndex);
                    }
@@ -127,7 +142,8 @@ public class TablePageBlock extends AbstractPageBlock {
                if (lastHeaderCell != null) {
                    cell.getHeaderCells().add(lastHeaderCell);
                }
-                if (!cell.getTextBlocks().isEmpty() && cell.getTextBlocks().get(0).getMostPopularWordStyle().equals("bold")) {
+                if (!cell.getTextBlocks().isEmpty() && cell.getTextBlocks()
+                        .get(0).getMostPopularWordStyle().equals("bold")) {
                    cell.setHeaderCell(true);
                }
            }
@@ -143,7 +159,7 @@ public class TablePageBlock extends AbstractPageBlock {
            for (int i = 0; i < unrotatedColCount; i++) { // rows
                List<Cell> lastRow = new ArrayList<>();
                for (int j = unrotatedRowCount - 1; j >= 0; j--) { // cols
-                    Cell cell = cells.get(new CellPosition(j, i));
+                    Cell cell = cellTreeMap.get(new CellPosition(j, i));
                    if (cell != null) {
                        lastRow.add(cell);
                    }
@@ -154,7 +170,7 @@ public class TablePageBlock extends AbstractPageBlock {
            for (int i = unrotatedColCount - 1; i >= 0; i--) { // rows
                List<Cell> lastRow = new ArrayList<>();
                for (int j = 0; j < unrotatedRowCount; j++) { // cols
-                    Cell cell = cells.get(new CellPosition(j, i));
+                    Cell cell = cellTreeMap.get(new CellPosition(j, i));
                    if (cell != null) {
                        lastRow.add(cell);
                    }
@@ -165,7 +181,7 @@ public class TablePageBlock extends AbstractPageBlock {
            for (int i = 0; i < unrotatedRowCount; i++) {
                List<Cell> lastRow = new ArrayList<>();
                for (int j = 0; j < unrotatedColCount; j++) {
-                    Cell cell = cells.get(new CellPosition(i, j)); // JAVA_8 use getOrDefault()
+                    Cell cell = cellTreeMap.get(new CellPosition(i, j)); // JAVA_8 use getOrDefault()
                    if (cell != null) {
                        lastRow.add(cell);
                    }
@@ -179,17 +195,6 @@ public class TablePageBlock extends AbstractPageBlock {
    }


-    private void add(Cell chunk, int row, int col) {
-
-        unrotatedRowCount = Math.max(unrotatedRowCount, row + 1);
-        unrotatedColCount = Math.max(unrotatedColCount, col + 1);
-
-        CellPosition cp = new CellPosition(row, col);
-        cells.put(cp, chunk);
-
-    }
-
-
    private void addCells(List<Cell> cells) {

        if (cells.isEmpty()) {
@@ -198,11 +203,12 @@ public class TablePageBlock extends AbstractPageBlock {

        cells.removeIf(cell -> cell.getWidth() < 1.1 || cell.getHeight() < 1.1);

-        List<List<Cell>> rowsOfCells = calculateStructure(cells);
+        List<List<Cell>> rowsOfCellsMatrix = calculateTableStructure(cells);

-        for (int i = 0; i < rowsOfCells.size(); i++) {
-            for (int j = 0; j < rowsOfCells.get(i).size(); j++) {
-                add(rowsOfCells.get(i).get(j), i, j);
+        for (int i = 0; i < rowsOfCellsMatrix.size(); i++) {
+            for (int j = 0; j < rowsOfCellsMatrix.get(i).size(); j++) {
+                addCellToRowAndCol(rowsOfCellsMatrix.get(i)
+                                           .get(j), i, j);
            }
        }

@@ -213,29 +219,36 @@ public class TablePageBlock extends AbstractPageBlock {
     * Calculates the structure of the table. For spanning rows and columns multiple cells with the same values will be inserted.
     *
     * @param cells The found cells
-     * @return TablePageBlock Structure
+     * @return TablePageBlock Structure as a rows of cells matrix
     */
-    private List<List<Cell>> calculateStructure(List<Cell> cells) {
-
-        List<List<Cell>> matrix = new ArrayList<>();
+    private List<List<Cell>> calculateTableStructure(List<Cell> cells) {

        if (cells.isEmpty()) {
-            return matrix;
+            return new ArrayList<>();
        }

        Set<Float> uniqueX = new HashSet<>();
        Set<Float> uniqueY = new HashSet<>();
-        cells.stream().filter(c -> !c.getTextBlocks().isEmpty() || c.getHeight() > 3 && c.getWidth() > 3).forEach(c -> {
-            uniqueX.add(c.getLeft());
-            uniqueX.add(c.getRight());
-            uniqueY.add(c.getBottom());
-            uniqueY.add(c.getTop());
-        });
+        cells.stream()
+                .filter(c -> !c.getTextBlocks().isEmpty() || c.getHeight() > 3 && c.getWidth() > 3)
+                .forEach(c -> {
+                    uniqueX.add(c.getLeft());
+                    uniqueX.add(c.getRight());
+                    uniqueY.add(c.getBottom());
+                    uniqueY.add(c.getTop());
+                });

-        var sortedUniqueX = uniqueX.stream().sorted().toList();
-        var sortedUniqueY = uniqueY.stream().sorted().toList();
+        var sortedUniqueX = uniqueX.stream()
+                .sorted()
+                .toList();
+        var sortedUniqueY = uniqueY.stream()
+                .sorted()
+                .toList();
+
+        List<List<Cell>> rowsOfCells = new ArrayList<>();

        Float prevY = null;
+
        for (Float y : sortedUniqueY) {

            List<Cell> row = new ArrayList<>();
@@ -246,24 +259,79 @@ public class TablePageBlock extends AbstractPageBlock {
                if (prevY != null && prevX != null) {
                    var cell = new Cell(new Point2D.Float(prevX, prevY), new Point2D.Float(x, y));

-                    var intersectionCell = cells.stream().filter(c -> cell.intersects(c) && cell.overlapRatio(c) > 0.1f).findFirst();
-                    intersectionCell.ifPresent(value -> cell.getTextBlocks().addAll(value.getTextBlocks()));
                    if (cell.hasMinimumSize()) {
+
+                        cells.stream()
+                                .filter(cell::nearlyIntersects)
+                                .forEach(intersectingCell -> cell.getTextBlocks().addAll(intersectingCell.getTextBlocks()));
+
                        row.add(cell);
                    }
                }
                prevX = x;
            }

-            if (prevY != null && prevX != null && !row.isEmpty()) {
-                matrix.add(row);
+            // exclude empty rows and rows where all text blocks are empty
+            if (prevY != null && prevX != null && !row.isEmpty() && !row.stream()
+                    .allMatch(cell -> cell.getTextBlocks().isEmpty())) {
+
+                rowsOfCells.add(row);
            }
            prevY = y;
        }

-        Collections.reverse(matrix);
+        Collections.reverse(rowsOfCells);
+
+        // now cells are removed which are part of a column without any text blocks
+        // this is done by first computing the inverse matrix which contains call columns of cells
+        // then the column indices that have to be removed are determined
+        List<List<Cell>> columnsOfCells = new ArrayList<>();
+        int maxRowLength = rowsOfCells.stream()
+                .map(List::size)
+                .max(java.util.Comparator.naturalOrder())
+                .orElse(0);
+        for (int i = 0; i < maxRowLength; i++) {
+            columnsOfCells.add(new ArrayList<>());
+        }
+
+        for (List<Cell> row : rowsOfCells) {
+            for (int j = 0; j < row.size(); j++) {
+                columnsOfCells.get(j).add(row.get(j));
+            }
+        }
+
+        List<Integer> columnIndicesToRemove = new ArrayList<>();
+        int columnIndex = 0;
+        for (List<Cell> col : columnsOfCells) {
+            if (col.stream()
+                    .allMatch(cell -> cell.getTextBlocks().isEmpty())) {
+                columnIndicesToRemove.add(columnIndex);
+            }
+            columnIndex++;
+        }
+        columnIndicesToRemove.sort(Collections.reverseOrder());
+
+        // update all rows so that the values of the empty columns get removed
+        var rowsOfCellsBefore = new ArrayList<>(rowsOfCells);
+        rowsOfCells = new ArrayList<>();
+        for (List<Cell> row : rowsOfCellsBefore) {
+            var updatedRow = new ArrayList<>(row);
+            columnIndicesToRemove.forEach(idxToRemove -> updatedRow.remove(updatedRow.get(idxToRemove)));
+            rowsOfCells.add(updatedRow);
+        }
+
+        return rowsOfCells;
+    }
+
+
+    private void addCellToRowAndCol(Cell cell, int row, int col) {
+
+        unrotatedRowCount = Math.max(unrotatedRowCount, row + 1);
+        unrotatedColCount = Math.max(unrotatedColCount, col + 1);
+
+        CellPosition cp = new CellPosition(row, col);
+        cellTreeMap.put(cp, cell);

-        return matrix;
    }


@@ -290,7 +358,7 @@ public class TablePageBlock extends AbstractPageBlock {
                            if (!first) {
                                sb.append("\n");
                            }
-                            sb.append('\"').append(textBlock.getText().replaceAll("\"", "\\\"")).append('\"');
+                            sb.append('\"').append(textBlock.getText().replaceAll("\"", "\"")).append('\"');
                            first = false;
                        }
                    }
@@ -17,7 +17,6 @@ import lombok.SneakyThrows;
@AllArgsConstructor
 public class RedTextPosition {

-    private String textMatrix;
    private float[] position;

    @JsonIgnore
@@ -47,8 +46,12 @@ public class RedTextPosition {
    private String fontName;


+    @JsonIgnore
+    private int textSequence;
+
+
    @SneakyThrows
-    public static RedTextPosition fromTextPosition(TextPosition textPosition) {
+    public static RedTextPosition fromTextPosition(TextPosition textPosition, int textSequence) {

        var pos = new RedTextPosition();
        BeanUtils.copyProperties(textPosition, pos);
@@ -56,8 +59,6 @@ public class RedTextPosition {

        pos.setFontSizeInPt(textPosition.getFontSizeInPt());

-        pos.setTextMatrix(textPosition.getTextMatrix().toString());
-
        var position = new float[4];

        position[0] = textPosition.getXDirAdj();
@@ -66,6 +67,7 @@ public class RedTextPosition {
        position[3] = textPosition.getHeightDir();

        pos.setPosition(position);
+        pos.setTextSequence(textSequence);
        return pos;
    }

@@ -73,7 +73,7 @@ public class TextPageBlock extends AbstractPageBlock {

        return sequences.get(0).getPageWidth();
    }
-    
+

    public static TextPageBlock merge(List<TextPageBlock> textBlocksToMerge) {

@@ -82,6 +82,7 @@ public class TextPageBlock extends AbstractPageBlock {
        return fromTextPositionSequences(sequences);
    }

+
    public static TextPageBlock fromTextPositionSequences(List<TextPositionSequence> wordBlockList) {

        TextPageBlock textBlock = null;
@@ -133,7 +134,6 @@ public class TextPageBlock extends AbstractPageBlock {
    }


-
    /**
     * Returns the minX value in pdf coordinate system.
     * Note: This needs to use Pdf Coordinate System where {0,0} rotated with the page rotation.
@@ -362,7 +362,29 @@ public class TextPageBlock extends AbstractPageBlock {
        }

        return TextNormalizationUtilities.removeHyphenLineBreaks(sb.toString());
+    }

+
+    public int getNumberOfLines() {
+
+        int numberOfLines = 1;
+        TextPositionSequence previous = null;
+        for (TextPositionSequence word : sequences) {
+            if (previous != null) {
+                if (word.getMaxYDirAdj() - previous.getMaxYDirAdj() > word.getTextHeight()) {
+                    numberOfLines++;
+                }
+            }
+            previous = word;
+        }
+        return numberOfLines;
+    }
+
+
+    @Override
+    public boolean isEmpty() {
+
+        return sequences.isEmpty();
    }

 }
@@ -9,8 +9,6 @@ import java.util.stream.Collectors;

 import org.apache.pdfbox.text.TextPosition;

-import com.dslplatform.json.JsonAttribute;
-import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Point;
 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Rectangle;

@@ -45,9 +43,9 @@ public class TextPositionSequence implements CharSequence {
    }


-    public TextPositionSequence(List<TextPosition> textPositions, int page, boolean isParagraphStart) {
+    public TextPositionSequence(List<TextPosition> textPositions, int page, boolean isParagraphStart, int textSequence) {

-        this.textPositions = textPositions.stream().map(RedTextPosition::fromTextPosition).collect(Collectors.toList());
+        this.textPositions = textPositions.stream().map(textPosition -> RedTextPosition.fromTextPosition(textPosition, textSequence)).collect(Collectors.toList());
        this.page = page;
        this.dir = TextDirection.fromDegrees(textPositions.get(0).getDir());
        this.rotation = textPositions.get(0).getRotation();
@@ -57,6 +55,17 @@ public class TextPositionSequence implements CharSequence {
    }


+    public TextPositionSequence(List<RedTextPosition> textPositions, int page) {
+
+        this.textPositions = textPositions;
+        this.page = page;
+        this.dir = TextDirection.fromDegrees(textPositions.get(0).getDir());
+        this.rotation = textPositions.get(0).getRotation();
+        this.pageHeight = textPositions.get(0).getPageHeight();
+        this.pageWidth = textPositions.get(0).getPageWidth();
+    }
+
+
    @Override
    public int length() {

@@ -124,9 +133,9 @@ public class TextPositionSequence implements CharSequence {
    }


-    public void add(TextPosition textPosition) {
+    public void add(TextPosition textPosition, int textSequence) {

-        this.textPositions.add(RedTextPosition.fromTextPosition(textPosition));
+        this.textPositions.add(RedTextPosition.fromTextPosition(textPosition, textSequence));

        this.dir = TextDirection.fromDegrees(textPositions.get(0).getDir());
        this.rotation = textPositions.get(0).getRotation();
@@ -142,8 +151,7 @@ public class TextPositionSequence implements CharSequence {
     *
     * @return the text direction adjusted minX value
     */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
    public float getMinXDirAdj() {

        return textPositions.get(0).getXDirAdj();
@@ -157,8 +165,7 @@ public class TextPositionSequence implements CharSequence {
     *
     * @return the text direction adjusted maxX value
     */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
    public float getMaxXDirAdj() {

        return textPositions.get(textPositions.size() - 1).getXDirAdj() + textPositions.get(textPositions.size() - 1).getWidthDirAdj() + HEIGHT_PADDING;
@@ -172,8 +179,7 @@ public class TextPositionSequence implements CharSequence {
     *
     * @return the text direction adjusted minY value. The upper border of the bounding box of the word.
     */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
    public float getMinYDirAdj() {

        return textPositions.get(0).getYDirAdj() - getTextHeight();
@@ -187,8 +193,7 @@ public class TextPositionSequence implements CharSequence {
     *
     * @return the text direction adjusted maxY value. The lower border of the bounding box of the word.
     */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
    public float getMaxYDirAdj() {

        return textPositions.get(0).getYDirAdj();
@@ -196,42 +201,38 @@ public class TextPositionSequence implements CharSequence {
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public float getTextHeight() {

        return textPositions.get(0).getHeightDir() + HEIGHT_PADDING;
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public float getHeight() {

        return getMaxYDirAdj() - getMinYDirAdj();
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public float getWidth() {

        return getMaxXDirAdj() - getMinXDirAdj();
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public String getFont() {

+        if (textPositions.get(0).getFontName() == null) {
+            return "none";
+        }
        return textPositions.get(0).getFontName().toLowerCase(Locale.ROOT).replaceAll(",bold", "").replaceAll(",italic", "");
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public String getFontStyle() {

+        if (textPositions.get(0).getFontName() == null) {
+            return "standard";
+        }
        String lowercaseFontName = textPositions.get(0).getFontName().toLowerCase(Locale.ROOT);

        if (lowercaseFontName.contains("bold") && lowercaseFontName.contains("italic")) {
@@ -243,20 +244,15 @@ public class TextPositionSequence implements CharSequence {
        } else {
            return "standard";
        }
-
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public float getFontSize() {

        return textPositions.get(0).getFontSizeInPt();
    }


-    @JsonIgnore
-    @JsonAttribute(ignore = true)
    public float getSpaceWidth() {

        return textPositions.get(0).getWidthOfSpace();
@@ -272,8 +268,7 @@ public class TextPositionSequence implements CharSequence {
     *
     * @return bounding box of the word in Pdf Coordinate System
     */
-    @JsonIgnore
-    @JsonAttribute(ignore = true)
+
    @SneakyThrows
    public Rectangle getRectangle() {

@@ -1,8 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.model.visualization;
-
-import java.awt.Color;
-import java.awt.geom.Line2D;
-
-public record ColoredLine(Line2D line, Color color) {
-
-}
@@ -1,7 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.model.visualization;
-
-import java.awt.geom.Point2D;
-
-public record PlacedText(String text, Point2D lineStart) {
-
-}
@@ -8,6 +8,7 @@ import java.util.Map;

 import org.springframework.stereotype.Service;

+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.PageInfo;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
 import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableServiceResponse;

@@ -24,24 +25,26 @@ public class CvTableParsingAdapter {
        Map<Integer, List<TableCells>> tableCells = new HashMap<>();
        tableServiceResponse.getData()
                .forEach(tableData -> tableCells.computeIfAbsent(tableData.getPageInfo().getNumber(), tableCell -> new ArrayList<>())
-                        .addAll(convertTableCells(tableData.getTableCells())));
+                        .addAll(convertTableCells(tableData.getTableCells(), tableData.getPageInfo())));

        return tableCells;
    }


-    private Collection<TableCells> convertTableCells(List<TableCells> tableCells) {
+    private Collection<TableCells> convertTableCells(List<TableCells> tableCells, PageInfo pageInfo) {

        List<TableCells> cvParsedTableCells = new ArrayList<>();

-        tableCells.forEach(t -> cvParsedTableCells.add(TableCells.builder()
-                .y0(t.getY0())
-                .x1(t.getX1())
-                .y1(t.getY1())
-                .x0(t.getX0())
-                .width(t.getWidth())
-                .height(t.getHeight())
-                .build()));
+        tableCells.stream()
+                .filter(cell -> cell.getWidth() < pageInfo.getWidth() * 0.98 && cell.getHeight() < pageInfo.getHeight() * 0.98)
+                .forEach(t -> cvParsedTableCells.add(TableCells.builder()
+                        .y0(t.getY0())
+                        .x1(t.getX1())
+                        .y1(t.getY1())
+                        .x0(t.getX0())
+                        .width(t.getWidth())
+                        .height(t.getHeight())
+                        .build()));

        return cvParsedTableCells;
    }
@@ -0,0 +1,83 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.adapter;
+
+import java.awt.geom.Rectangle2D;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.ImageType;
+import com.knecon.fforesight.service.layoutparser.processor.model.image.ClassifiedImage;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingBox;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResponse;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.VisualLayoutParsingResult;
+
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+
+@Service
+@RequiredArgsConstructor
+@Slf4j
+public class VisualLayoutParsingAdapter {
+
+    private static String SIGNATURES = "signature";
+
+    public Map<Integer, List<VisualLayoutParsingResult>> buildExtractedTablesPerPage(VisualLayoutParsingResponse visualLayoutParsingResponse) {
+
+        Map<Integer, List<VisualLayoutParsingResult>> tableCells = new HashMap<>();
+        visualLayoutParsingResponse.getData()
+                .forEach(tableData -> tableCells.computeIfAbsent(tableData.getPage_idx(), tableCell -> new ArrayList<>()).addAll(convertTableCells(tableData.getBoxes())));
+
+        return tableCells;
+
+    }
+
+    public Map<Integer, List<ClassifiedImage>> buildExtractedSignaturesPerPage(VisualLayoutParsingResponse visualLayoutParsingResponse) {
+
+        Map<Integer, List<ClassifiedImage>> signatures = new HashMap<>();
+            visualLayoutParsingResponse.getData().forEach(tableData -> signatures.computeIfAbsent(tableData.getPage_idx(), tableCell -> new ArrayList<>()).addAll(convertSignatures(tableData.getPage_idx(), tableData.getBoxes())));
+
+            return signatures;
+    }
+
+
+    public List<VisualLayoutParsingResult> convertTableCells(List<VisualLayoutParsingBox> tableObjects) {
+
+        List<VisualLayoutParsingResult> parsedTableCells = new ArrayList<>();
+
+        tableObjects.stream().forEach(t -> {
+            VisualLayoutParsingResult result = new VisualLayoutParsingResult();
+            result.setX0(t.getBox().getX1());
+            result.setX1(t.getBox().getX2());
+            result.setY0(t.getBox().getY1());
+            result.setY1(t.getBox().getY2());
+            result.setWidth(result.getX1() - result.getX0());
+            result.setHeight(result.getY1() - result.getY0());
+            result.setLabel(t.getLabel());
+            parsedTableCells.add(result);
+        });
+
+        return parsedTableCells;
+
+    }
+
+    public List<ClassifiedImage> convertSignatures(int pageNumber, List<VisualLayoutParsingBox> tableObjects) {
+
+        List<ClassifiedImage> signatures = new ArrayList<>();
+
+        tableObjects.stream().forEach(t -> {
+            if(t.getLabel().equals(SIGNATURES)) {
+                ClassifiedImage signature = new ClassifiedImage(new Rectangle2D.Float(t.getBox().getX1(),t.getBox().getY1(),t.getBox().getX2() - t.getBox().getX1(),t.getBox().getY2() - t.getBox().getY1()),
+                        ImageType.SIGNATURE,false,pageNumber);
+
+                signatures.add(signature);
+            }
+        });
+
+        return signatures;
+
+    }
+
+}
@@ -0,0 +1,18 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
+
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+public class VisualLayoutParsingBox {
+
+    private VisualLayoutParsingBoxValue box;
+    private String label;
+    private float probability;
+
+}
@@ -0,0 +1,19 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
+
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+public class VisualLayoutParsingBoxValue {
+
+    private float x1;
+    private float y1;
+    private float x2;
+    private float y2;
+
+}
@@ -0,0 +1,21 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
+
+import java.util.List;
+
+
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+public class VisualLayoutParsingData {
+
+    private int page_idx;
+
+    private List<VisualLayoutParsingBox> boxes;
+
+}
@@ -0,0 +1,23 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
+
+import java.util.List;
+
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+public class VisualLayoutParsingResponse {
+
+    private String dossierId;
+    private String fileId;
+    private String targetFileExtension;
+    private String responseFileExtension;
+    private String X_TENANT_ID;
+    private List<VisualLayoutParsingData> data;
+
+}
@@ -0,0 +1,22 @@
+package com.knecon.fforesight.service.layoutparser.processor.python_api.model.table;
+
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+public class VisualLayoutParsingResult {
+
+    private float x0;
+    private float y0;
+    private float x1;
+    private float y1;
+    private float width;
+    private float height;
+    private String label;
+
+}
@@ -1,8 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.processor.services;

+import java.util.Comparator;
 import java.util.List;

-import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import org.springframework.stereotype.Service;

 import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Point;
@@ -13,72 +13,70 @@ import com.knecon.fforesight.service.layoutparser.processor.model.Classification
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.FloatFrequencyCounter;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.utils.MarkedContentUtils;
 import com.knecon.fforesight.service.layoutparser.processor.utils.PositionUtils;

@Service
 public class BodyTextFrameService {

+    private static final float RULING_HEIGHT_THRESHOLD = 0.15f; // multiplied with page height. Header/Footer Rulings must be within that border of the page.
+    private static final float RULING_WIDTH_THRESHOLD = 0.75f; // multiplied with page width. Header/Footer Rulings must be at least that wide.

    public void setBodyTextFrames(ClassificationDocument classificationDocument, LayoutParsingType layoutParsingType) {

        Rectangle bodyTextFrame = calculateBodyTextFrame(classificationDocument.getPages(), classificationDocument.getFontSizeCounter(), false, layoutParsingType);
        Rectangle landscapeBodyTextFrame = calculateBodyTextFrame(classificationDocument.getPages(), classificationDocument.getFontSizeCounter(), true, layoutParsingType);
        for (ClassificationPage page : classificationDocument.getPages()) {
+//            var updatedBodyTextFrame = getBodyTextFrameFromRulings(page, bodyTextFrame, landscapeBodyTextFrame);
            setBodyTextFrameAdjustedToPage(page, bodyTextFrame, landscapeBodyTextFrame);
        }
    }


-/*
-    private Rectangle calculateBodyTextFrameByRulings(List<ClassificationPage> pages) {
+    private Rectangle getBodyTextFrameFromRulings(ClassificationPage page, Rectangle bodyTextFrame, Rectangle landscapeBodyTextFrame) {

-        Map<ClassificationPage, List<Ruling>> potentialHeaderRulingsPerPage = new HashMap<>();
-        Map<ClassificationPage, List<Ruling>> potentialFooterRulingsPerPage = new HashMap<>();
-
-        for (var page : pages) {
-            potentialHeaderRulingsPerPage.put(page,
-                    page.getCleanRulings()
-                            .getHorizontal()
-                            .stream()
-                            .filter(ruling -> ruling.getY1() > page.getPageHeight() * 0.8)
-                            .filter(ruling -> ruling.getWidth() > 0.6 * page.getPageWidth())
-                            .toList());
-            potentialFooterRulingsPerPage.put(page,
-                    page.getCleanRulings()
-                            .getHorizontal()
-                            .stream()
-                            .filter(ruling -> ruling.getY1() < page.getPageHeight() * 0.2)
-                            .filter(ruling -> ruling.getWidth() > 0.6 * page.getPageWidth())
-                            .toList());
+        List<Ruling> potentialFooterRulings = getPotentialFooterRulings(page);
+        List<Ruling> potentialHeaderRulings = getPotentialHeaderRulings(page);
+        var x = bodyTextFrame.getTopLeft().getX();
+        var y = bodyTextFrame.getTopLeft().getY();
+        var w = bodyTextFrame.getWidth();
+        var h = bodyTextFrame.getHeight();
+        if (!potentialFooterRulings.isEmpty()) {
+            h = y + h - potentialFooterRulings.get(0).getTop();
+            y = potentialFooterRulings.get(0).getTop();
        }
+        if (!potentialHeaderRulings.isEmpty()) {
+            h = potentialHeaderRulings.get(0).getBottom() - bodyTextFrame.getTopLeft().getY();
+        }
+        return new Rectangle(new Point(x, y), w, h, page.getPageNumber());
+    }

-        Optional<Ruling> headerRuling = potentialHeaderRulingsPerPage.values()
+
+    private List<Ruling> getPotentialFooterRulings(ClassificationPage page) {
+
+        return page.getCleanRulings()
+                .getHorizontal()
                .stream()
-                .flatMap(Collection::stream)
-                .filter(ruling -> potentialHeaderRulingsPerPage.values()
-                        .stream()
-                        .filter(rulingsPerPage -> rulingsPerPage.stream().anyMatch(ruling::almostMatches))
-                        .count() > pages.size() * RULING_THRESHOLD_FACTOR)
-                .min(Comparator.comparingDouble(Ruling::getY1));
+                .filter(ruling -> ruling.getY1() < page.getPageHeight() * RULING_HEIGHT_THRESHOLD)
+                .filter(ruling -> ruling.getWidth() > RULING_WIDTH_THRESHOLD * page.getPageWidth())
+                .sorted(Comparator.comparingDouble(Ruling::getTop))
+                .toList();
+    }

-        Optional<Ruling> footerRuling = potentialFooterRulingsPerPage.values()
-                .stream()
-                .flatMap(Collection::stream)
-                .filter(ruling -> potentialHeaderRulingsPerPage.values()
-                        .stream()
-                        .filter(rulingsPerPage -> rulingsPerPage.stream().anyMatch(ruling::almostMatches))
-                        .count() > pages.size() * RULING_THRESHOLD_FACTOR)
-                .max(Comparator.comparingDouble(Ruling::getY1));

-        double maxY = headerRuling.isPresent() ? headerRuling.get().y1 : pages.stream().mapToDouble(ClassificationPage::getPageHeight).max().orElse(Double.MAX_VALUE);
-        double minY = footerRuling.map(ruling -> ruling.y1).orElse(0F);
-        double maxX = pages.stream().mapToDouble(ClassificationPage::getPageWidth).max().orElse(Double.MAX_VALUE);
+    private List<Ruling> getPotentialHeaderRulings(ClassificationPage page) {

-        return new Rectangle(new Point((float) maxX, (float) maxY), (float) 0, (float) minY, -1);
+        return page.getCleanRulings()
+                .getHorizontal()
+                .stream()
+                .filter(ruling -> ruling.getY1() > page.getPageHeight() * (1 - RULING_HEIGHT_THRESHOLD))
+                .filter(ruling -> ruling.getWidth() > RULING_WIDTH_THRESHOLD * page.getPageWidth())
+                .sorted(Comparator.comparingDouble(Ruling::getBottom).reversed())
+                .toList();
    }
-*/


    /**
@@ -129,10 +127,10 @@ public class BodyTextFrameService {
     * @param landscape               Calculate for landscape or portrait
     * @return Rectangle of the text frame
     */
-    private Rectangle calculateBodyTextFrame(List<ClassificationPage> pages,
-                                             FloatFrequencyCounter documentFontSizeCounter,
-                                             boolean landscape,
-                                             LayoutParsingType layoutParsingType) {
+    protected Rectangle calculateBodyTextFrame(List<ClassificationPage> pages,
+                                               FloatFrequencyCounter documentFontSizeCounter,
+                                               boolean landscape,
+                                               LayoutParsingType layoutParsingType) {

        float approximateHeaderLineCount;
        if (layoutParsingType.equals(LayoutParsingType.TAAS)) {
@@ -1,7 +1,6 @@
 package com.knecon.fforesight.service.layoutparser.processor.services;

 import java.io.IOException;
-import java.io.InputStream;
 import java.util.Collection;
 import java.util.LinkedList;
 import java.util.List;
@@ -26,35 +25,34 @@ public class PageContentExtractor {
    public List<PageContents> getSortedPageContents(String filename) throws IOException {

        List<PageContents> textPositionSequencesPerPage = new LinkedList<>();
-        try (InputStream inputStream = new ClassPathResource(filename).getInputStream()) {
+        ClassPathResource pdfResource = new ClassPathResource(filename);

-            try (PDDocument pdDocument = Loader.loadPDF(inputStream)) {
+        try (PDDocument pdDocument = Loader.loadPDF(pdfResource.getFile())) {

-                for (int pageNumber = 1; pageNumber < pdDocument.getNumberOfPages() + 1; pageNumber++) {
+            for (int pageNumber = 1; pageNumber < pdDocument.getNumberOfPages() + 1; pageNumber++) {

-                    PDFLinesTextStripper stripper = new PDFLinesTextStripper();
-                    PDPage pdPage = pdDocument.getPage(pageNumber - 1);
-                    stripper.setPageNumber(pageNumber);
-                    stripper.setSortByPosition(true);
-                    stripper.setStartPage(pageNumber);
-                    stripper.setEndPage(pageNumber);
-                    stripper.setPdpage(pdPage);
-                    stripper.getText(pdDocument);
+                PDFLinesTextStripper stripper = new PDFLinesTextStripper();
+                PDPage pdPage = pdDocument.getPage(pageNumber - 1);
+                stripper.setPageNumber(pageNumber);
+                stripper.setSortByPosition(true);
+                stripper.setStartPage(pageNumber);
+                stripper.setEndPage(pageNumber);
+                stripper.setPdpage(pdPage);
+                stripper.getText(pdDocument);

-                    Map<Float, List<TextPositionSequence>> sortedTextPositionSequencesPerDir = stripper.getTextPositionSequences()
-                            .stream()
-                            .collect(Collectors.groupingBy(textPositionSequence -> textPositionSequence.getDir().getDegrees()));
+                Map<Float, List<TextPositionSequence>> sortedTextPositionSequencesPerDir = stripper.getTextPositionSequences()
+                        .stream()
+                        .collect(Collectors.groupingBy(textPositionSequence -> textPositionSequence.getDir().getDegrees()));

-                    var sortedTextPositionSequences = sortByDirAccordingToPageRotation(sortedTextPositionSequencesPerDir, pdPage.getRotation());
+                var sortedTextPositionSequences = sortByDirAccordingToPageRotation(sortedTextPositionSequencesPerDir, pdPage.getRotation());

-                    textPositionSequencesPerPage.add(new PageContents(sortedTextPositionSequences,
-                            RectangleTransformations.toRectangle2D(pdPage.getCropBox()),
-                            RectangleTransformations.toRectangle2D(pdPage.getMediaBox()),
-                            stripper.getRulings()));
-                }
+                textPositionSequencesPerPage.add(new PageContents(sortedTextPositionSequences,
+                        RectangleTransformations.toRectangle2D(pdPage.getCropBox()),
+                        RectangleTransformations.toRectangle2D(pdPage.getMediaBox()),
+                        stripper.getRulings()));
            }
        }
-
+    
        return textPositionSequencesPerPage;
    }

@@ -12,9 +12,9 @@ import java.util.Map;

 import org.springframework.stereotype.Service;

-import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.CleanRulings;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
+import com.knecon.fforesight.service.layoutparser.processor.python_api.model.table.TableCells;
 import com.knecon.fforesight.service.layoutparser.processor.utils.DoubleComparisons;

 import lombok.RequiredArgsConstructor;
@@ -25,10 +25,14 @@ import lombok.extern.slf4j.Slf4j;
@RequiredArgsConstructor
 public class RulingCleaningService {

-    public CleanRulings getCleanRulings(List<TableCells> tableCells, List<Ruling> rulings, float minCharWidth, float maxCharHeight) {
+    private static final float THRESHOLD_Y = 6;
+    private static final float THRESHOLD_X = 2;
+
+
+    public CleanRulings getCleanRulings(List<TableCells> tableCells, List<Ruling> rulings) {

        if (!rulings.isEmpty()) {
-            snapPoints(rulings, minCharWidth, maxCharHeight);
+            snapPoints(rulings);
        }

        List<Ruling> vrs = new ArrayList<>();
@@ -53,14 +57,11 @@ public class RulingCleaningService {
        }
        List<Ruling> horizontalRulingLines = collapseOrientedRulings(hrs);

-        return CleanRulings.builder()
-                .vertical(verticalRulingLines)
-                .horizontal(horizontalRulingLines)
-                .build();
+        return CleanRulings.builder().vertical(verticalRulingLines).horizontal(horizontalRulingLines).build();
    }


-    public void snapPoints(List<? extends Line2D.Float> rulings, float xThreshold, float yThreshold) {
+    public void snapPoints(List<? extends Line2D.Float> rulings) {

        // collect points and keep a Line -> p1,p2 map
        Map<Line2D.Float, Point2D[]> linesToPoints = new HashMap<>();
@@ -81,7 +82,7 @@ public class RulingCleaningService {

        for (Point2D p : points.subList(1, points.size() - 1)) {
            List<Point2D> last = groupedPoints.get(groupedPoints.size() - 1);
-            if (Math.abs(p.getX() - last.get(0).getX()) < xThreshold) {
+            if (Math.abs(p.getX() - last.get(0).getX()) < THRESHOLD_X) {
                groupedPoints.get(groupedPoints.size() - 1).add(p);
            } else {
                groupedPoints.add(new ArrayList<>(Collections.singletonList(p)));
@@ -108,7 +109,7 @@ public class RulingCleaningService {

        for (Point2D p : points.subList(1, points.size() - 1)) {
            List<Point2D> last = groupedPoints.get(groupedPoints.size() - 1);
-            if (Math.abs(p.getY() - last.get(0).getY()) < yThreshold) {
+            if (Math.abs(p.getY() - last.get(0).getY()) < THRESHOLD_Y) {
                groupedPoints.get(groupedPoints.size() - 1).add(p);
            } else {
                groupedPoints.add(new ArrayList<>(Collections.singletonList(p)));
@@ -1,146 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.services;
-
-import java.awt.geom.Rectangle2D;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.function.BiConsumer;
-import java.util.function.BinaryOperator;
-import java.util.function.Function;
-import java.util.function.Supplier;
-import java.util.stream.Collector;
-import java.util.stream.Stream;
-
-import org.springframework.stereotype.Service;
-
-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.Point;
-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.section.CellRectangle;
-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.section.SectionGrid;
-import com.iqser.red.service.persistence.service.v1.api.shared.model.redactionlog.section.SectionRectangle;
-import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.SemanticNode;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;
-
-import lombok.RequiredArgsConstructor;
-
-@Service
-@RequiredArgsConstructor
-public class SectionGridCreatorService {
-
-    public SectionGrid createSectionGrid(Document document) {
-
-        Map<Integer, List<SectionRectangle>> sectionBBox = document.streamAllSubNodesOfType(NodeType.SECTION).map(SemanticNode::getBBox).collect(new SectionGridCollector());
-        Map<Integer, List<SectionRectangle>> paragraphBBox = document.streamAllSubNodesOfType(NodeType.PARAGRAPH).map(SemanticNode::getBBox).collect(new SectionGridCollector());
-        Map<Integer, List<SectionRectangle>> headlineBBox = document.streamAllSubNodesOfType(NodeType.HEADLINE).map(SemanticNode::getBBox).collect(new SectionGridCollector());
-        Map<Integer, List<SectionRectangle>> tableBBox = document.streamAllSubNodesOfType(NodeType.TABLE).map(node -> (Table) node).collect(new TableGridCollector());
-        var sectionGrid = new SectionGrid();
-
-        sectionGrid.setRectanglesPerPage(mergeMapsByConcatenatingLists(//
-                mergeMapsByConcatenatingLists(paragraphBBox, headlineBBox), //
-                mergeMapsByConcatenatingLists(sectionBBox, tableBBox)));
-
-        return sectionGrid;
-    }
-
-
-    private static abstract class GridCollector<T> implements Collector<T, Map<Integer, List<SectionRectangle>>, Map<Integer, List<SectionRectangle>>> {
-
-        @Override
-        public Supplier<Map<Integer, List<SectionRectangle>>> supplier() {
-
-            return HashMap::new;
-        }
-
-
-        @Override
-        public Function<Map<Integer, List<SectionRectangle>>, Map<Integer, List<SectionRectangle>>> finisher() {
-
-            return Function.identity();
-        }
-
-
-        @Override
-        public BinaryOperator<Map<Integer, List<SectionRectangle>>> combiner() {
-
-            return SectionGridCreatorService::mergeMapsByConcatenatingLists;
-        }
-
-
-        @Override
-        public Set<Characteristics> characteristics() {
-
-            return Set.of(Characteristics.IDENTITY_FINISH, Characteristics.CONCURRENT, Characteristics.UNORDERED);
-        }
-
-    }
-
-    private static class TableGridCollector extends GridCollector<Table> {
-
-        @Override
-        public BiConsumer<Map<Integer, List<SectionRectangle>>, Table> accumulator() {
-
-            return (map, table) -> table.getPages()
-                    .forEach(page -> map.merge(page.getNumber(), List.of(toSectionRectangle(table, page, table.getPages().size())), SectionGridCreatorService::concatLists));
-        }
-
-
-        private static SectionRectangle toSectionRectangle(Table table, Page page, int numberOfParts) {
-
-            Rectangle2D rect = table.getBBox().get(page);
-            List<CellRectangle> tableCellRectangles = table.streamTableCells()
-                    .map(TableCell::getBBox)
-                    .map(map -> map.get(page))
-                    .filter(Objects::nonNull)
-                    .map(rectangle2D -> new CellRectangle(new Point((float) rectangle2D.getX(), (float) rectangle2D.getY()),
-                            (float) rectangle2D.getWidth(),
-                            (float) rectangle2D.getHeight()))
-                    .toList();
-            return new SectionRectangle(new Point((float) rect.getX(), (float) rect.getY()),
-                    (float) rect.getWidth(),
-                    (float) rect.getHeight(),
-                    1,
-                    numberOfParts,
-                    tableCellRectangles);
-        }
-
-    }
-
-    private static class SectionGridCollector extends GridCollector<Map<Page, Rectangle2D>> {
-
-        @Override
-        public BiConsumer<Map<Integer, List<SectionRectangle>>, Map<Page, Rectangle2D>> accumulator() {
-
-            return (mapToKeep, mapToMerge) -> mapToMerge.forEach((page, rectangle) -> mapToKeep.merge(page.getNumber(),
-                    List.of(toSectionRectangle(rectangle, mapToMerge.values().size())),
-                    SectionGridCreatorService::concatLists));
-
-        }
-
-
-        private static SectionRectangle toSectionRectangle(Rectangle2D rect, int numberOfParts) {
-
-            return new SectionRectangle(new Point((float) rect.getX(), (float) rect.getY()), (float) rect.getWidth(), (float) rect.getHeight(), 1, numberOfParts, null);
-        }
-
-    }
-
-
-    private static Map<Integer, List<SectionRectangle>> mergeMapsByConcatenatingLists(Map<Integer, List<SectionRectangle>> mapToKeep,
-                                                                                      Map<Integer, List<SectionRectangle>> mapToMerge) {
-
-        mapToMerge.forEach((page, rectangle) -> mapToKeep.merge(page, rectangle, SectionGridCreatorService::concatLists));
-        return mapToKeep;
-    }
-
-
-    private static List<SectionRectangle> concatLists(List<SectionRectangle> l1, List<SectionRectangle> l2) {
-
-        return Stream.concat(l1.stream(), l2.stream()).toList();
-    }
-
-}
@@ -240,7 +240,7 @@ public class SectionsBuilderService {
    }


-    private ClassificationSection buildTextBlock(List<AbstractPageBlock> wordBlockList, String lastHeadline) {
+    public ClassificationSection buildTextBlock(List<AbstractPageBlock> wordBlockList, String lastHeadline) {

        ClassificationSection section = new ClassificationSection();

@@ -5,14 +5,12 @@ import java.util.ArrayList;
 import java.util.Comparator;
 import java.util.HashMap;
 import java.util.HashSet;
-import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
 import java.util.Set;

 import org.springframework.stereotype.Service;

-import com.knecon.fforesight.service.layoutparser.internal.api.queue.LayoutParsingType;
 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
 import com.knecon.fforesight.service.layoutparser.processor.model.table.Cell;
@@ -26,45 +24,63 @@ import com.knecon.fforesight.service.layoutparser.processor.utils.DoubleComparis
@Service
 public class TableExtractionService {

-    private static final Comparator<Point2D> X_FIRST_POINT_COMPARATOR = (arg0, arg1) -> {
+    private static final int MAX_TABLE_OUTER_POINT_TOLERANCE = 10;
+    private static final int MAX_TABLE_CONTAINED_CELLS_WITH_TEXT = 1;
+    private static final float SPREADSHEET_AREA_TOLERANCE = 0.001f;
+
+    private static final Comparator<Point2D> X_FIRST_POINT_COMPARATOR = (point1, point2) -> {

        int rv = 0;
-        float arg0X = DoubleComparisons.round(arg0.getX(), 2);
-        float arg0Y = DoubleComparisons.round(arg0.getY(), 2);
-        float arg1X = DoubleComparisons.round(arg1.getX(), 2);
-        float arg1Y = DoubleComparisons.round(arg1.getY(), 2);
+        float point1X = DoubleComparisons.round(point1.getX(), 2);
+        float point1Y = DoubleComparisons.round(point1.getY(), 2);
+        float point2X = DoubleComparisons.round(point2.getX(), 2);
+        float point2Y = DoubleComparisons.round(point2.getY(), 2);

-        if (arg0X > arg1X) {
+        if (point1X > point2X) {
            rv = 1;
-        } else if (arg0X < arg1X) {
+        } else if (point1X < point2X) {
            rv = -1;
-        } else if (arg0Y > arg1Y) {
+        } else if (point1Y > point2Y) {
            rv = 1;
-        } else if (arg0Y < arg1Y) {
+        } else if (point1Y < point2Y) {
            rv = -1;
        }
        return rv;
    };
-    private static final Comparator<Point2D> POINT_COMPARATOR = (arg0, arg1) -> {
+    private static final Comparator<Point2D> Y_FIRST_POINT_COMPARATOR = (point1, point2) -> {

        int rv = 0;
-        float arg0X = DoubleComparisons.round(arg0.getX(), 2);
-        float arg0Y = DoubleComparisons.round(arg0.getY(), 2);
-        float arg1X = DoubleComparisons.round(arg1.getX(), 2);
-        float arg1Y = DoubleComparisons.round(arg1.getY(), 2);
+        float point1X = DoubleComparisons.round(point1.getX(), 2);
+        float point1Y = DoubleComparisons.round(point1.getY(), 2);
+        float point2X = DoubleComparisons.round(point2.getX(), 2);
+        float point2Y = DoubleComparisons.round(point2.getY(), 2);

-        if (arg0Y > arg1Y) {
+        if (point1Y > point2Y) {
            rv = 1;
-        } else if (arg0Y < arg1Y) {
+        } else if (point1Y < point2Y) {
            rv = -1;
-        } else if (arg0X > arg1X) {
+        } else if (point1X > point2X) {
            rv = 1;
-        } else if (arg0X < arg1X) {
+        } else if (point1X < point2X) {
            rv = -1;
        }
        return rv;
    };

+    private static final Comparator<Cell> CELL_SIZE_COMPARATOR = (cell1, cell2) -> {
+
+        Double cell1Size = cell1.getHeight() * cell1.getWidth();
+        Double cell2Size = cell2.getHeight() * cell2.getWidth();
+        return cell1Size.compareTo(cell2Size);
+    };
+
+    private static final Comparator<Rectangle> RECTANGLE_SIZE_COMPARATOR = (rect1, rect2) -> {
+
+        Double rect1Size = rect1.getHeight() * rect1.getWidth();
+        Double rect2Size = rect2.getHeight() * rect2.getWidth();
+        return rect1Size.compareTo(rect2Size);
+    };
+

    /**
     * Finds tables on a page and moves textblocks into cells of the found tables.
@@ -79,21 +95,18 @@ public class TableExtractionService {
     * @param cleanRulings The lines used to build the table.
     * @param page         Page object that contains textblocks and statistics.
     */
-    public void extractTables(CleanRulings cleanRulings, ClassificationPage page, LayoutParsingType layoutParsingType) {

-        List<Cell> cells = findCells(cleanRulings.getHorizontal(), cleanRulings.getVertical(), layoutParsingType);
+    public void extractTables(CleanRulings cleanRulings, ClassificationPage page) {

-        List<TextPageBlock> toBeRemoved = new ArrayList<>();
+        List<Cell> cells = findCells(cleanRulings.getHorizontal(), cleanRulings.getVertical());
+        // sort cells by size (height * width) ascending so that textBlocks are always assigned to the smallest cells that contain them
+        cells.sort(CELL_SIZE_COMPARATOR);

        for (AbstractPageBlock abstractPageBlock : page.getTextBlocks()) {
            TextPageBlock textBlock = (TextPageBlock) abstractPageBlock;
            for (Cell cell : cells) {
-                if (cell.hasMinimumSize() && cell.intersects(textBlock.getPdfMinX(),
-                        textBlock.getPdfMinY(),
-                        textBlock.getPdfMaxX() - textBlock.getPdfMinX(),
-                        textBlock.getPdfMaxY() - textBlock.getPdfMinY())) {
+                if (cell.hasMinimumSize() && doesCellContainTextBlock(cell, textBlock)) {
                    cell.addTextBlock(textBlock);
-                    toBeRemoved.add(textBlock);
                    break;
                }
            }
@@ -102,56 +115,85 @@ public class TableExtractionService {
        cells = new ArrayList<>(new HashSet<>(cells));
        DoubleComparisons.sort(cells, Rectangle.ILL_DEFINED_ORDER);

-        List<Rectangle> spreadsheetAreas = findSpreadsheetsFromCells(cells).stream().filter(r -> r.getWidth() > 0f && r.getHeight() > 0f).toList();
+        List<Rectangle> spreadsheetAreas = findSpreadsheetsFromCells(cells);
+        // sort spreadsheetAreas by size (height * width) ascending so that cells are placed in the smallest tables first
+        // this way no cell duplication occurs when tables are contained in other tables and only the most inner table contains the cells
+        spreadsheetAreas.sort(RECTANGLE_SIZE_COMPARATOR);

        List<TablePageBlock> tables = new ArrayList<>();
        for (Rectangle area : spreadsheetAreas) {

-            List<Cell> overlappingCells = new ArrayList<>();
+            List<Cell> containedCells = new ArrayList<>();
            for (Cell c : cells) {
-                if (c.hasMinimumSize() && c.intersects(area)) {
-                    overlappingCells.add(c);
+                if (c.hasMinimumSize() && area.contains(c)) {
+                    containedCells.add(c);
                }
            }
-            tables.add(new TablePageBlock(overlappingCells, area, page.getRotation()));
+
+            var containedCellsWithText = containedCells.stream()
+                    .filter(cell -> !cell.getTextBlocks().isEmpty())
+                    .count();
+
+            // verify if table would contain fewer cells with text than the threshold allows
+            if (containedCellsWithText >= MAX_TABLE_CONTAINED_CELLS_WITH_TEXT) {
+                tables.add(new TablePageBlock(containedCells, area, page.getRotation()));
+                cells.removeAll(containedCells);
+            }
        }

        for (TablePageBlock table : tables) {
            int position = -1;

-            Iterator<AbstractPageBlock> itty = page.getTextBlocks().iterator();
-            while (itty.hasNext()) {
-                AbstractPageBlock textBlock = itty.next();
-                if (textBlock instanceof TextPageBlock ? table.containsBlock((TextPageBlock) textBlock) : table.contains(textBlock) && position == -1) {
-                    position = page.getTextBlocks().indexOf(textBlock);
+            for (AbstractPageBlock pageBlock : page.getTextBlocks()) {
+                if (pageBlock instanceof TextPageBlock ? table.containsBlock((TextPageBlock) pageBlock) : table.contains(pageBlock) && position == -1) {
+                    position = page.getTextBlocks().indexOf(pageBlock);
                }
            }
            if (position != -1) {
                page.getTextBlocks().add(position, table);
+
+                var toBeRemoved = table.getCells()
+                        .stream()
+                        .map(Cell::getTextBlocks)
+                        .flatMap(List::stream)
+                        .toList();
+                // remove text blocks from the page that were also added with the table (from its contained cells)
+                page.getTextBlocks().removeAll(toBeRemoved);
            }
        }
-
-        page.getTextBlocks().removeAll(toBeRemoved);
    }


-    public List<Cell> findCells(List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines, LayoutParsingType layoutParsingType) {
+    private boolean doesCellContainTextBlock(Cell cell, TextPageBlock textBlock) {

-        if (layoutParsingType.equals(LayoutParsingType.TAAS)) {
-            // TODO: breaks some tables, for example "1 Abamectin Prr.pdf" try to fix this upstream in RulingCleaningService
-            for (Ruling r : horizontalRulingLines) {
-                if (r.getX2() < r.getX1()) {
-                    double a = r.getX2();
-                    r.x2 = (float) r.getX1();
-                    r.x1 = (float) a;
-                }
+        double x = textBlock.getPdfMinX();
+        double y = textBlock.getPdfMinY();
+        double w = textBlock.getPdfMaxX() - textBlock.getPdfMinX();
+        double h = textBlock.getPdfMaxY() - textBlock.getPdfMinY();
+        if (cell.isEmpty() || w <= 0 || h <= 0) {
+            return false;
+        }
+        double x0 = cell.getX();
+        double y0 = cell.getY();
+        return (x >= x0 - 2 && y >= y0 - 2 && (x + w) <= x0 + cell.getWidth() + 2 && (y + h) <= y0 + cell.getHeight() + 2);
+    }
+
+
+    private List<Cell> findCells(List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
+
+        // Fix for 211.pdf
+        for (Ruling r : horizontalRulingLines) {
+            if (r.getX2() < r.getX1()) {
+                double a = r.getX2();
+                r.x2 = (float) r.getX1();
+                r.x1 = (float) a;
            }
        }

        List<Cell> cellsFound = new ArrayList<>();
        Map<Point2D, Ruling[]> intersectionPoints = Ruling.findIntersections(horizontalRulingLines, verticalRulingLines);
        List<Point2D> intersectionPointsList = new ArrayList<>(intersectionPoints.keySet());
-        intersectionPointsList.sort(POINT_COMPARATOR);
+        intersectionPointsList.sort(Y_FIRST_POINT_COMPARATOR);

        for (int i = 0; i < intersectionPointsList.size(); i++) {
            Point2D topLeft = intersectionPointsList.get(i);
@@ -177,13 +219,14 @@ public class TableExtractionService {
                    continue;
                }
                for (Point2D yPoint : yPoints) {
-                    // is there an horizontal edge b/w topLeft and yPoint ?
+                    // is there a horizontal edge b/w topLeft and yPoint ?
                    if (!hv[0].equals(intersectionPoints.get(yPoint)[0])) {
                        continue;
                    }
                    Point2D btmRight = new Point2D.Float((float) yPoint.getX(), (float) xPoint.getY());
-                    if (intersectionPoints.containsKey(btmRight) && intersectionPoints.get(btmRight)[0].equals(intersectionPoints.get(xPoint)[0]) && intersectionPoints.get(btmRight)[1].equals(
-                            intersectionPoints.get(yPoint)[1])) {
+                    if (intersectionPoints.containsKey(btmRight)
+                        && intersectionPoints.get(btmRight)[0].equals(intersectionPoints.get(xPoint)[0])
+                        && intersectionPoints.get(btmRight)[1].equals(intersectionPoints.get(yPoint)[1])) {
                        cellsFound.add(new Cell(topLeft, btmRight));
                        break outer;
                    }
@@ -205,7 +248,6 @@ public class TableExtractionService {
        Set<Point2D> pointSet = new HashSet<>();
        Map<Point2D, Point2D> edgesH = new HashMap<>();
        Map<Point2D, Point2D> edgesV = new HashMap<>();
-        int i = 0;

        for (Rectangle cell : cells) {
            for (Point2D pt : cell.getPoints()) {
@@ -222,8 +264,9 @@ public class TableExtractionService {
        pointsSortX.sort(X_FIRST_POINT_COMPARATOR);
        // Y first sort
        List<Point2D> pointsSortY = new ArrayList<>(pointSet);
-        pointsSortY.sort(POINT_COMPARATOR);
+        pointsSortY.sort(Y_FIRST_POINT_COMPARATOR);

+        int i = 0;
        while (i < pointSet.size()) {
            float currY = (float) pointsSortY.get(i).getY();
            while (i < pointSet.size() && DoubleComparisons.feq(pointsSortY.get(i).getY(), currY)) {
@@ -248,7 +291,8 @@ public class TableExtractionService {
        Point2D nextVertex;
        while (!edgesH.isEmpty()) {
            ArrayList<PolygonVertex> polygon = new ArrayList<>();
-            Point2D first = edgesH.keySet().iterator().next();
+            Point2D first = edgesH.keySet()
+                    .iterator().next();
            polygon.add(new PolygonVertex(first, Direction.HORIZONTAL));
            edgesH.remove(first);

@@ -292,7 +336,14 @@ public class TableExtractionService {
                bottom = (float) Math.max(bottom, pt.point.getY());
                right = (float) Math.max(right, pt.point.getX());
            }
-            rectangles.add(new Rectangle(top, left, right - left, bottom - top));
+
+            // do not add polygons with too many outer points as they are unlikely to be tables
+            if (poly.size() <= MAX_TABLE_OUTER_POINT_TOLERANCE) {
+                rectangles.add(new Rectangle(top - SPREADSHEET_AREA_TOLERANCE,
+                                             left - SPREADSHEET_AREA_TOLERANCE,
+                                             right - left + 2 * SPREADSHEET_AREA_TOLERANCE,
+                                             bottom - top + 2 * SPREADSHEET_AREA_TOLERANCE));
+            }
        }

        return rectangles;
@@ -0,0 +1,310 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.blockification;
+
+import static java.util.stream.Collectors.toSet;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.stream.Collectors;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
+import com.knecon.fforesight.service.layoutparser.processor.model.FloatFrequencyCounter;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.TablePageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.StringFrequencyCounter;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.DocstrumSegmentationService;
+import com.knecon.fforesight.service.layoutparser.processor.utils.QuickSort;
+import com.knecon.fforesight.service.layoutparser.processor.utils.RulingTextDirAdjustUtil;
+import com.knecon.fforesight.service.layoutparser.processor.utils.TextPositionSequenceComparator;
+
+import lombok.RequiredArgsConstructor;
+
+@SuppressWarnings("all")
+@Service
+@RequiredArgsConstructor
+public class DocstrumBlockificationService {
+
+    private final DocstrumSegmentationService docstrumSegmentationService;
+
+    static final float THRESHOLD = 2f;
+
+
+    public ClassificationPage blockify(List<TextPositionSequence> textPositions, List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines, boolean columnWise) {
+
+        List<AbstractPageBlock> abstractPageBlocks = new ArrayList<>();
+        var zones = docstrumSegmentationService.segmentPage(textPositions, columnWise);
+        zones.forEach(zone -> {
+
+            List<TextPositionSequence> textPositionSequences = new ArrayList<>();
+            zone.getLines().forEach(line -> {
+                line.getWords().forEach(word -> {
+                    textPositionSequences.add(new TextPositionSequence(word.getTextPositions(), word.getPage()));
+                });
+            });
+
+            abstractPageBlocks.addAll(splitZonesAtRulings(textPositionSequences, horizontalRulingLines, verticalRulingLines));
+//            abstractPageBlocks.add(buildTextBlock(textPositionSequences, 0));
+        });
+
+        return new ClassificationPage(abstractPageBlocks);
+    }
+
+
+    public void combineBlocks(ClassificationPage page) {
+
+        mergeZones(page.getTextBlocks());
+
+        TextPageBlock previous = new TextPageBlock();
+        ListIterator<AbstractPageBlock> itty = page.getTextBlocks().listIterator();
+        while (itty.hasNext()) {
+            AbstractPageBlock block = itty.next();
+            if (block instanceof TablePageBlock) {
+                continue;
+            }
+            TextPageBlock current = (TextPageBlock) block;
+
+            if (previous != null && !previous.getSequences().isEmpty()) {
+
+                if (current.getDir() == previous.getDir() && (Math.abs(previous.getMaxY() - current.getMaxY()) < THRESHOLD || Math.abs(previous.getMinY() - current.getMinY()) < THRESHOLD) && (previous.getNumberOfLines() >= 2 && current.getNumberOfLines() >= 2 /* && current.getNumberOfLines() <= 10 */ && previous.getNumberOfLines() <= current.getNumberOfLines())) {
+                    previous.getSequences().addAll(current.getSequences());
+                    previous = buildTextBlock(previous.getSequences(), 0);
+                    itty.remove();
+
+                    // Might be a left/right mapping add one sorted as well
+                    var sortedDublicate = buildTextBlock(previous.getSequences().stream().sorted(new TextPositionSequenceComparator()).collect(Collectors.toList()), 0);
+                    itty.add(sortedDublicate);
+                    continue;
+                }
+
+                if (current.getDir() == previous.getDir() && (Math.abs(previous.getMaxY() - current.getMaxY()) < THRESHOLD || Math.abs(previous.getMinY() - current.getMinY()) < THRESHOLD) && (previous.getNumberOfLines() == 1 && current.getNumberOfLines() >= 1  /* && current.getNumberOfLines() <= 10 */ || previous.getNumberOfLines() == 2 && current.getNumberOfLines() == 1)) {
+                    previous.getSequences().addAll(current.getSequences());
+                    previous = buildTextBlock(previous.getSequences(), 0);
+                    itty.remove();
+                    continue;
+                }
+
+                if (current.getDir() == previous.getDir() && previous.containsBlock(current, THRESHOLD)) {
+                    previous.getSequences().addAll(current.getSequences());
+                    QuickSort.sort(previous.getSequences(), new TextPositionSequenceComparator());
+                    previous = buildTextBlock(previous.getSequences(), 0);
+                    itty.remove();
+                    continue;
+                }
+
+            }
+            previous = current;
+        }
+    }
+
+
+    private void mergeZones(List<AbstractPageBlock> zones) {
+
+        ListIterator<AbstractPageBlock> itty = zones.listIterator();
+
+        while (itty.hasNext()) {
+            AbstractPageBlock block = itty.next();
+            if (block instanceof TablePageBlock) {
+                continue;
+            }
+
+            TextPageBlock current = (TextPageBlock) block;
+
+            List<AbstractPageBlock> toBeRemoved = new ArrayList<>();
+            for (AbstractPageBlock innerZone : zones) {
+                if (innerZone == current) {
+                    continue;
+                }
+                if (innerZone instanceof TablePageBlock) {
+                    continue;
+                }
+
+                TextPageBlock inner = (TextPageBlock) innerZone;
+
+                if (current.getDir() == inner.getDir() && current.almostIntersects(inner, 0, 0)) {
+
+                    current.getSequences().addAll(inner.getSequences());
+                    current = buildTextBlock(inner.getSequences().stream().sorted(new TextPositionSequenceComparator()).collect(Collectors.toList()), 0);
+                }
+            }
+            zones.removeAll(toBeRemoved);
+        }
+    }
+
+
+    public List<AbstractPageBlock> splitZonesAtRulings(List<TextPositionSequence> textPositions, List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {
+
+        int indexOnPage = 0;
+        List<TextPositionSequence> chunkWords = new ArrayList<>();
+        List<AbstractPageBlock> chunkBlockList = new ArrayList<>();
+
+        float minX = 1000, maxX = 0, minY = 1000, maxY = 0;
+        TextPositionSequence prev = null;
+
+        Float splitX1 = null;
+        for (TextPositionSequence word : textPositions) {
+
+            boolean isSplitByRuling = isSplitByRuling(minX, minY, maxX, maxY, word, horizontalRulingLines, verticalRulingLines);
+            boolean splitByDir = prev != null && !prev.getDir().equals(word.getDir());
+
+            if (prev != null && (splitByDir || isSplitByRuling)) {
+
+                TextPageBlock cb1 = buildTextBlock(chunkWords, indexOnPage);
+                indexOnPage++;
+
+                chunkBlockList.add(cb1);
+                chunkWords = new ArrayList<>();
+
+                minX = 1000;
+                maxX = 0;
+                minY = 1000;
+                maxY = 0;
+                prev = null;
+            }
+
+            chunkWords.add(word);
+
+            prev = word;
+            if (word.getMinXDirAdj() < minX) {
+                minX = word.getMinXDirAdj();
+            }
+            if (word.getMaxXDirAdj() > maxX) {
+                maxX = word.getMaxXDirAdj();
+            }
+            if (word.getMinYDirAdj() < minY) {
+                minY = word.getMinYDirAdj();
+            }
+            if (word.getMaxYDirAdj() > maxY) {
+                maxY = word.getMaxYDirAdj();
+            }
+        }
+
+        TextPageBlock cb1 = buildTextBlock(chunkWords, indexOnPage);
+        if (cb1 != null) {
+            chunkBlockList.add(cb1);
+        }
+
+        return chunkBlockList;
+    }
+
+
+    private boolean equalsWithThreshold(float f1, float f2) {
+
+        return Math.abs(f1 - f2) < THRESHOLD;
+    }
+
+
+    private TextPageBlock buildTextBlock(List<TextPositionSequence> wordBlockList, int indexOnPage) {
+
+        TextPageBlock textBlock = null;
+
+        FloatFrequencyCounter lineHeightFrequencyCounter = new FloatFrequencyCounter();
+        FloatFrequencyCounter fontSizeFrequencyCounter = new FloatFrequencyCounter();
+        FloatFrequencyCounter spaceFrequencyCounter = new FloatFrequencyCounter();
+        StringFrequencyCounter fontFrequencyCounter = new StringFrequencyCounter();
+        StringFrequencyCounter styleFrequencyCounter = new StringFrequencyCounter();
+
+        for (TextPositionSequence wordBlock : wordBlockList) {
+
+            lineHeightFrequencyCounter.add(wordBlock.getTextHeight());
+            fontSizeFrequencyCounter.add(wordBlock.getFontSize());
+            spaceFrequencyCounter.add(wordBlock.getSpaceWidth());
+            fontFrequencyCounter.add(wordBlock.getFont());
+            styleFrequencyCounter.add(wordBlock.getFontStyle());
+
+            if (textBlock == null) {
+                textBlock = new TextPageBlock(wordBlock.getMinXDirAdj(),
+                        wordBlock.getMaxXDirAdj(),
+                        wordBlock.getMinYDirAdj(),
+                        wordBlock.getMaxYDirAdj(),
+                        wordBlockList,
+                        wordBlock.getRotation());
+            } else {
+                TextPageBlock spatialEntity = textBlock.union(wordBlock);
+                textBlock.resize(spatialEntity.getMinX(), spatialEntity.getMinY(), spatialEntity.getWidth(), spatialEntity.getHeight());
+            }
+        }
+
+        if (textBlock != null) {
+            textBlock.setMostPopularWordFont(fontFrequencyCounter.getMostPopular());
+            textBlock.setMostPopularWordStyle(styleFrequencyCounter.getMostPopular());
+            textBlock.setMostPopularWordFontSize(fontSizeFrequencyCounter.getMostPopular());
+            textBlock.setMostPopularWordHeight(lineHeightFrequencyCounter.getMostPopular());
+            textBlock.setMostPopularWordSpaceWidth(spaceFrequencyCounter.getMostPopular());
+            textBlock.setHighestFontSize(fontSizeFrequencyCounter.getHighest());
+        }
+
+        if (textBlock != null && textBlock.getSequences() != null && textBlock.getSequences().stream().map(t -> round(t.getMinYDirAdj(), 3)).collect(toSet()).size() == 1) {
+            textBlock.getSequences().sort(Comparator.comparing(TextPositionSequence::getMinXDirAdj));
+        }
+        return textBlock;
+    }
+
+
+    private boolean isSplitByRuling(float minX,
+                                    float minY,
+                                    float maxX,
+                                    float maxY,
+                                    TextPositionSequence word,
+                                    List<Ruling> horizontalRulingLines,
+                                    List<Ruling> verticalRulingLines) {
+
+        return isSplitByRuling(maxX,
+                minY,
+                word.getMinXDirAdj(),
+                word.getMinYDirAdj(),
+                verticalRulingLines,
+                word.getDir().getDegrees(),
+                word.getPageWidth(),
+                word.getPageHeight()) //
+                || isSplitByRuling(minX,
+                minY,
+                word.getMinXDirAdj(),
+                word.getMaxYDirAdj(),
+                horizontalRulingLines,
+                word.getDir().getDegrees(),
+                word.getPageWidth(),
+                word.getPageHeight()) //
+                || isSplitByRuling(maxX,
+                minY,
+                word.getMinXDirAdj(),
+                word.getMinYDirAdj(),
+                horizontalRulingLines,
+                word.getDir().getDegrees(),
+                word.getPageWidth(),
+                word.getPageHeight()) //
+                || isSplitByRuling(minX,
+                minY,
+                word.getMinXDirAdj(),
+                word.getMaxYDirAdj(),
+                verticalRulingLines,
+                word.getDir().getDegrees(),
+                word.getPageWidth(),
+                word.getPageHeight());
+    }
+
+
+    private boolean isSplitByRuling(float previousX2, float previousY1, float currentX1, float currentY1, List<Ruling> rulingLines, float dir, float pageWidth, float pageHeight) {
+
+        for (Ruling ruling : rulingLines) {
+            var line = RulingTextDirAdjustUtil.convertToDirAdj(ruling, dir, pageWidth, pageHeight);
+            if (line.intersectsLine(previousX2, previousY1, currentX1, currentY1)) {
+                return true;
+            }
+        }
+        return false;
+    }
+
+
+    private double round(float value, int decimalPoints) {
+
+        var d = Math.pow(10, decimalPoints);
+        return Math.round(value * d) / d;
+    }
+
+}
@@ -1,13 +1,8 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.blockification;

-import java.util.ArrayList;
-import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.regex.Matcher;
-import java.util.regex.Pattern;

-import org.springframework.stereotype.Service;
+// TODO: figure out, why this fails the build
+// import static com.knecon.fforesight.service.layoutparser.processor.services.factory.SearchTextWithTextPositionFactory.HEIGHT_PADDING;

 import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.ClassificationPage;
@@ -16,14 +11,25 @@ import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RulingTextDirAdjustUtil;
+import org.springframework.stereotype.Service;
+
+import java.util.*;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;

@Service
@SuppressWarnings("all")
 public class TaasBlockificationService {

    private static final float THRESHOLD = 1f;
-    private static final float Y_GAP_SPLIT_HEIGHT_MODIFIER = 1.25f;
+    private static final float Y_GAP_SPLIT_HEIGHT_MODIFIER = 1.25f; // multiplied with text height
+    private static final float INTERSECTS_Y_THRESHOLD = 4;// 2 * HEIGHT_PADDING // This is exactly 2 times our position height padding. This is required to find boxes that are visually intersecting.
    private static final int X_GAP_SPLIT_CONSTANT = 50;
+    public static final int X_ALIGNMENT_THRESHOLD = 1;
+    public static final int NEGATIVE_X_GAP_THRESHOLD = -5;
+
+    private Pattern listIdentifier = Pattern.compile("^(?:(?:[1-9]|1\\d|20|[ivxlc]|[a-z])\\s*(?:[.)]))|\\uF0B7", Pattern.CASE_INSENSITIVE);


    /**
@@ -39,14 +45,28 @@ public class TaasBlockificationService {
    public ClassificationPage blockify(List<TextPositionSequence> textPositions, List<Ruling> horizontalRulingLines, List<Ruling> verticalRulingLines) {

        List<TextPageBlock> classificationTextBlocks = constructFineGranularTextPageBlocks(textPositions, horizontalRulingLines, verticalRulingLines);
-
-        classificationTextBlocks = mergeFineGranularTextPageBlocks(classificationTextBlocks);
+        classificationTextBlocks = mergeTextPageBlocksAligningX(classificationTextBlocks);
+        classificationTextBlocks = mergeIntersectingTextBlocksUntilConvergence(classificationTextBlocks);

        return new ClassificationPage(new ArrayList<>(classificationTextBlocks.stream().map(classificationTextBlock -> (AbstractPageBlock) classificationTextBlock).toList()));
    }


-    private List<TextPageBlock> mergeFineGranularTextPageBlocks(List<TextPageBlock> classificationTextBlocks) {
+    private List<TextPageBlock> mergeIntersectingTextBlocksUntilConvergence(List<TextPageBlock> classificationTextBlocks) {
+
+        int currentSize = classificationTextBlocks.size();
+        while (true) {
+            classificationTextBlocks = mergeTextPageBlocksAlmostIntersecting(classificationTextBlocks);
+            if (classificationTextBlocks.size() == currentSize) {
+                break;
+            }
+            currentSize = classificationTextBlocks.size();
+        }
+        return classificationTextBlocks;
+    }
+
+
+    private List<TextPageBlock> mergeTextPageBlocksAligningX(List<TextPageBlock> classificationTextBlocks) {

        if (classificationTextBlocks.isEmpty()) {
            return new ArrayList<>();
@@ -55,16 +75,29 @@ public class TaasBlockificationService {
        List<TextPageBlock> currentTextBlocksToMerge = new LinkedList<>();
        textBlocksToMerge.add(currentTextBlocksToMerge);
        TextPageBlock previousTextBlock = null;
+        Float lastLineGap = null;
        for (TextPageBlock currentTextBlock : classificationTextBlocks) {
            if (previousTextBlock == null) {
                currentTextBlocksToMerge.add(currentTextBlock);
                previousTextBlock = currentTextBlock;
                continue;
            }
-            boolean alignsXRight = Math.abs(currentTextBlock.getPdfMaxX() - previousTextBlock.getPdfMaxX()) < 1;
-            boolean smallYGap = Math.abs(currentTextBlock.getPdfMaxY() - previousTextBlock.getPdfMinY()) < 5;
-            if (alignsXRight && smallYGap) {
+
+
+            Matcher listIdentifierPattern = listIdentifier.matcher(currentTextBlock.getText());
+            boolean isListIdentifier = listIdentifierPattern.find();
+
+            boolean yGap = Math.abs(currentTextBlock.getPdfMaxY() - previousTextBlock.getPdfMinY()) < previousTextBlock.getMostPopularWordHeight() * Y_GAP_SPLIT_HEIGHT_MODIFIER;
+
+            boolean sameFont = previousTextBlock.getMostPopularWordFont().equals(currentTextBlock.getMostPopularWordFont()) && previousTextBlock.getMostPopularWordFontSize() == currentTextBlock.getMostPopularWordFontSize();
+//            boolean yGap = previousTextBlock != null && currentTextBlock.getMinYDirAdj() - maxY > Math.min(word.getHeight(), prev.getHeight()) * Y_GAP_SPLIT_HEIGHT_MODIFIER;
+
+            boolean alignsXRight = Math.abs(currentTextBlock.getPdfMaxX() - previousTextBlock.getPdfMaxX()) < X_ALIGNMENT_THRESHOLD;
+            boolean alignsXLeft = Math.abs(currentTextBlock.getPdfMinX() - previousTextBlock.getPdfMinX()) < X_ALIGNMENT_THRESHOLD;
+//            boolean smallYGap = Math.abs(currentTextBlock.getPdfMaxY() - previousTextBlock.getPdfMinY()) < yGap;
+            if (yGap && sameFont && !isListIdentifier) {
                currentTextBlocksToMerge.add(currentTextBlock);
+
            } else {
                currentTextBlocksToMerge = new LinkedList<>();
                currentTextBlocksToMerge.add(currentTextBlock);
@@ -76,6 +109,23 @@ public class TaasBlockificationService {
    }


+    private List<TextPageBlock> mergeTextPageBlocksAlmostIntersecting(List<TextPageBlock> textPageBlocks) {
+
+        Set<TextPageBlock> alreadyMerged = new HashSet<>();
+        List<List<TextPageBlock>> textBlocksToMerge = new LinkedList<>();
+        for (TextPageBlock textPageBlock : textPageBlocks) {
+            if (alreadyMerged.contains(textPageBlock)) {
+                continue;
+            }
+            alreadyMerged.add(textPageBlock);
+            textBlocksToMerge.add(Stream.concat(Stream.of(textPageBlock),
+                            textPageBlocks.stream().filter(textPageBlock2 -> textPageBlock.almostIntersects(textPageBlock2, INTERSECTS_Y_THRESHOLD, 0) && !alreadyMerged.contains(textPageBlock2)).peek(alreadyMerged::add))
+                    .toList());
+        }
+        return textBlocksToMerge.stream().map(TextPageBlock::merge).toList();
+    }
+
+
    private void assignOrientations(List<TextPageBlock> classificationTextBlocks) {

        Iterator<TextPageBlock> itty = classificationTextBlocks.iterator();
@@ -128,8 +178,8 @@ public class TaasBlockificationService {


    private List<TextPageBlock> constructFineGranularTextPageBlocks(List<TextPositionSequence> textPositions,
-                                                                                        List<Ruling> horizontalRulingLines,
-                                                                                        List<Ruling> verticalRulingLines) {
+                                                                    List<Ruling> horizontalRulingLines,
+                                                                    List<Ruling> verticalRulingLines) {

        int indexOnPage = 0;
        List<TextPositionSequence> wordClusterToCombine = new ArrayList<>();
@@ -138,18 +188,18 @@ public class TaasBlockificationService {
        float minX = 1000, maxX = 0, minY = 1000, maxY = 0;
        TextPositionSequence prev = null;
        // TODO: make static final constant
-        var listIdentitifier = Pattern.compile("\\b(?:[1-9]|1\\d|20|[ivxlc]|[a-z])\\s*(?:[.)])", Pattern.CASE_INSENSITIVE);
+

        boolean wasSplitted = false;
        Float splitX1 = null;
        for (TextPositionSequence word : textPositions) {

-            Matcher listIdentifierPattern = listIdentitifier.matcher(word.toString());
+            Matcher listIdentifierPattern = listIdentifier.matcher(word.toString());

            boolean yGap = prev != null && word.getMinYDirAdj() - maxY > Math.min(word.getHeight(), prev.getHeight()) * Y_GAP_SPLIT_HEIGHT_MODIFIER;
            boolean sameLine = prev != null && equalsWithThreshold(prev.getMinYDirAdj(), word.getMinYDirAdj());
            boolean positiveXGapInline = prev != null && maxX + X_GAP_SPLIT_CONSTANT < word.getMinXDirAdj() && sameLine;
-            boolean negativeXGap = prev != null && word.getMinXDirAdj() - minX < -5;
+            boolean negativeXGap = prev != null && word.getMinXDirAdj() - minX < NEGATIVE_X_GAP_THRESHOLD;
            boolean startFromTop = prev != null && word.getMinYDirAdj() < prev.getMinYDirAdj() - prev.getTextHeight();
            boolean newLineAfterSplit = prev != null && word.getMinYDirAdj() != prev.getMinYDirAdj() && wasSplitted && splitX1 != word.getMinXDirAdj();
            boolean splitByRuling = isSplitByRuling(minX, minY, maxX, maxY, word, horizontalRulingLines, verticalRulingLines);
@@ -164,7 +214,7 @@ public class TaasBlockificationService {

                Orientation prevOrientation = null;
                if (!classificationTextBlocks.isEmpty()) {
-                    prevOrientation = classificationTextBlocks.get(classificationTextBlocks.size() - 1).getOrientation();
+                    prevOrientation = classificationTextBlocks.get(classificationTextBlocks.size() - X_ALIGNMENT_THRESHOLD).getOrientation();
                }

                TextPageBlock classificationTextBlock = TextPageBlock.fromTextPositionSequences(wordClusterToCombine);
@@ -0,0 +1,59 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextDirection;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service.LineBuilderService;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service.NearestNeighbourService;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service.ReadingOrderService;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service.SpacingService;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service.ZoneBuilderService;
+
+import lombok.RequiredArgsConstructor;
+
+@Service
+@RequiredArgsConstructor
+public class DocstrumSegmentationService {
+
+    private final NearestNeighbourService nearestNeighbourService;
+    private final SpacingService spacingService;
+    private final LineBuilderService lineBuilderService;
+    private final ZoneBuilderService zoneBuilderService;
+    private final ReadingOrderService readingOrderService;
+
+
+    public List<Zone> segmentPage(List<TextPositionSequence> textPositions, boolean columnWise) {
+
+        List<Zone> zones = new ArrayList<>();
+        zones.addAll(computeZones(textPositions, TextDirection.ZERO));
+        zones.addAll(computeZones(textPositions, TextDirection.QUARTER_CIRCLE));
+        zones.addAll(computeZones(textPositions, TextDirection.HALF_CIRCLE));
+        zones.addAll(computeZones(textPositions, TextDirection.THREE_QUARTER_CIRCLE));
+
+        return readingOrderService.resolve(zones, columnWise);
+    }
+
+
+    private List<Zone> computeZones(List<TextPositionSequence> textPositions, TextDirection direction) {
+
+        var positions = textPositions.stream().filter(t -> t.getDir() == direction).map(TextPositionSequence::getTextPositions).flatMap(List::stream).toList();
+
+        var characters = positions.stream().map(Character::new).collect(Collectors.toList());
+
+        nearestNeighbourService.findNearestNeighbors(characters);
+
+        var characterSpacing = spacingService.computeCharacterSpacing(characters);
+        var lineSpacing = Math.min(spacingService.computeLineSpacing(characters), 20);
+
+        var lines = lineBuilderService.buildLines(characters, characterSpacing, lineSpacing);
+        return zoneBuilderService.buildZones(lines, characterSpacing, lineSpacing);
+    }
+
+}
@@ -0,0 +1,32 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+public class AngleFilter {
+
+    protected double lowerAngle;
+    protected double upperAngle;
+
+
+    public AngleFilter(double lowerAngle, double upperAngle) {
+
+        if (lowerAngle < -Math.PI / 2) {
+            lowerAngle += Math.PI;
+        }
+        if (upperAngle >= Math.PI / 2) {
+            upperAngle -= Math.PI;
+        }
+
+        this.lowerAngle = lowerAngle;
+        this.upperAngle = upperAngle;
+    }
+
+
+    public boolean matches(Neighbor neighbor) {
+
+        if (lowerAngle <= upperAngle) {
+            return lowerAngle <= neighbor.getAngle() && neighbor.getAngle() < upperAngle;
+        } else {
+            return lowerAngle <= neighbor.getAngle() || neighbor.getAngle() < upperAngle;
+        }
+    }
+
+}
@@ -0,0 +1,56 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import java.awt.geom.Rectangle2D;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.AbstractPageBlock;
+
+import lombok.Data;
+
+@Data
+public abstract class BoundingBox {
+
+    private Rectangle2D bBox;
+
+
+    public double getX() {
+
+        return bBox.getX();
+    }
+
+
+    public double getY() {
+
+        return bBox.getY();
+    }
+
+
+    public double getWidth() {
+
+        return bBox.getWidth();
+    }
+
+
+    public double getHeight() {
+
+        return bBox.getHeight();
+    }
+
+
+    public double getArea() {
+
+        return (bBox.getHeight() * bBox.getWidth());
+    }
+
+
+    public boolean contains(Rectangle2D contained, double tolerance) {
+
+        return bBox.getX() <= contained.getX() + tolerance && bBox.getY() <= contained.getY() + tolerance && bBox.getX() + bBox.getWidth() >= contained.getX() + contained.getWidth() - tolerance && bBox.getY() + bBox.getHeight() >= contained.getY() + contained.getHeight() - tolerance;
+    }
+
+
+    public boolean intersectsY(BoundingBox other) {
+
+        return this.getBBox().getMinY() <= other.getBBox().getMaxY() && this.getBBox().getMaxY() >= other.getBBox().getMinY();
+    }
+
+}
@@ -0,0 +1,84 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.text.RedTextPosition;
+
+import lombok.Data;
+
+@Data
+public class Character {
+
+    private final double x;
+    private final double y;
+    private final RedTextPosition textPosition;
+
+    private List<Neighbor> neighbors = new ArrayList<>();
+
+
+    public Character(RedTextPosition chunk) {
+
+        this.x = chunk.getXDirAdj() + chunk.getWidthDirAdj() / 2;
+        this.y = chunk.getYDirAdj() + chunk.getHeightDir() / 2;
+        this.textPosition = chunk;
+    }
+
+
+    public double getHeight() {
+
+        return textPosition.getHeightDir();
+    }
+
+
+    public double distance(Character character) {
+
+        double dx = getX() - character.getX();
+        double dy = getY() - character.getY();
+        return Math.sqrt(dx * dx + dy * dy);
+    }
+
+
+    public double horizontalDistance(Character character) {
+
+        return Math.abs(getX() - character.getX());
+    }
+
+
+    public double verticalDistance(Character character) {
+
+        return Math.abs(getY() - character.getY());
+    }
+
+
+    public double overlappingDistance(Character other) {
+
+        double[] xs = new double[4];
+        double s = Math.sin(-0), c = Math.cos(-0);
+        xs[0] = c * x - s * y;
+        xs[1] = c * (x + textPosition.getWidthDirAdj()) - s * (y + textPosition.getHeightDir());
+        xs[2] = c * other.x - s * other.y;
+        xs[3] = c * (other.x + other.textPosition.getWidthDirAdj()) - s * (other.y + other.textPosition.getHeightDir());
+        boolean overlapping = xs[1] >= xs[2] && xs[3] >= xs[0];
+        Arrays.sort(xs);
+        return Math.abs(xs[2] - xs[1]) * (overlapping ? 1 : -1);
+    }
+
+
+    public void setNeighbors(List<Neighbor> neighbors) {
+
+        this.neighbors = neighbors;
+    }
+
+
+    public double angle(Character character) {
+
+        if (getX() > character.getX()) {
+            return Math.atan2(getY() - character.getY(), getX() - character.getX());
+        } else {
+            return Math.atan2(character.getY() - getY(), character.getX() - getX());
+        }
+    }
+
+}
@@ -0,0 +1,194 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import java.util.AbstractSet;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.Map;
+import java.util.NoSuchElementException;
+import java.util.Set;
+
+public class DisjointSets<E> implements Iterable<Set<E>> {
+
+    private final Map<E, Entry<E>> map = new HashMap<>();
+
+
+    public DisjointSets(Collection<? extends E> collection) {
+
+        for (E element : collection) {
+            map.put(element, new Entry<E>(element));
+        }
+    }
+
+
+    public boolean areTogether(E e1, E e2) {
+
+        return map.get(e1).findRepresentative() == map.get(e2).findRepresentative();
+    }
+
+
+    public void union(E e1, E e2) {
+
+        Entry<E> r1 = map.get(e1).findRepresentative();
+        Entry<E> r2 = map.get(e2).findRepresentative();
+        if (r1 != r2) {
+            if (r1.size <= r2.size) {
+                r2.mergeWith(r1);
+            } else {
+                r1.mergeWith(r2);
+            }
+        }
+    }
+
+
+    @Override
+    public Iterator<Set<E>> iterator() {
+
+        return new Iterator<>() {
+
+            private final Iterator<Entry<E>> iterator = map.values().iterator();
+            private Entry<E> nextRepresentative;
+
+            {
+                findNextRepresentative();
+            }
+
+            @Override
+            public boolean hasNext() {
+
+                return nextRepresentative != null;
+            }
+
+
+            @Override
+            public Set<E> next() {
+
+                if (nextRepresentative == null) {
+                    throw new NoSuchElementException();
+                }
+                Set<E> result = nextRepresentative.asSet();
+                findNextRepresentative();
+                return result;
+            }
+
+
+            private void findNextRepresentative() {
+
+                while (iterator.hasNext()) {
+                    Entry<E> candidate = iterator.next();
+                    if (candidate.isRepresentative()) {
+                        nextRepresentative = candidate;
+                        return;
+                    }
+                }
+                nextRepresentative = null;
+            }
+
+
+            @Override
+            public void remove() {
+
+                throw new UnsupportedOperationException();
+            }
+
+        };
+    }
+
+
+    private static class Entry<E> {
+
+        private int size = 1;
+        private final E value;
+        private Entry<E> parent = this;
+        private Entry<E> next = null;
+        private Entry<E> last = this;
+
+
+        Entry(E value) {
+
+            this.value = value;
+        }
+
+
+        void mergeWith(Entry<E> otherRepresentative) {
+
+            size += otherRepresentative.size;
+            last.next = otherRepresentative;
+            last = otherRepresentative.last;
+            otherRepresentative.parent = this;
+        }
+
+
+        Entry<E> findRepresentative() {
+
+            Entry<E> representative = parent;
+            while (representative.parent != representative) {
+                representative = representative.parent;
+            }
+            for (Entry<E> entry = this; entry != representative; ) {
+                Entry<E> nextEntry = entry.parent;
+                entry.parent = representative;
+                entry = nextEntry;
+            }
+            return representative;
+        }
+
+
+        boolean isRepresentative() {
+
+            return parent == this;
+        }
+
+
+        Set<E> asSet() {
+
+            return new AbstractSet<E>() {
+
+                @Override
+                public Iterator<E> iterator() {
+
+                    return new Iterator<E>() {
+
+                        private Entry<E> nextEntry = findRepresentative();
+
+
+                        @Override
+                        public boolean hasNext() {
+
+                            return nextEntry != null;
+                        }
+
+
+                        @Override
+                        public E next() {
+
+                            if (nextEntry == null) {
+                                throw new NoSuchElementException();
+                            }
+                            E result = nextEntry.value;
+                            nextEntry = nextEntry.next;
+                            return result;
+                        }
+
+
+                        @Override
+                        public void remove() {
+
+                            throw new UnsupportedOperationException();
+                        }
+
+                    };
+                }
+
+
+                @Override
+                public int size() {
+
+                    return findRepresentative().size;
+                }
+            };
+        }
+
+    }
+
+}
@@ -0,0 +1,91 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+public class Histogram {
+
+    private static final double EPSILON = 1.0e-6;
+    private final double min;
+    private final double resolution;
+    private double[] frequencies;
+
+
+    public Histogram(double minValue, double maxValue, double resolution) {
+
+        this.min = minValue - EPSILON;
+        double delta = maxValue - minValue + 2 * EPSILON;
+        int size = Math.max(1, (int) Math.round((maxValue - minValue) / resolution));
+        this.resolution = delta / size;
+        this.frequencies = new double[size];
+    }
+
+
+    public void kernelSmooth(double[] kernel) {
+
+        double[] newFrequencies = new double[frequencies.length];
+        int shift = (kernel.length - 1) / 2;
+        for (int i = 0; i < kernel.length; i++) {
+            int jStart = Math.max(0, i - shift);
+            int jEnd = Math.min(frequencies.length, frequencies.length + i - shift);
+            for (int j = jStart; j < jEnd; j++) {
+                newFrequencies[j - i + shift] += kernel[i] * frequencies[j];
+            }
+        }
+        frequencies = newFrequencies;
+    }
+
+
+    public double[] createGaussianKernel(double length, double stdDeviation) {
+
+        int r = (int) Math.round(length / resolution) / 2;
+        stdDeviation /= resolution;
+
+        int size = 2 * r + 1;
+        double[] kernel = new double[size];
+        double sum = 0;
+        double b = 2 * stdDeviation * stdDeviation;
+        double a = 1 / Math.sqrt(Math.PI * b);
+        for (int i = 0; i < size; i++) {
+            kernel[i] = a * Math.exp(-(i - r) * (i - r) / b);
+            sum += kernel[i];
+        }
+        for (int i = 0; i < size; i++) {
+            kernel[i] /= sum;
+        }
+        return kernel;
+    }
+
+
+    public void gaussianSmooth(double windowLength, double stdDeviation) {
+
+        kernelSmooth(createGaussianKernel(windowLength, stdDeviation));
+    }
+
+
+    public void add(double value) {
+
+        frequencies[(int) ((value - min) / resolution)] += 1.0;
+    }
+
+
+    public int getSize() {
+
+        return frequencies.length;
+    }
+
+
+    public double getPeakValue() {
+
+        int peakIndex = 0;
+        for (int i = 1; i < frequencies.length; i++) {
+            if (frequencies[i] > frequencies[peakIndex]) {
+                peakIndex = i;
+            }
+        }
+        int peakEndIndex = peakIndex + 1;
+        final double EPS = 0.0001;
+        while (peakEndIndex < frequencies.length && Math.abs(frequencies[peakEndIndex] - frequencies[peakIndex]) < EPS) {
+            peakEndIndex++;
+        }
+        return ((double) peakIndex + peakEndIndex) / 2 * resolution + min;
+    }
+
+}
@@ -0,0 +1,165 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import java.awt.geom.Rectangle2D;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+
+import lombok.Data;
+
+@Data
+public class Line extends BoundingBox {
+
+    private static final double WORD_DISTANCE_MULTIPLIER = 0.2;
+
+    private final double x0;
+    private final double y0;
+
+    private final double x1;
+    private final double y1;
+
+    private final double height;
+
+    private final List<Character> characters;
+    private final List<TextPositionSequence> words = new ArrayList<>();
+
+
+    public Line(List<Character> characters, double wordSpacing) {
+
+        this.characters = characters;
+
+        if (characters.size() >= 2) {
+            // linear regression
+            double sx = 0.0, sxx = 0.0, sxy = 0.0, sy = 0.0;
+            for (Character character : characters) {
+                sx += character.getX();
+                sxx += character.getX() * character.getX();
+                sxy += character.getX() * character.getY();
+                sy += character.getY();
+            }
+            double b = (characters.size() * sxy - sx * sy) / (characters.size() * sxx - sx * sx);
+            double a = (sy - b * sx) / characters.size();
+
+            this.x0 = characters.get(0).getX();
+            this.y0 = a + b * this.x0;
+            this.x1 = characters.get(characters.size() - 1).getX();
+            this.y1 = a + b * this.x1;
+        } else {
+            Character character = characters.get(0);
+            double dx = character.getTextPosition().getWidthDirAdj() / 3;
+            double dy = dx * Math.tan(0);
+            this.x0 = character.getX() - dx;
+            this.x1 = character.getX() + dx;
+            this.y0 = character.getY() - dy;
+            this.y1 = character.getY() + dy;
+        }
+        height = computeHeight();
+        computeWords(wordSpacing * WORD_DISTANCE_MULTIPLIER);
+        buildBBox();
+    }
+
+
+    public double getAngle() {
+
+        return Math.atan2(y1 - y0, x1 - x0);
+    }
+
+
+    public double getLength() {
+
+        return Math.sqrt((x0 - x1) * (x0 - x1) + (y0 - y1) * (y0 - y1));
+    }
+
+
+    private double computeHeight() {
+
+        double sum = 0.0;
+        for (Character component : characters) {
+            sum += component.getHeight();
+        }
+        return sum / characters.size();
+    }
+
+
+    public double angularDifference(Line j) {
+
+        double diff = Math.abs(getAngle() - j.getAngle());
+        if (diff <= Math.PI / 2) {
+            return diff;
+        } else {
+            return Math.PI - diff;
+        }
+    }
+
+
+    public double horizontalDistance(Line other) {
+
+        double[] xs = new double[4];
+        xs[0] = x0;
+        xs[1] = x1;
+        xs[2] = other.x0;
+        xs[3] = other.x1;
+        boolean overlapping = xs[1] >= xs[2] && xs[3] >= xs[0];
+        Arrays.sort(xs);
+        return Math.abs(xs[2] - xs[1]) * (overlapping ? 1 : -1);
+    }
+
+
+    public double verticalDistance(Line other) {
+
+        double ym = (y0 + y1) / 2;
+        double yn = (other.y0 + other.y1) / 2;
+        return Math.abs(ym - yn) / Math.sqrt(1);
+    }
+
+
+    private void computeWords(double wordSpacing) {
+
+        TextPositionSequence word = new TextPositionSequence();
+        Character previous = null;
+        for (Character current : characters) {
+            if (previous != null) {
+                double dist = current.getTextPosition().getXDirAdj() - previous.getTextPosition().getXDirAdj() - previous.getTextPosition().getWidthDirAdj();
+                if (dist > wordSpacing) {
+                    words.add(word);
+                    word = new TextPositionSequence();
+                }
+            }
+            word.getTextPositions().add(current.getTextPosition());
+            previous = current;
+        }
+        words.add(word);
+    }
+
+
+    private void buildBBox() {
+
+        double minX = Double.POSITIVE_INFINITY;
+        double minY = Double.POSITIVE_INFINITY;
+        double maxX = Double.NEGATIVE_INFINITY;
+        double maxY = Double.NEGATIVE_INFINITY;
+
+        for (Character character : characters) {
+
+            minX = Math.min(minX, character.getTextPosition().getXDirAdj());
+            minY = Math.min(minY, character.getTextPosition().getYDirAdj());
+            maxX = Math.max(maxX, character.getTextPosition().getXDirAdj() + character.getTextPosition().getWidthDirAdj());
+            maxY = Math.max(maxY, character.getTextPosition().getYDirAdj() + character.getTextPosition().getHeightDir());
+
+        }
+
+        this.setBBox(new Rectangle2D.Double(minX, minY, maxX - minX, maxY - minY));
+    }
+
+
+    public String toString() {
+
+        StringBuilder sb = new StringBuilder();
+        words.forEach(word -> sb.append(word.toString()).append(" "));
+        return sb.toString().trim();
+    }
+
+}
+
@@ -0,0 +1,36 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import lombok.Getter;
+
+public class Neighbor {
+
+    @Getter
+    private final double distance;
+    @Getter
+    private final double angle;
+    private final Character originCharacter;
+    @Getter
+    private final Character character;
+
+
+    public Neighbor(Character neighbor, Character origin) {
+
+        this.distance = neighbor.distance(origin);
+        this.angle = neighbor.angle(origin);
+        this.character = neighbor;
+        this.originCharacter = origin;
+    }
+
+
+    public double getHorizontalDistance() {
+
+        return character.horizontalDistance(originCharacter);
+    }
+
+
+    public double getVerticalDistance() {
+
+        return character.verticalDistance(originCharacter);
+    }
+
+}
@@ -0,0 +1,51 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model;
+
+import java.awt.geom.Rectangle2D;
+import java.util.Comparator;
+import java.util.List;
+
+import lombok.Data;
+
+@Data
+public class Zone extends BoundingBox {
+
+    private List<Line> lines;
+
+    private int readingOrder = -1;
+
+    public Zone(List<Line> lines) {
+
+        lines.sort(Comparator.comparingDouble(Line::getY));
+        this.lines = lines;
+        buildBBox();
+    }
+
+
+    public void buildBBox() {
+
+        double minX = Double.POSITIVE_INFINITY;
+        double minY = Double.POSITIVE_INFINITY;
+        double maxX = Double.NEGATIVE_INFINITY;
+        double maxY = Double.NEGATIVE_INFINITY;
+
+        for (Line line : lines) {
+
+            minX = Math.min(minX, line.getX());
+            minY = Math.min(minY, line.getY());
+            maxX = Math.max(maxX, line.getX() + line.getWidth());
+            maxY = Math.max(maxY, line.getY() + line.getHeight());
+
+        }
+
+        this.setBBox(new Rectangle2D.Double(minX, minY, maxX - minX, maxY - minY));
+    }
+
+
+    public String toString() {
+
+        StringBuilder sb = new StringBuilder();
+        lines.forEach(line -> sb.append(line.toString()).append("\n"));
+        return sb.toString().trim();
+    }
+
+}
@@ -0,0 +1,59 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder;
+
+public enum IntervalRelations {
+    // Unknown interval relations.
+    UNKNOWN,
+
+    // X takes place before Y.
+    // |____X____|......................
+    // ......................|____Y____|
+    PRECEDES,
+
+    // X meets Y.
+    // |____X____|.................
+    // ................|____Y____|
+    MEETS,
+
+    // X overlaps with Y.
+    // |______X______|.................
+    // ................|______Y______|
+    OVERLAPS,
+
+    // X starts Y.
+    // |____X____|.................
+    // |_____Y_____|..............
+    STARTS,
+
+    // X during Y.
+    // ........|____X____|.........
+    // .....|______Y______|.....
+    DURING,
+
+    // X finishes Y.
+    // .................|____X____|
+    // ..............|_____Y_____|
+    FINISHES,
+
+    // Inverse precedes.
+    PRECEDES_INVERSE,
+
+    // Inverse meets.
+    MEETS_INVERSE,
+
+    // Inverse overlaps.
+    OVERLAPS_INVERSE,
+
+    // Inverse Starts.
+    STARTS_INVERSE,
+
+    // Inverse during.
+    DURING_INVERSE,
+
+    // Inverse finishes.
+    FINISHES_INVERSE,
+
+    // X is equal to Y.
+    // ..........|____X____|............
+    // ..........|____Y____|............
+    EQUALS
+}
@@ -0,0 +1,11 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder;
+
+import java.util.Collection;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+
+public interface ReadingOrderDetector {
+
+    Collection<Zone> get(Collection<Zone> zones);
+
+}
@@ -0,0 +1,10 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder;
+
+public enum SpatialReasoningRules {
+    // In western culture the reading order is from left to right and from top to bottom.
+    BASIC,
+    // The diagonal direction 'left-bottom to top-right' cannot be present among the Basic relations allowed.
+    ROW_WISE,
+    // The diagonal direction 'right-top to bottom-left' cannot be present among the Basic relations allowed.
+    COLUMN_WISE;
+}
@@ -0,0 +1,261 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.tuple.ImmutablePair;
+import org.apache.commons.lang3.tuple.Pair;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+
+import lombok.Getter;
+
+public class UnsupervisedReadingOrderDetector {
+
+    private boolean useRenderingOrder = true;
+    @Getter
+    private SpatialReasoningRules spatialReasoningRule = SpatialReasoningRules.COLUMN_WISE;
+    private double tolerance = 5;
+    private ZoneComparator zoneComparator;
+
+
+    public boolean useRenderingOrder() {
+
+        return useRenderingOrder;
+    }
+
+
+    public UnsupervisedReadingOrderDetector() {
+
+        configureComparator();
+    }
+
+
+    public UnsupervisedReadingOrderDetector(double tolerance, SpatialReasoningRules spatialReasoningRule, boolean useRenderingOrder) {
+
+        this.tolerance = tolerance;
+        this.spatialReasoningRule = spatialReasoningRule;
+        this.useRenderingOrder = useRenderingOrder;
+
+        configureComparator();
+
+    }
+
+
+    public void configureComparator() {
+
+        switch (spatialReasoningRule) {
+            case COLUMN_WISE:
+                if (useRenderingOrder) {
+                    zoneComparator = (Zone z1, Zone z2, double t) -> getBeforeInReadingColumnWise(z1, z2, t) || getBeforeInRendering(z1, z2);
+                } else {
+                    zoneComparator = this::getBeforeInReadingColumnWise;
+                }
+                break;
+
+            case ROW_WISE:
+                if (useRenderingOrder) {
+                    zoneComparator = (Zone z1, Zone z2, double t) -> getBeforeInReadingRowWise(z1, z2, t) || getBeforeInRendering(z1, z2);
+                } else {
+                    zoneComparator = this::getBeforeInReadingRowWise;
+                }
+                break;
+
+            case BASIC:
+            default:
+                if (useRenderingOrder) {
+                    zoneComparator = (Zone z1, Zone z2, double t) -> getBeforeInReading(z1, z2, t) || getBeforeInRendering(z1, z2);
+                } else {
+                    zoneComparator = this::getBeforeInReading;
+                }
+                break;
+        }
+    }
+
+
+    public List<Zone> get(List<Zone> zones) {
+
+        int readingOrder = 0;
+        Map<Integer, List<Integer>> graph = buildGraph(zones);
+
+        List<Zone> orderedZones = new ArrayList<>();
+
+        while (!graph.isEmpty()) {
+            int maxCount = graph.values()
+                    .stream()
+                    .mapToInt(List::size)
+                    .max()
+                    .orElse(0);
+
+            Map.Entry<Integer, List<Integer>> current = graph.entrySet()
+                    .stream()
+                    .filter(entry -> entry.getValue().size() == maxCount)
+                    .findFirst()
+                    .orElse(null);
+
+            if (current != null) {
+                int index = current.getKey();
+                graph.remove(index);
+
+                for (List<Integer> valueList : graph.values()) {
+                    valueList.remove(Integer.valueOf(index));
+                }
+
+                Zone zone = zones.get(index);
+                zone.setReadingOrder(readingOrder++);
+                orderedZones.add(zone);
+            }
+        }
+
+        return orderedZones;
+    }
+
+
+    private Map<Integer, List<Integer>> buildGraph(List<Zone> zones) {
+
+        Map<Integer, List<Integer>> graph = new HashMap<>();
+
+        for (int i = 0; i < zones.size(); i++) {
+            graph.put(i, new ArrayList<>());
+        }
+
+        for (int i = 0; i < zones.size(); i++) {
+            Zone zone1 = zones.get(i);
+            for (int j = 0; j < zones.size(); j++) {
+                if (i == j) {
+                    continue;
+                }
+                Zone zone2 = zones.get(j);
+
+                if (zoneComparator.isBefore(zone1, zone2, tolerance)) {
+                    graph.get(i).add(j);
+                }
+            }
+        }
+
+        return graph;
+    }
+
+
+    private boolean getBeforeInRendering(Zone z1, Zone z2) {
+
+        double avgTextSequenceZ1 = z1.getLines()
+                .stream()
+                .flatMap(line -> line.getCharacters()
+                        .stream())
+                .map(character -> character.getTextPosition().getTextSequence())
+                .collect(Collectors.averagingDouble(Integer::intValue));
+
+        double avgTextSequenceZ2 = z2.getLines()
+                .stream()
+                .flatMap(line -> line.getCharacters()
+                        .stream())
+                .map(character -> character.getTextPosition().getTextSequence())
+                .collect(Collectors.averagingDouble(Integer::intValue));
+
+        return avgTextSequenceZ1 < avgTextSequenceZ2;
+    }
+
+
+    private boolean getBeforeInReading(Zone z1, Zone z2, double tolerance) {
+
+        IntervalRelations xRelation = getIntervalRelationX(z1, z2, tolerance).get(0);
+        IntervalRelations yRelation = getIntervalRelationY(z1, z2, tolerance).get(0);
+
+        return xRelation == IntervalRelations.PRECEDES
+               || yRelation == IntervalRelations.PRECEDES
+               || xRelation == IntervalRelations.MEETS
+               || yRelation == IntervalRelations.MEETS
+               || xRelation == IntervalRelations.OVERLAPS
+               || yRelation == IntervalRelations.OVERLAPS;
+    }
+
+
+    private boolean getBeforeInReadingColumnWise(Zone z1, Zone z2, double tolerance) {
+
+        IntervalRelations xRelation = getIntervalRelationX(z1, z2, tolerance).get(0);
+        IntervalRelations yRelation = getIntervalRelationY(z1, z2, tolerance).get(0);
+
+        return getIntervalRelations(xRelation, yRelation);
+    }
+
+    private static boolean getIntervalRelations(IntervalRelations relation1, IntervalRelations relation2) {
+
+        return relation1 == IntervalRelations.PRECEDES //
+               || relation1 == IntervalRelations.MEETS //
+               || relation1 == IntervalRelations.OVERLAPS && //
+                  (relation2 == IntervalRelations.PRECEDES //
+                   || relation2 == IntervalRelations.MEETS //
+                   || relation2 == IntervalRelations.OVERLAPS) //
+               || ((relation2 == IntervalRelations.PRECEDES || relation2 == IntervalRelations.MEETS || relation2 == IntervalRelations.OVERLAPS) && //
+                   (relation1 == IntervalRelations.STARTS //
+                    || relation1 == IntervalRelations.FINISHES_INVERSE //
+                    || relation1 == IntervalRelations.EQUALS //
+                    || relation1 == IntervalRelations.DURING //
+                    || relation1 == IntervalRelations.DURING_INVERSE //
+                    || relation1 == IntervalRelations.FINISHES //
+                    || relation1 == IntervalRelations.STARTS_INVERSE //
+                    || relation1 == IntervalRelations.OVERLAPS_INVERSE));
+    }
+
+
+
+    private boolean getBeforeInReadingRowWise(Zone z1, Zone z2, double tolerance) {
+
+        IntervalRelations xRelations = getIntervalRelationX(z1, z2, tolerance).get(0);
+        IntervalRelations yRelations = getIntervalRelationY(z1, z2, tolerance).get(0);
+
+        return getIntervalRelations(yRelations, xRelations);
+    }
+
+
+    private static List<IntervalRelations> getIntervalRelationX(Zone z1, Zone z2, double t) {
+
+        return getIntervalRelation(new ImmutablePair<>(z1.getX(), z1.getX() + z1.getWidth()), new ImmutablePair<>(z2.getX(), z2.getX() + z2.getWidth()), t);
+    }
+
+
+    private static List<IntervalRelations> getIntervalRelationY(Zone z1, Zone z2, double t) {
+
+        return getIntervalRelation(new ImmutablePair<>(z1.getY(), z1.getY() + z1.getHeight()), new ImmutablePair<>(z2.getY(), z2.getY() + z2.getHeight()), t);
+
+    }
+
+
+    private static List<IntervalRelations> getIntervalRelation(Pair<Double, Double> a, Pair<Double, Double> b, double t) {
+
+        var intervalRelations = getIntervalRelation(a, b, t, false);
+        intervalRelations.addAll(getIntervalRelation(b, a, t, true));
+        if ((b.getLeft() - t <= a.getLeft() && a.getLeft() <= b.getLeft() + t) && (b.getRight() - t <= a.getRight() && a.getRight() <= b.getRight() + t)) {
+            intervalRelations.add(IntervalRelations.EQUALS);
+        }
+        return intervalRelations;
+
+
+    }
+
+
+    private static List<IntervalRelations> getIntervalRelation(Pair<Double, Double> a, Pair<Double, Double> b, double t, boolean inverse) {
+
+        List<IntervalRelations> intervalRelations = new ArrayList<>();
+        if (a.getRight() < b.getLeft() - t) {
+            intervalRelations.add(inverse ? IntervalRelations.PRECEDES_INVERSE : IntervalRelations.PRECEDES);
+        }  if (b.getLeft() - t <= a.getRight() && a.getRight() <= b.getLeft() + t) {
+            intervalRelations.add(inverse ? IntervalRelations.MEETS_INVERSE : IntervalRelations.MEETS);
+        }  if (a.getLeft() < b.getLeft() - t && (b.getLeft() + t < a.getRight() && a.getRight() < b.getRight() - t)) {
+            intervalRelations.add(inverse ? IntervalRelations.OVERLAPS_INVERSE : IntervalRelations.OVERLAPS);
+        }  if ((b.getLeft() - t <= a.getLeft() && a.getLeft() <= b.getLeft() + t) && a.getRight() < b.getRight() - t) {
+            intervalRelations.add(inverse ? IntervalRelations.STARTS_INVERSE : IntervalRelations.STARTS);
+        }  if (a.getLeft() > b.getLeft() + t && a.getRight() < b.getRight() + t) {
+            intervalRelations.add(inverse ? IntervalRelations.DURING_INVERSE : IntervalRelations.DURING);
+        }  if (a.getLeft() > b.getLeft() + t && (b.getRight() - t <= a.getRight() && a.getRight() <= b.getRight() + t)) {
+            intervalRelations.add(inverse ? IntervalRelations.FINISHES_INVERSE : IntervalRelations.FINISHES);
+        }
+        return intervalRelations;
+    }
+
+}
@@ -0,0 +1,8 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+
+@FunctionalInterface
+public interface ZoneComparator {
+    boolean isBefore(Zone zone1, Zone zone2, double tolerance);
+}
@@ -0,0 +1,52 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.List;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.AngleFilter;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.DisjointSets;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Line;
+
+@Service
+public class LineBuilderService {
+
+    private static final double CHARACTER_SPACING_DISTANCE_MULTIPLIER = 3.5;
+    private static final double MAX_VERTICAL_CHARACTER_DISTANCE = 0.67;
+    private static final double ANGLE_TOLERANCE = Math.PI / 6;
+
+
+    public List<Line> buildLines(List<Character> characters, double characterSpacing, double lineSpacing) {
+
+        double maxHorizontalDistance = characterSpacing * CHARACTER_SPACING_DISTANCE_MULTIPLIER;
+        double maxVerticalDistance = lineSpacing * MAX_VERTICAL_CHARACTER_DISTANCE;
+
+        DisjointSets<Character> sets = new DisjointSets<>(characters);
+        AngleFilter filter = new AngleFilter(-ANGLE_TOLERANCE, ANGLE_TOLERANCE);
+
+        characters.forEach(character -> {
+            character.getNeighbors().forEach(neighbor -> {
+                double x = neighbor.getHorizontalDistance() / maxHorizontalDistance;
+                double y = neighbor.getVerticalDistance() / maxVerticalDistance;
+                if (character.getTextPosition().getDir() == neighbor.getCharacter().getTextPosition().getDir() && filter.matches(neighbor) && Math.pow(x, 2) + Math.pow(y,
+                        2) <= 1) {
+                    sets.union(character, neighbor.getCharacter());
+                }
+            });
+        });
+
+        List<Line> lines = new ArrayList<>();
+        sets.forEach(group -> {
+            List<Character> lineCharacters = new ArrayList<>(group);
+//            QuickSort.sort(lineCharacters, new CharacterComparator());
+            lineCharacters.sort(Comparator.comparingDouble(Character::getX));
+            lines.add(new Line(lineCharacters, characterSpacing));
+        });
+
+        return lines;
+    }
+
+}
@@ -0,0 +1,78 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.List;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Neighbor;
+
+@Service
+public class NearestNeighbourService {
+
+    private static final int NUMBER_OF_NEIGHBOURS = 8;
+    private static final double STEP = 16.0;
+
+
+    public void findNearestNeighbors(List<Character> characters) {
+
+        if (characters.isEmpty()) {
+            return;
+        }
+
+        characters.sort(Comparator.comparingDouble(Character::getX));
+
+        int maxNeighborCount = NUMBER_OF_NEIGHBOURS;
+        if (characters.size() <= NUMBER_OF_NEIGHBOURS) {
+            maxNeighborCount = characters.size() - 1;
+        }
+
+        for (int i = 0; i < characters.size(); i++) {
+
+            List<Neighbor> candidates = new ArrayList<>();
+
+            int start = i;
+            int end = i + 1;
+
+            double distance = Double.POSITIVE_INFINITY;
+
+            for (double searchDistance = 0; searchDistance < distance; ) {
+
+                searchDistance += STEP;
+                boolean newCandidatesFound = false;
+
+                while (start > 0 && characters.get(i).getX() - characters.get(start - 1).getX() < searchDistance) {
+                    start--;
+                    candidates.add(new Neighbor(characters.get(start), characters.get(i)));
+                    clearLeastDistant(candidates, maxNeighborCount);
+                    newCandidatesFound = true;
+                }
+
+                while (end < characters.size() && characters.get(end).getX() - characters.get(i).getX() < searchDistance) {
+                    candidates.add(new Neighbor(characters.get(end), characters.get(i)));
+                    clearLeastDistant(candidates, maxNeighborCount);
+                    end++;
+                    newCandidatesFound = true;
+                }
+
+                if (newCandidatesFound && candidates.size() >= maxNeighborCount) {
+                    distance = candidates.get(maxNeighborCount - 1).getDistance();
+                }
+            }
+            clearLeastDistant(candidates, maxNeighborCount);
+            characters.get(i).setNeighbors(new ArrayList<>(candidates));
+        }
+    }
+
+
+    private void clearLeastDistant(List<Neighbor> candidates, int maxNeighborCount) {
+
+        if (candidates.size() > maxNeighborCount) {
+            candidates.sort(Comparator.comparingDouble(Neighbor::getDistance));
+            candidates.remove(candidates.remove(candidates.size() - 1));
+        }
+    }
+
+}
@@ -0,0 +1,167 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.BoundingBox;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder.SpatialReasoningRules;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.readingorder.UnsupervisedReadingOrderDetector;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.utils.DoubleUtils;
+
+@Service
+public class ReadingOrderService {
+
+    private static final double THRESHOLD = 1;
+
+
+    public List<Zone> resolveNew(List<Zone> zones, boolean columnWise) {
+
+        if (zones.isEmpty() || zones.size() == 1) {
+            return zones;
+        }
+
+        SpatialReasoningRules spatialReasoningRules = columnWise ? SpatialReasoningRules.COLUMN_WISE : SpatialReasoningRules.ROW_WISE;
+
+        var unsupervisedReadingOrderDetector = new UnsupervisedReadingOrderDetector(1, spatialReasoningRules, false);
+
+        return unsupervisedReadingOrderDetector.get(zones);
+    }
+
+
+    public List<Zone> resolve(List<Zone> zones, boolean columnWise) {
+
+        if (zones.isEmpty() || zones.size() == 1) {
+            return zones;
+        }
+
+        Map<Long, Integer> histogram = new HashMap<>();
+
+        for (Zone zone : zones) {
+            long minY = Math.round(zone.getBBox().getMinY());
+            long maxY = Math.round(zone.getBBox().getMaxY());
+            for (long i = minY; i <= maxY; i++) {
+                histogram.put(i, histogram.getOrDefault(i, 0) + 1);
+            }
+        }
+
+        columnWise = histogram.values()
+                             .stream()
+                             .mapToInt(Integer::intValue).average()
+                             .orElse(1) > 1.5;
+
+        if (!columnWise) {
+            zones.sort(Comparator.comparing(BoundingBox::getY, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD))
+                               .thenComparing(BoundingBox::getX, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD)));
+            return zones;
+        }
+
+        return resolveMultiColumnReadingOder(zones);
+    }
+
+
+    private List<Zone> resolveMultiColumnReadingOder(List<Zone> zones) {
+
+        // Simple reading order resolver for multi column page layout as described here : https://pub.towardsai.net/advanced-rag-02-unveiling-pdf-parsing-b84ae866344e
+        // TODO implement a more fancy reading order resolver see https://github.com/BobLd/DocumentLayoutAnalysis/blob/master/README.md#reading-order
+
+        double minX = Double.POSITIVE_INFINITY;
+        double maxX = Double.NEGATIVE_INFINITY;
+
+        for (Zone zone : zones) {
+            if (zone.getX() < minX) {
+                minX = zone.getX();
+            }
+            if (zone.getX() + zone.getWidth() > maxX) {
+                maxX = zone.getX() + zone.getWidth();
+            }
+        }
+
+        double midLineXCoordinate = (minX + maxX) / 2;
+
+        List<Zone> leftOf = new ArrayList<>();
+        List<Zone> rightOf = new ArrayList<>();
+        List<Zone> middle = new ArrayList<>();
+        for (Zone zone : zones) {
+            if (zone.getX() < midLineXCoordinate && zone.getX() + zone.getWidth() < midLineXCoordinate) {
+                leftOf.add(zone);
+            } else if (zone.getX() > midLineXCoordinate && zone.getX() + zone.getWidth() > midLineXCoordinate) {
+                rightOf.add(zone);
+            } else {
+                middle.add(zone);
+            }
+        }
+
+        List<Zone> leftNotIntersecting = new ArrayList<>();
+        for (Zone leftZone : leftOf) {
+            boolean intersects = false;
+            for (Zone rightZone : rightOf) {
+                if (leftZone.intersectsY(rightZone)) {
+                    intersects = true;
+                    break;
+                }
+            }
+            if (!intersects) {
+                leftNotIntersecting.add(leftZone);
+            }
+        }
+
+        List<Zone> rightNotIntersecting = new ArrayList<>();
+        for (Zone rightZone : rightOf) {
+            boolean intersects = false;
+            for (Zone leftZone : leftOf) {
+                if (rightZone.intersectsY(leftZone)) {
+                    intersects = true;
+                    break;
+                }
+            }
+            if (!intersects) {
+                rightNotIntersecting.add(rightZone);
+            }
+        }
+
+        leftOf.removeAll(leftNotIntersecting);
+        rightOf.removeAll(rightNotIntersecting);
+
+        middle.addAll(leftNotIntersecting);
+        middle.addAll(rightNotIntersecting);
+
+        leftOf.sort(Comparator.comparing(BoundingBox::getY, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD))
+                            .thenComparing(BoundingBox::getX, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD)));
+
+        rightOf.sort(Comparator.comparing(BoundingBox::getY, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD))
+                             .thenComparing(BoundingBox::getX, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD)));
+
+        middle.sort(Comparator.comparing(BoundingBox::getY, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD))
+                            .thenComparing(BoundingBox::getX, (o1, o2) -> DoubleUtils.compareDouble(o1, o2, THRESHOLD)));
+
+        List<Zone> sortedZones = new ArrayList<>();
+        sortedZones.addAll(leftOf);
+        sortedZones.addAll(rightOf);
+
+        ListIterator<Zone> itty = middle.listIterator();
+
+        while (itty.hasNext()) {
+            Zone current = itty.next();
+            for (int i = 0; i < sortedZones.size(); i++) {
+                if (current.getY() < sortedZones.get(i).getY()) {
+                    sortedZones.add(i, current);
+                    itty.remove();
+                    break;
+                }
+            }
+        }
+
+        sortedZones.addAll(middle);
+
+        return sortedZones;
+    }
+
+}
@@ -0,0 +1,56 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service;
+
+import java.util.List;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.AngleFilter;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Histogram;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Neighbor;
+
+@Service
+public class SpacingService {
+
+    private static final double SPACING_HISTOGRAM_RESOLUTION = 0.5;
+    private static final double SPACING_HISTOGRAM_SMOOTHING_LENGTH = 2.5;
+    private static final double SPACING_HIST_SMOOTHING_STANDARD_DEVIATION = 0.5;
+    private static final double ANGLE_TOLERANCE = Math.PI / 6;
+
+
+    public double computeCharacterSpacing(List<Character> characters) {
+
+        return computeSpacing(characters, 0);
+    }
+
+
+    public double computeLineSpacing(List<Character> characters) {
+
+        return computeSpacing(characters, Math.PI / 2);
+    }
+
+
+    private double computeSpacing(List<Character> characters, double angle) {
+
+        double maxDistance = Double.NEGATIVE_INFINITY;
+
+        for (Character character : characters) {
+            for (Neighbor neighbor : character.getNeighbors()) {
+                maxDistance = Math.max(maxDistance, neighbor.getDistance());
+            }
+        }
+        Histogram histogram = new Histogram(0, maxDistance, SPACING_HISTOGRAM_RESOLUTION);
+        AngleFilter angleFilter = new AngleFilter(angle - ANGLE_TOLERANCE, angle + ANGLE_TOLERANCE);
+        for (Character character : characters) {
+            for (Neighbor neighbor : character.getNeighbors()) {
+                if (angleFilter.matches(neighbor)) {
+                    histogram.add(neighbor.getDistance());
+                }
+            }
+        }
+
+        histogram.gaussianSmooth(SPACING_HISTOGRAM_SMOOTHING_LENGTH, SPACING_HIST_SMOOTHING_STANDARD_DEVIATION);
+        return histogram.getPeakValue();
+    }
+
+}
@@ -0,0 +1,190 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.service;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.Set;
+
+import org.springframework.stereotype.Service;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.DisjointSets;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Line;
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Zone;
+
+@Service
+public class ZoneBuilderService {
+
+    private static final double MIN_HORIZONTAL_DISTANCE_MULTIPLIER = -0.5;
+    private static final double MAX_VERTICAL_DISTANCE_MULTIPLIER = 1.2;
+
+    private static final double MIN_HORIZONTAL_MERGE_DISTANCE_MULTIPLIER = -3.0;
+
+    private static final double MAX_VERTICAL_MERGE_DISTANCE_MULTIPLIER = 0.5;
+
+    private static final double MIN_LINE_SIZE_SCALE = 0.9;
+
+    private static final double MAX_LINE_SIZE_SCALE = 2.5;
+
+    private static final double ANGLE_TOLERANCE = Math.PI / 6;
+
+    private static final int MAX_ZONES = 300;
+
+    private static final double MAX_VERTICAL_MERGE_DISTANCE = 0.5;
+
+
+    public List<Zone> buildZones(List<Line> lines, double characterSpacing, double lineSpacing) {
+
+        double minHorizontalDistance = characterSpacing * MIN_HORIZONTAL_DISTANCE_MULTIPLIER;
+        double maxVerticalDistance = lineSpacing * MAX_VERTICAL_DISTANCE_MULTIPLIER;
+        double minHorizontalMergeDistance = characterSpacing * MIN_HORIZONTAL_MERGE_DISTANCE_MULTIPLIER;
+        double maxVerticalMergeDistance = lineSpacing * MAX_VERTICAL_MERGE_DISTANCE_MULTIPLIER;
+
+        DisjointSets<Line> sets = new DisjointSets<>(lines);
+
+        double meanHeight = calculateMeanHeight(lines);
+
+        lines.forEach(outerLine -> //
+                lines.forEach(innerLine -> {
+
+                    double scale = Math.min(outerLine.getHeight(), innerLine.getHeight()) / meanHeight;
+                    scale = Math.max(MIN_LINE_SIZE_SCALE, Math.min(scale, MAX_LINE_SIZE_SCALE));
+
+                    if (!sets.areTogether(outerLine, innerLine) && outerLine.angularDifference(innerLine) <= ANGLE_TOLERANCE) {
+
+                        double horizontalDistance = outerLine.horizontalDistance(innerLine) / scale;
+                        double verticalDistance = outerLine.verticalDistance(innerLine) / scale;
+
+                        if (minHorizontalDistance <= horizontalDistance && verticalDistance <= maxVerticalDistance //
+                                || minHorizontalMergeDistance <= horizontalDistance && verticalDistance <= maxVerticalMergeDistance) {
+                            sets.union(outerLine, innerLine);
+                        }
+                    }
+                }));
+
+        List<Zone> zones = new ArrayList<>();
+        sets.forEach(group -> {
+            zones.add(new Zone(new ArrayList<>(group)));
+        });
+
+//        List<Zone> mergedZones = mergeZones(zones);
+
+        List<Zone> finalZones = zones;
+
+        if (finalZones.size() > MAX_ZONES) {
+            List<Line> oneZoneLines = new ArrayList<>();
+            for (Zone zone : finalZones) {
+                oneZoneLines.addAll(zone.getLines());
+            }
+            return List.of(mergeLinesInZone(oneZoneLines, characterSpacing, lineSpacing));
+        }
+
+        return finalZones;
+    }
+
+
+    private List<Zone> mergeZones(List<Zone> zones) {
+
+        ListIterator<Zone> itty = zones.listIterator();
+
+        while (itty.hasNext()) {
+
+            Zone current = itty.next();
+
+            for (Zone inner : zones) {
+                if (inner == current) {
+                    continue;
+                }
+                if (current.getBBox().intersects(inner.getBBox())) {
+                    inner.getLines().addAll(current.getLines());
+                    inner.buildBBox();
+                    itty.remove();
+                    break;
+                }
+            }
+        }
+
+        return zones;
+    }
+
+
+    private double calculateMeanHeight(List<Line> lines) {
+
+        double meanHeight = 0.0;
+        double weights = 0.0;
+        for (Line line : lines) {
+            double weight = line.getLength();
+            meanHeight += line.getHeight() * weight;
+            weights += weight;
+        }
+        meanHeight /= weights;
+        return meanHeight;
+    }
+
+
+    private List<Zone> mergeLinesInZones(List<Zone> zones, double characterSpacing, double lineSpacing) {
+
+        List<Zone> merged = new ArrayList<>();
+        for (Zone zone : zones) {
+            merged.add(mergeLinesInZone(zone.getLines(), characterSpacing, lineSpacing));
+        }
+        return merged;
+    }
+
+
+    private Zone mergeLinesInZone(List<Line> lines, double characterSpacing, double lineSpacing) {
+
+        double maxHorizontalDistance = 0;
+        double minVerticalDistance = 0;
+        double maxVerticalDistance = lineSpacing * MAX_VERTICAL_MERGE_DISTANCE;
+
+        DisjointSets<Line> sets = new DisjointSets<>(lines);
+
+        lines.forEach(outer -> {
+
+            lines.forEach(inner -> {
+                if (inner != outer) {
+
+                    double horizontalDistance = outer.horizontalDistance(inner);
+                    double verticalDistance = outer.verticalDistance(inner);
+
+                    if (horizontalDistance <= maxHorizontalDistance && minVerticalDistance <= verticalDistance && verticalDistance <= maxVerticalDistance) {
+                        sets.union(outer, inner);
+                    } else if (minVerticalDistance <= verticalDistance && verticalDistance <= maxVerticalDistance && Math.abs(horizontalDistance - Math.min(outer.getLength(),
+                            inner.getLength())) < 0.1) {
+                        boolean characterOverlap = false;
+                        int overlappingCount = 0;
+                        for (Character outerCharacter : outer.getCharacters()) {
+                            for (Character innerCharacter : inner.getCharacters()) {
+                                double characterOverlapDistance = outerCharacter.overlappingDistance(innerCharacter);
+                                if (characterOverlapDistance > 2) {
+                                    characterOverlap = true;
+                                }
+                                if (characterOverlapDistance > 0) {
+                                    overlappingCount++;
+                                }
+                            }
+                        }
+                        if (!characterOverlap && overlappingCount <= 2) {
+                            sets.union(outer, inner);
+                        }
+                    }
+                }
+            });
+        });
+
+        List<Line> outputZone = new ArrayList<>();
+        for (Set<Line> group : sets) {
+            List<Character> components = new ArrayList<>();
+            for (Line line : group) {
+                components.addAll(line.getCharacters());
+            }
+            components.sort(Comparator.comparingDouble(Character::getX));
+
+            outputZone.add(new Line(components, characterSpacing));
+        }
+        return new Zone(outputZone);
+    }
+
+}
@@ -0,0 +1,40 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.utils;
+
+import java.util.Comparator;
+
+import com.knecon.fforesight.service.layoutparser.processor.services.docstrum.model.Character;
+
+public class CharacterComparator implements Comparator<Character> {
+
+    @Override
+    public int compare(Character pos1, Character pos2) {
+        // only compare text that is in the same direction
+        int cmp1 = Float.compare(pos1.getTextPosition().getDir(), pos2.getTextPosition().getDir());
+        if (cmp1 != 0) {
+            return cmp1;
+        }
+
+        // get the text direction adjusted coordinates
+        float x1 = pos1.getTextPosition().getXDirAdj();
+        float x2 = pos2.getTextPosition().getXDirAdj();
+
+        float pos1YBottom = pos1.getTextPosition().getYDirAdj();
+        float pos2YBottom = pos2.getTextPosition().getYDirAdj();
+
+        // note that the coordinates have been adjusted so 0,0 is in upper left
+        float pos1YTop = pos1YBottom - pos1.getTextPosition().getHeightDir();
+        float pos2YTop = pos2YBottom - pos2.getTextPosition().getHeightDir();
+
+        float yDifference = Math.abs(pos1YBottom - pos2YBottom);
+
+        // we will do a simple tolerance comparison
+        if (yDifference < .1 || pos2YBottom >= pos1YTop && pos2YBottom <= pos1YBottom || pos1YBottom >= pos2YTop && pos1YBottom <= pos2YBottom) {
+            return Float.compare(x1, x2);
+        } else if (pos1YBottom < pos2YBottom) {
+            return -1;
+        } else {
+            return 1;
+        }
+    }
+
+}
@@ -0,0 +1,18 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.utils;
+
+public class DoubleUtils {
+
+    public static int compareDouble(double d1, double d2, double precision) {
+
+        if (Double.isNaN(d1) || Double.isNaN(d2)) {
+            return Double.compare(d1, d2);
+        }
+        if (precision == 0) {
+            precision = 1;
+        }
+        long i1 = Math.round(d1 / precision);
+        long i2 = Math.round(d2 / precision);
+        return Long.compare(i1, i2);
+    }
+
+}
@@ -0,0 +1,39 @@
+package com.knecon.fforesight.service.layoutparser.processor.services.docstrum.utils;
+
+import java.util.Comparator;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+
+public class ReadingOrderHelper {
+
+    public static List<TextPositionSequence> orderByReadingOrder(List<TextPositionSequence> words) {
+        if (words.size() <= 1) {
+            return words;
+        }
+
+        int textOrientation = words.get(0).getRotation();
+
+        switch (textOrientation) {
+            case 0:
+                return words.stream()
+                        .sorted(Comparator.comparingDouble(w -> w.getRectangle().getTopLeft().getX()))
+                        .collect(Collectors.toList());
+            case 90:
+                return words.stream()
+                        .sorted((w1, w2) -> -Double.compare(w1.getRectangle().getTopLeft().getY(), w2.getRectangle().getTopLeft().getY()))
+                        .collect(Collectors.toList());
+            case 180:
+                return words.stream()
+                        .sorted((w1, w2) -> -Double.compare(w1.getRectangle().getTopLeft().getX(), w2.getRectangle().getTopLeft().getX()))
+                        .collect(Collectors.toList());
+            case 270:
+                return words.stream()
+                        .sorted(Comparator.comparingDouble(w -> w.getRectangle().getTopLeft().getY()))
+                        .collect(Collectors.toList());
+            default:
+                throw new IllegalArgumentException("Not sure what to do with this text rotation...");
+        }
+    }
+}
@@ -64,7 +64,7 @@ public class DocumentGraphFactory {

    private void addSections(ClassificationDocument document, Context context) {

-        document.getSections().forEach(section -> SectionNodeFactory.addSection(null, section.getPageBlocks(), section.getImages(), context));
+        document.getSections().forEach(section -> SectionNodeFactory.addSection(null, section.getNonEmptyPageBlocks(), section.getImages(), context));
    }


@@ -133,7 +133,7 @@ public class SearchTextWithTextPositionFactory {

    private static void addTextPositionWithFontType(RedTextPosition currentTextPosition, String fontType, List<Integer> fontTypePositions, int stringIdx) {

-        if (currentTextPosition.getFontName().toLowerCase(Locale.ROOT).contains(fontType)) {
+        if (currentTextPosition.getFontName() != null && currentTextPosition.getFontName().toLowerCase(Locale.ROOT).contains(fontType)) {
            fontTypePositions.add(stringIdx);
        }
    }
@@ -1,6 +1,7 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.mapper;

 import java.awt.geom.Rectangle2D;
+import java.util.Collections;
 import java.util.HashMap;
 import java.util.Locale;
 import java.util.Map;
@@ -8,6 +9,7 @@ import java.util.Map;
 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.DocumentStructure;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Image;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.ImageType;
+import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Page;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;

@@ -92,28 +92,28 @@ public class LegacyPDFStreamEngine extends PDFStreamEngine {
     */
    LegacyPDFStreamEngine() throws IOException {

-        addOperator(new BeginText());
-        addOperator(new Concatenate());
-        addOperator(new DrawObject()); // special text version
-        addOperator(new EndText());
-        addOperator(new SetGraphicsStateParameters());
-        addOperator(new Save());
-        addOperator(new Restore());
-        addOperator(new NextLine());
-        addOperator(new SetCharSpacing());
-        addOperator(new MoveText());
-        addOperator(new MoveTextSetLeading());
-        addOperator(new SetFontAndSize());
-        addOperator(new ShowText());
-        addOperator(new ShowTextAdjusted());
-        addOperator(new SetTextLeading());
-        addOperator(new SetMatrix());
-        addOperator(new SetTextRenderingMode());
-        addOperator(new SetTextRise());
-        addOperator(new SetWordSpacing());
-        addOperator(new SetTextHorizontalScaling());
-        addOperator(new ShowTextLine());
-        addOperator(new ShowTextLineAndSpace());
+        addOperator(new BeginText(this));
+        addOperator(new Concatenate(this));
+        addOperator(new DrawObject(this)); // special text version
+        addOperator(new EndText(this));
+        addOperator(new SetGraphicsStateParameters(this));
+        addOperator(new Save(this));
+        addOperator(new Restore(this));
+        addOperator(new NextLine(this));
+        addOperator(new SetCharSpacing(this));
+        addOperator(new MoveText(this));
+        addOperator(new MoveTextSetLeading(this));
+        addOperator(new SetFontAndSize(this));
+        addOperator(new ShowText(this));
+        addOperator(new ShowTextAdjusted(this));
+        addOperator(new SetTextLeading(this));
+        addOperator(new SetMatrix(this));
+        addOperator(new SetTextRenderingMode(this));
+        addOperator(new SetTextRise(this));
+        addOperator(new SetWordSpacing(this));
+        addOperator(new SetTextHorizontalScaling(this));
+        addOperator(new ShowTextLine(this));
+        addOperator(new ShowTextLineAndSpace(this));

        // load additional glyph list for Unicode mapping
        String path = "/org/apache/pdfbox/resources/glyphlist/additional.txt";
@@ -1,18 +1,34 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.parsing;

-import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.RedTextPosition;
-import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
-import lombok.Getter;
-import lombok.Setter;
-import lombok.SneakyThrows;
-import lombok.extern.slf4j.Slf4j;
+import java.awt.color.CMMException;
+import java.awt.geom.Point2D;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
 import org.apache.pdfbox.contentstream.operator.Operator;
 import org.apache.pdfbox.contentstream.operator.OperatorName;
-import org.apache.pdfbox.contentstream.operator.color.*;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingColor;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingColorN;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingColorSpace;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingDeviceCMYKColor;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingDeviceGrayColor;
+import org.apache.pdfbox.contentstream.operator.color.SetNonStrokingDeviceRGBColor;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingColor;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingColorN;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingColorSpace;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingDeviceCMYKColor;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingDeviceGrayColor;
+import org.apache.pdfbox.contentstream.operator.color.SetStrokingDeviceRGBColor;
 import org.apache.pdfbox.contentstream.operator.markedcontent.BeginMarkedContentSequenceWithProperties;
 import org.apache.pdfbox.contentstream.operator.markedcontent.EndMarkedContentSequence;
-import org.apache.pdfbox.contentstream.operator.state.*;
+import org.apache.pdfbox.contentstream.operator.state.SetFlatness;
+import org.apache.pdfbox.contentstream.operator.state.SetLineCapStyle;
+import org.apache.pdfbox.contentstream.operator.state.SetLineDashPattern;
+import org.apache.pdfbox.contentstream.operator.state.SetLineJoinStyle;
+import org.apache.pdfbox.contentstream.operator.state.SetLineMiterLimit;
+import org.apache.pdfbox.contentstream.operator.state.SetLineWidth;
+import org.apache.pdfbox.contentstream.operator.state.SetRenderingIntent;
 import org.apache.pdfbox.contentstream.operator.text.SetFontAndSize;
 import org.apache.pdfbox.cos.COSBase;
 import org.apache.pdfbox.cos.COSNumber;
@@ -21,11 +37,14 @@ import org.apache.pdfbox.pdmodel.PDPage;
 import org.apache.pdfbox.pdmodel.graphics.color.PDColor;
 import org.apache.pdfbox.text.TextPosition;

-import java.awt.color.CMMException;
-import java.awt.geom.Point2D;
-import java.io.IOException;
-import java.util.ArrayList;
-import java.util.List;
+import com.knecon.fforesight.service.layoutparser.processor.model.table.Ruling;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.RedTextPosition;
+import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
+
+import lombok.Getter;
+import lombok.Setter;
+import lombok.SneakyThrows;
+import lombok.extern.slf4j.Slf4j;

@Getter
@Slf4j
@@ -36,11 +55,6 @@ public class PDFLinesTextStripper extends PDFTextStripper {
    private final List<Ruling> graphicsPath = new ArrayList<>();
    @Setter
    protected PDPage pdpage;
-    private int minCharWidth;
-    private int maxCharWidth;
-    private int minCharHeight;
-    private int maxCharHeight;
-

    private float path_x;
    private float path_y;
@@ -52,32 +66,30 @@ public class PDFLinesTextStripper extends PDFTextStripper {
    public PDFLinesTextStripper() throws IOException {

        super();
-        this.addOperator(new SetStrokingColorSpace());
-        this.addOperator(new SetNonStrokingColorSpace());
-        this.addOperator(new SetLineDashPattern());
-        this.addOperator(new SetStrokingDeviceGrayColor());
-        this.addOperator(new SetNonStrokingDeviceGrayColor());
-        this.addOperator(new SetFlatness());
-        this.addOperator(new SetLineJoinStyle());
-        this.addOperator(new SetLineCapStyle());
-        this.addOperator(new SetStrokingDeviceCMYKColor());
-        this.addOperator(new SetNonStrokingDeviceCMYKColor());
-        this.addOperator(new SetLineMiterLimit());
-        this.addOperator(new SetStrokingDeviceRGBColor());
-        this.addOperator(new SetNonStrokingDeviceRGBColor());
-        this.addOperator(new SetRenderingIntent());
-        this.addOperator(new SetStrokingColor());
-        this.addOperator(new SetNonStrokingColor());
-        this.addOperator(new SetStrokingColorN());
-        this.addOperator(new SetNonStrokingColorN());
-        this.addOperator(new SetFontAndSize());
-        this.addOperator(new SetLineWidth());
-
-
-        addOperator(new BeginMarkedContentSequenceWithProperties());
-//        addOperator(new BeginMarkedContentSequence());
-        addOperator(new EndMarkedContentSequence());
+        this.addOperator(new SetStrokingColorSpace(this));
+        this.addOperator(new SetNonStrokingColorSpace(this));
+        this.addOperator(new SetLineDashPattern(this));
+        this.addOperator(new SetStrokingDeviceGrayColor(this));
+        this.addOperator(new SetNonStrokingDeviceGrayColor(this));
+        this.addOperator(new SetFlatness(this));
+        this.addOperator(new SetLineJoinStyle(this));
+        this.addOperator(new SetLineCapStyle(this));
+        this.addOperator(new SetStrokingDeviceCMYKColor(this));
+        this.addOperator(new SetNonStrokingDeviceCMYKColor(this));
+        this.addOperator(new SetLineMiterLimit(this));
+        this.addOperator(new SetStrokingDeviceRGBColor(this));
+        this.addOperator(new SetNonStrokingDeviceRGBColor(this));
+        this.addOperator(new SetRenderingIntent(this));
+        this.addOperator(new SetStrokingColor(this));
+        this.addOperator(new SetNonStrokingColor(this));
+        this.addOperator(new SetStrokingColorN(this));
+        this.addOperator(new SetNonStrokingColorN(this));
+        this.addOperator(new SetFontAndSize(this));
+        this.addOperator(new SetLineWidth(this));

+        addOperator(new BeginMarkedContentSequenceWithProperties(this));
+//        addOperator(new BeginMarkedContentSequence(this));
+        addOperator(new EndMarkedContentSequence(this));
    }


@@ -225,52 +237,44 @@ public class PDFLinesTextStripper extends PDFTextStripper {
        int startIndex = 0;
        RedTextPosition previous = null;

+        float direction = -1;
        for (int i = 0; i <= textPositions.size() - 1; i++) {

+            if (direction == -1) {
+                direction = textPositions.get(i).getDir();
+            }
+
            if (!textPositionSequences.isEmpty()) {
                previous = textPositionSequences.get(textPositionSequences.size() - 1)
                        .getTextPositions()
                        .get(textPositionSequences.get(textPositionSequences.size() - 1).getTextPositions().size() - 1);
            }

-            int charWidth = (int) textPositions.get(i).getWidthDirAdj();
-            if (charWidth < minCharWidth) {
-                minCharWidth = charWidth;
-            }
-            if (charWidth > maxCharWidth) {
-                maxCharWidth = charWidth;
-            }
-
-            int charHeight = (int) textPositions.get(i).getHeightDir();
-            if (charHeight < minCharHeight) {
-                minCharHeight = charHeight;
-            }
-            if (charWidth > maxCharHeight) {
-                maxCharHeight = charHeight;
-            }
-
            if (i == 0 && (textPositions.get(i).getUnicode().equals(" ") || textPositions.get(i).getUnicode().equals("\u00A0") || textPositions.get(i).getUnicode().equals("\t"))) {
                startIndex++;
                continue;
            }

-            // Strange but sometimes this is happening, for example: Metolachlor2.pdf
-            if (i > 0 && textPositions.get(i).getXDirAdj() < textPositions.get(i - 1).getXDirAdj()) {
+            if (textPositions.get(i).getDir() != direction && startIndex != i) {
                List<TextPosition> sublist = textPositions.subList(startIndex, i);
-                if (!(sublist.isEmpty() || sublist.size() == 1 && (sublist.get(0).getUnicode().equals(" ") || sublist.get(0).getUnicode().equals("\u00A0") || sublist.get(0)
-                        .getUnicode()
-                        .equals("\t")))) {
-                    textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart));
+                textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart, textPositionSequences.size()));
+                startIndex = i;
+                direction = textPositions.get(i).getDir();
+            }
+
+            // Strange but sometimes this is happening, for example: Metolachlor2.pdf
+            if (checkIfCurrentPositionIsToTheRightOfPreviousPosition(i, textPositions)) {
+                List<TextPosition> sublist = textPositions.subList(startIndex, i);
+                if (checkIfSequenceContainsOnlyWhitespaces(sublist)) {
+                    textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart, textPositionSequences.size()));
                }
                startIndex = i;
            }

            if (textPositions.get(i).getRotation() == 0 && i > 0 && textPositions.get(i).getX() > textPositions.get(i - 1).getEndX() + 1) {
                List<TextPosition> sublist = textPositions.subList(startIndex, i);
-                if (!(sublist.isEmpty() || sublist.size() == 1 && (sublist.get(0).getUnicode().equals(" ") || sublist.get(0).getUnicode().equals("\u00A0") || sublist.get(0)
-                        .getUnicode()
-                        .equals("\t")))) {
-                    textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart));
+                if (checkIfSequenceContainsOnlyWhitespaces(sublist)) {
+                    textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart, textPositionSequences.size()));
                }
                startIndex = i;
            }
@@ -279,18 +283,15 @@ public class PDFLinesTextStripper extends PDFTextStripper {
                    .getUnicode()
                    .equals("\t")) && i <= textPositions.size() - 2) {
                List<TextPosition> sublist = textPositions.subList(startIndex, i);
-                if (!(sublist.isEmpty() || sublist.size() == 1 && (sublist.get(0).getUnicode().equals(" ") || sublist.get(0).getUnicode().equals("\u00A0") || sublist.get(0)
-                        .getUnicode()
-                        .equals("\t")))) {
+                if (checkIfSequenceContainsOnlyWhitespaces(sublist)) {

                    // Remove false sequence ends (whitespaces)
-                    if (previous != null && sublist.get(0).getYDirAdj() == previous.getYDirAdj() && sublist.get(0)
-                            .getXDirAdj() - (previous.getXDirAdj() + previous.getWidthDirAdj()) < 0.01) {
+                    if (checkIfGapSizeBetweenCharactersSmallerThanMaximum(previous, sublist, 0.01f)) {
                        for (TextPosition t : sublist) {
-                            textPositionSequences.get(textPositionSequences.size() - 1).add(t);
+                            textPositionSequences.get(textPositionSequences.size() - 1).add(t, textPositionSequences.size());
                        }
                    } else {
-                        textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart));
+                        textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, i == textPositions.size() - 1 && isParagraphStart, textPositionSequences.size()));
                    }
                }
                startIndex = i + 1;
@@ -310,23 +311,40 @@ public class PDFLinesTextStripper extends PDFTextStripper {
            if (previous != null && sublist.get(0).getYDirAdj() == previous.getYDirAdj() && sublist.get(0)
                    .getXDirAdj() - (previous.getXDirAdj() + previous.getWidthDirAdj()) < 0.01) {
                for (TextPosition t : sublist) {
-                    textPositionSequences.get(textPositionSequences.size() - 1).add(t);
+                    textPositionSequences.get(textPositionSequences.size() - 1).add(t, textPositionSequences.size());
                }
            } else {
-                textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, isParagraphStart));
+                textPositionSequences.add(new TextPositionSequence(sublist, pageNumber, isParagraphStart, textPositionSequences.size()));
            }
        }
        super.writeString(text);
    }


+    public boolean checkIfCurrentPositionIsToTheRightOfPreviousPosition(int i, List<TextPosition> textPositions) {
+
+        return i > 0 && textPositions.get(i).getXDirAdj() < textPositions.get(i - 1).getXDirAdj();
+    }
+
+
+    public boolean checkIfSequenceContainsOnlyWhitespaces(List<TextPosition> sublist) {
+
+        return !(sublist.isEmpty() || sublist.size() == 1 && (sublist.get(0).getUnicode().equals(" ") || sublist.get(0).getUnicode().equals("\u00A0") || sublist.get(0)
+                .getUnicode()
+                .equals("\t")));
+    }
+
+
+    public boolean checkIfGapSizeBetweenCharactersSmallerThanMaximum(RedTextPosition previous, List<TextPosition> sublist, float maximumGapSize) {
+
+        return previous != null && sublist.get(0).getYDirAdj() == previous.getYDirAdj() && sublist.get(0)
+                .getXDirAdj() - (previous.getXDirAdj() + previous.getWidthDirAdj()) < maximumGapSize;
+    }
+
+
    @Override
    public String getText(PDDocument doc) throws IOException {

-        minCharWidth = Integer.MAX_VALUE;
-        maxCharWidth = 0;
-        minCharHeight = Integer.MAX_VALUE;
-        maxCharHeight = 0;
        textPositionSequences.clear();
        rulings.clear();
        graphicsPath.clear();
@@ -43,7 +43,8 @@ import org.apache.pdfbox.pdmodel.interactive.documentnavigation.outline.PDOutlin
 import org.apache.pdfbox.pdmodel.interactive.pagenavigation.PDThreadBead;
 import org.apache.pdfbox.text.TextPosition;
 import org.apache.pdfbox.text.TextPositionComparator;
-import org.apache.pdfbox.util.QuickSort;
+
+import com.knecon.fforesight.service.layoutparser.processor.utils.QuickSort;

 /**
 * This is just a copy except i only adjusted lines 594-607 cause this is a bug in Pdfbox.
@@ -1710,7 +1711,7 @@ public class PDFTextStripper extends LegacyPDFStreamEngine {
        int numberOfStrings = line.size();
        for (int i = 0; i < numberOfStrings; i++) {
            WordWithTextPositions word = line.get(i);
-            word.getTextPositions().sort(Comparator.comparing(TextPosition::getX));
+            word.getTextPositions().sort(Comparator.comparing(TextPosition::getXDirAdj));
            writeString(word.getText(), word.getTextPositions(), isParagraphEnd && i == numberOfStrings - 1);
            if (i < numberOfStrings - 1) {
                writeWordSeparator();
@@ -1,12 +1,11 @@
 package com.knecon.fforesight.service.layoutparser.processor.services.visualization;

-import static com.knecon.fforesight.service.layoutparser.processor.services.visualization.ViewerDocumentService.LINE_WIDTH;
-
 import java.awt.Color;
 import java.awt.geom.Line2D;
 import java.awt.geom.Point2D;
 import java.awt.geom.Rectangle2D;
 import java.awt.geom.RectangularShape;
+import java.io.File;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.Comparator;
@@ -24,25 +23,59 @@ import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Pa
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.SemanticNode;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Table;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.TableCell;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredLine;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.LayoutGrid;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.PlacedText;
 import com.knecon.fforesight.service.layoutparser.processor.utils.RectangleTransformations;
+import com.knecon.fforesight.service.viewerdoc.ContentStreams;
+import com.knecon.fforesight.service.viewerdoc.model.ColoredLine;
+import com.knecon.fforesight.service.viewerdoc.model.ColoredRectangle;
+import com.knecon.fforesight.service.viewerdoc.model.LayoutGrid;
+import com.knecon.fforesight.service.viewerdoc.model.PlacedText;
+import com.knecon.fforesight.service.viewerdoc.model.Standard14EmbeddableFont;
+import com.knecon.fforesight.service.viewerdoc.model.Visualizations;
+import com.knecon.fforesight.service.viewerdoc.service.ViewerDocumentService;
+
+import io.micrometer.observation.annotation.Observed;
+import lombok.AccessLevel;
+import lombok.RequiredArgsConstructor;
+import lombok.SneakyThrows;
+import lombok.experimental.FieldDefaults;

@Service
+@RequiredArgsConstructor
+@FieldDefaults(level = AccessLevel.PRIVATE, makeFinal = true)
 public class LayoutGridService {

-    private static final Color INNER_LINES_COLOR = new Color(255, 175, 175);
-    private static final Color PARAGRAPH_COLOR = new Color(70, 130, 180);
-    public static final Color TABLE_COLOR = new Color(102, 205, 170);
-    public static final Color SECTION_COLOR = new Color(50, 50, 50);
-    public static final Color HEADLINE_COLOR = new Color(162, 56, 56);
-    public static final Color HEADER_COLOR = new Color(171, 131, 6);
-    public static final Color IMAGE_COLOR = new Color(253, 63, 146);
+    ViewerDocumentService viewerDocumentService;
+
+    static float FONT_SIZE = 10f;
+    static float LINE_WIDTH = 1f;
+    static Standard14EmbeddableFont FONT = Standard14EmbeddableFont.helvetica();
+
+    static Color INNER_LINES_COLOR = new Color(255, 175, 175);
+    static Color PARAGRAPH_COLOR = new Color(70, 130, 180);
+    static Color TABLE_COLOR = new Color(102, 205, 170);
+    static Color SECTION_COLOR = new Color(50, 50, 50);
+    static Color HEADLINE_COLOR = new Color(162, 56, 56);
+    static Color HEADER_COLOR = new Color(171, 131, 6);
+    static Color IMAGE_COLOR = new Color(253, 63, 146);


-    public LayoutGrid createLayoutGrid(Document document) {
+    @SneakyThrows
+    @Observed(name = "ViewerDocumentService", contextualName = "create-viewer-document")
+    public void addLayoutGrid(File originFile, Document document, File destinationFile, boolean layerVisibilityDefaultValue) {
+
+        LayoutGrid layoutGrid = createLayoutGrid(document);
+
+        viewerDocumentService.addVisualizationsOnPage(originFile,
+                destinationFile,
+                Visualizations.builder()
+                        .layer(ContentStreams.KNECON_LAYOUT)
+                        .visualizationsOnPages(layoutGrid.getVisualizationsPerPages())
+                        .layerVisibilityDefaultValue(layerVisibilityDefaultValue)
+                        .build());
+    }
+
+
+    private LayoutGrid createLayoutGrid(Document document) {

        LayoutGrid layoutGrid = new LayoutGrid(document.getNumberOfPages());
        document.streamAllSubNodes().forEach(semanticNode -> {
@@ -103,11 +136,11 @@ public class LayoutGridService {
            List<ColoredLine> coloredLines = layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getColoredLines();
            xs.forEach(x -> {
                Line2D line = new Line2D.Double(new Point2D.Double(x, tableBBox.getMaxY()), new Point2D.Double(x, tableBBox.getMinY()));
-                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR));
+                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR, LINE_WIDTH));
            });
            ys.forEach(y -> {
                Line2D line = new Line2D.Double(new Point2D.Double(tableBBox.getMinX(), y), new Point2D.Double(tableBBox.getMaxX(), y));
-                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR));
+                coloredLines.add(new ColoredLine(line, INNER_LINES_COLOR, LINE_WIDTH));
            });
        }
    }
@@ -135,9 +168,9 @@ public class LayoutGridService {
            List<Line2D> lines = createLinesFromRectangle(r, firstPage.getRotation());
            // add string to top line
            var firstLine = lines.remove(0);
-            coloredLines.add(new ColoredLine(firstLine, color));
+            coloredLines.add(new ColoredLine(firstLine, color, LINE_WIDTH));
            for (Line2D line : lines) {
-                coloredLines.add(new ColoredLine(line, color));
+                coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
            }
            return;
        }
@@ -152,6 +185,7 @@ public class LayoutGridService {
    }


+    @SneakyThrows
    private void addPlacedText(Page page, Rectangle2D textBBox, String s, LayoutGrid layoutGrid) {

        Point2D.Float upperLeftCorner = switch (page.getRotation()) {
@@ -161,7 +195,8 @@ public class LayoutGridService {
            default -> new Point2D.Float((float) (textBBox.getMinX()), (float) textBBox.getMaxY());
        };
        var placedTexts = layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getPlacedTexts();
-        placedTexts.add(new PlacedText(s, upperLeftCorner));
+        upperLeftCorner.setLocation(upperLeftCorner.getX() - ((FONT.getStringWidth(s) / 1000) * FONT_SIZE + (2 * LINE_WIDTH) + 4), upperLeftCorner.getY() - FONT_SIZE);
+        placedTexts.add(PlacedText.textFacingUp(s, upperLeftCorner, FONT_SIZE, Color.BLACK, FONT));
    }


@@ -176,9 +211,9 @@ public class LayoutGridService {
        midPageLines.remove(1);
        // add string to left line
        var leftLine = midPageLines.remove(1);
-        coloredLines.add(new ColoredLine(leftLine, color));
+        coloredLines.add(new ColoredLine(leftLine, color, LINE_WIDTH));
        for (Line2D line : midPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
        }
    }

@@ -192,9 +227,9 @@ public class LayoutGridService {
        lastPageLines.remove(0);
        // add string to left line
        var leftLine = lastPageLines.remove(2);
-        coloredLines.add(new ColoredLine(leftLine, color));
+        coloredLines.add(new ColoredLine(leftLine, color, LINE_WIDTH));
        for (Line2D line : lastPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
        }
    }

@@ -208,9 +243,9 @@ public class LayoutGridService {
        firstPageLines.remove(2);
        // add string to top line
        var firstLine = firstPageLines.remove(0);
-        coloredLines.add(new ColoredLine(firstLine, color));
+        coloredLines.add(new ColoredLine(firstLine, color, LINE_WIDTH));
        for (Line2D line : firstPageLines) {
-            coloredLines.add(new ColoredLine(line, color));
+            coloredLines.add(new ColoredLine(line, color, LINE_WIDTH));
        }
    }

@@ -276,7 +311,10 @@ public class LayoutGridService {
    private void addAsRectangle(SemanticNode semanticNode, LayoutGrid layoutGrid, Color color) {

        semanticNode.getBBox()
-                .forEach((page, textBBox) -> layoutGrid.getVisualizationsPerPages().get(page.getNumber() - 1).getColoredRectangles().add(new ColoredRectangle(textBBox, color)));
+                .forEach((page, textBBox) -> layoutGrid.getVisualizationsPerPages()
+                        .get(page.getNumber() - 1)
+                        .getColoredRectangles()
+                        .add(new ColoredRectangle(textBBox, color, LINE_WIDTH)));
    }

 }
@@ -1,179 +0,0 @@
-package com.knecon.fforesight.service.layoutparser.processor.services.visualization;
-
-import java.awt.geom.AffineTransform;
-import java.awt.geom.Rectangle2D;
-import java.io.IOException;
-import java.io.OutputStream;
-import java.util.HashSet;
-import java.util.Set;
-
-import org.apache.pdfbox.cos.COSDictionary;
-import org.apache.pdfbox.cos.COSName;
-import org.apache.pdfbox.pdmodel.PDDocument;
-import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
-import org.apache.pdfbox.pdmodel.PDPage;
-import org.apache.pdfbox.pdmodel.PDPageContentStream;
-import org.apache.pdfbox.pdmodel.PDResources;
-import org.apache.pdfbox.pdmodel.font.PDFont;
-import org.apache.pdfbox.pdmodel.font.PDType1Font;
-import org.apache.pdfbox.pdmodel.graphics.optionalcontent.PDOptionalContentGroup;
-import org.apache.pdfbox.pdmodel.graphics.optionalcontent.PDOptionalContentProperties;
-import org.apache.pdfbox.pdmodel.graphics.state.PDExtendedGraphicsState;
-import org.apache.pdfbox.util.Matrix;
-import org.springframework.stereotype.Service;
-
-import com.knecon.fforesight.service.layoutparser.processor.model.graph.nodes.Document;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredLine;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.ColoredRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.FilledRectangle;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.LayoutGrid;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.PlacedText;
-import com.knecon.fforesight.service.layoutparser.processor.model.visualization.VisualizationsOnPage;
-
-import lombok.RequiredArgsConstructor;
-import lombok.SneakyThrows;
-import lombok.extern.slf4j.Slf4j;
-
-@Slf4j
-@Service
-@RequiredArgsConstructor
-public class ViewerDocumentService {
-
-    private static final String layerName = "Layout grid";
-
-    private static final int FONT_SIZE = 10;
-    public static final float LINE_WIDTH = 1f;
-
-    private final LayoutGridService layoutGridService;
-
-
-    @SneakyThrows
-    public void createViewerDocument(PDDocument pdDocument, Document document, OutputStream outputStream) {
-
-        log.info("Start Viewer Document Creation");
-        LayoutGrid layoutGrid = layoutGridService.createLayoutGrid(document);
-        // PDDocument.save() is very slow, since it actually traverses the entire pdf and writes a new one.
-        // If we collect all COSDictionaries we changed and tell it explicitly to only add the changed ones by using saveIncremental it's very fast.
-        Set<COSDictionary> dictionariesToUpdate = new HashSet<>();
-        PDOptionalContentGroup layer = addLayerToDocument(pdDocument, dictionariesToUpdate);
-        PDFont font = PDType1Font.HELVETICA;
-
-        for (int pageNumber = 0; pageNumber < pdDocument.getNumberOfPages(); pageNumber++) {
-            PDPage pdPage = pdDocument.getPage(pageNumber);
-
-            AffineTransform textDeRotationMatrix = getTextDeRotationTransform(pdPage);
-            addLayerToPageRessources(pdPage);
-
-            // We need to save the graphics state before, such that our appended content cannot be affected by previous content streams with side effects,
-            // e.g. not escaped matrix transformations.
-            escapePreviousContents(pdDocument, pdPage);
-
-            VisualizationsOnPage visualizationsOnPage = layoutGrid.getVisualizationsPerPages().get(pageNumber);
-            assert pageNumber == visualizationsOnPage.getPageNumber();
-            // We need to append to the content stream, otherwise the content could be overlapped by following content.
-            try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.APPEND, true)) {
-
-                contentStream.beginMarkedContent(COSName.OC, layer);
-                contentStream.saveGraphicsState();
-
-                contentStream.setLineWidth(LINE_WIDTH);
-                for (ColoredLine coloredLine : visualizationsOnPage.getColoredLines()) {
-                    contentStream.setStrokingColor(coloredLine.color());
-                    contentStream.moveTo((float) coloredLine.line().getX1(), (float) coloredLine.line().getY1());
-                    contentStream.lineTo((float) coloredLine.line().getX2(), (float) coloredLine.line().getY2());
-                    contentStream.stroke();
-                }
-                for (ColoredRectangle coloredRectangle : visualizationsOnPage.getColoredRectangles()) {
-                    contentStream.setStrokingColor(coloredRectangle.color());
-                    Rectangle2D r = coloredRectangle.rectangle2D();
-                    contentStream.addRect((float) r.getX(), (float) r.getY(), (float) r.getWidth(), (float) r.getHeight());
-                    contentStream.stroke();
-                }
-                for (FilledRectangle filledRectangle : visualizationsOnPage.getFilledRectangles()) {
-                    contentStream.setNonStrokingColor(filledRectangle.color());
-                    PDExtendedGraphicsState graphicsState = new PDExtendedGraphicsState();
-                    graphicsState.setNonStrokingAlphaConstant(filledRectangle.alpha());
-                    contentStream.setGraphicsStateParameters(graphicsState);
-                    Rectangle2D r = filledRectangle.rectangle2D();
-                    contentStream.addRect((float) r.getX(), (float) r.getY(), (float) r.getWidth(), (float) r.getHeight());
-                    contentStream.fill();
-                }
-                for (PlacedText placedText : visualizationsOnPage.getPlacedTexts()) {
-                    contentStream.setFont(font, FONT_SIZE);
-                    contentStream.beginText();
-                    Matrix textMatrix = new Matrix((float) textDeRotationMatrix.getScaleX(),
-                            (float) textDeRotationMatrix.getShearX(),
-                            (float) textDeRotationMatrix.getShearY(),
-                            (float) textDeRotationMatrix.getScaleY(),
-                            (float) placedText.lineStart().getX(),
-                            (float) placedText.lineStart().getY());
-                    textMatrix.translate(-((font.getStringWidth(placedText.text()) / 1000) * FONT_SIZE + (2 * LINE_WIDTH) + 4), -FONT_SIZE);
-                    contentStream.setTextMatrix(textMatrix);
-                    contentStream.showText(placedText.text());
-                    contentStream.endText();
-                }
-                contentStream.restoreGraphicsState();
-                contentStream.endMarkedContent();
-            }
-            dictionariesToUpdate.add(pdPage.getCOSObject());
-            dictionariesToUpdate.add(pdPage.getResources().getCOSObject());
-        }
-        dictionariesToUpdate.add(pdDocument.getDocumentInformation().getCOSObject());
-        pdDocument.saveIncremental(outputStream, dictionariesToUpdate);
-        log.info("Saved Viewer Document");
-    }
-
-
-    private static void addLayerToPageRessources(PDPage pdPage) {
-
-        PDResources resources = pdPage.getResources();
-        if (resources == null) {
-            resources = new PDResources();
-            pdPage.setResources(resources);
-        }
-    }
-
-
-    private static void escapePreviousContents(PDDocument pdDocument, PDPage pdPage) throws IOException {
-
-        try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.PREPEND, false)) {
-            contentStream.saveGraphicsState();
-        }
-        try (var contentStream = new PDPageContentStream(pdDocument, pdPage, PDPageContentStream.AppendMode.APPEND, false)) {
-            contentStream.restoreGraphicsState();
-        }
-    }
-
-
-    private static PDOptionalContentGroup addLayerToDocument(PDDocument pdDocument, Set<COSDictionary> dictionariesToUpdate) {
-
-        PDDocumentCatalog catalog = pdDocument.getDocumentCatalog();
-        PDOptionalContentProperties ocprops = catalog.getOCProperties();
-        if (ocprops == null) {
-            ocprops = new PDOptionalContentProperties();
-            catalog.setOCProperties(ocprops);
-        }
-        PDOptionalContentGroup layer = null;
-        if (ocprops.hasGroup(layerName)) {
-            layer = ocprops.getGroup(layerName);
-        } else {
-            layer = new PDOptionalContentGroup(layerName);
-            ocprops.addGroup(layer);
-        }
-        ocprops.setGroupEnabled(layer, false);
-        dictionariesToUpdate.add(catalog.getCOSObject());
-        return layer;
-    }
-
-
-    private static AffineTransform getTextDeRotationTransform(PDPage page) {
-
-        return AffineTransform.getQuadrantRotateInstance(switch (page.getRotation()) {
-            case 90 -> 3;
-            case 180 -> 2;
-            case 270 -> 1;
-            default -> 0;
-        });
-    }
-
-}
@@ -9,6 +9,7 @@ import org.apache.pdfbox.text.TextPosition;

 import java.awt.geom.Rectangle2D;
 import java.util.Collection;
+import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 import java.util.stream.Collectors;
@@ -19,10 +20,10 @@ public class MarkedContentUtils {
    public static final String HEADER = "Header";
    public static final String FOOTER = "Footer";

-    public Rectangle2D getMarkedContentBboxPerLine(List<PDMarkedContent> markedContents, String subtype) {
+    public List<Rectangle2D> getMarkedContentBboxPerLine(List<PDMarkedContent> markedContents, String subtype) {

        if (markedContents == null) {
-            return null;
+            return Collections.emptyList();
        }

        var markedContentByYPosition = markedContents.stream()
@@ -37,19 +38,19 @@ public class MarkedContentUtils {
                .collect(Collectors.groupingBy(TextPosition::getY));

        if (markedContentByYPosition.isEmpty()) {
-            return null;
+            return Collections.emptyList();
        }

-        return RectangleTransformations.rectangle2DBBox(markedContentByYPosition.values().stream()
+        return markedContentByYPosition.values().stream()
                .map(textPositions -> new TextPositionSequence(textPositions.stream()
-                        .toList(), 0, true)
+                        .toList(), 0, true, 0)
                        .getRectangle())
-                .map(t -> new Rectangle2D.Float(t.getTopLeft().getX(), t.getTopLeft().getY() - Math.abs(t.getHeight()), t.getWidth(), Math.abs(t.getHeight()))).collect(Collectors.toList()));
+                .map(t -> new Rectangle2D.Float(t.getTopLeft().getX(), t.getTopLeft().getY() - Math.abs(t.getHeight()), t.getWidth(), Math.abs(t.getHeight()))).collect(Collectors.toList());
    }


-    public boolean intersects(TextPageBlock textBlock, Map<String, Rectangle2D> markedContentBboxPerType, String type) {
-        return markedContentBboxPerType.get(type) != null && markedContentBboxPerType.get(type).intersects(textBlock.getPdfMinX(), textBlock.getPdfMinY(), textBlock.getWidth(), textBlock.getHeight());
+    public boolean intersects(TextPageBlock textBlock, Map<String, List<Rectangle2D>> markedContentBboxPerType, String type) {
+        return markedContentBboxPerType.get(type) != null && markedContentBboxPerType.get(type).stream().anyMatch(rectangle -> rectangle.intersects(textBlock.getPdfMinX(), textBlock.getPdfMinY(), textBlock.getWidth(), textBlock.getHeight()));
    }

 }
@@ -12,6 +12,7 @@ import org.apache.pdfbox.pdmodel.PDDocument;
 import org.apache.pdfbox.pdmodel.PDPage;
 import org.apache.pdfbox.pdmodel.PDPageContentStream;
 import org.apache.pdfbox.pdmodel.font.PDType1Font;
+import org.apache.pdfbox.pdmodel.font.Standard14Fonts;

 import com.knecon.fforesight.service.layoutparser.internal.api.data.redaction.NodeType;
 import com.knecon.fforesight.service.layoutparser.processor.model.graph.DocumentTree;
@@ -71,7 +72,7 @@ public class PdfVisualisationUtility {

        contentStream.beginText();
        contentStream.newLineAtOffset((float) location.getX(), (float) location.getY());
-        contentStream.setFont(PDType1Font.HELVETICA, 10);
+        contentStream.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 10);
        contentStream.showText(string);
        contentStream.endText();
        contentStream.close();
@@ -19,9 +19,10 @@ public final class PositionUtils {

        double threshold = textBlock.getMostPopularWordHeight() * 3;

-        if (textBlock.getPdfMinX() + threshold > btf.getTopLeft().getX() && textBlock.getPdfMaxX() - threshold < btf.getTopLeft()
-                .getX() + btf.getWidth() && textBlock.getPdfMinY() + threshold > btf.getTopLeft().getY() && textBlock.getPdfMaxY() - threshold < btf.getTopLeft()
-                .getY() + btf.getHeight()) {
+        if (textBlock.getPdfMinX() + threshold > btf.getTopLeft().getX()
+                && textBlock.getPdfMaxX() - threshold < btf.getTopLeft().getX() + btf.getWidth()
+                && textBlock.getPdfMinY() + threshold > btf.getTopLeft().getY()
+                && textBlock.getPdfMaxY() - threshold < btf.getTopLeft().getY() + btf.getHeight()) {
            return true;
        } else {
            return false;
@@ -30,8 +30,6 @@ public class TableMergingUtility {
            if (consecutiveTable.getColCount() == originalTablePageBlock.getColCount() && !hasTableHeader(consecutiveTable) && outerBoundaryAlignsX(originalTablePageBlock,
                    consecutiveTable)) {
                consecutiveTablesWithSameColCountAndHeaders.add(consecutiveTable);
-            } else {
-                break;
            }
        }
        return Stream.concat(Stream.of(originalTablePageBlock), consecutiveTablesWithSameColCountAndHeaders.stream()).toList();
@@ -3,7 +3,6 @@ package com.knecon.fforesight.service.layoutparser.processor.utils;
 import java.util.List;
 import java.util.stream.Collectors;

-import org.apache.pdfbox.util.QuickSort;

 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPageBlock;
 import com.knecon.fforesight.service.layoutparser.processor.model.text.TextPositionSequence;
@@ -3,40 +3,53 @@ import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
 plugins {
    application
    id("com.knecon.fforesight.java-conventions")
-    id("org.springframework.boot") version "3.1.2"
+    id("org.springframework.boot") version "3.1.5"
    id("io.spring.dependency-management") version "1.1.3"
    id("org.sonarqube") version "4.3.0.3225"
-    id("io.freefair.lombok") version "8.2.2"
+    id("io.freefair.lombok") version "8.4"
 //    id("org.graalvm.buildtools.native") version "0.9.23"
 }

 description = "layoutparser-service-server"

+configurations {
+    all {
+        exclude(group = "commons-logging", module = "commons-logging")
+        exclude(group = "org.springframework.boot", module = "spring-boot-starter-log4j2")
+        exclude(group = "com.iqser.red.commons", module = "logging-commons")
+    }
+}
+
+
 val jacksonVersion = "2.15.2"
-val pdfBoxVersion = "3.0.0-RC1"
+val pdfBoxVersion = "3.0.0"
+val springBootStarterVersion = "3.1.5"

 dependencies {
    implementation(project(":layoutparser-service-processor"))
    implementation(project(":layoutparser-service-internal-api"))

-    implementation("com.iqser.red.commons:storage-commons:2.36.0")
-    implementation("com.knecon.fforesight:tenant-commons:0.10.0")
-    implementation("org.springframework.boot:spring-boot-starter-actuator:3.1.2")
-    implementation("com.amazonaws:aws-java-sdk-s3:1.12.533")
-    implementation("org.springframework.boot:spring-boot-starter-amqp:3.1.2")
+    implementation("com.iqser.red.commons:storage-commons:2.45.0")
+    implementation("com.knecon.fforesight:tenant-commons:0.21.0")
+    implementation("com.knecon.fforesight:tracing-commons:0.5.0")
+    implementation("org.springframework.boot:spring-boot-starter-actuator:${springBootStarterVersion}")
+    implementation("org.springframework.boot:spring-boot-starter-amqp:${springBootStarterVersion}")
+    implementation("com.amazonaws:aws-java-sdk-s3:1.12.536")
    implementation("org.springframework.cloud:spring-cloud-starter-openfeign:4.0.4")
+    implementation("net.logstash.logback:logstash-logback-encoder:7.4")

    // for integration testing only
+    testImplementation(project(":viewer-doc-processor"))
    testImplementation(project(":layoutparser-service-internal-api"))

-    testImplementation("org.springframework.boot:spring-boot-starter-amqp:3.1.2")
-    testImplementation("com.iqser.red.service:persistence-service-shared-api-v1:2.138.0")
+    testImplementation("org.springframework.boot:spring-boot-starter-amqp:${springBootStarterVersion}")
+    testImplementation("org.springframework.boot:spring-boot-starter-test:${springBootStarterVersion}")
+    testImplementation("com.iqser.red.service:persistence-service-shared-api-v1:2.144.0")
    testImplementation("com.iqser.red.commons:jackson-commons:1.3.0")
    testImplementation("com.fasterxml.jackson.module:jackson-module-afterburner:${jacksonVersion}")
    testImplementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${jacksonVersion}")
    testImplementation("org.apache.pdfbox:pdfbox:${pdfBoxVersion}")
    testImplementation("org.apache.pdfbox:pdfbox-tools:${pdfBoxVersion}")
-    testImplementation("org.springframework.boot:spring-boot-starter-test:3.1.2")
 }

 /*
@@ -5,6 +5,7 @@ import org.springframework.boot.actuate.autoconfigure.security.servlet.Managemen
 import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
 import org.springframework.boot.autoconfigure.SpringBootApplication;
 import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;
+import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Import;

 import com.amazonaws.services.s3.model.metrics.MetricsConfiguration;
@@ -13,8 +14,11 @@ import com.knecon.fforesight.service.layoutparser.processor.LayoutParsingService
 import com.knecon.fforesight.service.layoutparser.server.queue.MessagingConfiguration;
 import com.knecon.fforesight.tenantcommons.MultiTenancyAutoConfiguration;

+import io.micrometer.observation.ObservationRegistry;
+import io.micrometer.observation.aop.ObservedAspect;
+
@ImportAutoConfiguration({MultiTenancyAutoConfiguration.class})
-@Import({MetricsConfiguration.class, StorageAutoConfiguration.class,  LayoutParsingServiceProcessorConfiguration.class, MessagingConfiguration.class})
+@Import({MetricsConfiguration.class, StorageAutoConfiguration.class, LayoutParsingServiceProcessorConfiguration.class, MessagingConfiguration.class})
@SpringBootApplication(exclude = {SecurityAutoConfiguration.class, ManagementWebSecurityAutoConfiguration.class})
 public class Application {

@@ -23,4 +27,11 @@ public class Application {
        SpringApplication.run(Application.class, args);
    }

+
+    @Bean
+    public ObservedAspect observedAspect(ObservationRegistry observationRegistry) {
+
+        return new ObservedAspect(observationRegistry);
+    }
+
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
maverickstuder	bb5b631950	RED-8666	2024-03-04 16:31:54 +01:00
maverickstuder	2567d89fbb	RED-8666	2024-03-04 09:23:30 +01:00
maverickstuder	aef1146e8f	RED-8666	2024-02-29 18:30:22 +01:00
maverickstuder	7f56ed15c8	RED-8666	2024-02-29 13:01:36 +01:00
maverickstuder	91401361e9	RED-8666	2024-02-28 17:49:23 +01:00
maverickstuder	2ab60195e4	RED-8666	2024-02-28 15:28:13 +01:00
Dominique Eifländer	32c877e8f7	RED-7141: Fixed problem with different text directions	2024-02-26 13:08:52 +01:00
Dominique Eifländer	385d4b399e	RED-7141: Improved basic block combination logic	2024-02-23 10:01:28 +01:00
Dominique Eifländer	d0e1af3a44	RED-7141: Improved basic block combination logic	2024-02-22 16:57:24 +01:00
Dominique Eifländer	d06933ed17	RED-7141: Added basic block combination logic	2024-02-22 15:50:41 +01:00
Dominique Eifländer	240ef82def	RED-7141: Implemented docstrum layout parsing	2024-02-22 13:04:08 +01:00
Maverick Studer	f4b6386e1c	Merge branch 'RED-8550-testing' into 'main' RED-8550: Faulty table recognition and text duplication leads to huge sections See merge request fforesight/layout-parser!102	2024-02-21 13:54:30 +01:00
Maverick Studer	1d64028158	RED-8550: Faulty table recognition and text duplication leads to huge sections	2024-02-21 13:54:30 +01:00
Yannik Hampe	0979a267d4	Merge branch 'RED-8481' into 'main' RED-8481: Use visual layout parsing to detect signatures See merge request fforesight/layout-parser!101	2024-02-15 13:07:06 +01:00
yhampe	cc77d19500	RED-8481: Use visual layout parsing to detect signatures addressed review comments	2024-02-15 13:01:30 +01:00
yhampe	fa048b2fe0	RED-8481: Use visual layout parsing to detect signatures addressed review comments	2024-02-15 12:19:26 +01:00
yhampe	bdf1161c91	RED-8481: Use visual layout parsing to detect signatures addressed review comments	2024-02-15 12:12:23 +01:00
yhampe	b4a225144d	RED-8481: Use visual layout parsing to detect signatures working on failing tests	2024-02-15 10:16:07 +01:00
yhampe	903b1c1fd4	RED-8481: Use visual layout parsing to detect signatures fixed failing tests because of null pointer	2024-02-15 09:27:07 +01:00
yhampe	c3e7582ee3	RED-8481: Use visual layout parsing to detect signatures fixed failing tests because of null pointer	2024-02-14 12:33:36 +01:00
yhampe	cfc5db45cd	RED-8481: Use visual layout parsing to detect signatures fixed failing tests because of null pointer	2024-02-14 12:24:32 +01:00
yhampe	fbd0196719	RED-8481: Use visual layout parsing to detect signatures implemented visuallayoutparsingresult	2024-02-14 12:16:37 +01:00
Kilian Schüttler	3c9049dc8a	Merge branch 'RED-8156' into 'main' RED-8156: refactor ViewerDocumentService as a dependency for ocr-service See merge request fforesight/layout-parser!99	2024-02-07 10:42:21 +01:00
Kilian Schuettler	015984891f	RED-8156: refactor ViewerDocumentService as a dependency for ocr-service * fix pmd	2024-02-06 17:17:26 +01:00
Kilian Schuettler	66fcb62833	RED-8156: refactor ViewerDocumentService as a dependency for ocr-service * fix pmd	2024-02-06 17:09:21 +01:00
Kilian Schuettler	48824f56a8	RED-8156: refactor ViewerDocumentService as a dependency for ocr-service * fix pmd	2024-02-06 17:06:53 +01:00
Kilian Schuettler	785628537f	RED-8156: refactor ViewerDocumentService as a dependency for ocr-service * various improvements to experimental parsing steps * added embed fonts functionality to viewer doc * fix checkstyle	2024-02-06 17:03:38 +01:00
Kilian Schuettler	23eb0c40a3	RED-8156: refactor ViewerDocumentService as a dependency for ocr-service * various improvements to experimental parsing steps * added embed fonts functionality to viewer doc	2024-02-06 16:59:51 +01:00
Dominique Eifländer	1b4aaf4454	Merge branch 'RED-8171' into 'main' RED-8171: Traces do not stop at @Async See merge request fforesight/layout-parser!98	2024-02-02 13:34:44 +01:00
Dominique Eifländer	e4f3557b36	RED-8171: Traces do not stop at @Async	2024-02-02 13:22:57 +01:00
Timo Bejan	9be3c86297	Merge branch 'RED-8085' into 'main' Red 8085 See merge request fforesight/layout-parser!96	2024-01-29 10:31:36 +01:00
Timo Bejan	88855de2da	Red 8085	2024-01-29 10:31:36 +01:00
Dominique Eifländer	368a75e985	Merge branch 'RED-8106' into 'main' RED-8106: Make documentdata serializable See merge request fforesight/layout-parser!95	2023-12-22 13:33:02 +01:00
Dominique Eifländer	12344d57b2	RED-8106: Make documentdata serializable	2023-12-21 13:42:25 +01:00
Dominique Eifländer	9e854379e7	Merge branch 'RED-1137' into 'main' RED-1137: Do not observe actuator endpoints See merge request fforesight/layout-parser!92	2023-12-20 14:11:31 +01:00
Dominique Eifländer	b779c72041	RED-1137: Do not observe actuator endpoints	2023-12-20 14:05:00 +01:00
Dominique Eifländer	760a809900	Merge branch 'RED-7384' into 'main' RED-7384: fixes for migration See merge request fforesight/layout-parser!91	2023-12-20 12:40:00 +01:00
Kilian Schüttler	ba1c7c07ab	RED-7384: fixes for migration	2023-12-20 12:40:00 +01:00
Dominique Eifländer	ca0cbbcb49	Merge branch 'RED-5223' into 'main' RED-5223: Use tracing-commons from fforesight See merge request fforesight/layout-parser!90	2023-12-13 16:05:01 +01:00
Dominique Eifländer	da2cdc288e	RED-5223: Use tracing-commons from fforesight	2023-12-13 15:31:26 +01:00
Dominique Eifländer	68da328889	Merge branch 'queueHotfix' into 'main' hotfix: removed dlq from response queue to be equal to persistence-service See merge request fforesight/layout-parser!89	2023-12-13 09:54:43 +01:00
Dominique Eifländer	711548d1a7	hotfix: removed dlq from response queue to be equal to persistence-service	2023-12-13 09:47:27 +01:00
Dominique Eifländer	2bddcdafee	Merge branch 'RED-5223' into 'main' RED-5223: Enabled tracing, upgrade spring, use logstash-logback-encoder for json logs See merge request fforesight/layout-parser!88	2023-12-11 15:13:22 +01:00
Dominique Eifländer	750ccf4ce2	RED-5223: Enabled tracing, upgrade spring, use logstash-logback-encoder for json logs	2023-12-11 15:06:23 +01:00
Ali Oezyetimoglu	57b5d3f48e	Merge branch 'RED-7714' into 'main' RED-7715 - Add log4j config to enable switching between json/line logs See merge request fforesight/layout-parser!87	2023-12-06 13:00:28 +01:00
Andrei Isvoran	d8c9659469	RED-7715 - Add log4j config to enable switching between json/line logs	2023-12-06 11:59:42 +02:00
Kilian Schüttler	30f060e36c	Merge branch 'enable-caching' into 'main' enable caching in build See merge request fforesight/layout-parser!84	2023-11-24 10:36:55 +01:00
Kilian Schuettler	53a5824e6c	enable caching in build	2023-11-24 10:24:50 +01:00
Dominique Eifländer	e2bcf971c9	Merge branch 'DM-589' into 'main' DM-589: Filter wrong detected cells that borders from rotation at scanning See merge request fforesight/layout-parser!83	2023-11-20 16:07:27 +01:00
Dominique Eifländer	dacc2f7f43	DM-589: Filter wrong detected cells that borders from rotation at scanning	2023-11-20 15:54:02 +01:00
Dominique Eifländer	144a9591a2	Merge branch 'TAAS-103-hotfix' into 'main' * added back in if statement See merge request fforesight/layout-parser!82	2023-11-16 12:48:48 +01:00
yhampe	207d9dec97	* added back in if statement * removed not needed commentar	2023-11-16 12:40:49 +01:00
Yannik Hampe	09ee90222e	Merge branch 'TAAS-103' into 'main' TAAS-103: Table Detection and rotated text See merge request fforesight/layout-parser!81	2023-11-16 09:13:41 +01:00
yhampe	1316a067fe	* removed double chechking for height of cell	2023-11-16 08:51:12 +01:00
yhampe	e203210ade	* removed not needed properties	2023-11-16 08:23:58 +01:00
yhampe	b25d46291a	* checkstyle	2023-11-16 08:12:47 +01:00
yhampe	84148d3b6e	* fixed tests	2023-11-16 07:51:08 +01:00
Dominique Eifländer	a6ba66b1aa	TAAS-103: Fixed values in wrong cells	2023-11-15 13:36:46 +01:00
yhampe	c3e69b2cdf	* fixed bug with incorrect empty cell count by adding threshhold to cell.contains	2023-11-15 10:44:47 +01:00
yhampe	f69331e7d8	*renamed page to firstPage in DocumentStructure and Table	2023-11-07 10:21:19 +01:00
yhampe	01493dc033	TAAS-103: Table Detection and rotated text * added page property to DocumentStructure to be able to get page of found tables * added a method to TableExtractionService to get the table area * added calculateMinCharWidthAndMaxCharHeightInsideTable to LayoutParsingPipeline to calculate the values based upon table area * refactored PDFLinesTextStripper for better readability *removed textMatrix from RedTextPosition as it is no longer needed	2023-11-07 08:47:28 +01:00
yhampe	459e0c8be7	TAAS-103:	2023-11-07 08:39:15 +01:00
Kilian Schüttler	1b1f777706	Merge branch 'RED-7806' into 'main' RED-7806 - Specific customer document cannot be processed See merge request fforesight/layout-parser!79	2023-10-25 10:50:34 +02:00
Corina Olariu	0e0a811f9d	RED-7806 - Specific customer document cannot be processed - add brackets	2023-10-25 11:36:54 +03:00
Corina Olariu	efa3d75479	RED-7806 - Specific customer document cannot be processed - check for font name null before using to avoid the NPE	2023-10-25 09:16:47 +03:00
Kilian Schüttler	9abdc6d44d	Merge branch 'RED-7434' into 'main' RED-7434 - Remove Section Grid entirely See merge request fforesight/layout-parser!78	2023-10-20 10:07:01 +02:00
Corina Olariu	3bab61c446	RED-7434 - Remove Section Grid entirely - remove sectionGrid relation (including SectionGridCreatorService) - update junit tests	2023-10-20 09:09:22 +03:00
Dominique Eifländer	d17517d3c3	Merge branch 'hotfix-bdr-doc' into 'main' hotfix: Fixed parsing for specific taas document See merge request fforesight/layout-parser!77	2023-10-18 16:12:15 +02:00
Dominique Eifländer	567cbc178b	hotfix: Fixed parsing for specific taas document	2023-10-17 15:52:19 +02:00
Kilian Schüttler	3c53772765	Merge branch 'RED-7759' into 'main' RED-7759: Upgraded storage-commons to newest windwos compatible version See merge request fforesight/layout-parser!76	2023-10-13 12:21:04 +02:00
Dominique Eifländer	8647cf5a18	RED-7759: Upgraded storage-commons to newest windwos compatible version	2023-10-13 12:15:22 +02:00
Kilian Schüttler	310c07b200	Merge branch 'RED-7607-WIP' into 'main' RED-7607 - Rotating pages leads to lost annotations (RM & DM) See merge request fforesight/layout-parser!75	2023-10-05 13:34:12 +02:00
Corina Olariu	daba0bf8a6	RED-7607 - Rotating pages leads to lost annotations (RM & DM) - remove finally clause	2023-10-04 17:46:46 +03:00
Corina Olariu	3839de215c	RED-7607 - Rotating pages leads to lost annotations (RM & DM) - rollback to getDir().getDegrees()	2023-10-04 15:27:13 +03:00
Corina Olariu	b4d68594f1	RED-7607 - Rotating pages leads to lost annotations (RM & DM) - use rotation instead of getDir().getDegrees()	2023-10-04 14:22:15 +03:00
Corina Olariu	99ed331a1e	RED-7607 - Rotating pages leads to lost annotations (RM & DM) - use getXDirAdj instead of getX - add fontSizeCounter for landscape pages also	2023-10-04 14:13:38 +03:00
Corina Olariu	f2c0991987	RED-7607 - Rotating pages leads to lost annotations (RM & DM) - fix PMD findings	2023-10-04 14:09:46 +03:00
Timo Bejan	b8ef55e6e2	Merge branch 'TAAS-104' into 'main' TAAS-104: merge visually intersecting Paragraphs See merge request fforesight/layout-parser!73	2023-09-05 17:08:40 +02:00
Kilian Schuettler	5792ff4a93	TAAS-104: merge visually intersecting Paragraphs * fix build	2023-09-05 16:54:23 +02:00
Kilian Schuettler	621c3f269d	TAAS-104: merge visually intersecting Paragraphs	2023-09-05 16:09:05 +02:00
Dominique Eifländer	8dba392904	Merge branch 'RED-7461' into 'main' RED-7461: Fixed wrong textblock classifation if footer is marked as header See merge request fforesight/layout-parser!72	2023-09-01 12:14:38 +02:00
deiflaender	306a53ea79	RED-7461: Fixed wrong textblock classifation if footer is marked as header	2023-09-01 12:07:47 +02:00
Kilian Schüttler	754fd8f933	Merge branch 'TAAS-89' into 'main' TAAS-89: added log entry and an end2end test See merge request fforesight/layout-parser!71	2023-08-31 14:40:48 +02:00
Kilian Schuettler	28ec4c9ccb	TAAS-89: added log entry and an end2end test	2023-08-31 14:28:18 +02:00
Kilian Schüttler	aed4a55787	Merge branch 'TAAS-89' into 'main' TAAS-89: fixed weird bug with empty sections See merge request fforesight/layout-parser!70	2023-08-31 12:00:18 +02:00
Kilian Schuettler	f87e2d75b5	TAAS-89: fixed weird bug with empty sections	2023-08-31 11:41:22 +02:00
Kilian Schüttler	de6760abc1	Merge branch 'TAAS-89' into 'main' TAAS-89: added some more documentation See merge request fforesight/layout-parser!69	2023-08-31 10:55:45 +02:00
Kilian Schuettler	261ef4c367	TAAS-89: added some more documentation * fixed weird bug with empty sections	2023-08-31 10:49:32 +02:00
Timo Bejan	11ba9c6bb9	Merge branch 'TAAS-89' into 'main' Added some documentation See merge request fforesight/layout-parser!64	2023-08-25 16:34:18 +02:00
Renovate Bot	b7c3d02978	Merge branch 'renovate/main-spring-boot' into 'main' Update spring boot to v3.1.3 (main) See merge request fforesight/layout-parser!63	2023-08-24 21:15:35 +02:00
Kilian Schuettler	bcf0bcbaf4	Added some documentation	2023-08-24 18:37:47 +02:00
Renovate Bot	84cde2a3db	Update spring boot to v3.1.3	2023-08-24 13:16:58 +00:00
Renovate Bot	6f2dd4f823	Merge branch 'renovate/main-com.iqser.red.commons-storage-commons-2.x' into 'main' Update dependency com.iqser.red.commons:storage-commons to v2.40.0 (main) See merge request fforesight/layout-parser!61	2023-08-24 09:17:14 +02:00
Renovate Bot	a909724217	Update dependency com.iqser.red.commons:storage-commons to v2.40.0	2023-08-24 04:16:37 +00:00
Renovate Bot	67a981e7a8	Merge branch 'renovate/main-aws-java-sdk-monorepo' into 'main' Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.536 (main) See merge request fforesight/layout-parser!60	2023-08-24 03:15:54 +02:00
Renovate Bot	0e93fdd515	Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.536	2023-08-23 22:17:08 +00:00
Renovate Bot	d464239f9b	Merge branch 'renovate/main-com.iqser.red.service-persistence-service-shared-api-v1-2.x' into 'main' Update dependency com.iqser.red.service:persistence-service-shared-api-v1 to v2.144.0 (main) See merge request fforesight/layout-parser!59	2023-08-24 00:16:53 +02:00
Renovate Bot	88a20924b9	Update dependency com.iqser.red.service:persistence-service-shared-api-v1 to v2.144.0	2023-08-23 19:14:51 +00:00
Renovate Bot	f89243472c	Merge branch 'renovate/main-com.iqser.red.commons-storage-commons-2.x' into 'main' Update dependency com.iqser.red.commons:storage-commons to v2.39.0 (main) See merge request fforesight/layout-parser!58	2023-08-23 09:17:15 +02:00
Renovate Bot	ad3612acd4	Update dependency com.iqser.red.commons:storage-commons to v2.39.0	2023-08-23 04:17:21 +00:00
Renovate Bot	630eee6bd7	Merge branch 'renovate/main-aws-java-sdk-monorepo' into 'main' Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.535 (main) See merge request fforesight/layout-parser!57	2023-08-23 03:13:39 +02:00
Renovate Bot	a951911ec8	Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.535	2023-08-22 22:16:17 +00:00
Renovate Bot	75e6b88705	Merge branch 'renovate/main-com.iqser.red.service-persistence-service-shared-api-v1-2.x' into 'main' Update dependency com.iqser.red.service:persistence-service-shared-api-v1 to v2.140.0 (main) See merge request fforesight/layout-parser!56	2023-08-22 21:18:37 +02:00
Renovate Bot	2e0adbdd9a	Update dependency com.iqser.red.service:persistence-service-shared-api-v1 to v2.140.0	2023-08-22 16:17:49 +00:00
Renovate Bot	b747742558	Merge branch 'renovate/main-com.iqser.red.commons-storage-commons-2.x' into 'main' Update dependency com.iqser.red.commons:storage-commons to v2.38.0 (main) See merge request fforesight/layout-parser!55	2023-08-22 15:17:43 +02:00
Renovate Bot	192c9976c1	Update dependency com.iqser.red.commons:storage-commons to v2.38.0	2023-08-22 10:17:05 +00:00
Dominique Eifländer	b251697492	Merge branch 'PDFBox-update' into 'main' upgrade PDFBox to 3.0.0 See merge request fforesight/layout-parser!52	2023-08-22 09:39:53 +02:00
Renovate Bot	22d6b25fe4	Merge branch 'renovate/main-aws-java-sdk-monorepo' into 'main' Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.534 (main) See merge request fforesight/layout-parser!53	2023-08-22 09:17:26 +02:00
Renovate Bot	e6bcd6fb2b	Update dependency com.amazonaws:aws-java-sdk-s3 to v1.12.534	2023-08-22 04:17:20 +00:00
Renovate Bot	2847adde22	Merge branch 'renovate/main-com.iqser.red.commons-storage-commons-2.x' into 'main' Update dependency com.iqser.red.commons:storage-commons to v2.37.0 (main) See merge request fforesight/layout-parser!54	2023-08-22 06:16:54 +02:00
Renovate Bot	7cf67d7121	Update dependency com.iqser.red.commons:storage-commons to v2.37.0	2023-08-22 01:17:23 +00:00
Kilian Schuettler	3a18923ef5	upgrade PDFBox to 3.0.0 * disable experimental ruling header stuff	2023-08-21 17:54:20 +02:00
Kilian Schuettler	2b15fd1d3c	RED-7461: improve header/footer recognition	2023-08-21 17:49:13 +02:00
Dominique Eifländer	3722fff476	Merge branch 'RED-7461' into 'main' Red 7461 See merge request fforesight/layout-parser!51	2023-08-21 17:08:10 +02:00