Marmelator/layout-parser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 289 Wiki Activity
365 Commits 48 Branches 295 Tags
f0a70a524202fb8375af03139033a2ef870eab52
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kilian Schuettler f0a70a5242 RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 11:09:23 +02:00
Kilian Schuettler 15ea385f4d RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 10:44:32 +02:00
Kilian Schuettler 08be18db2d RED-8825: general improvements 
* some more refactoring
 2024-04-29 20:09:53 +02:00
Kilian Schuettler 64209255cb RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:24:15 +02:00
Kilian Schuettler 4761d2e1a2 RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:22:33 +02:00
Kilian Schuettler 1916e626df RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:15:19 +02:00
Kilian Schuettler e4663ac8db RED-8825: added split by ruling into every step of docstrum 2024-04-29 15:54:56 +02:00
Kilian Schuettler 6a691183dc RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:56 +02:00
Kilian Schuettler 3dd215288a RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:53 +02:00
Kilian Schüttler 6fb1a0bef3 Merge branch 'RED-8992' into 'main' 
RED-8992 - Enable to add annotation on header with line breaks

See merge request fforesight/layout-parser!143
 2024-04-25 13:03:40 +02:00
Kilian Schüttler d38d023485 Merge branch 'RED-7384' into 'main' 
Red 7384

See merge request fforesight/layout-parser!140
 2024-04-23 12:13:21 +02:00
Kilian Schüttler c1afe9b11f Red 7384 2024-04-23 12:13:19 +02:00
Kilian Schuettler f256f9b30f RED-8995: unclassified text might be missing from document data 
* treat TablePageBlock.OTHER like PARAGRAPH (no special treatment)
 2024-04-18 17:42:34 +02:00
Kilian Schüttler c4d9c5df02 Merge branch 'RED-8747-fp' into 'main' 
RED-8747 - Entities not merged properly - fp

See merge request fforesight/layout-parser!131
 2024-04-09 16:30:02 +02:00
Kilian Schüttler bf6a0d770b Merge branch 'RED-8799' into 'main' 
RED-8799: LayoutGrid is wrong draw for some tables

See merge request fforesight/layout-parser!126
 2024-04-04 15:23:12 +02:00
Kilian Schuettler f18bda1d4e RED-8799: LayoutGrid is wrong draw for some tables 2024-04-04 13:33:22 +02:00
Kilian Schüttler c324d3815e Merge branch 'RED-8550-pre-backup' into 'main' 
RED-8550: Faulty table recognition and text duplication leads to huge sections

See merge request fforesight/layout-parser!105
 2024-02-28 16:13:56 +01:00
Kilian Schüttler e7bf607663 Merge branch 'hotfix' into 'main' 
hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written

See merge request fforesight/layout-parser!104
 2024-02-26 12:32:14 +01:00
Kilian Schuettler f4d789311c hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written 2024-02-26 12:24:15 +01:00
Kilian Schüttler 3c9049dc8a Merge branch 'RED-8156' into 'main' 
RED-8156: refactor ViewerDocumentService as a dependency for ocr-service

See merge request fforesight/layout-parser!99
 2024-02-07 10:42:21 +01:00
Kilian Schuettler 015984891f RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:17:26 +01:00
Kilian Schuettler 66fcb62833 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:09:21 +01:00
Kilian Schuettler 48824f56a8 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:06:53 +01:00
Kilian Schuettler 785628537f RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* various improvements to experimental parsing steps
* added embed fonts functionality to viewer doc
* fix checkstyle
 2024-02-06 17:03:38 +01:00
Kilian Schuettler 23eb0c40a3 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* various improvements to experimental parsing steps
* added embed fonts functionality to viewer doc
 2024-02-06 16:59:51 +01:00
Kilian Schüttler ba1c7c07ab RED-7384: fixes for migration 2023-12-20 12:40:00 +01:00
Kilian Schüttler 30f060e36c Merge branch 'enable-caching' into 'main' 
enable caching in build

See merge request fforesight/layout-parser!84
 2023-11-24 10:36:55 +01:00
Kilian Schuettler 53a5824e6c enable caching in build 2023-11-24 10:24:50 +01:00
Kilian Schüttler 1b1f777706 Merge branch 'RED-7806' into 'main' 
RED-7806 - Specific customer document cannot be processed

See merge request fforesight/layout-parser!79
 2023-10-25 10:50:34 +02:00
Kilian Schüttler 9abdc6d44d Merge branch 'RED-7434' into 'main' 
RED-7434 - Remove Section Grid entirely

See merge request fforesight/layout-parser!78
 2023-10-20 10:07:01 +02:00
Kilian Schüttler 3c53772765 Merge branch 'RED-7759' into 'main' 
RED-7759: Upgraded storage-commons to newest windwos compatible version

See merge request fforesight/layout-parser!76
 2023-10-13 12:21:04 +02:00
Kilian Schüttler 310c07b200 Merge branch 'RED-7607-WIP' into 'main' 
RED-7607 - Rotating pages leads to lost annotations (RM & DM)

See merge request fforesight/layout-parser!75
 2023-10-05 13:34:12 +02:00
Kilian Schuettler 5792ff4a93 TAAS-104: merge visually intersecting Paragraphs 
* fix build
 2023-09-05 16:54:23 +02:00
Kilian Schuettler 621c3f269d TAAS-104: merge visually intersecting Paragraphs 2023-09-05 16:09:05 +02:00
Kilian Schüttler 754fd8f933 Merge branch 'TAAS-89' into 'main' 
TAAS-89: added log entry and an end2end test

See merge request fforesight/layout-parser!71
 2023-08-31 14:40:48 +02:00
Kilian Schuettler 28ec4c9ccb TAAS-89: added log entry and an end2end test 2023-08-31 14:28:18 +02:00
Kilian Schüttler aed4a55787 Merge branch 'TAAS-89' into 'main' 
TAAS-89: fixed weird bug with empty sections

See merge request fforesight/layout-parser!70
 2023-08-31 12:00:18 +02:00
Kilian Schuettler f87e2d75b5 TAAS-89: fixed weird bug with empty sections 2023-08-31 11:41:22 +02:00
Kilian Schüttler de6760abc1 Merge branch 'TAAS-89' into 'main' 
TAAS-89: added some more documentation

See merge request fforesight/layout-parser!69
 2023-08-31 10:55:45 +02:00
Kilian Schuettler 261ef4c367 TAAS-89: added some more documentation 
* fixed weird bug with empty sections
 2023-08-31 10:49:32 +02:00
Kilian Schuettler bcf0bcbaf4 Added some documentation 2023-08-24 18:37:47 +02:00
Kilian Schuettler 3a18923ef5 upgrade PDFBox to 3.0.0 
* disable experimental ruling header stuff
 2023-08-21 17:54:20 +02:00
Kilian Schuettler 2b15fd1d3c RED-7461: improve header/footer recognition 2023-08-21 17:49:13 +02:00
Kilian Schüttler 457f7d9c66 Merge branch 'RED-7158' into 'main' 
RED-7158: fix for all page rotations

See merge request fforesight/layout-parser!35
 2023-08-15 15:07:04 +02:00
Kilian Schuettler 0387cdd143 RED-7158: fix for all page rotations 
* also make lines thinner
 2023-08-15 14:55:41 +02:00
Kilian Schüttler 5c6898b975 Merge branch 'RED-7158' into 'main' 
RED-7158: add layoutgrid into new ViewerDocument as optional content

See merge request fforesight/layout-parser!34
 2023-08-15 13:22:06 +02:00
Kilian Schuettler b7b273b47d RED-7158: layout grid 
* downgraded storage-commons to working version
 2023-08-15 13:15:44 +02:00
Kilian Schuettler 9aa9cb2d54 RED-7158: add layoutgrid into new ViewerDocument as optional content 
* set layer to invisible by default
 2023-08-15 13:14:16 +02:00
Kilian Schüttler d854125867 Merge branch 'RED-7158' into 'main' 
RED-7158: add layoutgrid into new ViewerDocument as optional content

See merge request fforesight/layout-parser!31
 2023-08-14 16:14:23 +02:00
Kilian Schuettler 63de8ef82d RED-7158: add layoutgrid into new ViewerDocument as optional content 
* downgraded storage-commons
 2023-08-14 16:07:11 +02:00
Kilian Schuettler ea0af08c31 RED-7851: add layoutgrid to new viewer document as optional content 2023-08-14 16:06:23 +02:00
Kilian Schuettler 0be6454a7e add script for pushing custom images 2023-08-10 15:47:52 +02:00
Kilian Schüttler 442c1dafea Merge branch 'update-pdfbox' into 'main' 
update PDFBox Version

See merge request fforesight/layout-parser!19
 2023-08-09 12:48:14 +02:00
Kilian Schüttler 33bc532eac Merge branch 'renovate/main-com.iqser.red.service-persistence-service-shared-api-v1-2.x' into 'main' 
Update dependency com.iqser.red.service:persistence-service-shared-api-v1 to v2.124.0 (main)

See merge request fforesight/layout-parser!18
 2023-08-09 12:42:38 +02:00
Kilian Schuettler 4bd6e7e343 update PDFBox Version 2023-08-09 12:41:28 +02:00
Kilian Schuettler 17259ed805 add renovate, fix checkstyle 2023-08-09 10:11:02 +02:00
Kilian Schuettler 4a5464d6aa Refactoring to make downstream refactoring easier 2023-08-04 15:16:36 +02:00
Kilian Schuettler 676f0c9d09 cleanup dependency versions 2023-08-01 10:27:14 +02:00
Kilian Schuettler ded00df11e fix build 2023-08-01 09:57:58 +02:00
Kilian Schuettler 286556cbb6 mark non nullable fields in request 2023-08-01 00:50:33 +02:00
Kilian Schuettler d6a74dc9f9 add field id to image data 2023-07-31 16:32:11 +02:00
Kilian Schuettler 2a55654fcf add simplifiedText 2023-07-31 15:30:03 +02:00
Kilian Schüttler 7496914b37 Merge branch 'RED-6725' into 'main' 
include openfeign for tenant-commons

See merge request fforesight/layout-parser!6
 2023-07-31 15:02:33 +02:00
Kilian Schuettler c8ace585e1 include openfeign for tenant-commons 2023-07-31 14:40:57 +02:00
Kilian Schüttler 69c5f80c8c Merge branch 'RED-6725' into 'main' 
Red 6725

See merge request fforesight/layout-parser!5
 2023-07-31 13:19:39 +02:00
Kilian Schuettler 79d27189fd move Properties names to DocumentStructure 2023-07-31 12:54:49 +02:00
Kilian Schuettler 75bac72c05 fix dsljson 2023-07-29 02:44:22 +02:00
Kilian Schuettler c5e6271dc3 fix fileIds 2023-07-29 01:59:32 +02:00
Kilian Schüttler 5561dd5e95 Merge branch 'RED-6725' into 'main' 
Red 6725

See merge request fforesight/layout-parser!4
 2023-07-28 17:46:40 +02:00
Kilian Schuettler 041b633742 use correct repo 2023-07-28 17:42:18 +02:00
Kilian Schuettler 715426bd3b remove hardcoded version 2023-07-28 17:34:59 +02:00
Kilian Schuettler 464b8053fe configure maven-publish plugin 2023-07-28 17:26:05 +02:00
Kilian Schuettler 2fece83c7c remove root build.gradle 2023-07-28 16:15:53 +02:00
Kilian Schuettler ad03ef1922 move queue to server package, so i can easily import processor as a library for redaction-service 2023-07-28 16:15:31 +02:00
Kilian Schuettler f72838b0be removed unnecessary dependency 2023-07-27 17:15:22 +02:00
Kilian Schuettler 6388898cc0 added comments for native build investigation 2023-07-27 17:12:16 +02:00
Kilian Schuettler 72d1e6271a more refactoring, added a comment 2023-07-27 14:35:40 +02:00
Kilian Schuettler 299b5be385 package refactoring in processor 2023-07-27 14:28:09 +02:00
Kilian Schüttler 2ea58f5e9f Merge branch 'fixGradleDeploy' into 'main' 
Fix gradle deploy

See merge request fforesight/layout-parser!3
 2023-07-27 13:42:10 +02:00
Kilian Schuettler 510ec7ce45 remove native build 2023-07-27 13:29:14 +02:00
Kilian Schuettler c186927e3d add reflection config 2023-07-27 13:21:01 +02:00
Kilian Schuettler 1a494b0dea add buildArgs to include logback at runtime 2023-07-27 13:00:26 +02:00
Kilian Schuettler 19552ddf69 added gradle.properties.kts 2023-07-27 12:32:10 +02:00
Kilian Schuettler 41267a0f98 ported to gradle 2023-07-27 12:27:30 +02:00
Kilian Schuettler 270129cd73 outputs almost equal current redaction-service in regards to RedactManager 
* 3/200 files have minimal whitespace/sorting errors, most likely rounding errors
 2023-07-25 18:12:57 +02:00
Kilian Schuettler a41c13fdd6 add LayoutParsingType to BodyTextFrameService 2023-07-25 17:20:52 +02:00
Kilian Schuettler 65ab5eca22 update to redaction-service state 2023-07-25 16:10:57 +02:00
Kilian Schuettler 143ebee25e move and fix layout tests from redaction-service 2023-07-24 19:43:25 +02:00
Kilian Schuettler 47fd8e05d1 rename Data classes 2023-07-24 18:36:27 +02:00
Kilian Schüttler 653f280fd1 Merge branch 'TAAS-41' into 'main' 
TAAS-41: TAAS Document Structure

See merge request fforesight/layout-parser!2
 2023-07-24 16:09:18 +02:00
Kilian Schuettler daa68f3fa6 TAAS-41: disable experimental tests 2023-07-24 16:07:27 +02:00
Kilian Schuettler ed66043856 TAAS-41: add test files 2023-07-24 16:04:51 +02:00
Kilian Schuettler 526b1c5ad3 TAAS-41: add (inactive) experimental services 2023-07-24 15:58:06 +02:00
Kilian Schuettler 241a32cb4f TAAS-41/ RED-6725: integrate layoutparser into redactmanager 2023-07-24 15:55:31 +02:00
Kilian Schuettler 15a6d46f5c RED-7081: getBBox() Performance Improvement 2023-07-13 13:01:01 +03:00
Kilian Schuettler 788613c92e TAAS-41: TAAS Document Structure 
* added linebreaks to ParagraphData
* moved List<String> cellText to List<ParagraphData> cellTexts
 2023-07-13 13:01:01 +03:00
Kilian Schuettler 7f0aa32d1b TAAS-41: TAAS Document Structure 
* added more testFiles
* hacked a workaround for CMMException
 2023-07-13 13:01:01 +03:00
Kilian Schuettler f08c4ced43 TAAS-41: TAAS Document Structure 
* changed TextPageBlock splitting
* changed Header and Footer Classification
* added TAAS Document Structure Prototype
 2023-07-13 13:01:01 +03:00
Kilian Schuettler df9cbdc036 RED-6725: Integrate new layout parser 
* ported current state from RedactManager
 2023-06-15 12:52:51 +02:00
Kilian Schuettler c912d27d92 RED-6009: add direct json output test 2023-04-19 12:22:19 +02:00