Marmelator/layout-parser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 289 Wiki Activity
506 Commits 48 Branches 295 Tags
9d2596e5ef13ef9adc0954659d0585e9e367f578
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kilian Schuettler 9d2596e5ef RED-10127: improve list classification 
* add one more format to list identification
* add 'ppb' to known units
* special case for headlines continuing with 14C after the identifier (quite often in some specific files)
 2024-10-14 17:21:44 +02:00
Kilian Schüttler e7b01161ac Merge branch 'feature/RED-10127' into 'main' 
RED-10127: add list classification

See merge request fforesight/layout-parser!237
 2024-10-10 10:50:10 +02:00
Kilian Schüttler 7b073eb4f3 RED-10127: add list classification 2024-10-10 10:50:10 +02:00
Kilian Schüttler 6c7442ac6d RED-10127: improve headline detection 2024-10-09 08:48:48 +02:00
Kilian Schüttler 469da38952 Red 9974: improce headline classification, fix font size calculation 2024-09-16 14:06:48 +02:00
Kilian Schuettler 8e165a41d7 hotfix: viewerDocService doesn't remove existing marked content 2024-09-11 16:34:21 +02:00
Kilian Schüttler ed7a701ad9 Merge branch 'RED-9975' into 'main' 
RED-9975: improve SuperSection handling

See merge request fforesight/layout-parser!223
 2024-09-11 13:38:09 +02:00
Kilian Schüttler 393103e074 RED-9975: improve SuperSection handling 2024-09-11 13:38:09 +02:00
Kilian Schüttler c726a643f0 Merge branch 'hotfix' into 'main' 
Hotfix: unmerge super large tables

See merge request fforesight/layout-parser!220
 2024-09-05 15:05:21 +02:00
Kilian Schüttler 519e95735c Hotfix: unmerge super large tables 2024-09-05 15:05:21 +02:00
Kilian Schüttler 9650195afd Merge branch 'hotfix-fp' into 'main' 
hotfix: add Java advanced imaging

See merge request fforesight/layout-parser!217
 2024-09-04 15:43:56 +02:00
Kilian Schuettler ce628a99f7 hotfix: add Java advanced imaging 2024-09-04 15:18:12 +02:00
Kilian Schüttler af45f2cd8c Merge branch 'RED-9964' into 'main' 
RED-9964: fix errors with images

See merge request fforesight/layout-parser!212
 2024-09-04 09:16:59 +02:00
Kilian Schuettler befb6b1df6 RED-9964: fix errors with images 2024-09-03 16:37:48 +02:00
Kilian Schüttler 8e14b74da2 Red 9975: fix outline detection 2024-09-02 09:02:36 +02:00
Kilian Schüttler 3b91639ea9 Merge branch 'RED-9964-fp' into 'main' 
RED-9964: don't merge tables on non-consecutive pages

See merge request fforesight/layout-parser!205
 2024-08-30 14:00:48 +02:00
Kilian Schüttler c5178ea5c2 RED-9964: don't merge tables on non-consecutive pages 2024-08-30 14:00:48 +02:00
Kilian Schüttler e3e9d16145 Merge branch 'RED-9975' into 'main' 
RED-9975: activate outline detection

See merge request fforesight/layout-parser!201
 2024-08-29 14:27:00 +02:00
Kilian Schuettler f6ca5a3c17 RED-9975: activate outline detection 2024-08-29 14:18:29 +02:00
Kilian Schüttler ab86714cb3 Merge branch 'RED-9975' into 'main' 
RED-9975: activate outline detection

See merge request fforesight/layout-parser!198
 2024-08-29 12:25:42 +02:00
Kilian Schuettler 8626b106d0 RED-9975: activate outline detection 2024-08-29 12:16:07 +02:00
Kilian Schüttler 8e115dcd8a RED-9760: change compareDouble to something sensible 2024-08-12 16:02:50 +02:00
Kilian Schuettler b0ae00aa02 hotfix: threshold adjustements 2024-08-12 14:52:18 +02:00
Kilian Schuettler d16377a24a hotfix: line comparison with center coordinates 2024-08-09 15:45:23 +02:00
Kilian Schüttler 6f6e8d5d4e Merge branch 'hotfix-reading-order' into 'main' 
hotfix reading order

See merge request fforesight/layout-parser!187
 2024-08-09 11:49:12 +02:00
Kilian Schüttler 69bcd4f68d hotfix reading order 2024-08-09 11:49:12 +02:00
Kilian Schüttler 370165dc59 Merge branch 'document-data-markdown' into 'main' 
CLARI: document-data-markdown

See merge request fforesight/layout-parser!181
 2024-07-18 17:19:44 +02:00
Kilian Schüttler ea18d3d307 Merge branch 'RED-8800' into 'main' 
RED-8800: adjust coordinates in BE to ignore cropbox

See merge request fforesight/layout-parser!179
 2024-07-15 17:45:13 +02:00
Kilian Schüttler 2726fc3fe1 RED-8800: adjust coordinates in BE to ignore cropbox 2024-07-15 17:45:13 +02:00
Kilian Schüttler 033279e261 Merge branch 'RED-9353' into 'main' 
RED-9353: refactor PDFTronViewerDocumentService

See merge request fforesight/layout-parser!178
 2024-07-15 12:54:17 +02:00
Kilian Schüttler ec0dd032c9 RED-9353: refactor PDFTronViewerDocumentService 2024-07-15 12:54:17 +02:00
Kilian Schüttler 3173610be5 Merge branch 'CLARI-003' into 'main' 
CLARI-003: add treeId to StructureObject

See merge request fforesight/layout-parser!176
 2024-07-02 11:37:22 +02:00
Kilian Schuettler e920eb5a78 CLARI-003: add treeId to StructureObject 2024-07-01 13:56:16 +02:00
Kilian Schüttler 7e4baea7e5 Merge branch 'RED-9353' into 'main' 
RED-9353: use azure ocr service

See merge request fforesight/layout-parser!175
 2024-07-01 11:13:27 +02:00
Kilian Schüttler 66d3433e04 RED-9353: use azure ocr service 2024-07-01 11:13:26 +02:00
Kilian Schüttler b216f02e15 Merge branch 'RED-9194' into 'main' 
RED-9194: roll back single digit headline change

See merge request fforesight/layout-parser!171
 2024-06-21 15:13:40 +02:00
Kilian Schuettler 2e2f30ba35 RED-9194: roll back single digit headline change 2024-06-21 14:42:30 +02:00
Kilian Schuettler 9f7ed974ec RED-9194: roll back single digit headline change 2024-06-21 14:41:30 +02:00
Kilian Schuettler 570a348a77 RED-9194: roll back single digit headline change 2024-06-21 14:39:47 +02:00
Kilian Schüttler 79795e408a Merge branch 'RED-9194' into 'main' 
RED-9194: allow single digit headline identifiers

See merge request fforesight/layout-parser!168
 2024-06-07 09:09:25 +02:00
Kilian Schuettler b719db86ab RED-9194: allow single digit headline identifiers 2024-06-06 16:32:05 +02:00
Kilian Schüttler e304a9f2d7 Merge branch 'RED-7074-le' into 'main' 
RED-7074: Design Subsection section tree structure algorithm

See merge request fforesight/layout-parser!166
 2024-06-06 13:22:14 +02:00
Kilian Schuettler 8648ed0952 hotifx for clarifynd 2024-05-15 14:02:02 +02:00
Kilian Schüttler 53f786b539 Merge branch 'RED-9149' into 'main' 
RED-9149 - Header and footer detection by page-association

See merge request fforesight/layout-parser!150
 2024-05-13 14:57:33 +02:00
Kilian Schuettler 6a65d7f9fc RED-8825: minor fixes 
* also added overrides via env variables
 2024-05-07 17:37:42 +02:00
Kilian Schuettler e935cc7b14 RED-8825: some fixes, and experimental column detector 2024-05-06 14:24:39 +02:00
Kilian Schüttler 07733d0855 Merge branch 'RED-8825' into 'main' 
RED-8825: improve layoutparsing

See merge request fforesight/layout-parser!132
 2024-05-03 12:03:03 +02:00
Kilian Schuettler abb249e966 RED-8825: general layoutparsing improvements 
* fix checkstyle
 2024-05-03 00:15:31 +02:00
Kilian Schuettler bcd1eb9afa RED-8825: general layoutparsing improvements 
* added test for table line classification
 2024-05-03 00:13:48 +02:00
Kilian Schuettler 60acbac53f RED-8825: general layoutparsing improvements 
* fixing a bunch of coordinates
 2024-05-03 00:06:29 +02:00
Kilian Schuettler a3decd292d RED-8825: general layoutparsing improvements 
* fix RulingCleaningService
 2024-05-02 23:00:22 +02:00
Kilian Schuettler b6f0a21886 RED-8825: general layoutparsing improvements 
* refactor all coordinates
 2024-05-02 21:01:25 +02:00
Kilian Schuettler d61cac8b4f RED-8825: general layoutparsing improvements 
* fix tests
 2024-04-30 16:06:22 +02:00
Kilian Schuettler ae46c5f1ca RED-8825: general layoutparsing improvements 
* fix tests
 2024-04-30 11:55:18 +02:00
Kilian Schuettler f0a70a5242 RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 11:09:23 +02:00
Kilian Schuettler 15ea385f4d RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 10:44:32 +02:00
Kilian Schuettler 08be18db2d RED-8825: general improvements 
* some more refactoring
 2024-04-29 20:09:53 +02:00
Kilian Schuettler 64209255cb RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:24:15 +02:00
Kilian Schuettler 4761d2e1a2 RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:22:33 +02:00
Kilian Schuettler 1916e626df RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:15:19 +02:00
Kilian Schuettler e4663ac8db RED-8825: added split by ruling into every step of docstrum 2024-04-29 15:54:56 +02:00
Kilian Schuettler 6a691183dc RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:56 +02:00
Kilian Schuettler 3dd215288a RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:53 +02:00
Kilian Schüttler 6fb1a0bef3 Merge branch 'RED-8992' into 'main' 
RED-8992 - Enable to add annotation on header with line breaks

See merge request fforesight/layout-parser!143
 2024-04-25 13:03:40 +02:00
Kilian Schüttler d38d023485 Merge branch 'RED-7384' into 'main' 
Red 7384

See merge request fforesight/layout-parser!140
 2024-04-23 12:13:21 +02:00
Kilian Schüttler c1afe9b11f Red 7384 2024-04-23 12:13:19 +02:00
Kilian Schuettler f256f9b30f RED-8995: unclassified text might be missing from document data 
* treat TablePageBlock.OTHER like PARAGRAPH (no special treatment)
 2024-04-18 17:42:34 +02:00
Kilian Schüttler c4d9c5df02 Merge branch 'RED-8747-fp' into 'main' 
RED-8747 - Entities not merged properly - fp

See merge request fforesight/layout-parser!131
 2024-04-09 16:30:02 +02:00
Kilian Schüttler bf6a0d770b Merge branch 'RED-8799' into 'main' 
RED-8799: LayoutGrid is wrong draw for some tables

See merge request fforesight/layout-parser!126
 2024-04-04 15:23:12 +02:00
Kilian Schuettler f18bda1d4e RED-8799: LayoutGrid is wrong draw for some tables 2024-04-04 13:33:22 +02:00
Kilian Schüttler c324d3815e Merge branch 'RED-8550-pre-backup' into 'main' 
RED-8550: Faulty table recognition and text duplication leads to huge sections

See merge request fforesight/layout-parser!105
 2024-02-28 16:13:56 +01:00
Kilian Schüttler e7bf607663 Merge branch 'hotfix' into 'main' 
hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written

See merge request fforesight/layout-parser!104
 2024-02-26 12:32:14 +01:00
Kilian Schuettler f4d789311c hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written 2024-02-26 12:24:15 +01:00
Kilian Schüttler 3c9049dc8a Merge branch 'RED-8156' into 'main' 
RED-8156: refactor ViewerDocumentService as a dependency for ocr-service

See merge request fforesight/layout-parser!99
 2024-02-07 10:42:21 +01:00
Kilian Schuettler 015984891f RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:17:26 +01:00
Kilian Schuettler 66fcb62833 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:09:21 +01:00
Kilian Schuettler 48824f56a8 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:06:53 +01:00
Kilian Schuettler 785628537f RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* various improvements to experimental parsing steps
* added embed fonts functionality to viewer doc
* fix checkstyle
 2024-02-06 17:03:38 +01:00
Kilian Schuettler 23eb0c40a3 RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* various improvements to experimental parsing steps
* added embed fonts functionality to viewer doc
 2024-02-06 16:59:51 +01:00
Kilian Schüttler ba1c7c07ab RED-7384: fixes for migration 2023-12-20 12:40:00 +01:00
Kilian Schüttler 30f060e36c Merge branch 'enable-caching' into 'main' 
enable caching in build

See merge request fforesight/layout-parser!84
 2023-11-24 10:36:55 +01:00
Kilian Schuettler 53a5824e6c enable caching in build 2023-11-24 10:24:50 +01:00
Kilian Schüttler 1b1f777706 Merge branch 'RED-7806' into 'main' 
RED-7806 - Specific customer document cannot be processed

See merge request fforesight/layout-parser!79
 2023-10-25 10:50:34 +02:00
Kilian Schüttler 9abdc6d44d Merge branch 'RED-7434' into 'main' 
RED-7434 - Remove Section Grid entirely

See merge request fforesight/layout-parser!78
 2023-10-20 10:07:01 +02:00
Kilian Schüttler 3c53772765 Merge branch 'RED-7759' into 'main' 
RED-7759: Upgraded storage-commons to newest windwos compatible version

See merge request fforesight/layout-parser!76
 2023-10-13 12:21:04 +02:00
Kilian Schüttler 310c07b200 Merge branch 'RED-7607-WIP' into 'main' 
RED-7607 - Rotating pages leads to lost annotations (RM & DM)

See merge request fforesight/layout-parser!75
 2023-10-05 13:34:12 +02:00
Kilian Schuettler 5792ff4a93 TAAS-104: merge visually intersecting Paragraphs 
* fix build
 2023-09-05 16:54:23 +02:00
Kilian Schuettler 621c3f269d TAAS-104: merge visually intersecting Paragraphs 2023-09-05 16:09:05 +02:00
Kilian Schüttler 754fd8f933 Merge branch 'TAAS-89' into 'main' 
TAAS-89: added log entry and an end2end test

See merge request fforesight/layout-parser!71
 2023-08-31 14:40:48 +02:00
Kilian Schuettler 28ec4c9ccb TAAS-89: added log entry and an end2end test 2023-08-31 14:28:18 +02:00
Kilian Schüttler aed4a55787 Merge branch 'TAAS-89' into 'main' 
TAAS-89: fixed weird bug with empty sections

See merge request fforesight/layout-parser!70
 2023-08-31 12:00:18 +02:00
Kilian Schuettler f87e2d75b5 TAAS-89: fixed weird bug with empty sections 2023-08-31 11:41:22 +02:00
Kilian Schüttler de6760abc1 Merge branch 'TAAS-89' into 'main' 
TAAS-89: added some more documentation

See merge request fforesight/layout-parser!69
 2023-08-31 10:55:45 +02:00
Kilian Schuettler 261ef4c367 TAAS-89: added some more documentation 
* fixed weird bug with empty sections
 2023-08-31 10:49:32 +02:00
Kilian Schuettler bcf0bcbaf4 Added some documentation 2023-08-24 18:37:47 +02:00
Kilian Schuettler 3a18923ef5 upgrade PDFBox to 3.0.0 
* disable experimental ruling header stuff
 2023-08-21 17:54:20 +02:00
Kilian Schuettler 2b15fd1d3c RED-7461: improve header/footer recognition 2023-08-21 17:49:13 +02:00
Kilian Schüttler 457f7d9c66 Merge branch 'RED-7158' into 'main' 
RED-7158: fix for all page rotations

See merge request fforesight/layout-parser!35
 2023-08-15 15:07:04 +02:00
Kilian Schuettler 0387cdd143 RED-7158: fix for all page rotations 
* also make lines thinner
 2023-08-15 14:55:41 +02:00
Kilian Schüttler 5c6898b975 Merge branch 'RED-7158' into 'main' 
RED-7158: add layoutgrid into new ViewerDocument as optional content

See merge request fforesight/layout-parser!34
 2023-08-15 13:22:06 +02:00