Marmelator/layout-parser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 289 Wiki Activity
531 Commits 48 Branches 295 Tags
6e5b1f19782f495e44bbf7ec808a55f912d51a8b
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kilian Schuettler 6e5b1f1978 RED-9139: move document to module in redaction-service 
* add feature version
 2024-11-14 16:39:48 +01:00
Kilian Schuettler cf846d18bc RED-9139: move document to module in redaction-service 
* add feature version
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 25c46f16ac RED-9139: move document to module in redaction-service 
* add feature version
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 96acefed78 RED-9139: move document to module in redaction-service 
* add TableOfContents node
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 366241e6c6 RED-9139: move document to module in redaction-service 
* add TableOfContents node
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 7f472ccc52 RED-9139: move document to module in redaction-service 
* add TableOfContents node
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 6f807c7d94 RED-9139: add new TableOfContents Node 
* rename previous TableOfContent to SectionTree
* added protobuf compile script
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 6e04c15f3d RED-9139: add new TableOfContents Node 
* rename previous TableOfContent to SectionTree
* added protobuf compile script
 2024-11-14 16:39:48 +01:00
Kilian Schuettler 1384584e2f RED-9139: more robust TOC detection 
* detect numbers in words, and not just whole words that are numbers
 2024-11-14 16:39:46 +01:00
Kilian Schuettler e58011e111 RED-9139: more robust TOC detection 
* detect numbers in words, and not just whole words that are numbers
 2024-11-14 16:39:21 +01:00
Kilian Schüttler a821570065 Merge branch 'RED-9139-bp' into 'main' 
RED-9139: more robust TOC detection

See merge request fforesight/layout-parser!254
 2024-11-13 10:54:39 +01:00
Kilian Schüttler 7ee1f9e360 RED-9139: more robust TOC detection 2024-11-13 10:54:39 +01:00
Kilian Schüttler f9b25c8157 Merge branch 'RED-10249' into 'main' 
RED-10249: regex found incorrectly due to wrong text sorting

See merge request fforesight/layout-parser!252
 2024-11-04 12:51:38 +01:00
Kilian Schüttler c90874da7a RED-10249: regex found incorrectly due to wrong text sorting 2024-11-04 12:51:37 +01:00
Kilian Schüttler 4683c696a5 Merge branch 'RED-10247' into 'main' 
RED-10247: dictionary entry not found in footer due to wrong text sorting

See merge request fforesight/layout-parser!251
 2024-10-25 18:30:35 +02:00
Kilian Schuettler 95c02ce3cf RED-10247: dictionary entry not found in footer due to wrong text sorting 2024-10-25 17:18:14 +02:00
Kilian Schüttler b2d62e32fe Merge branch 'RED-10270-fp' into 'main' 
RED-10270: fix NumberFormatException

See merge request fforesight/layout-parser!248
 2024-10-24 17:14:47 +02:00
Kilian Schuettler 65c1f03ea3 RED-10270: fix NumberFormatException 2024-10-24 10:59:05 +02:00
Kilian Schüttler 2219519a2b Merge branch 'RED-10127' into 'main' 
RED-10127: rename TextPositionSequence to Word

See merge request fforesight/layout-parser!244
 2024-10-18 12:20:15 +02:00
Kilian Schüttler af05218e37 RED-10127: rename TextPositionSequence to Word 2024-10-18 12:20:15 +02:00
Kilian Schüttler 736f531df3 Merge branch 'hotfix' into 'main' 
Hotfix

See merge request fforesight/layout-parser!243
 2024-10-18 12:12:15 +02:00
Kilian Schüttler c64445d54b Hotfix 2024-10-18 12:12:15 +02:00
Kilian Schüttler af29233b10 Merge branch 'feature/RED-10127' into 'main' 
RED-10127: add more units

See merge request fforesight/layout-parser!242
 2024-10-15 09:57:21 +02:00
Kilian Schuettler 5f04b45554 RED-10127: add more units 2024-10-15 09:47:39 +02:00
Kilian Schüttler 6c41533f0b Merge branch 'feature/RED-10127' into 'main' 
RED-10127: improve list classification

See merge request fforesight/layout-parser!240
 2024-10-14 17:34:33 +02:00
Kilian Schuettler 9d2596e5ef RED-10127: improve list classification 
* add one more format to list identification
* add 'ppb' to known units
* special case for headlines continuing with 14C after the identifier (quite often in some specific files)
 2024-10-14 17:21:44 +02:00
Kilian Schüttler e7b01161ac Merge branch 'feature/RED-10127' into 'main' 
RED-10127: add list classification

See merge request fforesight/layout-parser!237
 2024-10-10 10:50:10 +02:00
Kilian Schüttler 7b073eb4f3 RED-10127: add list classification 2024-10-10 10:50:10 +02:00
Kilian Schüttler 6c7442ac6d RED-10127: improve headline detection 2024-10-09 08:48:48 +02:00
Kilian Schüttler 469da38952 Red 9974: improce headline classification, fix font size calculation 2024-09-16 14:06:48 +02:00
Kilian Schuettler 8e165a41d7 hotfix: viewerDocService doesn't remove existing marked content 2024-09-11 16:34:21 +02:00
Kilian Schüttler ed7a701ad9 Merge branch 'RED-9975' into 'main' 
RED-9975: improve SuperSection handling

See merge request fforesight/layout-parser!223
 2024-09-11 13:38:09 +02:00
Kilian Schüttler 393103e074 RED-9975: improve SuperSection handling 2024-09-11 13:38:09 +02:00
Kilian Schüttler c726a643f0 Merge branch 'hotfix' into 'main' 
Hotfix: unmerge super large tables

See merge request fforesight/layout-parser!220
 2024-09-05 15:05:21 +02:00
Kilian Schüttler 519e95735c Hotfix: unmerge super large tables 2024-09-05 15:05:21 +02:00
Kilian Schüttler 9650195afd Merge branch 'hotfix-fp' into 'main' 
hotfix: add Java advanced imaging

See merge request fforesight/layout-parser!217
 2024-09-04 15:43:56 +02:00
Kilian Schuettler ce628a99f7 hotfix: add Java advanced imaging 2024-09-04 15:18:12 +02:00
Kilian Schüttler af45f2cd8c Merge branch 'RED-9964' into 'main' 
RED-9964: fix errors with images

See merge request fforesight/layout-parser!212
 2024-09-04 09:16:59 +02:00
Kilian Schuettler befb6b1df6 RED-9964: fix errors with images 2024-09-03 16:37:48 +02:00
Kilian Schüttler 8e14b74da2 Red 9975: fix outline detection 2024-09-02 09:02:36 +02:00
Kilian Schüttler 3b91639ea9 Merge branch 'RED-9964-fp' into 'main' 
RED-9964: don't merge tables on non-consecutive pages

See merge request fforesight/layout-parser!205
 2024-08-30 14:00:48 +02:00
Kilian Schüttler c5178ea5c2 RED-9964: don't merge tables on non-consecutive pages 2024-08-30 14:00:48 +02:00
Kilian Schüttler e3e9d16145 Merge branch 'RED-9975' into 'main' 
RED-9975: activate outline detection

See merge request fforesight/layout-parser!201
 2024-08-29 14:27:00 +02:00
Kilian Schuettler f6ca5a3c17 RED-9975: activate outline detection 2024-08-29 14:18:29 +02:00
Kilian Schüttler ab86714cb3 Merge branch 'RED-9975' into 'main' 
RED-9975: activate outline detection

See merge request fforesight/layout-parser!198
 2024-08-29 12:25:42 +02:00
Kilian Schuettler 8626b106d0 RED-9975: activate outline detection 2024-08-29 12:16:07 +02:00
Kilian Schüttler 8e115dcd8a RED-9760: change compareDouble to something sensible 2024-08-12 16:02:50 +02:00
Kilian Schuettler b0ae00aa02 hotfix: threshold adjustements 2024-08-12 14:52:18 +02:00
Kilian Schuettler d16377a24a hotfix: line comparison with center coordinates 2024-08-09 15:45:23 +02:00
Kilian Schüttler 6f6e8d5d4e Merge branch 'hotfix-reading-order' into 'main' 
hotfix reading order

See merge request fforesight/layout-parser!187
 2024-08-09 11:49:12 +02:00
Kilian Schüttler 69bcd4f68d hotfix reading order 2024-08-09 11:49:12 +02:00
Kilian Schüttler 370165dc59 Merge branch 'document-data-markdown' into 'main' 
CLARI: document-data-markdown

See merge request fforesight/layout-parser!181
 2024-07-18 17:19:44 +02:00
Kilian Schüttler ea18d3d307 Merge branch 'RED-8800' into 'main' 
RED-8800: adjust coordinates in BE to ignore cropbox

See merge request fforesight/layout-parser!179
 2024-07-15 17:45:13 +02:00
Kilian Schüttler 2726fc3fe1 RED-8800: adjust coordinates in BE to ignore cropbox 2024-07-15 17:45:13 +02:00
Kilian Schüttler 033279e261 Merge branch 'RED-9353' into 'main' 
RED-9353: refactor PDFTronViewerDocumentService

See merge request fforesight/layout-parser!178
 2024-07-15 12:54:17 +02:00
Kilian Schüttler ec0dd032c9 RED-9353: refactor PDFTronViewerDocumentService 2024-07-15 12:54:17 +02:00
Kilian Schüttler 3173610be5 Merge branch 'CLARI-003' into 'main' 
CLARI-003: add treeId to StructureObject

See merge request fforesight/layout-parser!176
 2024-07-02 11:37:22 +02:00
Kilian Schuettler e920eb5a78 CLARI-003: add treeId to StructureObject 2024-07-01 13:56:16 +02:00
Kilian Schüttler 7e4baea7e5 Merge branch 'RED-9353' into 'main' 
RED-9353: use azure ocr service

See merge request fforesight/layout-parser!175
 2024-07-01 11:13:27 +02:00
Kilian Schüttler 66d3433e04 RED-9353: use azure ocr service 2024-07-01 11:13:26 +02:00
Kilian Schüttler b216f02e15 Merge branch 'RED-9194' into 'main' 
RED-9194: roll back single digit headline change

See merge request fforesight/layout-parser!171
 2024-06-21 15:13:40 +02:00
Kilian Schuettler 2e2f30ba35 RED-9194: roll back single digit headline change 2024-06-21 14:42:30 +02:00
Kilian Schuettler 9f7ed974ec RED-9194: roll back single digit headline change 2024-06-21 14:41:30 +02:00
Kilian Schuettler 570a348a77 RED-9194: roll back single digit headline change 2024-06-21 14:39:47 +02:00
Kilian Schüttler 79795e408a Merge branch 'RED-9194' into 'main' 
RED-9194: allow single digit headline identifiers

See merge request fforesight/layout-parser!168
 2024-06-07 09:09:25 +02:00
Kilian Schuettler b719db86ab RED-9194: allow single digit headline identifiers 2024-06-06 16:32:05 +02:00
Kilian Schüttler e304a9f2d7 Merge branch 'RED-7074-le' into 'main' 
RED-7074: Design Subsection section tree structure algorithm

See merge request fforesight/layout-parser!166
 2024-06-06 13:22:14 +02:00
Kilian Schuettler 8648ed0952 hotifx for clarifynd 2024-05-15 14:02:02 +02:00
Kilian Schüttler 53f786b539 Merge branch 'RED-9149' into 'main' 
RED-9149 - Header and footer detection by page-association

See merge request fforesight/layout-parser!150
 2024-05-13 14:57:33 +02:00
Kilian Schuettler 6a65d7f9fc RED-8825: minor fixes 
* also added overrides via env variables
 2024-05-07 17:37:42 +02:00
Kilian Schuettler e935cc7b14 RED-8825: some fixes, and experimental column detector 2024-05-06 14:24:39 +02:00
Kilian Schüttler 07733d0855 Merge branch 'RED-8825' into 'main' 
RED-8825: improve layoutparsing

See merge request fforesight/layout-parser!132
 2024-05-03 12:03:03 +02:00
Kilian Schuettler abb249e966 RED-8825: general layoutparsing improvements 
* fix checkstyle
 2024-05-03 00:15:31 +02:00
Kilian Schuettler bcd1eb9afa RED-8825: general layoutparsing improvements 
* added test for table line classification
 2024-05-03 00:13:48 +02:00
Kilian Schuettler 60acbac53f RED-8825: general layoutparsing improvements 
* fixing a bunch of coordinates
 2024-05-03 00:06:29 +02:00
Kilian Schuettler a3decd292d RED-8825: general layoutparsing improvements 
* fix RulingCleaningService
 2024-05-02 23:00:22 +02:00
Kilian Schuettler b6f0a21886 RED-8825: general layoutparsing improvements 
* refactor all coordinates
 2024-05-02 21:01:25 +02:00
Kilian Schuettler d61cac8b4f RED-8825: general layoutparsing improvements 
* fix tests
 2024-04-30 16:06:22 +02:00
Kilian Schuettler ae46c5f1ca RED-8825: general layoutparsing improvements 
* fix tests
 2024-04-30 11:55:18 +02:00
Kilian Schuettler f0a70a5242 RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 11:09:23 +02:00
Kilian Schuettler 15ea385f4d RED-8825: general improvements 
* some more refactoring
 * fixed text ruling classification for vertical text
 * shrunk min graphics size
 2024-04-30 10:44:32 +02:00
Kilian Schuettler 08be18db2d RED-8825: general improvements 
* some more refactoring
 2024-04-29 20:09:53 +02:00
Kilian Schuettler 64209255cb RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:24:15 +02:00
Kilian Schuettler 4761d2e1a2 RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:22:33 +02:00
Kilian Schuettler 1916e626df RED-8825: general improvements 
* classify rulings as underline/striketrough
* improve performance of CleanRulings.lineBetween
* use lineBetween where possible
* wip, still todo:
 - Header/Footer by Ruling for all rotations
 - actually the ticket, optimizing layoutparsing for documine
 2024-04-29 17:15:19 +02:00
Kilian Schuettler e4663ac8db RED-8825: added split by ruling into every step of docstrum 2024-04-29 15:54:56 +02:00
Kilian Schuettler 6a691183dc RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:56 +02:00
Kilian Schuettler 3dd215288a RED-8825: improve layoutparsing 
* added improved debugging capabilities to viewer-doc
* refactored coordinates (wip)
* refactored line intersection algorithm
* removed cropbox correction from pdfbox text positions
 2024-04-29 15:54:53 +02:00
Kilian Schüttler 6fb1a0bef3 Merge branch 'RED-8992' into 'main' 
RED-8992 - Enable to add annotation on header with line breaks

See merge request fforesight/layout-parser!143
 2024-04-25 13:03:40 +02:00
Kilian Schüttler d38d023485 Merge branch 'RED-7384' into 'main' 
Red 7384

See merge request fforesight/layout-parser!140
 2024-04-23 12:13:21 +02:00
Kilian Schüttler c1afe9b11f Red 7384 2024-04-23 12:13:19 +02:00
Kilian Schuettler f256f9b30f RED-8995: unclassified text might be missing from document data 
* treat TablePageBlock.OTHER like PARAGRAPH (no special treatment)
 2024-04-18 17:42:34 +02:00
Kilian Schüttler c4d9c5df02 Merge branch 'RED-8747-fp' into 'main' 
RED-8747 - Entities not merged properly - fp

See merge request fforesight/layout-parser!131
 2024-04-09 16:30:02 +02:00
Kilian Schüttler bf6a0d770b Merge branch 'RED-8799' into 'main' 
RED-8799: LayoutGrid is wrong draw for some tables

See merge request fforesight/layout-parser!126
 2024-04-04 15:23:12 +02:00
Kilian Schuettler f18bda1d4e RED-8799: LayoutGrid is wrong draw for some tables 2024-04-04 13:33:22 +02:00
Kilian Schüttler c324d3815e Merge branch 'RED-8550-pre-backup' into 'main' 
RED-8550: Faulty table recognition and text duplication leads to huge sections

See merge request fforesight/layout-parser!105
 2024-02-28 16:13:56 +01:00
Kilian Schüttler e7bf607663 Merge branch 'hotfix' into 'main' 
hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written

See merge request fforesight/layout-parser!104
 2024-02-26 12:32:14 +01:00
Kilian Schuettler f4d789311c hotfix: double viewerdoc writes in rare cases lead to some contentstreams not being written 2024-02-26 12:24:15 +01:00
Kilian Schüttler 3c9049dc8a Merge branch 'RED-8156' into 'main' 
RED-8156: refactor ViewerDocumentService as a dependency for ocr-service

See merge request fforesight/layout-parser!99
 2024-02-07 10:42:21 +01:00
Kilian Schuettler 015984891f RED-8156: refactor ViewerDocumentService as a dependency for ocr-service 
* fix pmd
 2024-02-06 17:17:26 +01:00