72 Commits

Author SHA1 Message Date
Isaac Riley
4c06206445 merge minor changes before branching 2022-02-28 08:07:08 +01:00
Isaac Riley
44d4eb5a98 format and add functions in post_processing.py missing from merge 2022-02-25 12:34:34 +01:00
Isaac Riley
fc4789101f merge hugo into main; deskew already merged 2022-02-25 11:20:40 +01:00
Isaac Riley
596aa2d5a3 prepare for merge; minor edits 2022-02-25 09:46:20 +01:00
Isaac Riley
2180ff924a make full demo 2022-02-23 13:41:57 +01:00
Isaac Riley
a09fe65ed2 minor changes to demo script 2022-02-22 16:04:11 +01:00
Isaac Riley
8ff5147ee4 change default deskew function from hough-line-based to pixel-histogram-based; use scipy.ndimage.rotation 2022-02-22 10:18:41 +01:00
Isaac Riley
59e082379c fix angle detection to make more sensitive to small angles; format with black 2022-02-21 16:52:24 +01:00
Isaac Riley
08cce36940 add deskew.py to utils, also demo 2022-02-21 14:53:09 +01:00
llocarnini
2a68e1b221 kernel size for morphology ex set bit higher, so less non-table structure are detected. Reduced the kernel size of the directional motion blurr and increased the treshold a little bit so narrow cells wont be split up.
Problem with the cell filtering for certain scanned pdfs detected.
2022-02-18 16:36:25 +01:00
llocarnini
723c6606e1 kernel size for morphology ex set bit higher, so less non-table structure are detected. Reduced the kernel size of the directional motion blurr and increased the treshold a little bit so narrow cells wont be split up.
Problem with the cell filtering for certain scanned pdfs detected.
2022-02-18 16:35:50 +01:00
llocarnini
d70781f4aa changed tolerance in adjacent1 function in postprocessing.y from 2 to 4
added function so vertical and horizontal components do not overlap the layout box of the table
2022-02-17 16:45:55 +01:00
llocarnini
57ca47f38d different approaches to isolate line components of tables in scanned pdf files. 2022-02-16 12:37:17 +01:00
llocarnini
c2faf7d00b adjusted isolation of vertical and horizontal components to be more robust to scanned pages; work in progress 2022-02-14 11:04:04 +01:00
llocarnini
885fc22f9d added changes to parse scanned pdfs 2022-02-11 15:59:54 +01:00
llocarnini
07907d45dd some changes to fix some minor bugs in table_parsing.py and post_processing.py 2022-02-10 10:56:03 +01:00
llocarnini
4964c8f5a1 some changes to fix some minor bugs in table_parsing.py and post_processing.py 2022-02-10 10:22:22 +01:00
Matthias Bisping
f7d3e39692 nix dolles 2022-02-08 15:05:12 +01:00
Matthias Bisping
87cecadb44 applied black 2022-02-06 21:27:39 +01:00
Matthias Bisping
295666c28f added todo comments 2022-02-06 21:25:01 +01:00
Matthias Bisping
90b8613bf8 filtering non-tables by bounding rect check WIP 2022-02-06 21:03:40 +01:00
Matthias Bisping
36284f9a78 removed obsolete lines 2022-02-06 20:01:00 +01:00
Matthias Bisping
0fc6cf8008 fixed bug in adjaceny test 2022-02-06 20:00:38 +01:00
Matthias Bisping
106b333dca filtering for connected cells... but does not quite work yet 2022-02-06 16:44:07 +01:00
Matthias Bisping
c9b2f6bf29 Pull request #9: Refactoring
Merge in RR/vidocp from refactoring to master

Squashed commit of the following:

commit 36a62a13e51148d2420cb12930e84d78629db6b0
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:54:53 2022 +0100

    refactoring

commit e652da1fa88a048f9a5211b4e8c0b96074fb5849
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:53:17 2022 +0100

    refactoring

commit d9567da428c81f9cd7971a657281df0a90166810
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:47:18 2022 +0100

    refactoring

commit 9d30009dceec0357db6499bfaffae8ce97718ee0
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:45:53 2022 +0100

    refactoring

commit e8863d67aaaff138fb088c4e496a91b6354cc059
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:42:45 2022 +0100

    refactoring

commit 89a99d3586db4fbafa743a45bdd02eaf0c1f341f
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:39:49 2022 +0100

    refactoring

commit aa66b6865b00b0490b9e7695a6bae386e6f96723
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:31:21 2022 +0100

    refactoring

commit 98d77cb522a08821c3a13ae2cffbe7239c654762
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:27:55 2022 +0100

    refactoring

commit fed3a7e4f1b8b7ca4e14f9e495459c26490fb50b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:26:16 2022 +0100

    refactoring

commit 504cafbd5d4bba183d9943b36c60548aae34e402
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:25:44 2022 +0100

    renaming

commit c9780a57e5a048529d36958ba678eddb11759cef
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:24:41 2022 +0100

    removed obsolete import

commit d555e86475e82024f8e1a5fc5b0ac70faa091ee1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 14:24:04 2022 +0100

    refactored figure detection once
2022-02-06 14:55:38 +01:00
Matthias Bisping
36a62a13e5 refactoring 2022-02-06 14:54:53 +01:00
Matthias Bisping
e652da1fa8 refactoring 2022-02-06 14:53:17 +01:00
Matthias Bisping
d9567da428 refactoring 2022-02-06 14:47:18 +01:00
Matthias Bisping
9d30009dce refactoring 2022-02-06 14:45:53 +01:00
Matthias Bisping
e8863d67aa refactoring 2022-02-06 14:42:45 +01:00
Matthias Bisping
89a99d3586 refactoring 2022-02-06 14:39:49 +01:00
Matthias Bisping
aa66b6865b refactoring 2022-02-06 14:31:21 +01:00
Matthias Bisping
98d77cb522 refactoring 2022-02-06 14:27:55 +01:00
Matthias Bisping
fed3a7e4f1 refactoring 2022-02-06 14:26:16 +01:00
Matthias Bisping
504cafbd5d renaming 2022-02-06 14:25:44 +01:00
Matthias Bisping
c9780a57e5 removed obsolete import 2022-02-06 14:24:41 +01:00
Matthias Bisping
d555e86475 refactored figure detection once 2022-02-06 14:24:04 +01:00
Matthias Bisping
8432cfe514 Pull request #8: figure detection
Merge in RR/vidocp from text_removal to master

Squashed commit of the following:

commit b65374c512ce9ba07fa522d591c83db3de5d7d55
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 01:03:12 2022 +0100

    readme updated

commit 1c1f7a395a00fa505cf19e1ad87d8c34faa6ef5b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 01:00:46 2022 +0100

    figure detection version 1 completed

commit f257660823ef8682e9fedda9921ad946ef2ade76
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 00:37:03 2022 +0100

    wip

commit 2e89b28f4a69da80570597c823b3b7a591788d0a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sun Feb 6 00:23:56 2022 +0100

    wip
2022-02-06 01:04:15 +01:00
Matthias Bisping
b82a294610 Pull request #7: Layout detection version 3
Merge in RR/vidocp from layout_detection_version_3 to master

Squashed commit of the following:

commit 262b1c14c0b8b164221d39fd286b20914d1a8e6a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 22:56:10 2022 +0100

    comment

commit 975dcdaae2b0e9bfcb075fe1c87adc48175c0d93
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 22:50:41 2022 +0100

    applied black

commit 49ba3b5f318a1b5d6bb39c0b53de5e237a87da96
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 22:48:44 2022 +0100

    improved layout parsing logic: filtering of included rects

commit d78ac24c10793f72b569c3c827834400b730888a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 22:36:49 2022 +0100

    improved layout parsing logic: filtering of overlaps, no sub-text regions
2022-02-05 22:58:51 +01:00
Matthias Bisping
7ceb534fa2 readme updated 2022-02-05 20:31:13 +01:00
Matthias Bisping
8a9b493c94 readme updated 2022-02-05 20:29:23 +01:00
Matthias Bisping
e62a88ff9a fixed import 2022-02-05 20:27:29 +01:00
Matthias Bisping
bb5707dc89 Pull request #6: added layout parsing logic
Merge in RR/vidocp from layout_detection_version_2 to master

Squashed commit of the following:

commit d443e95ad8143bed3efc74d9e38640498d8d16bf
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 20:16:13 2022 +0100

    readme updated

commit 953ad696932454ce851544ed016f9e64bcc12080
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 20:14:59 2022 +0100

    added layot parsing logic
2022-02-05 20:17:14 +01:00
Matthias Bisping
00748a8ac0 Pull request #5: Table parsing version 2
Merge in RR/vidocp from table_parsing_version_2 to master

Squashed commit of the following:

commit af136ca10cf96f99699e409000ff598ce90c192e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:34:01 2022 +0100

    readme updated

commit 13ca7b1b03cb2bf7b3c8ef5821c1f8fa9ec532a0
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:32:11 2022 +0100

    drawing color standardized

commit 654e961c62ddc0f512074e8238d7fa88f0ea227e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:22:57 2022 +0100

    refactoring

commit 964c17a36f7bbc1376dfe68f4ea90462d676e215
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:07:16 2022 +0100

    readme updated

commit 4470969b35bb76e68cc41947fa02e63100b30ce9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:05:35 2022 +0100

    readme updated

commit a6c6bdb1e71a778a3c21a628cfb30acc5bc6086f
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:05:21 2022 +0100

    readme updated

commit e178793dd69b720adefe7533312314e4c405f975
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 18:03:45 2022 +0100

    readme updated

commit 443163864bab56930c2ef735c0aaafddd2561ead
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Sat Feb 5 17:59:03 2022 +0100

    implememted clean solution for parsing open tables. still needs final refactoring.
2022-02-05 19:32:47 +01:00
Matthias Bisping
224360c823 readme updated 2022-02-05 16:23:42 +01:00
Matthias Bisping
9507bffd89 readme updated 2022-02-05 16:20:23 +01:00
Matthias Bisping
e454938c1e readme updated 2022-02-05 16:20:05 +01:00
Matthias Bisping
5491f511e2 readme updated 2022-02-05 16:19:27 +01:00
Matthias Bisping
5fec1db1e6 readme updated 2022-02-05 16:18:01 +01:00
Matthias Bisping
d5f445b5e1 readme updated 2022-02-05 16:17:39 +01:00