217 Commits

Author SHA1 Message Date
Julius Unverfehrt
08c0096c07 Pull request #38: upgrade references
Merge in RR/cv-analysis from RED-6118-multi-tenancy-patch to master

* commit '233c6facfd75771885ae87c79b57bcb53c71d6e7':
  upgrade references
1.16.0
2023-02-16 16:47:40 +01:00
Julius Unverfehrt
233c6facfd upgrade references RED-6118-multi-tenancy-patch_3 2023-02-16 16:45:19 +01:00
Francisco Schulz
4ce6c9bdc9 Pull request #37: RED-5277 fix heartbeat issue
Merge in RR/cv-analysis from RED-5277-fix-heartbeat-issue to master

* commit '5bb9282da6aa1d75182c2172c601bed534099b0f':
  use python 3.8 in build
  update serve.py to work with new pyinfra version
  update reference to pyinfra
1.15.0 RED-6118-multi-tenancy-patch_2
2023-02-16 11:06:06 +01:00
Francisco Schulz
5bb9282da6 use python 3.8 in build RED-5277-fix-heartbeat-issue_3 2023-02-16 11:00:06 +01:00
Francisco Schulz
eef371e2a8 update serve.py to work with new pyinfra version 2023-02-16 10:47:13 +01:00
Francisco Schulz
ad45e2c1da update reference to pyinfra 2023-02-16 10:46:55 +01:00
Matthias Bisping
6909e48b60 Pull request #36: Poetrify
Merge in RR/cv-analysis from poetrify to master

* commit '35d046c454cc199515fee87ec3463a5fe106e986':
  Rollback
  Copy paste and adjust build code from rule-engine repo
  Poetrify Dockerfile
  Rollback
  Update requirements file
  Add dvc-data-pull fixture
  Adjust pdf2img dependency branch
  Adjust type hints for new lower python version
  Initialize poetry and add dependencies
1.14.0
2023-01-03 15:22:35 +01:00
Matthias Bisping
35d046c454 Rollback
Porting build logic to poetry not economical right now.
poetrify_9
2023-01-03 11:38:33 +01:00
Matthias Bisping
f0bba5bbdb Copy paste and adjust build code from rule-engine repo
That build code is more recent and the current build code fails.
2023-01-03 10:51:26 +01:00
Matthias Bisping
aee7ee9b97 Poetrify Dockerfile 2023-01-03 10:41:09 +01:00
Matthias Bisping
64639bce6d Rollback poetrify_6 2023-01-03 10:15:13 +01:00
Matthias Bisping
16698428f3 Update requirements file 2023-01-03 10:11:01 +01:00
Matthias Bisping
04f100386d Add dvc-data-pull fixture
This allows the tests that depend on input files to run without the need
to manuallt dvc-pull them beforehand.
poetrify_4
2023-01-03 10:01:03 +01:00
Matthias Bisping
fcb2786697 Adjust pdf2img dependency branch poetrify_3 2023-01-02 15:57:06 +01:00
Matthias Bisping
7bbe459208 Adjust type hints for new lower python version poetrify_2 2023-01-02 15:46:35 +01:00
Matthias Bisping
d52b2bfe22 Initialize poetry and add dependencies 2023-01-02 15:45:47 +01:00
Julius Unverfehrt
506ed789f7 add explorative script for hierarichal layout parsing 1.13.0 2022-12-13 11:16:15 +01:00
Julius Unverfehrt
d14960da08 Pull request #35: RED-5527 update pdf2img with skip_pages_without_images flag, add ENV for this setting for table parsing
Merge in RR/cv-analysis from RED-5527 to master

Squashed commit of the following:

commit 1748095ce45f4a76e9d3d8547d9fc70f7deae2fb
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Nov 8 15:15:16 2022 +0100

    RED-5527 update pdf2img with skip_pages_without_images flag, add ENV for this setting for table parsing
1.12.0
2022-11-08 15:16:53 +01:00
Francisco Schulz
8c5a979cc8 Pull request #34: update pyinfra
Merge in RR/cv-analysis from fschulz/update-to-new-pyinfra-version to master

* commit '555d1973b7a80a38e0893412ba0e0c0e2af4a520':
  update pyinfra
1.11.0
2022-10-13 15:33:31 +02:00
Francisco Schulz
555d1973b7 update pyinfra fschulz/update-to-new-pyinfra-version_4 2022-10-13 15:29:57 +02:00
Julius Unverfehrt
b26253120c Pull request #33: Fix response coords
Merge in RR/cv-analysis from fix-response-coords to master

Squashed commit of the following:

commit 0c6178a564b48abc43f129f81d93091a277fc64a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Oct 6 14:53:02 2022 +0200

    update tests

commit 46ad8737593df976555e4f60db8dc7947784d46d
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Oct 6 14:40:25 2022 +0200

    rename script

commit f541311d0aae22d5b76ba3c2580aada662812557
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Oct 6 14:40:11 2022 +0200

    response now returns natural page index, update pdf2image to correct response coordinates
1.10.0 fschulz/update-to-new-pyinfra-version_2
2022-10-06 14:56:28 +02:00
Viktor Seifert
559a86e7a3 Pull request #32: RED-5324: Update pyinfra to include storage-region fix
Merge in RR/cv-analysis from RED-5324 to master

* commit 'dcffacbb4568e2d8eebe20e0dd90efa8a9731f84':
  RED-5324: Update pyinfra to include storage-region fix
1.9.0
2022-09-30 16:12:46 +02:00
Viktor Seifert
dcffacbb45 RED-5324: Update pyinfra to include storage-region fix RED-5324_2 2022-09-30 16:07:46 +02:00
Christoph Schabert
7706cfe973 Pull request #31: hotfix: fix key prepare
Merge in RR/cv-analysis from hotfix/keyPrep to master

Squashed commit of the following:

commit 58961a319b985cb5d658d867459340eafc0e7c04
Author: cschabert <christoph.schabert@iqser.com>
Date:   Tue Sep 20 11:25:15 2022 +0200

    hotfix: fix key prepare

commit d937ff7c7d5824e8a75956102bfe7cf24fb27305
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Wed Aug 24 15:22:10 2022 +0200

    Pull request #30: RED-5009 update pyinfra to support message rejection on unobtainable files

    Merge in RR/cv-analysis from RED-5009-update-pyinfra to master

    Squashed commit of the following:

    commit fe46f92494b7f00db2884e0f11cd1f4cc29d1675
    Merge: 35d1675 95cab33
    Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
    Date:   Wed Aug 24 15:20:16 2022 +0200

        Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis into RED-5009-update-pyinfra

    commit 35d16759eb747467ce8deb88f8d953da0d4dc630
    Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
    Date:   Wed Aug 24 15:19:41 2022 +0200

        RED-5009 update pyinfra to support message rejection on unobtainable files
1.8.0
2022-09-20 11:56:35 +02:00
Julius Unverfehrt
09ba101ce7 Pull request #30: RED-5009 update pyinfra to support message rejection on unobtainable files
Merge in RR/cv-analysis from RED-5009-update-pyinfra to master

Squashed commit of the following:

commit fe46f92494b7f00db2884e0f11cd1f4cc29d1675
Merge: 35d1675 95cab33
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 24 15:20:16 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis into RED-5009-update-pyinfra

commit 35d16759eb747467ce8deb88f8d953da0d4dc630
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 24 15:19:41 2022 +0200

    RED-5009 update pyinfra to support message rejection on unobtainable files
1.7.0
2022-08-24 15:22:10 +02:00
lillian locarnini
95cab33f19 Pull request #29: Evaluate layout detection
Merge in RR/cv-analysis from evaluate_layout_detection to master

Squashed commit of the following:

commit 8ec2f69fc61d1e15bd502b0a2c1f720cbec2b34e
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Tue Aug 23 15:07:21 2022 +0200

    repaired is_not_included() logic (did drop the outer rectangle, not the included)

commit 97be081d1e60989313924ceac0bfb3062229411e
Merge: 2c28fa2 2b5c4f1
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Tue Aug 23 14:28:14 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis into evaluate_layout_detection

commit 2c28fa280b7eff922c715245fffe69702c7e6742
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Tue Aug 23 13:50:17 2022 +0200

    del print statements

commit c60121fc4faebc5de556ec0ab7a3af4f815f7ce1
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Mon Aug 22 10:51:52 2022 +0200

    few changes to connect_rects.py

commit a99719905d58cbe856fa020177abd7e317c1d072
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Thu Aug 18 08:37:12 2022 +0200

    layout parsing improved with connect_rects.py

commit d693688a0f0d63395cfd36645de7b3417f64de30
Author: llocarnini <lillian.locarnini@iqser.com>
Date:   Tue Aug 2 09:31:19 2022 +0200

    removed vizlogger instances
1.6.0
2022-08-23 15:09:51 +02:00
Julius Unverfehrt
2b5c4f1e45 Pull request #28: queue callback: add storage lookup for input file, add should_publish flag to signal processing success to queue manager
Merge in RR/cv-analysis from RED-5009-extend-callback to master

Squashed commit of the following:

commit aa9871e11ca56c721024d702351a3b2d51d4c69d
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 23 10:31:03 2022 +0200

    queue callback: add storage lookup for input file, add should_publish flag to signal processing success to queue manager
1.5.0
2022-08-23 10:43:34 +02:00
Julius Unverfehrt
ce3b460217 add executible rights 1.4.0 RED-5009-extend-callback_2 2022-08-18 09:10:03 +02:00
Julius Unverfehrt
a45c7858ee move script 1.3.0 2022-08-18 09:08:17 +02:00
Julius Unverfehrt
952a719ae6 add missing script 1.2.0 2022-08-18 09:06:42 +02:00
Julius Unverfehrt
b14a341cfc readd annotate_pdf script 1.0.0 1.1.0 2022-08-17 14:53:07 +02:00
Julius Unverfehrt
010e029d9b update pdf2image 2022-08-17 13:10:59 +02:00
Julius Unverfehrt
309ae0d57b Pull request #27: Image service compat
Merge in RR/cv-analysis from image-service-compat to master

Squashed commit of the following:

commit 397d12a96a6b78de762f7b3a80a72427f5f51e97
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 16 16:14:40 2022 +0200

    update pdf2image, adjust response format for table-parsing & figure-detection

commit f2061bda8d25d64de974e97f36148dea29af50d9
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Aug 15 08:56:39 2022 +0200

    add script to save figure detection data that can be used for image-service pipeline script
2022-08-16 17:04:05 +02:00
Julius Unverfehrt
20267f2715 remove nested list for figure detection response 2022-08-12 12:34:21 +02:00
Julius Unverfehrt
954c2f17a5 Pull request #26: adjust response for table parsing to be per page based
Merge in RR/cv-analysis from adjust-signature-to-per-page to master

Squashed commit of the following:

commit 1142a350d537453a7ac35d97bfa00bd2a64c4871
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Aug 12 11:09:04 2022 +0200

    adjust response for table parsing to be per page based
2022-08-12 11:46:24 +02:00
Julius Unverfehrt
ea25b57dd9 update pdf2image module 2022-08-10 14:17:57 +02:00
Julius Unverfehrt
7d7cc6026a update scripts to work with image service and show jsons from storage 2022-08-10 10:35:45 +02:00
Julius Unverfehrt
59a0a61708 Pull request #25: Pdf2image
Merge in RR/cv-analysis from pdf2image to master

Squashed commit of the following:

commit 1353f54d2dceb0a79b1f81bfa2c035f5a454275a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 10 09:07:31 2022 +0200

    add deRotation and transformation vie rectanglePlus

commit 51459dbf57a86e3eac66ec0da02de40dc1b68796
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 9 08:53:50 2022 +0200

    add derotation and to pdf coords transformation to cv-analysis output

commit 733991e2f5a4664205b2f7cc756cebcbc9ee3930
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Aug 8 15:15:13 2022 +0200

    update pipline with detrotation logic WIP
2022-08-10 09:17:59 +02:00
Christoph Schabert
e13430727b Pull request #24: RED-4758: Adjust buildjob
Merge in RR/cv-analysis from RED-4758 to master

* commit 'a29bc775e6db13d4af34ce0170b70a7fe962d84e':
  RED-4758: Adjust buildjob
2022-08-03 15:43:51 +02:00
cschabert
a29bc775e6 RED-4758: Adjust buildjob RED-4758_3 2022-08-03 15:04:27 +02:00
Julius Unverfehrt
016abe46de Pull request #23: Add pdf2image module
Merge in RR/cv-analysis from add-pdf2image-module to master

Squashed commit of the following:

commit 13355e2dd006fae9ee05c2d00acbbc8b38fd1e8e
Merge: eaf4627 edbda58
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 13:35:27 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis into add-pdf2image-module

commit eaf462768787642889d496203034d017c4ec959b
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 13:26:58 2022 +0200

    update build scripts

commit d429c713f4e5e74afca81c2354e8125bf389b865
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 13:11:07 2022 +0200

    purge target

commit 349b81c5db724bf70d6f31b58ded2b5414216bfe
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 13:07:58 2022 +0200

    Revert "extinguish target"

    This reverts commit d2bd4cefde0648d2487839b0344509b984435273.

commit d2bd4cefde0648d2487839b0344509b984435273
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 12:57:50 2022 +0200

    extinguish target

commit 5f6cc713db31e3e16c8e7f13a59804c86b5d77d7
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 11:58:52 2022 +0200

    refactor

commit 576019378a39b580b816d9eb7957774f1faf48b9
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 11:52:04 2022 +0200

    add test for adjustesd server analysis pipeline logic

commit bdf0121929d6941cbba565055f37df7970925c79
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 11:30:17 2022 +0200

    update analysis pipline logic to use imported pdf2image

commit f7cef98d5e6d7b95517bbd047dd3e958acebb3d8
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 2 11:04:34 2022 +0200

    add pdf2image as git submodule
master_37
2022-08-02 13:36:50 +02:00
Isaac Riley
edbda58837 Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis master_36 2022-08-02 12:13:02 +02:00
Isaac Riley
beb40da3b1 Pull request #22: add single-cell filtering to table parsing and increase tolerance parameter to 7; refactor postprocessing to use the Rectangles data structure
Merge in RR/cv-analysis from remove_isolated to master

Squashed commit of the following:

commit 2613ed1615d1b69b3e4f2acea197993a91d00561
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Aug 2 10:17:33 2022 +0200

    add single-cell filtering to table parsing and increase tolerance parameter to 7; refactored postprocessing to use the Rectangles data structure
master_35
2022-08-02 10:54:13 +02:00
Isaac Riley
b33dcd83a5 Revert "Pull request #21: move rotation logic to before cv-analysis, so that cv-analysis only needs to operate on portrait images and matrix rotation logic can be dropped"
This reverts commit de921e308f7e0c6d5686b14ca132910bce0bad17.
2022-07-29 08:50:06 +02:00
Julius Unverfehrt
de921e308f Pull request #21: move rotation logic to before cv-analysis, so that cv-analysis only needs to operate on portrait images and matrix rotation logic can be dropped
Merge in RR/cv-analysis from rotation-logic-refactor to master

Squashed commit of the following:

commit 684dd140cbfc9fbebe9beb8c13b52a2d131c9932
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jul 27 14:22:58 2022 +0200

    move rotation logic to before cv-analysis, so that cv-analysis only needs to operate on portrait images and matrix rotation logic can be dropped
0.2.0 master_33
2022-07-27 14:28:21 +02:00
Isaac Riley
9d98945ff9 Pull request #20: New pyinfra
Merge in RR/cv-analysis from new_pyinfra to master

Squashed commit of the following:

commit f7a01a90aad1c402ac537de5bdf15df628ad54df
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jul 27 10:40:59 2022 +0200

    fix typo

commit ff4d549fac5b612c2d391ae85823c5eca1e91916
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jul 27 10:34:04 2022 +0200

    adjust build scripts for new pyinfra

commit ecd70f60d46406d8b6cc7f36a1533d706c917ca8
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jul 27 09:42:55 2022 +0200

    simplify logging by using default configurations

commit 20193c14c940eed2b0a7a72058167e26064119d0
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Jul 26 17:16:57 2022 +0200

    tidy-up, refactor config logic to not dependent on external files

commit d8069cd4d404a570bb04a04278161669d1c83332
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Jul 26 15:14:59 2022 +0200

    update pyinfra

commit c3bc11037cca9baf016043ab997c566f5b4a2586
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Jul 26 15:09:14 2022 +0200

    repair tests

commit 6f4e4f2863ee16ae056c1d432f663858c5f10221
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Jul 26 14:52:38 2022 +0200

    updated server logic to work with new pyinfra; update scripts for pyinfra as submodule

commit 2a18dba81de5ee84d0bdf0e77f478693e8d8aef4
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Jul 26 14:10:41 2022 +0200

    formatting

commit d87ce9328de9aa2341228af9b24473d5e583504e
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Tue Jul 26 14:10:11 2022 +0200

    make server logic compatible with new pyinfra
master_32
2022-07-27 10:50:10 +02:00
Isaac Riley
1618909d8e Merge branch 'master' of ssh://git.iqser.com:2222/rr/cv-analysis 2022-07-26 13:13:29 +02:00
Isaac Riley
29c8d204e1 small fix to annotate.py 2022-07-26 13:03:34 +02:00
Julius Unverfehrt
a871fa3bd3 Pull request #19: Refactor evaluate
Merge in RR/cv-analysis from refactor-evaluate to master

Squashed commit of the following:

commit cde03a492452610322f8b7d3eb804a51afb76d81
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jul 22 12:37:36 2022 +0200

    add optional show analysis metadata dict

commit fb8bb9e2afa7767f2560f865516295be65f97f20
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jul 22 12:13:18 2022 +0200

    add script to evaluate runtime per page for all cv-analysis operations for multiple PDFs

commit 721e823e2ec38aae3fea51d01e2135fc8f228d94
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jul 22 10:30:31 2022 +0200

    refactor

commit a453753cfa477e162e5902ce191ded61cb678337
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jul 22 10:19:24 2022 +0200

    add logic to transform result coordinates accordingly to page rotation, update annotation script to use this logic

commit 71c09758d0fb763a2c38c6871e1d9bf51f2e7c41
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jul 21 15:57:49 2022 +0200

    introduce pipeline for image conversion, analysis and result formatting

commit aef252a41b9658dd0c4f55aa2d9f84de933586e0
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jul 21 15:57:38 2022 +0200

    introduce pipeline for image conversion, analysis and result formatting
master_26
2022-07-22 15:11:40 +02:00
Julius Unverfehrt
e7b28f5bda Pull request #18: Remove pil
Merge in RR/cv-analysis from remove_pil to master

Squashed commit of the following:

commit 83c8d88f3d48404251470176c70979ee75ae068b
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jul 21 10:51:51 2022 +0200

    remove deprecated server tests

commit cebc03b5399ac257a74036b41997201f882f5b74
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jul 21 10:51:08 2022 +0200

    remove deprecated server tests

commit ce2845b0c51f001b7b5b8b195d6bf7e034ec4e39
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jul 20 17:05:00 2022 +0200

    repair tests to work without pillow WIP

commit 023fdab8322f28359a24c63e32635a3d0deccbe4
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Wed Jul 20 16:40:36 2022 +0200

    fixed typo

commit 33850ca83a175f74789ae6b9bebd057ed84b7fb3
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Wed Jul 20 16:38:37 2022 +0200

    fixed import from refactored open_img.py

commit dbc6d345f074e538948e2c4f94ebed8a5ef520bc
Author: Isaac Riley <Isaac.Riley@iqser.com>
Date:   Wed Jul 20 16:32:42 2022 +0200

    removed PIL from production code, now inly in scripts
2022-07-21 13:25:00 +02:00