Marmelator/image-classification-service
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 29 Wiki Activity
188 Commits 4 Branches 400 Tags
Commit Graph

29 Commits

Author SHA1 Message Date
Julius Unverfehrt
79455f0dd6 Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction into RED-6189-bugfix 2023-02-13 17:23:07 +01:00
Julius Unverfehrt
2bc9c24f6a revert refactoring changes 
- revert functional refactoring changes to be able
to determine where the error described in the ticket comes from
- change array normalization to dimensionally
sparse arrays to reduce memory consumption
 2023-02-13 13:53:35 +01:00
Julius Unverfehrt
ea301b4df2 Pull request #40: replace trace log level by debug 
Merge in RR/image-prediction from adjust-falsy-loglevel to master

Squashed commit of the following:

commit 66794acb1a64be6341f98c7c0ce0bc202634a9f4
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Feb 10 10:15:41 2023 +0100

    replace trace log level by debug

    - trace method is not supported by buld-in logging module
 2023-02-10 10:18:38 +01:00
Matthias Bisping
5cdf93b923 Pull request #39: RED-6084 Improve image extraction speed 
Merge in RR/image-prediction from RED-6084-adhoc-scanned-pages-filtering-refactoring to master

Squashed commit of the following:

commit bd6d83e7363b1c1993babcceb434110a6312c645
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Thu Feb 9 16:08:25 2023 +0100

    Tweak logging

commit 55bdd48d2a3462a8b4a6b7194c4a46b21d74c455
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Thu Feb 9 15:47:31 2023 +0100

    Update dependencies

commit 970275b25708c05e4fbe78b52aa70d791d5ff17a
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Thu Feb 9 15:35:37 2023 +0100

    Refactoring

    Make alpha channel check monadic to streamline error handling

commit e99e97e23fd8ce16f9a421d3e5442fccacf71ead
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Tue Feb 7 14:32:29 2023 +0100

    Refactoring

    - Rename
    - Refactor image extraction functions

commit 76b1b0ca2401495ec03ba2b6483091b52732eb81
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Tue Feb 7 11:55:30 2023 +0100

    Refactoring

commit cb1c461049d7c43ec340302f466447da9f95a499
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Tue Feb 7 11:44:01 2023 +0100

    Refactoring

commit 092069221a85ac7ac19bf838dcbc7ab1fde1e12b
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Tue Feb 7 10:18:53 2023 +0100

    Add to-do

commit 3cea4dad2d9703b8c79ddeb740b66a3b8255bb2a
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Tue Feb 7 10:11:35 2023 +0100

    Refactoring

    - Rename
    - Add typehints everywhere

commit 865e0819a14c420bc2edff454d41092c11c019a4
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 19:38:57 2023 +0100

    Add type explanation

commit 01d3d5d33f1ccb05aea1cec1d1577572b1a4deaa
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 19:37:49 2023 +0100

    Formatting

commit dffe1c18fc3a322a6b08890d4438844e8122faaf
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 19:34:13 2023 +0100

    [WIP] Either refactoring

    Add alternative formulation for monadic chain

commit 066cf17add404a313520cd794c06e3264cf971c9
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 18:40:30 2023 +0100

    [WIP] Either refactoring

commit f53f0fea298cdab88deb090af328b34d37e0198e
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 18:18:34 2023 +0100

    [WIP] Either refactoring

    Propagate error and metadata

commit 274a5f56d4fcb9c67fac5cf43e9412ec1ab5179e
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 17:51:35 2023 +0100

    [WIP] Either refactoring

    Fix test assertion

commit 3235a857f6e418e50484cbfff152b0f63efb2f53
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 16:57:31 2023 +0100

    [WIP] Either-refactoring

    Replace Maybe with Either to allow passing on error information or
    metadata which otherwise get sucked up by Nothing.

commit 89989543d87490f8b20a0a76055605d34345e8f4
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 16:12:40 2023 +0100

    [WIP] Monadic refactoring

    Integrate image validation step into monadic chain.

    At the moment we lost the error information through this. Refactoring to
    Either monad can bring it back.

commit 022bd4856a51aa085df5fe983fd77b99b53d594c
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 15:16:41 2023 +0100

    [WIP] Monadic refactoring

commit ca3898cb539607c8c3dd01c57e60211a5fea8a7d
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 15:10:34 2023 +0100

    [WIP] Monadic refactoring

commit d8f37bed5cbd6bdd2a0b52bae46fcdbb50f9dff2
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 15:09:51 2023 +0100

    [WIP] Monadic refactoring

commit 906fee0e5df051f38076aa1d2725e52a182ade13
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Feb 6 15:03:35 2023 +0100

    [WIP] Monadic refactoring

... and 35 more commits
 2023-02-10 08:33:13 +01:00
Julius Unverfehrt
4d43e385c5 replace image extraction logic final 2023-02-06 09:43:28 +01:00
Julius Unverfehrt
bd0279ddd1 introduce normalizing function for image extraction 2023-02-03 12:25:27 +01:00
Julius Unverfehrt
2995d5ee48 refactoring 2023-02-03 11:14:14 +01:00
Julius Unverfehrt
eff1bb4124 adjust behavior of filtering of invalid images 2023-02-03 09:04:02 +01:00
Julius Unverfehrt
978f48e8f9 add ad hoc logic for bad xref handling 2023-02-02 15:39:44 +01:00
Julius Unverfehrt
94652aafe4 beautify 2023-02-02 15:26:33 +01:00
Julius Unverfehrt
c4416636c0 beautify 2023-02-02 14:10:32 +01:00
Julius Unverfehrt
c0b41e77b8 implement ad hoc channel count detection for new image extraction 2023-02-02 13:57:56 +01:00
Julius Unverfehrt
73f7491c8f improve performance 
- disable scanned page filter, since dropping these disables the
computation of the images hash and the frontend OCR hint, which are both
wanted
- optimize image extraction by using arrays instead of byte streams for
the conversion to PIL images
 2023-02-02 13:37:03 +01:00
Julius Unverfehrt
2385584dcb refactor scanned page filtering 2023-02-01 15:49:36 +01:00
Julius Unverfehrt
b880e892ec refactor scanned page filtering WIP 2023-02-01 15:47:40 +01:00
Julius Unverfehrt
8c7349c2d1 refactor scanned page filtering WIP 2023-02-01 15:36:16 +01:00
Julius Unverfehrt
c55777e339 refactor scanned page filtering WIP 2023-02-01 15:16:12 +01:00
Julius Unverfehrt
0f440bdb09 refactor scanned page filtering WIP 2023-02-01 15:14:27 +01:00
Julius Unverfehrt
436a32ad2b refactor scanned page filtering WIP 2023-02-01 15:07:35 +01:00
Julius Unverfehrt
9ec6cc19ba refactor scanned page filtering WIP 2023-02-01 14:53:26 +01:00
Julius Unverfehrt
2d385b0a73 refactor scanned page filtering WIP 2023-02-01 14:38:55 +01:00
Julius Unverfehrt
5bd5e0cf2b refactor 
- reduce code duplication by adapting functions of the module
- use the modules enums for image metadata
- improve readabilty of the scanned page detection heuristic
 2023-02-01 12:43:59 +01:00
Julius Unverfehrt
876260f403 improve the readability of variable names and docstrings 2023-02-01 10:08:36 +01:00
Julius Unverfehrt
368c54a8be clean-up filter logic 
- Logic adapted so that it can potentially be
easily removed again from the extraction logic
 2023-02-01 08:49:30 +01:00
Julius Unverfehrt
1490d27308 introduce adhoc filter for scanned pages 2023-01-31 17:18:28 +01:00
Julius Unverfehrt
521222eb96 Pull request #31: RED-5202 port hotfixes 
Merge in RR/image-prediction from RED-5202-port-hotfixes to master

Squashed commit of the following:

commit c1b92270354c764861da0f7782348e9cd0725d76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Sep 12 13:28:44 2022 +0200

    fixed statefulness issue with os.environ in tests

commit ad9c5657fe93079d5646ba2b70fa091e8d2daf76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Mon Sep 12 13:04:55 2022 +0200

    - Adapted response formatting logic for threshold maps passed via env vars.
    - Added test for reading threshold maps and values from env vars.

commit c60e8cd6781b8e0c3ec69ccd0a25375803de26f0
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Sep 12 11:38:01 2022 +0200

    add parser for environment variables WIP

commit 101b71726c697f30ec9298ba62d2203bd7da2efb
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Sep 12 09:52:33 2022 +0200

    Add typehints, make custom page quotient breach function private since the intention of outsourcing it from build_image_info is to make it testable seperately

commit 04aee4e62781e78cd54c6d20e961dcd7bf1fc081
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Sep 12 09:25:59 2022 +0200

    DotIndexable default get method exception made more specific

commit 4584e7ba66400033dc5f1a38473b644eeb11e67c
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Sep 12 08:55:05 2022 +0200

    RED-5202 port temporary broken image handling so the hotfix won't be lost by upgrading the service. A proper solution is still desirable (see RED-5148)

commit 5f99622646b3f6d3a842aebef91ff8e082072cd6
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Sep 12 08:47:02 2022 +0200

    RED-5202 add per class customizable max image to page quotient setting for signatures, default is 0.4. Can be overwritten by , set to null to use default value or set to value that should be used.
 2022-09-12 14:49:56 +02:00
Matthias Bisping
f787b957f8 Pull request #18: Docstrfix 
Merge in RR/image-prediction from docstrfix to master

Squashed commit of the following:

commit 8ccb07037074cc88ba5b72e4bedd5bc346eb0256
Merge: 77cd0a8 5d611d5
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:50:52 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction into docstrfix

commit 77cd0a860a69bfb8f4390dabdca23455b340bd9e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:46:25 2022 +0200

    fixed docstring

commit eb53464ca9f1ccf881d90ece592ad50226decd7a
Merge: 4efb9c7 fd0e4dc
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jun 21 15:22:03 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction

commit 4efb9c79b10f23fa556ce43c8e7f05944dae1af6
Merge: 84a8b0a 9f18ef9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu May 12 11:51:30 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction

commit 84a8b0a290081616240c3876f8db8a1ae8592096
Merge: 1624ee4 6030f40
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu May 12 10:18:56 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction

commit 1624ee40376b84a4519025343f913120c464407a
Author: Matthias Bisping <Matthias.Bisping@iqser.com>
Date:   Mon Apr 25 16:51:13 2022 +0200

    Pull request #11: fixed assignment

    Merge in RR/image-prediction from image_prediction_service_overhaul_xref_and_empty_result_fix_fix to master

    Squashed commit of the following:

    commit 7312e57d1127b081bfdc6e96311e8348d3f8110d
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 16:45:12 2022 +0200

        logging setup changed

    commit 955e353d74f414ee2d57b234bdf84d32817d14bf
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 16:37:52 2022 +0200

        fixed assignment
 2022-07-04 11:57:01 +02:00
Matthias Bisping
1e65d672d7 Pull request #10: fixed check for analysis result validity 
Merge in RR/image-prediction from result_check_fix to master

Squashed commit of the following:

commit 8352e657ef1f399ca0fe6a89e7a4c7fc4bd0701d
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 16:23:24 2022 +0200

    logging setup changed

commit 956706378b7d7f6daa574b86eb29636797c05bba
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 16:17:19 2022 +0200

    error handling for bad xrefs

commit e7d229c0d70574cae316a841ab1377fae625ab15
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 14:07:19 2022 +0200

    fixed check for analysis result validity
 2022-04-25 16:28:26 +02:00
Matthias Bisping
ddd8d4685e Pull request #9: Tdd refactoring 
Merge in RR/image-prediction from tdd_refactoring to master

Squashed commit of the following:

commit f6c64430007590f5d2b234a7f784e26025d06484
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 12:18:47 2022 +0200

    renaming

commit 8f40b51282191edf3e2a5edcd6d6acb388ada453
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 12:07:18 2022 +0200

    adjusted expetced output for alpha channel in response

commit 7e666302d5eadb1e84b70cae27e8ec6108d7a135
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:52:51 2022 +0200

    added alpha channel check result to response

commit a6b9f64b51cd888fc0c427a38bd43ae2ae2cb051
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:27:57 2022 +0200

    readme updated

commit 0d06ad657e3c21dcef361c53df37b05aba64528b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:19:35 2022 +0200

    readme updated and config

commit 75748a1d82f0ebdf3ad7d348c6d820c8858aa3cb
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:19:26 2022 +0200

    refactoring

commit 60101337828d11f5ee5fed0d8c4ec80cde536d8a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:18:23 2022 +0200

    multiple reoutes for prediction

commit c8476cb5f55e470b831ae4557a031a2c1294eb86
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:17:49 2022 +0200

    add banner.txt to container

commit 26ef5fce8a9bc015f1c35f32d40e8bea50a96454
Author: Matthias Bisping <Matthias.Bisping@iqser.com>
Date:   Mon Apr 25 10:08:49 2022 +0200

    Pull request #8: Pipeline refactoring

    Merge in RR/image-prediction from pipeline_refactoring to tdd_refactoring

    Squashed commit of the following:

    commit 6989fcb3313007b7eecf4bba39077fcde6924a9a
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 09:49:49 2022 +0200

        removed obsolete module

    commit 7428aeee37b11c31cffa597c85b018ba71e79a1d
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 09:45:45 2022 +0200

        refactoring

    commit 0dcd3894154fdf34bd3ba4ef816362434474f472
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 08:57:21 2022 +0200

        refactoring; removed obsolete extractor-classifier

    commit 1078aa81144f4219149b3fcacdae8b09c4b905c0
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:18:10 2022 +0200

        removed obsolete imports

    commit 71f61fc5fc915da3941cf5ed5d9cc90fccc49031
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:16:25 2022 +0200

        comment changed

    commit b582726cd1de233edb55c5a76c91e99f9dd3bd13
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:12:11 2022 +0200

        refactoring

    commit 8abc9010048078868b235d6793ac6c8b20abb985
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:25:47 2022 +0200

        formatting

    commit 2c87c419fe3185a25c27139e7fcf79f60971ad24
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:24:05 2022 +0200

        formatting

    commit 50b161192db43a84464125c6d79650225e1010d6
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:20:18 2022 +0200

        refactoring

    commit 9a1446cccfa070852a5d9c0bdbc36037b82541fc
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:04:57 2022 +0200

        refactoring

    commit 6c10b55ff8e61412cb2fe5a5625e660ecaf1d7d1
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 19:48:05 2022 +0200

        refactoring

    commit 72e785e3e31c132ab352119e9921725f91fac9e2
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 19:43:39 2022 +0200

        refactoring

    commit f036ee55e6747daf31e3929bdc2d93dc5f2a56ca
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Wed Apr 20 18:30:41 2022 +0200

        refactoring pipeline WIP

commit 120721f5f1a7e910c0c2ebc79dc87c2908794c80
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:39:58 2022 +0200

    rm debug ls

commit 81226d4f8599af0db0e9718fbb1789cfad91a855
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:28:27 2022 +0200

    no compose down

commit 943f7799d49b6a6b0fed985a76ed4fe725dfaeef
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:22:17 2022 +0200

    coverage combine

commit d4cd96607157ea414db417cfd7133f56cb56afe1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 14:43:09 2022 +0200

    model builder path in mlruns adjusted

commit 5b90bb47c3421feb6123c179eb68d1125d58ff1e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:56:58 2022 +0200

    dvc pull in test running script

commit a935cacf2305a4a78a15ff571f368962f4538369
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:50:36 2022 +0200

    no clean working dir

commit ba09df7884485b8ab8efbf42a8058de9af60c75c
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:43:22 2022 +0200

    debug ls

commit 71263a9983dbfe2060ef5b74de7cc2cbbad43416
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 09:11:03 2022 +0200

    debug ls

commit 41fbadc331e65e4ffe6d053e2d925e5e0543d8b7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 20:08:08 2022 +0200

    debug echo

commit bb19698d640b3a99ea404e5b4b06d719a9bfe9e9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 20:01:59 2022 +0200

    skip server predict test

commit 5094015a87fc0976c9d3ff5d1f4c6fdbd96b7eae
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 19:05:50 2022 +0200

    sonar stage after build stage

... and 253 more commits
 2022-04-25 12:25:41 +02:00