Integrate image validation step into monadic chain.
At the moment we lost the error information through this. Refactoring to
Either monad can bring it back.
- Rename metadata -> metadatum in some more places to make it clear that
it is the metadata of a single image in that context
- Re-order function definitions according to caller hierarchy
- disable scanned page filter, since dropping these disables the
computation of the images hash and the frontend OCR hint, which are both
wanted
- optimize image extraction by using arrays instead of byte streams for
the conversion to PIL images
- reduce code duplication by adapting functions of the module
- use the modules enums for image metadata
- improve readabilty of the scanned page detection heuristic
Merge in RR/image-prediction from fschulz/update-to-new-pyinfra-version to master
* commit 'd63f8c4eaf39ef7346188b585fb9d968de72db87':
update dependencies
Merge in RR/image-prediction from RED-5324 to master
* commit 'c72ef26a6caac8d87cdc08dd19dbe235247129d4':
RED-5324: Update pyinfra to include storage-region fix
Merge in RR/image-prediction from RED-4206-fix-unwanted-restart-bug to master
Squashed commit of the following:
commit 3dfe7b861816ef9019103e16a23efd97a08fb617
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Thu Sep 22 13:53:32 2022 +0200
RED-4206 wrap queue callback in process to manage memory allocation with the operating system and force deallocation after processing.
Merge in RR/image-prediction from RED-5202-port-hotfixes to master
Squashed commit of the following:
commit 9674901235264de6b74d679fd39a52775ac4aee1
Merge: ec2ab89 9763d2c
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:55:58 2022 +0200
Merge remote-tracking branch 'origin' into RED-5202-port-hotfixes
commit ec2ab890b8307942d147d6b8b236f6a3c1d0aebc
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:49:17 2022 +0200
swap case when the log is printed for env var parsing
commit aaa02ea35e9c1b3b307116d7e3e32c93fd79ef5d
Merge: 5d87066 521222e
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:28:39 2022 +0200
Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction into RED-5202-port-hotfixes
commit 5d87066b40b28f919b1346f5e5396b46445b4e00
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:25:01 2022 +0200
remove warning log for non existent non default env var
commit 23c61ef49ef918b29952150d4a6e61b99d60ac64
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:14:19 2022 +0200
make env var parser discrete
commit c1b92270354c764861da0f7782348e9cd0725d76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date: Mon Sep 12 13:28:44 2022 +0200
fixed statefulness issue with os.environ in tests
commit ad9c5657fe93079d5646ba2b70fa091e8d2daf76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date: Mon Sep 12 13:04:55 2022 +0200
- Adapted response formatting logic for threshold maps passed via env vars.
- Added test for reading threshold maps and values from env vars.
commit c60e8cd6781b8e0c3ec69ccd0a25375803de26f0
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 11:38:01 2022 +0200
add parser for environment variables WIP
commit 101b71726c697f30ec9298ba62d2203bd7da2efb
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 09:52:33 2022 +0200
Add typehints, make custom page quotient breach function private since the intention of outsourcing it from build_image_info is to make it testable seperately
commit 04aee4e62781e78cd54c6d20e961dcd7bf1fc081
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 09:25:59 2022 +0200
DotIndexable default get method exception made more specific
commit 4584e7ba66400033dc5f1a38473b644eeb11e67c
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 08:55:05 2022 +0200
RED-5202 port temporary broken image handling so the hotfix won't be lost by upgrading the service. A proper solution is still desirable (see RED-5148)
commit 5f99622646b3f6d3a842aebef91ff8e082072cd6
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 08:47:02 2022 +0200
RED-5202 add per class customizable max image to page quotient setting for signatures, default is 0.4. Can be overwritten by , set to null to use default value or set to value that should be used.
Merge in RR/image-prediction from RED-5202-port-hotfixes to master
Squashed commit of the following:
commit aaa02ea35e9c1b3b307116d7e3e32c93fd79ef5d
Merge: 5d87066 521222e
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:28:39 2022 +0200
Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction into RED-5202-port-hotfixes
commit 5d87066b40b28f919b1346f5e5396b46445b4e00
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:25:01 2022 +0200
remove warning log for non existent non default env var
commit 23c61ef49ef918b29952150d4a6e61b99d60ac64
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 15:14:19 2022 +0200
make env var parser discrete
commit c1b92270354c764861da0f7782348e9cd0725d76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date: Mon Sep 12 13:28:44 2022 +0200
fixed statefulness issue with os.environ in tests
commit ad9c5657fe93079d5646ba2b70fa091e8d2daf76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date: Mon Sep 12 13:04:55 2022 +0200
- Adapted response formatting logic for threshold maps passed via env vars.
- Added test for reading threshold maps and values from env vars.
commit c60e8cd6781b8e0c3ec69ccd0a25375803de26f0
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 11:38:01 2022 +0200
add parser for environment variables WIP
commit 101b71726c697f30ec9298ba62d2203bd7da2efb
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 09:52:33 2022 +0200
Add typehints, make custom page quotient breach function private since the intention of outsourcing it from build_image_info is to make it testable seperately
commit 04aee4e62781e78cd54c6d20e961dcd7bf1fc081
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 09:25:59 2022 +0200
DotIndexable default get method exception made more specific
commit 4584e7ba66400033dc5f1a38473b644eeb11e67c
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 08:55:05 2022 +0200
RED-5202 port temporary broken image handling so the hotfix won't be lost by upgrading the service. A proper solution is still desirable (see RED-5148)
commit 5f99622646b3f6d3a842aebef91ff8e082072cd6
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Mon Sep 12 08:47:02 2022 +0200
RED-5202 add per class customizable max image to page quotient setting for signatures, default is 0.4. Can be overwritten by , set to null to use default value or set to value that should be used.
