9 Commits

Author SHA1 Message Date
Julius Unverfehrt
98dc001123 revert adhoc figure detection changes
- revert pipeline and serve logic to pre figure detection data for image
extraction changes: figure detection data as input not supported for now
2023-01-30 12:41:22 +01:00
Julius Unverfehrt
520eee26e3 Pull request #22: Integrate image extraction new pyinfra
Merge in RR/image-prediction from integrate-image-extraction-new-pyinfra to master

Squashed commit of the following:

commit 8470c065c71ea2a985aadfc399fb32c693e3a90f
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Aug 18 09:19:52 2022 +0200

    add key script

commit 8f6eb1e79083fb32fb7bedac640c10b6fd411899
Merge: 27fd7de c1b9629
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Aug 18 09:17:50 2022 +0200

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/image-prediction into integrate-image-extraction-new-pyinfra

commit 27fd7de39a59d0d88fbddb471dd7797b61223ece
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 17 13:15:09 2022 +0200

    update pyinfra

commit ca58f85642598dc15e286074982e7cedae9a1355
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Aug 16 16:16:10 2022 +0200

    update pdf2image-service

commit f43795cee0e211e14ac5f9296b01d440ae759c55
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Aug 15 10:32:02 2022 +0200

    update pipeline script to also work with figure detection metadata

commit 2b2da1b60ce56fb006cf2f6b65aeda9774391b2a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Aug 12 13:37:48 2022 +0200

    add new pyinfra, add optional image classifcation under key dataCV if figure metadata is present on storage

commit bae25bedbd3a262a9d00e18a1b19f4ee6f1eb924
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 10 13:27:41 2022 +0200

    tidy-up

commit 287b0ebc8a952e506185d13508eaa386d0420704
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 10 12:57:35 2022 +0200

    update server logic for new pyinfra, add extraction from scanned PDF with figure detection logic

commit 3225cefaa25e4559b105397bc06c867a22806ba8
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 10 10:37:31 2022 +0200

    integrate new pyinfra logic

commit 46926078342b0680a7416560bb69bec037cf8038
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 3 13:15:27 2022 +0200

    add image extraction for scanned PDFs WIP

commit 1b3b11b6f9044d44cb9a822a78197a2ebc6f306a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Aug 3 09:41:06 2022 +0200

    add pyinfra and pdf2image as git submodule
2022-08-18 09:20:48 +02:00
Matthias Bisping
ddd8d4685e Pull request #9: Tdd refactoring
Merge in RR/image-prediction from tdd_refactoring to master

Squashed commit of the following:

commit f6c64430007590f5d2b234a7f784e26025d06484
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 12:18:47 2022 +0200

    renaming

commit 8f40b51282191edf3e2a5edcd6d6acb388ada453
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 12:07:18 2022 +0200

    adjusted expetced output for alpha channel in response

commit 7e666302d5eadb1e84b70cae27e8ec6108d7a135
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:52:51 2022 +0200

    added alpha channel check result to response

commit a6b9f64b51cd888fc0c427a38bd43ae2ae2cb051
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:27:57 2022 +0200

    readme updated

commit 0d06ad657e3c21dcef361c53df37b05aba64528b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:19:35 2022 +0200

    readme updated and config

commit 75748a1d82f0ebdf3ad7d348c6d820c8858aa3cb
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:19:26 2022 +0200

    refactoring

commit 60101337828d11f5ee5fed0d8c4ec80cde536d8a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:18:23 2022 +0200

    multiple reoutes for prediction

commit c8476cb5f55e470b831ae4557a031a2c1294eb86
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Apr 25 11:17:49 2022 +0200

    add banner.txt to container

commit 26ef5fce8a9bc015f1c35f32d40e8bea50a96454
Author: Matthias Bisping <Matthias.Bisping@iqser.com>
Date:   Mon Apr 25 10:08:49 2022 +0200

    Pull request #8: Pipeline refactoring

    Merge in RR/image-prediction from pipeline_refactoring to tdd_refactoring

    Squashed commit of the following:

    commit 6989fcb3313007b7eecf4bba39077fcde6924a9a
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 09:49:49 2022 +0200

        removed obsolete module

    commit 7428aeee37b11c31cffa597c85b018ba71e79a1d
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 09:45:45 2022 +0200

        refactoring

    commit 0dcd3894154fdf34bd3ba4ef816362434474f472
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Mon Apr 25 08:57:21 2022 +0200

        refactoring; removed obsolete extractor-classifier

    commit 1078aa81144f4219149b3fcacdae8b09c4b905c0
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:18:10 2022 +0200

        removed obsolete imports

    commit 71f61fc5fc915da3941cf5ed5d9cc90fccc49031
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:16:25 2022 +0200

        comment changed

    commit b582726cd1de233edb55c5a76c91e99f9dd3bd13
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Fri Apr 22 17:12:11 2022 +0200

        refactoring

    commit 8abc9010048078868b235d6793ac6c8b20abb985
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:25:47 2022 +0200

        formatting

    commit 2c87c419fe3185a25c27139e7fcf79f60971ad24
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:24:05 2022 +0200

        formatting

    commit 50b161192db43a84464125c6d79650225e1010d6
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:20:18 2022 +0200

        refactoring

    commit 9a1446cccfa070852a5d9c0bdbc36037b82541fc
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 21:04:57 2022 +0200

        refactoring

    commit 6c10b55ff8e61412cb2fe5a5625e660ecaf1d7d1
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 19:48:05 2022 +0200

        refactoring

    commit 72e785e3e31c132ab352119e9921725f91fac9e2
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Thu Apr 21 19:43:39 2022 +0200

        refactoring

    commit f036ee55e6747daf31e3929bdc2d93dc5f2a56ca
    Author: Matthias Bisping <matthias.bisping@iqser.com>
    Date:   Wed Apr 20 18:30:41 2022 +0200

        refactoring pipeline WIP

commit 120721f5f1a7e910c0c2ebc79dc87c2908794c80
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:39:58 2022 +0200

    rm debug ls

commit 81226d4f8599af0db0e9718fbb1789cfad91a855
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:28:27 2022 +0200

    no compose down

commit 943f7799d49b6a6b0fed985a76ed4fe725dfaeef
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 15:22:17 2022 +0200

    coverage combine

commit d4cd96607157ea414db417cfd7133f56cb56afe1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 14:43:09 2022 +0200

    model builder path in mlruns adjusted

commit 5b90bb47c3421feb6123c179eb68d1125d58ff1e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:56:58 2022 +0200

    dvc pull in test running script

commit a935cacf2305a4a78a15ff571f368962f4538369
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:50:36 2022 +0200

    no clean working dir

commit ba09df7884485b8ab8efbf42a8058de9af60c75c
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 10:43:22 2022 +0200

    debug ls

commit 71263a9983dbfe2060ef5b74de7cc2cbbad43416
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Apr 20 09:11:03 2022 +0200

    debug ls

commit 41fbadc331e65e4ffe6d053e2d925e5e0543d8b7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 20:08:08 2022 +0200

    debug echo

commit bb19698d640b3a99ea404e5b4b06d719a9bfe9e9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 20:01:59 2022 +0200

    skip server predict test

commit 5094015a87fc0976c9d3ff5d1f4c6fdbd96b7eae
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Apr 19 19:05:50 2022 +0200

    sonar stage after build stage

... and 253 more commits
2022-04-25 12:25:41 +02:00
Julius Unverfehrt
eb18ae8719 Pull request #5: Tests&Fixes
Merge in RR/image-prediction from tests to master

Squashed commit of the following:

commit 1776e3083c97025e699d579f936dd0cc6e1fe152
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 13:54:27 2022 +0100

    blacckkkyykykykyk

commit 4c9e6c38bdcea7d81008bf9dfcfcdd19d199da6a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 13:53:40 2022 +0100

    add predicting as subprocess, add workaround for keras not working if the model was loaded in different process

commit 530de2ff8979c70aa22f06edf297864787e0cc79
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 13:36:23 2022 +0100

    refactor

commit 130d0e8b23e0375a6fd240ac8aa00492c341a716
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 13:34:54 2022 +0100

    add minimal not working example for keras bug in multiprocess process

commit 2589598b052f680fd702df4f60d56a55778474a9
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 11:13:45 2022 +0100

    test

commit eb6f211f02bc184e7f92d6b4d53c91da34ab9f2f
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 11:07:32 2022 +0100

    hardcoded test

commit 3e9bfac5cf9b2e09340e2c2c5b24a800925bcd60
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 11:01:21 2022 +0100

    test

commit 3d9c4d8856522cc2a22b2a7b9ea64d34629eb2c1
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 10:57:03 2022 +0100

    change test

commit 58ca784d6c56fd63734062d0c40b6b39550cf7d7
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 10:21:38 2022 +0100

    fix test

commit 6faad5ad5b6ef59bb5ef701b57d4c4addd17de0e
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Mar 21 10:00:28 2022 +0100

    add predictor test

commit 3fbca0ac23821568a8afa904a8fb33ab0679f129
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 18 13:04:13 2022 +0100

    refactor folder structure

commit 90e3058c7124394a9f229d50278e57194f3d875d
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 18 12:58:02 2022 +0100

    add response test

commit 2a2deffd0b461ec5161009b3923623152f4c8f44
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 18 12:56:32 2022 +0100

    add test infrastructure
2022-03-23 11:49:05 +01:00
cdietrich
372d6645d7 tidy up repo 2022-03-02 10:15:08 +01:00
cdietrich
684aca364f adapt service-container to image-service-v2 2022-03-01 14:17:37 +01:00
Matthias Bisping
3e2cb94060 Pull request #9: Docker image tuning, batching of pdf pages and misc other
Merge in RR/fb_detr_prediction_container from docker-image-tuning to master

Squashed commit of the following:

commit 9b30e6317aaf892fcb6f87275d03e2efb76954bf
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Feb 21 15:17:01 2022 +0100

    applied black

commit 84a57ac29723910dbc2c4d8ccce58c9d3131a305
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Feb 21 15:15:15 2022 +0100

    refactorig of tqdm

commit b26c52765c58125826099072d510a39baabce73e
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Mon Feb 21 14:52:11 2022 +0100

    correcting versioning of docker-compose

commit 23752eec0d95cc543f15a86c78bd8531ebfdde7d
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Feb 21 14:49:26 2022 +0100

    put tqdm progress in different place

commit e2e109ea7125c90f5b15ec374f3cbfef41e2ee9e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Feb 21 11:51:09 2022 +0100

    fixed batching index bug

commit 6ca508ac55dd02ded356617653f580099e1cf186
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Feb 21 11:37:34 2022 +0100

    batching WIP

commit 0ceb7c1415b10230397f4860ac4e314d44bfbfd1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 19:21:02 2022 +0100

    debug mode for webserver renamed

commit 617f07a0296ad3efc85b6ee52d1641cdfa22d3d3
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 18:25:39 2022 +0100

    refactoring, better logging, added compose file for local testing

commit a24f799614e22481dd20b578c354e33474bec5c0
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 17:31:14 2022 +0100

    updated submodule

commit 67b64606e081373e5c30ccf5bfafcb91dcc9a74e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 17:29:20 2022 +0100

    cleanup: better config; refactoring; renaming

commit c3a1ab560879d6a1e6ce003c74a07d62175316f7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 15:55:35 2022 +0100

    tweaked dockerfiles

commit 43f7a32265243bc0f110bd307325b5404e8726a8
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 18 15:02:49 2022 +0100

    added dockerignore
2022-02-21 15:36:38 +01:00
Matthias Bisping
8ebbe0e6a7 Pull request #2: Non max supprs
Merge in RR/fb_detr_prediction_container from non_max_supprs to master

Squashed commit of the following:

commit 9cc31b70e39412b3613a117228554608d947dbb5
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 4 17:41:00 2022 +0100

    refactoring, renaming

commit ebc37299df598b71f7569d8e8473bdb66bbbbd1a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 4 17:34:26 2022 +0100

    renaming

commit d694866e1e98e6129f37eaf4c1950b962fed437f
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 4 17:33:07 2022 +0100

    applied black

commit 381fe2dbf5d88f008d87bd807b84174376c5bcfe
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Feb 4 17:32:22 2022 +0100

    duplicate detection removal completed

commit ef2bab300322da3b12326d470f1c41263779e4a0
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Fri Feb 4 09:58:49 2022 +0100

    box merging algo  WIP

commit d770e56a7f31a28dea635816cae3b7b75fed0e24
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Fri Feb 4 09:37:17 2022 +0100

    refactor & box dropping working but algo is faulty & drops too much WIP

commit 289848871caadb4438f889b8a030f30cfb64201a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 23:56:04 2022 +0100

    non max supprs WIP

commit 2f1ec100b2d33409e9178af8d53218b57d9bb0e2
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 13:32:22 2022 +0100

    changed Flask to not listen on public IP
2022-02-04 17:45:55 +01:00
Matthias Bisping
e4dc6631b5 Pull request #1: Setup
Merge in RR/fb_detr_prediction_container from setup to master

Squashed commit of the following:

commit 7fae4878d4250676367b7201fa163a4b67f79f84
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 11:22:12 2022 +0100

    readded annotation to client

commit ff788030f6b3b342919a7fd31dfa66940033d7e1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 11:15:16 2022 +0100

    applied black

commit 3521444f678950a2772b725c6964751e0e655736
Merge: 4080aff 51d6597
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 10:39:11 2022 +0100

    Merge branch 'setup' of ssh://git.iqser.com:2222/rr/fb_detr_prediction_container into setup

commit 4080affd21a02ad32c61fbd2027511f51a202d63
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Feb 3 10:39:02 2022 +0100

    added poppler-utils download to Dockerfile, since pdf2image only is a wrapper for it

commit 51d6597b056ae9ac693280f65a3f37d46b1276cf
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Thu Feb 3 09:43:35 2022 +0100

    Structure change for local backbone lookup (working now)

commit ac314d5148d6e026c67f00df45a8bbc70c15b52d
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Thu Feb 3 09:35:41 2022 +0100

    env bug fixed

commit 1c3221fe4956911b29fd8fede8d07dcdefad06d8
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Thu Feb 3 09:23:55 2022 +0100

    ENV correctly set now

commit 58069440583f1f78cfb2fb796fa4dc4a63e2916a
Author: Julius Unverfehrt <Julius.Unverfehrt@iqser.com>
Date:   Thu Feb 3 08:41:29 2022 +0100

    ENV for local torch model lookup set

commit f0501cf0bf904793e8e04afbd3d80ee84af9d981
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 18:28:44 2022 +0100

    changed host and port for flask

commit 986fda22f6656b10930628d0d284995b33ea2df5
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 17:33:07 2022 +0100

    added debug webserver method

commit 64b857ce53757ec2b7e7c327962fa65b551603a0
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 16:59:11 2022 +0100

    moved utils into module; fixed open-cv (maybe)

commit c62ada183135e12b41a29c6822472e33698f947f
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 15:55:10 2022 +0100

    made bash scripts executable

commit 982bdd7503c14fcf1776ae10c38589475199545e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 15:35:16 2022 +0100

    service building logic added (WIP)

commit 46e5e3b8e67e54ecedaeee4765a3437f08fa4b17
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 14:37:28 2022 +0100

    applied black

commit ad93130e66d2e87bc86b2bf1de6234f3c037df48
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 14:36:09 2022 +0100

    fixed formatting (w, h -> x2, y2); added drawing logic to caller mock

commit df76f033599e66aaa52143f5e2b156530f643df9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 13:54:34 2022 +0100

    page indices in predictions

commit 5e87c57dff752419486d1a44de9a734e3f840816
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 13:17:34 2022 +0100

    service main loop WIP (working in basic version)

commit ba5ec3d57621d090201413309126955940602be9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 13:03:52 2022 +0100

    service main loop WIP

commit 77266f6982ec826eadcdd8a18c5ccf0fc380611b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 11:24:27 2022 +0100

    fixed bug for self.classes == None

commit 858ef7589d6914ad503660a3ddc5e75bf72a6bb7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Feb 2 11:09:11 2022 +0100

    removed 'postprocessors' argument and attribute

... and 32 more commits
2022-02-03 11:44:11 +01:00