Marmelator/pyinfra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity
350 Commits 9 Branches 411 Tags
Commit Graph

350 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Julius Unverfehrt
a1bfec765c Pull request #43: Image prediction v2 support 
Merge in RR/pyinfra from image-prediction-v2-support to 2.0.0

Squashed commit of the following:

commit 37c536324e847357e86dd9b72d1e07ad792ed90f
Merge: 77d1db8 01bfb1d
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Jul 11 13:53:56 2022 +0200

    Merge branch '2.0.0' of ssh://git.iqser.com:2222/rr/pyinfra into image-prediction-v2-support

commit 77d1db8e8630de8822c124eb39f4cd817ed1d3e1
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Jul 11 13:07:41 2022 +0200

    add operation assignment via config if operation is not defined by caller

commit 36c8ca48a8c6151f713c093a23de110901ba6b02
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Jul 11 10:33:34 2022 +0200

    refactor nothing part 2

commit f6cd0ef986802554dd544b9b7a24073d3b3f05b5
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Mon Jul 11 10:28:49 2022 +0200

    refactor nothing

commit 1e70d49531e89613c70903be49290b94ee014f65
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jul 6 17:42:12 2022 +0200

    enable docker-compose fixture

commit 9fee32cecdd120cfac3e065fb8ad2b4f37b49226
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jul 6 17:40:35 2022 +0200

    added 'multi' key to actual operation configurations

commit 4287f6d9878dd361489b8490eafd06f81df472ce
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jul 6 16:56:12 2022 +0200

    removed debug prints

commit 23a533e8f99222c7e598fb0864f65e9aa3508a3b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jul 6 16:31:50 2022 +0200

    completed correcting / cleaning upload and download logic with regard to operations and ids. next: remove debug code

commit 33246d1ff94989d2ea70242c7ae2e58afa4d35c1
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jul 6 14:37:17 2022 +0200

    corrected / cleaned upload and download logic with regard to operations and ids

commit 7f2b4e882022c6843cb2f80df202caa495c54ee9
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jul 5 18:41:07 2022 +0200

    partially decomplected file descriptor manager from concrete and non-generic descriptor code

commit 40b892da17670dae3b8eba1700877c1dcf219852
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jul 5 09:53:46 2022 +0200

    typo

commit ec4fa8e6f4551ff1f8d4f78c484b7a260f274898
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jul 5 09:52:41 2022 +0200

    typo

commit 701b43403c328161fd96a73ce388a66035cca348
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 17:26:53 2022 +0200

    made adjustments for image classification with pyinfra 2.x; added related fixmes

commit 7a794bdcc987631cdc4d89b5620359464e2e018e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 13:05:26 2022 +0200

    removed obsolete imports

commit 3fc6a7ef5d0172dbce1c4292d245eced2f378b5a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:47:12 2022 +0200

    enable docker-compose fixture

commit 36d8d3bc851b06d94cf12a73048a00a67ef79c42
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:46:53 2022 +0200

    renaming

commit 3bf00d11cd041dff325b66f13fcd00d3ce96b8b5
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 30 12:47:57 2022 +0200

    refactoring: added cached pipeline factory

commit 90e735852af2f86e35be845fabf28494de952edb
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 13:47:08 2022 +0200

    renaming

commit 93b3d4b202b41183ed8cabe193a4bfa03f520787
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 13:25:03 2022 +0200

    further refactored server setup code: moving and decomplecting

commit 8b2ed83c7ade5bd811cb045d56fbfb0353fa385e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 12:53:09 2022 +0200

    refactored server setup code: factored out and decoupled operation registry and prometheus summary registry

... and 6 more commits
 2022-07-11 14:17:59 +02:00
Julius Unverfehrt
01bfb1d668 Pull request #40: 2.0.0 documentation 
Merge in RR/pyinfra from 2.0.0-documentation to 2.0.0

Squashed commit of the following:

commit 7a794bdcc987631cdc4d89b5620359464e2e018e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 13:05:26 2022 +0200

    removed obsolete imports

commit 3fc6a7ef5d0172dbce1c4292d245eced2f378b5a
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:47:12 2022 +0200

    enable docker-compose fixture

commit 36d8d3bc851b06d94cf12a73048a00a67ef79c42
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jul 4 11:46:53 2022 +0200

    renaming

commit 3bf00d11cd041dff325b66f13fcd00d3ce96b8b5
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 30 12:47:57 2022 +0200

    refactoring: added cached pipeline factory

commit 90e735852af2f86e35be845fabf28494de952edb
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 13:47:08 2022 +0200

    renaming

commit 93b3d4b202b41183ed8cabe193a4bfa03f520787
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 13:25:03 2022 +0200

    further refactored server setup code: moving and decomplecting

commit 8b2ed83c7ade5bd811cb045d56fbfb0353fa385e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 29 12:53:09 2022 +0200

    refactored server setup code: factored out and decoupled operation registry and prometheus summary registry

commit da2dce762bdd6889165fbb320dc9ee8a0bd089b2
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jun 28 19:40:04 2022 +0200

    adjusted test target

commit 70df7911b9b92f4b72afd7d4b33ca2bbf136295e
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Tue Jun 28 19:32:38 2022 +0200

    minor refactoring

commit 0937b63dc000346559bde353381304b273244109
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jun 27 13:59:59 2022 +0200

    support for empty operation suffix

commit 5e56917970962a2e69bbd66a324bdb4618c040bd
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jun 27 12:52:36 2022 +0200

    minor refactoring

commit 40665a7815ae5927b3877bda14fb77deef37d667
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jun 27 10:57:04 2022 +0200

    optimization: prefix filtering via storage API for get_all_object_names

commit af0892a899d09023eb0e61eecb63e03dc2fd3b60
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Mon Jun 27 10:55:47 2022 +0200

    topological sorting of definitions by caller hierarchy
 2022-07-11 09:09:44 +02:00
Matthias Bisping
94254e1681 Pull request #38: 2.0.0 input output file pattern for download strategy 
Merge in RR/pyinfra from 2.0.0-input-output-file-pattern-for-download-strategy to 2.0.0

Squashed commit of the following:

commit c7ce79ebbeace6a8cb7925ed69eda2d7cd2a4783
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jun 24 12:35:29 2022 +0200

    refactor

commit 80f04e544962760adb2dc60c9dd03ccca22167d6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Jun 24 11:06:10 2022 +0200

    refactoring of component factory, callback and client-pipeline getter

commit 6c024e1a789e1d55f0739c6846e5c02e8b7c943d
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 20:04:10 2022 +0200

    operations section in config cleaned up; added upload formatter

commit c85800aefc224967cea591c1ec4cf1aaa3ac8215
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 19:22:51 2022 +0200

    refactoring; removed obsolete config entries and code

commit 4be125952d82dc868935c8c73ad87fd8f0bd1d6c
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 19:14:47 2022 +0200

    removed obsolete code

commit ac69a5c8e3f1e2fd7e828a17eeab97984f4f9746
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 18:58:41 2022 +0200

    refactoring: rm dl strat module

commit efd36d0fc4f8f36d267bfa9d35415811fe723ccc
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 18:33:51 2022 +0200

    refactoring: multi dl strat -> downloader, rm single dl strat

commit afffdeb993500a6abdb6fe85a549e3d6e97e9ee7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 16:39:22 2022 +0200

    operations section in config cleaned up

commit 671129af3e343490e0fb277a2b0329aa3027fd73
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jun 23 16:09:16 2022 +0200

    rename prometheus metric name to include service name

commit 932a3e314b382315492aecab95b1f02f2916f8a6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 14:43:23 2022 +0200

    cleaned up file descr mngr

commit 79350b4ce71fcd095ed6a5e1d3a598ea246fae53
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 12:26:15 2022 +0200

    refactoring WIP: moving response stratgey logic into storage strategy (needs to be refactored as well, later) and file descr mngr. Here the moved code needs to be cleaned up.

commit 7e48c66f0c378b25a433a4034eefdc8a0957e775
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 12:00:48 2022 +0200

    refactoring; removed operation / response folder from output path

commit 8e6cbdaf23c48f6eeb52512b7f382d5727e206d6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 11:08:37 2022 +0200

    refactoring; added operation -> file pattern mapping to file descr mngr (mainly for self-documentaton purposes)

commit 2c80d7cec0cc171e099e5b13aadd2ae0f9bf4f02
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:59:57 2022 +0200

    refactoring: introduced input- and output-file specific methods to file descr mngr

commit ecced37150eaac3008cc1b01b235e5f7135e504b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:43:26 2022 +0200

    refactoring

commit 3828341e98861ff8d63035ee983309ad5064bb30
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:42:46 2022 +0200

    refactoring

commit 9a7c412523d467af40feb6924823ca89e28aadfe
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 17:04:54 2022 +0200

    add prometheus metric name for default operation

commit d207b2e274ba53b2a21a18c367bb130fb05ee1cd
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 17:02:55 2022 +0200

    Merge config

commit d3fdf36b12d8def18810454765e731599b833bfc
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 22 17:01:12 2022 +0200

    added fixmes / todos

commit f49d0b9cb7764473ef9d127bc5d88525a4a16a23
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 16:28:25 2022 +0200

    update script

... and 47 more commits
2.0.0_27 		2022-06-24 12:59:26 +02:00
Julius Unverfehrt
0d87c60fce parametrize download strategy 2.0.0_25 2.0.2 2022-06-20 11:43:33 +02:00
Julius Unverfehrt
2dff7d62aa remove duplicate pickup metrics 2.0.0_24 2022-06-20 10:27:36 +02:00
Julius Unverfehrt
e9424aee04 remove duplicate pickup metrics 2.0.0_23 2022-06-20 08:29:33 +02:00
Julius Unverfehrt
9d73f42982 remove duplicate pickup metrics 2.0.0_22 2022-06-20 08:19:51 +02:00
Matthias Bisping
4aef3316a3 renaming 2.0.0_21 2.0.1 2022-06-15 15:14:17 +02:00
Matthias Bisping
41172d6abb formatting 2022-06-15 15:13:46 +02:00
Matthias Bisping
71af6f703b using function local registry for prometheus 2022-06-15 15:13:10 +02:00
Julius Unverfehrt
a5ff59069a Merge branch '2.0.0' of ssh://git.iqser.com:2222/rr/pyinfra into 2.0.0 2022-06-15 14:56:06 +02:00
Julius Unverfehrt
965d79b08f add prometheus endpoint to analysis server 2022-06-15 14:52:22 +02:00
Matthias Bisping
ca6a2f8d32 fixed docker fixture 2022-06-15 14:14:38 +02:00
Matthias Bisping
86eb3a6f7e enanbled docker fixture 2022-06-15 14:01:56 +02:00
Matthias Bisping
87cf1ad189 removed obsolete imports 2022-06-15 14:00:36 +02:00
Matthias Bisping
7865a767c7 added type hint 2022-06-15 14:00:09 +02:00
Matthias Bisping
3897e44378 Merge branch '2.0.0' of ssh://git.iqser.com:2222/rr/pyinfra into 2.0.0 2022-06-15 12:25:55 +02:00
Matthias Bisping
1558398c56 made object name construction logic part of download strategies 2022-06-15 12:25:27 +02:00
Matthias Bisping
8537d4af50 made object name construction logic part of download strategies 2022-06-15 12:02:41 +02:00
Matthias Bisping
116c2b8924 changed default target file extension 2022-06-15 10:31:05 +02:00
Matthias Bisping
45f04590cc removed obsolete code 2022-06-15 10:25:58 +02:00
Matthias Bisping
bb729b6c26 wrapped retry decortaor, so retry behaviour can be controlled via config and set to a lower value for tests to save time 2022-06-15 10:25:53 +02:00
Matthias Bisping
24be8d3f13 test config options for logging and docker; changed object name construction 2022-06-15 09:59:47 +02:00
Matthias Bisping
147416bfad pin minio and rabbitmq again 2.0.0_2 file_extensions_and_index_handler_via_config_22 2022-06-14 17:05:33 +02:00
Matthias Bisping
c8fb15b9f7 rm retry decorator on clear_bucket, unpin minio 2022-06-14 17:02:22 +02:00
Matthias Bisping
771df7c78d make bucket before running test; rabbitmq 3.9 again 2022-06-14 16:58:27 +02:00
Matthias Bisping
f9972a95a7 fixed minio version 2022-06-14 16:44:46 +02:00
Matthias Bisping
83e1b5f029 added retry to clear bucket 2022-06-14 16:34:10 +02:00
Matthias Bisping
c1b5cbeb51 logging setup changed 2022-06-14 16:28:45 +02:00
Matthias Bisping
4fcc89f938 s3 backend fixture no longer needs to not come last 2022-06-14 15:40:23 +02:00
Matthias Bisping
d1242aee6c enable docker-compose fixture 2022-06-14 15:33:34 +02:00
Matthias Bisping
0442ecd7b3 Merge branch 'file_extensions_and_index_handler_via_config' of ssh://git.iqser.com:2222/rr/pyinfra into file_extensions_and_index_handler_via_config 2022-06-14 15:33:14 +02:00
Matthias Bisping
e64ade3135 added comments to new config params 2022-06-14 15:33:11 +02:00
Julius Unverfehrt
ace919d078 set xfail for broken tests, set docker-compose rabbitmq version to version running on production server 2022-06-14 15:10:08 +02:00
Matthias Bisping
d179fdede6 consumer test runs again...? 2022-06-14 14:17:08 +02:00
Julius Unverfehrt
9b975b759b set xfail for broken tests 2022-06-14 14:06:06 +02:00
Julius Unverfehrt
c033d98acd adjust test for new return type of visitor, add download strategy parameter to config 2022-06-14 12:33:56 +02:00
Julius Unverfehrt
bb7e631f91 introduce flag to distinguish between server side tests and complete integration tests 2022-06-14 11:56:47 +02:00
Julius Unverfehrt
d8b5be9e72 refactoring 2022-06-14 11:26:46 +02:00
Julius Unverfehrt
2954bbc1ad refactoring 2022-06-14 11:21:31 +02:00
Matthias Bisping
a69f613fe6 completed multi download to single response logic. but broke pipeline test again, maybe? 2022-06-14 11:08:03 +02:00
Matthias Bisping
fa3b08aef5 added dependencies 2022-06-13 15:38:14 +02:00
Matthias Bisping
14ab23b2cc fixed bug in operation wrapper returning a tuple instead of an singleton-iterable with a tuple in one of the return-cases. 2022-06-13 15:36:17 +02:00
Matthias Bisping
8a64e5d868 narrowed down the pytest bug: n_items interacts with s3_backend: when n_items has more than one entry, s3_backend must not be the last decorator 2022-06-13 10:36:26 +02:00
Matthias Bisping
051cea3ded found bug in pytest fixture setup causing serve_test to fail (s3 backend fixture function vs s3 backend decorator fixture) 2022-06-13 10:15:33 +02:00
Matthias Bisping
40bc8c2c2c debugging of queue problem, when queue is not consumed by skipping a test configuration WIP 2022-06-13 09:49:10 +02:00
Matthias Bisping
9962651d88 download strategy WIP: added 1 -> n upload logic 2022-06-10 14:06:19 +02:00
Matthias Bisping
249c6203b2 download strategy WIP 2022-06-10 13:26:43 +02:00
Matthias Bisping
3a3c497383 added logic for uploading response files to a folder, which defaults to the name of the operation used 2022-06-10 13:03:40 +02:00
Matthias Bisping
13b6388e5a fixed default identifier type 2022-06-09 16:24:11 +02:00