Marmelator/pyinfra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity
348 Commits 9 Branches 411 Tags
Commit Graph

38 Commits

Author SHA1 Message Date
Matthias Bisping
94254e1681 Pull request #38: 2.0.0 input output file pattern for download strategy 
Merge in RR/pyinfra from 2.0.0-input-output-file-pattern-for-download-strategy to 2.0.0

Squashed commit of the following:

commit c7ce79ebbeace6a8cb7925ed69eda2d7cd2a4783
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Jun 24 12:35:29 2022 +0200

    refactor

commit 80f04e544962760adb2dc60c9dd03ccca22167d6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Fri Jun 24 11:06:10 2022 +0200

    refactoring of component factory, callback and client-pipeline getter

commit 6c024e1a789e1d55f0739c6846e5c02e8b7c943d
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 20:04:10 2022 +0200

    operations section in config cleaned up; added upload formatter

commit c85800aefc224967cea591c1ec4cf1aaa3ac8215
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 19:22:51 2022 +0200

    refactoring; removed obsolete config entries and code

commit 4be125952d82dc868935c8c73ad87fd8f0bd1d6c
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 19:14:47 2022 +0200

    removed obsolete code

commit ac69a5c8e3f1e2fd7e828a17eeab97984f4f9746
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 18:58:41 2022 +0200

    refactoring: rm dl strat module

commit efd36d0fc4f8f36d267bfa9d35415811fe723ccc
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 18:33:51 2022 +0200

    refactoring: multi dl strat -> downloader, rm single dl strat

commit afffdeb993500a6abdb6fe85a549e3d6e97e9ee7
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 16:39:22 2022 +0200

    operations section in config cleaned up

commit 671129af3e343490e0fb277a2b0329aa3027fd73
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Jun 23 16:09:16 2022 +0200

    rename prometheus metric name to include service name

commit 932a3e314b382315492aecab95b1f02f2916f8a6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 14:43:23 2022 +0200

    cleaned up file descr mngr

commit 79350b4ce71fcd095ed6a5e1d3a598ea246fae53
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 12:26:15 2022 +0200

    refactoring WIP: moving response stratgey logic into storage strategy (needs to be refactored as well, later) and file descr mngr. Here the moved code needs to be cleaned up.

commit 7e48c66f0c378b25a433a4034eefdc8a0957e775
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 12:00:48 2022 +0200

    refactoring; removed operation / response folder from output path

commit 8e6cbdaf23c48f6eeb52512b7f382d5727e206d6
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 11:08:37 2022 +0200

    refactoring; added operation -> file pattern mapping to file descr mngr (mainly for self-documentaton purposes)

commit 2c80d7cec0cc171e099e5b13aadd2ae0f9bf4f02
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:59:57 2022 +0200

    refactoring: introduced input- and output-file specific methods to file descr mngr

commit ecced37150eaac3008cc1b01b235e5f7135e504b
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:43:26 2022 +0200

    refactoring

commit 3828341e98861ff8d63035ee983309ad5064bb30
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Thu Jun 23 10:42:46 2022 +0200

    refactoring

commit 9a7c412523d467af40feb6924823ca89e28aadfe
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 17:04:54 2022 +0200

    add prometheus metric name for default operation

commit d207b2e274ba53b2a21a18c367bb130fb05ee1cd
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 17:02:55 2022 +0200

    Merge config

commit d3fdf36b12d8def18810454765e731599b833bfc
Author: Matthias Bisping <matthias.bisping@iqser.com>
Date:   Wed Jun 22 17:01:12 2022 +0200

    added fixmes / todos

commit f49d0b9cb7764473ef9d127bc5d88525a4a16a23
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Jun 22 16:28:25 2022 +0200

    update script

... and 47 more commits
 2022-06-24 12:59:26 +02:00
Matthias Bisping
45f04590cc removed obsolete code 2022-06-15 10:25:58 +02:00
Julius Unverfehrt
bb7e631f91 introduce flag to distinguish between server side tests and complete integration tests 2022-06-14 11:56:47 +02:00
Matthias Bisping
a69f613fe6 completed multi download to single response logic. but broke pipeline test again, maybe? 2022-06-14 11:08:03 +02:00
Matthias Bisping
14ab23b2cc fixed bug in operation wrapper returning a tuple instead of an singleton-iterable with a tuple in one of the return-cases. 2022-06-13 15:36:17 +02:00
Matthias Bisping
40bc8c2c2c debugging of queue problem, when queue is not consumed by skipping a test configuration WIP 2022-06-13 09:49:10 +02:00
Matthias Bisping
9962651d88 download strategy WIP: added 1 -> n upload logic 2022-06-10 14:06:19 +02:00
Matthias Bisping
7e46a66698 fixed bug in stream_pages: return -> yield 2022-06-09 14:58:42 +02:00
Matthias Bisping
eb81e96400 added extraction test case (with page index) to serving test 2022-06-03 13:13:45 +02:00
Matthias Bisping
c944cdb1a7 refactoring: splitting source data from encoded data in data fixture 2022-05-23 13:53:50 +02:00
Matthias Bisping
6cb13051eb fixed following bugs: 
- upper() did yield instead of return
 - metdadata was not repeated when zipping with results generator
 - since test metadata was empty dict,  target data was therefore empty always, since results were zipped with {}
 - hence added check for target lengths > 0
 - fixed return value of queued stream function dispatcher; only returned first item of 1 -> n results
 2022-05-17 21:48:16 +02:00
Matthias Bisping
9c262e7138 non-rest pipeline works again 2022-05-17 10:27:32 +02:00
Matthias Bisping
89f562aa71 refactoring: move 2022-05-16 14:58:19 +02:00
Matthias Bisping
5d2b71d647 target data fixture and test for flat stream buffer on different data 2022-05-16 13:21:31 +02:00
Matthias Bisping
7adbdefb4e refactoring: skipping invalid parameter combinations 2022-05-16 12:18:04 +02:00
Matthias Bisping
948575d199 renaming 2022-05-16 11:43:48 +02:00
Matthias Bisping
5b30a32fff added client pipeline without webserver backend 2022-05-09 00:43:21 +02:00
Matthias Bisping
dca3eaaa54 fixed buffering and result streaming: all items are yielded individually and computed on demand now 2022-05-06 12:14:09 +02:00
Matthias Bisping
1b04c46853 removed code related to eager endpoint 2022-05-05 17:25:42 +02:00
Matthias Bisping
68c24c863f removed eager endpoint (/process) 2022-05-05 17:21:47 +02:00
Matthias Bisping
bd5fe82e06 refactoring: rename 2022-05-05 15:47:25 +02:00
Matthias Bisping
b62652957a refactoring: pipeline 2022-05-05 15:15:05 +02:00
Matthias Bisping
531ff8d3e0 refactoring; fixed sender 2022-05-04 16:57:08 +02:00
Matthias Bisping
a301876ab9 refactoring: metadata argument as iterable instead of dict 2022-05-04 14:55:04 +02:00
Matthias Bisping
cf0a877569 refactoring 2022-05-04 10:37:46 +02:00
Matthias Bisping
c315247625 parametrized number of pages for pdf fixture 2022-05-03 15:47:11 +02:00
Matthias Bisping
29fb0dda30 improved 1 -> n test and explanation 2022-05-03 15:43:30 +02:00
Matthias Bisping
8575567890 refactoring 2022-05-03 14:25:01 +02:00
Matthias Bisping
77f23a2185 data nesting harmonized for 1 -> 1 and 1 -> n; pdf page extraction (1 -> n) working for non-batched usage 2022-04-29 17:22:54 +02:00
Matthias Bisping
3172a00aaa refactoring 2022-04-29 17:08:34 +02:00
Matthias Bisping
c9f26000d7 test parametrization for number of input items and buffer size 2022-04-27 18:38:43 +02:00
Matthias Bisping
67c4bac4b7 sync 2022-04-27 17:45:48 +02:00
Matthias Bisping
ab5839a126 refactoring; extended partial posting by image payload data 2022-04-27 16:29:52 +02:00
Matthias Bisping
fa4e5e5e0e refactoring; made test dynamic relative to import 2022-04-27 14:15:52 +02:00
Matthias Bisping
7419612c21 partial request by manual receiver buffering V1 2022-04-26 19:54:37 +02:00
Julius Unverfehrt
656bc7cd63 explore partial responses 2022-04-26 16:34:58 +02:00
Matthias Bisping
5a948ef7ad partial response test WIP 2022-04-26 16:15:54 +02:00
Matthias Bisping
b7882d4452 refactoring: introduced sub-conftest files 2022-04-26 13:13:26 +02:00