Marmelator/pyinfra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity
411 Commits 9 Branches 411 Tags
Commit Graph

411 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matthias Bisping
0104395790 removed obsolete code 2022-05-31 13:50:44 +02:00
Matthias Bisping
18a9683ddb operation field in queue message WIP 2022-05-31 13:47:40 +02:00
Julius Unverfehrt
6d31cbe635 Pull request #37: RED-4124 PyInfra now tries three times to download an unobtainable object before republishing message, removed unreadable stack trace print if message is put on dead letter queue since allready logged where the exeption is raised. 
Merge in RR/pyinfra from RED-4124-s3-retry to master

Squashed commit of the following:

commit 40e4dc3712fc692115b8274226430a431bf0192f
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue May 31 12:59:45 2022 +0200

    RED-4124 PyInfra now tries three times to download an unobtainable object before republishing message, removed unreadable stack trace print if message is put on dead letter queue since allready logged where the exeption is raised.
master_61 master_62 		2022-05-31 13:13:18 +02:00
Matthias Bisping
ae2509dc59 modified visitor and queue manager for 1 -> n (1 request to n response messages) 2022-05-30 13:04:12 +02:00
Matthias Bisping
bf9f6ba8e2 tweaked response upload related logic and repaired visitor tests that were broken by new visitor code written for accomodating the aggregation storage strategy 2022-05-30 12:10:14 +02:00
Matthias Bisping
868a53b23f response file path depending on response metadata and request page index complete 2022-05-25 17:37:18 +02:00
Matthias Bisping
2d1ec16714 modified serve test to use components from fixtures; response file path depending on response metadata and request page index WIP 2022-05-25 16:56:08 +02:00
Matthias Bisping
9e2ed6a9f9 fix: data was doubly encoded and hence always triggering the immediate upload path 2022-05-24 14:51:24 +02:00
Matthias Bisping
ab56c9a173 added todo comment for modifying acknowledging logic at some point to allow input buffering to take effect 2022-05-24 14:39:39 +02:00
Matthias Bisping
298d8d3e2c metadata as part of storage item test works 2022-05-23 15:59:56 +02:00
Matthias Bisping
0842ec0d91 metadata as part of storage item WIP 2022-05-23 15:36:18 +02:00
Matthias Bisping
c944cdb1a7 refactoring: splitting source data from encoded data in data fixture 2022-05-23 13:53:50 +02:00
Matthias Bisping
7b998cdaf6 refactoring 2022-05-23 13:14:18 +02:00
Matthias Bisping
426967ee46 refactoring 2022-05-23 11:39:30 +02:00
Matthias Bisping
54ca81d577 moved parameter combination based test skipping into operation factory 2022-05-23 11:28:35 +02:00
Matthias Bisping
13888524fb refactoring 2022-05-23 10:40:28 +02:00
Matthias Bisping
a7ffaeb18f changed return value of file name listing function for storages to return strings of filenames rather than tuples of bucket name and file name 2022-05-23 10:19:07 +02:00
Matthias Bisping
c97393f690 skipping undefined combinations for analysis_task 2022-05-23 10:07:28 +02:00
Matthias Bisping
7ff466e0ea added test for empty return-data operation, like classification 2022-05-23 09:58:43 +02:00
Matthias Bisping
02b0009219 data AND metadata is being uploaded instead of data only 2022-05-18 16:35:18 +02:00
Matthias Bisping
cf13f67394 refactoring: serve_test can now be run with input_data_items like image, pdf etc 2022-05-18 10:56:14 +02:00
Matthias Bisping
0ab86206ec fixed bug introduced by overwritng 'body' as explanatory variable within try-block, which resultet in republish() receiving parsed body, instead of body as bytes 2022-05-18 10:32:54 +02:00
Matthias Bisping
35542f994c integration test for lazy pipeline 2022-05-18 09:24:12 +02:00
Matthias Bisping
fb712af7c6 storag aggregation strategy working in first version 2022-05-17 22:30:54 +02:00
Matthias Bisping
6cb13051eb fixed following bugs: 
- upper() did yield instead of return
 - metdadata was not repeated when zipping with results generator
 - since test metadata was empty dict,  target data was therefore empty always, since results were zipped with {}
 - hence added check for target lengths > 0
 - fixed return value of queued stream function dispatcher; only returned first item of 1 -> n results
 2022-05-17 21:48:16 +02:00
Matthias Bisping
456cb4157d refactoring: move 2022-05-17 17:27:58 +02:00
Matthias Bisping
6945760045 refactoring: move 2022-05-17 15:59:04 +02:00
Julius Unverfehrt
4f18dbc60d Pull request #36: RED-4049 fix: Pyinfra now retries to reconnect after AMPQ connection loss and eventually throws an error if unsuccessful, forcing container restart. 
Merge in RR/pyinfra from RED-4049-AMQP-reconnect to master

Squashed commit of the following:

commit 1baafb781c9f623faf4223043e1185fd4ecbbe0e
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue May 17 14:36:47 2022 +0200

    RED-4049 fix: Pyinfra now retries to reconnect after AMPQ connection loss and eventually throws an error if unsuccessful, forcing container restart.
1.0.7 		2022-05-17 14:41:03 +02:00
Matthias Bisping
47f1d77c03 renaming 2022-05-17 12:12:43 +02:00
Matthias Bisping
fb325ce43d renaming 2022-05-17 12:10:24 +02:00
Matthias Bisping
5590669939 pipelin laziness test works again 2022-05-17 10:44:43 +02:00
Matthias Bisping
9c262e7138 non-rest pipeline works again 2022-05-17 10:27:32 +02:00
Matthias Bisping
e5a4e7e994 applied black 2022-05-16 15:00:09 +02:00
Matthias Bisping
89f562aa71 refactoring: move 2022-05-16 14:58:19 +02:00
Matthias Bisping
1074f44b30 no buffer capacity test; commented out probably dead codde -- removing next 2022-05-16 14:34:37 +02:00
Matthias Bisping
96bf831b00 refactoring: move 2022-05-16 13:31:11 +02:00
Matthias Bisping
5d2b71d647 target data fixture and test for flat stream buffer on different data 2022-05-16 13:21:31 +02:00
Matthias Bisping
d12124b2d5 refactoring 2022-05-16 13:04:37 +02:00
Matthias Bisping
7adbdefb4e refactoring: skipping invalid parameter combinations 2022-05-16 12:18:04 +02:00
Matthias Bisping
a1c292a485 refactoring: pulled core operation taking only data out from operation taking data and metadata 2022-05-16 11:53:52 +02:00
Matthias Bisping
948575d199 renaming 2022-05-16 11:43:48 +02:00
Matthias Bisping
2070f300c9 refactoring: queued stream function returns first of generator 2022-05-16 10:28:53 +02:00
Matthias Bisping
092a0e2964 renaming 2022-05-13 17:16:27 +02:00
Matthias Bisping
40777ae609 refactoring: simplyfing lazy processor to queued function WIP 2022-05-13 16:59:50 +02:00
Matthias Bisping
2434e0ea55 refactoring; stream buffer tests 2022-05-13 16:38:20 +02:00
Matthias Bisping
08ad83b6a5 renaming 2022-05-13 15:04:01 +02:00
Matthias Bisping
9870aa38d1 renaming 2022-05-13 15:02:05 +02:00
Matthias Bisping
3b7605772e refactoring 2022-05-13 12:42:07 +02:00
Matthias Bisping
1acf16dc91 refactoring: flat stream buffer now takes over stream buffer flushing 2022-05-13 12:29:18 +02:00
Matthias Bisping
c09e5df23e refactoring: introduced flat stream buffer class 2022-05-13 12:13:59 +02:00