Marmelator/pyinfra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity
295 Commits 9 Branches 411 Tags
Commit Graph

295 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matthias Bisping
8d209d63c7 Merge branch 'master' of ssh://git.iqser.com:2222/rr/pyinfra into fixing_consumer_tests 2022-06-07 15:44:39 +02:00
Matthias Bisping
0dee98b23d consumer test adjustment WIP 2022-06-07 15:11:38 +02:00
Matthias Bisping
91701929e5 adjusted stream buffer test for core-operations taking tuples now 2022-06-07 15:08:58 +02:00
Matthias Bisping
c55e41f2d8 refactoring; tweaked json-blob-parser; added standardization case for decodable strings as storage items 2022-06-07 14:55:40 +02:00
Matthias Bisping
f718b2f7ef parser composer checks for either-type 2022-06-03 16:34:37 +02:00
Matthias Bisping
730bdfb220 refactoring 2022-06-03 16:26:12 +02:00
Matthias Bisping
e48fa85784 replaced trial and error logic for parsing blobs in visitor with parser composer instance 2022-06-03 16:18:56 +02:00
Matthias Bisping
6e5af4092e added parsers and parser composer for clean handling of storage blobs in the context of interpreting downloaded blobs 2022-06-03 16:13:15 +02:00
Matthias Bisping
ea2d3223fb parsing strategy error handling for bytes as not an encoded string 2022-06-03 14:55:04 +02:00
Matthias Bisping
26573eeda3 introduced parsing strategy for storage blobs as part of the queue visitor 2022-06-03 14:49:19 +02:00
Matthias Bisping
7730950b50 cleaning up standardization method for downloaded storage items (WIP) 2022-06-03 14:08:40 +02:00
Matthias Bisping
9a47388017 adjusted param fixtures for serve test 2022-06-03 13:48:33 +02:00
Matthias Bisping
8a2b60a8f5 applied black 2022-06-03 13:42:52 +02:00
Matthias Bisping
9232385dea modified core operations to return metadata for better classification mock test 2022-06-03 13:40:59 +02:00
Matthias Bisping
eb81e96400 added extraction test case (with page index) to serving test 2022-06-03 13:13:45 +02:00
Julius Unverfehrt
e7ee0cda42 add compression for storage item before upload, update script for extraction 2022-06-02 15:49:28 +02:00
Julius Unverfehrt
90f8f9da36 update script for extraction 2022-06-02 15:12:33 +02:00
Julius Unverfehrt
c2d7127a84 add log for Consumer Error, fix page index hash function 2022-06-02 14:15:03 +02:00
Matthias Bisping
bfe8bbb8cb reorganized queue message metadata in request 2022-06-01 16:00:46 +02:00
Matthias Bisping
ecff50ae7c submit endpoint is now 'submit' or 'operation' 2022-06-01 11:46:51 +02:00
Matthias Bisping
7a1b215d69 removed obsolete code 2022-06-01 11:37:47 +02:00
Matthias Bisping
01ce914417 generalizing server setup from operations WIP 2022-06-01 10:58:44 +02:00
Matthias Bisping
2b72174605 Merge branch 'integration_test_in_order_develop_aggregation_stratgey' of ssh://git.iqser.com:2222/rr/pyinfra into integration_test_in_order_develop_aggregation_stratgey 2022-05-31 18:40:47 +02:00
Matthias Bisping
586871a26f added queue message body to analysis input dict 2022-05-31 18:40:40 +02:00
Julius Unverfehrt
187055e5eb adjust mock_client script for conversion 2022-05-31 18:19:41 +02:00
Matthias Bisping
3046b4dc26 misc minor fixes while integrating with pdf2image 2022-05-31 17:58:28 +02:00
Matthias Bisping
dd591bd24b removed obsolete code 2022-05-31 16:23:14 +02:00
Matthias Bisping
12fa52f38c removed obsolete code 2022-05-31 16:22:27 +02:00
Matthias Bisping
043fa1ee53 removed obsolete code 2022-05-31 16:22:11 +02:00
Matthias Bisping
1fa6bbdbc6 removed obsolete code 2022-05-31 16:19:37 +02:00
Matthias Bisping
93747d0f63 test param adjustment 2022-05-31 16:14:48 +02:00
Matthias Bisping
dc4f578e94 reorganized serve-test to use only default-objects instead of test-object 2022-05-31 16:13:47 +02:00
Matthias Bisping
93da0d12bb refactoring: added paramater 'n' to consume_and_publish 2022-05-31 15:30:23 +02:00
Matthias Bisping
a688cbd7bd refactoring 2022-05-31 13:57:20 +02:00
Matthias Bisping
0104395790 removed obsolete code 2022-05-31 13:50:44 +02:00
Matthias Bisping
18a9683ddb operation field in queue message WIP 2022-05-31 13:47:40 +02:00
Julius Unverfehrt
6d31cbe635 Pull request #37: RED-4124 PyInfra now tries three times to download an unobtainable object before republishing message, removed unreadable stack trace print if message is put on dead letter queue since allready logged where the exeption is raised. 
Merge in RR/pyinfra from RED-4124-s3-retry to master

Squashed commit of the following:

commit 40e4dc3712fc692115b8274226430a431bf0192f
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue May 31 12:59:45 2022 +0200

    RED-4124 PyInfra now tries three times to download an unobtainable object before republishing message, removed unreadable stack trace print if message is put on dead letter queue since allready logged where the exeption is raised.
master_61 master_62 		2022-05-31 13:13:18 +02:00
Matthias Bisping
ae2509dc59 modified visitor and queue manager for 1 -> n (1 request to n response messages) 2022-05-30 13:04:12 +02:00
Matthias Bisping
bf9f6ba8e2 tweaked response upload related logic and repaired visitor tests that were broken by new visitor code written for accomodating the aggregation storage strategy 2022-05-30 12:10:14 +02:00
Matthias Bisping
868a53b23f response file path depending on response metadata and request page index complete 2022-05-25 17:37:18 +02:00
Matthias Bisping
2d1ec16714 modified serve test to use components from fixtures; response file path depending on response metadata and request page index WIP 2022-05-25 16:56:08 +02:00
Matthias Bisping
9e2ed6a9f9 fix: data was doubly encoded and hence always triggering the immediate upload path 2022-05-24 14:51:24 +02:00
Matthias Bisping
ab56c9a173 added todo comment for modifying acknowledging logic at some point to allow input buffering to take effect 2022-05-24 14:39:39 +02:00
Matthias Bisping
298d8d3e2c metadata as part of storage item test works 2022-05-23 15:59:56 +02:00
Matthias Bisping
0842ec0d91 metadata as part of storage item WIP 2022-05-23 15:36:18 +02:00
Matthias Bisping
c944cdb1a7 refactoring: splitting source data from encoded data in data fixture 2022-05-23 13:53:50 +02:00
Matthias Bisping
7b998cdaf6 refactoring 2022-05-23 13:14:18 +02:00
Matthias Bisping
426967ee46 refactoring 2022-05-23 11:39:30 +02:00
Matthias Bisping
54ca81d577 moved parameter combination based test skipping into operation factory 2022-05-23 11:28:35 +02:00
Matthias Bisping
13888524fb refactoring 2022-05-23 10:40:28 +02:00