Matthias Bisping
|
c85800aefc
|
refactoring; removed obsolete config entries and code
|
2022-06-23 19:22:51 +02:00 |
|
Matthias Bisping
|
ac69a5c8e3
|
refactoring: rm dl strat module
|
2022-06-23 18:58:41 +02:00 |
|
Matthias Bisping
|
efd36d0fc4
|
refactoring: multi dl strat -> downloader, rm single dl strat
|
2022-06-23 18:33:51 +02:00 |
|
Matthias Bisping
|
afffdeb993
|
operations section in config cleaned up
|
2022-06-23 16:39:22 +02:00 |
|
Matthias Bisping
|
932a3e314b
|
cleaned up file descr mngr
|
2022-06-23 14:43:23 +02:00 |
|
Matthias Bisping
|
7e48c66f0c
|
refactoring; removed operation / response folder from output path
|
2022-06-23 12:00:48 +02:00 |
|
Matthias Bisping
|
2c80d7cec0
|
refactoring: introduced input- and output-file specific methods to file descr mngr
|
2022-06-23 10:59:57 +02:00 |
|
Matthias Bisping
|
fb0b64f8d5
|
added hash method to config, so lru_cache can distinguish between different configs. but something is broken again in te fixture setup. using many_to_n = [True, False] hangs, while independently both pass. the consumer does not get requests in the former case, probably.
|
2022-06-22 15:21:14 +02:00 |
|
Matthias Bisping
|
1adeb4038a
|
removed direct download strategy usages in test setups
|
2022-06-22 13:56:12 +02:00 |
|
Matthias Bisping
|
bb6b28bb4e
|
refactoring; single dl strat now uses file descriptor manager
|
2022-06-22 12:30:36 +02:00 |
|
Matthias Bisping
|
23876501dc
|
refactoring: de-hardcorded operation by refactoring queue message metadata fixture
|
2022-06-22 12:00:42 +02:00 |
|
Matthias Bisping
|
af88e2d7d4
|
file pattern matching WIP: using of different operations to select file pattern config section -- semi-hardcoded multi input file operation. next: refactor fixtures in order to de-hardcode
|
2022-06-22 11:18:55 +02:00 |
|
Matthias Bisping
|
5728961b26
|
file pattern matching WIP: pulled out file descriptor logic from multi dl strat and put into dedicated class; added single dl strat as default for dl strat in visitor
|
2022-06-22 10:18:41 +02:00 |
|
Matthias Bisping
|
473362b86f
|
file pattern matching WIP: reimplememyed logic in a way that allows unifying upload (for tests) and download file descriptors.
|
2022-06-21 18:11:52 +02:00 |
|
Matthias Bisping
|
3f03e25bd3
|
refactoring; added operations to test config + overwriting main config in components fixture
|
2022-06-21 16:59:43 +02:00 |
|
Matthias Bisping
|
21daab2404
|
introduced component factory to wrap compnent getters into a config parametrized clojure
|
2022-06-21 14:21:56 +02:00 |
|
Matthias Bisping
|
562e383bdd
|
file pattern matching WIP: added first bit of file pattern matching logic to multi dl strat
|
2022-06-21 14:02:51 +02:00 |
|
Matthias Bisping
|
8e3badd053
|
removed .*FileExtension fields from everywhere but mock_client script
|
2022-06-21 10:21:31 +02:00 |
|
Matthias Bisping
|
2362619bef
|
fixed bug introduced by aggr strat refac: singleton lists are now unpacked before upload via default merge function
|
2022-06-20 16:36:40 +02:00 |
|
Matthias Bisping
|
343637aaf7
|
- modified aggr strat to produce only one response message, which now contains a list of response files instead of a single response file
- introduced response formatter
|
2022-06-20 15:22:59 +02:00 |
|
Matthias Bisping
|
3a6e9281ce
|
fixed bug in check for empty data in refactored aggregation strategy... was correct before
|
2022-06-20 14:45:09 +02:00 |
|
Matthias Bisping
|
0d455422b7
|
added aggregation strategy test + fixed bug in check for empty data in refactored aggregation strategy
|
2022-06-20 14:03:01 +02:00 |
|
Matthias Bisping
|
5392d03295
|
refactoring; removed dispatch callback from aggregation strategy
|
2022-06-20 12:45:59 +02:00 |
|
Matthias Bisping
|
2c902f3129
|
refactored visitor init
|
2022-06-19 00:06:24 +02:00 |
|
Matthias Bisping
|
02da828268
|
wrapped gzip.(de)compress
|
2022-06-18 23:15:22 +02:00 |
|
Matthias Bisping
|
8ab13fe4fb
|
comment changed; logging level of blob parser set to below debug
|
2022-06-18 22:42:21 +02:00 |
|
Matthias Bisping
|
f7e4953a4e
|
refactored visitor module: split into various submodules
|
2022-06-18 22:41:09 +02:00 |
|
Matthias Bisping
|
ca6a2f8d32
|
fixed docker fixture
|
2022-06-15 14:14:38 +02:00 |
|
Matthias Bisping
|
86eb3a6f7e
|
enanbled docker fixture
|
2022-06-15 14:01:56 +02:00 |
|
Matthias Bisping
|
1558398c56
|
made object name construction logic part of download strategies
|
2022-06-15 12:25:27 +02:00 |
|
Matthias Bisping
|
45f04590cc
|
removed obsolete code
|
2022-06-15 10:25:58 +02:00 |
|
Matthias Bisping
|
bb729b6c26
|
wrapped retry decortaor, so retry behaviour can be controlled via config and set to a lower value for tests to save time
|
2022-06-15 10:25:53 +02:00 |
|
Matthias Bisping
|
24be8d3f13
|
test config options for logging and docker; changed object name construction
|
2022-06-15 09:59:47 +02:00 |
|
Matthias Bisping
|
771df7c78d
|
make bucket before running test; rabbitmq 3.9 again
|
2022-06-14 16:58:27 +02:00 |
|
Matthias Bisping
|
c1b5cbeb51
|
logging setup changed
|
2022-06-14 16:28:45 +02:00 |
|
Matthias Bisping
|
4fcc89f938
|
s3 backend fixture no longer needs to not come last
|
2022-06-14 15:40:23 +02:00 |
|
Matthias Bisping
|
d1242aee6c
|
enable docker-compose fixture
|
2022-06-14 15:33:34 +02:00 |
|
Matthias Bisping
|
e64ade3135
|
added comments to new config params
|
2022-06-14 15:33:11 +02:00 |
|
Matthias Bisping
|
d179fdede6
|
consumer test runs again...?
|
2022-06-14 14:17:08 +02:00 |
|
Julius Unverfehrt
|
9b975b759b
|
set xfail for broken tests
|
2022-06-14 14:06:06 +02:00 |
|
Julius Unverfehrt
|
c033d98acd
|
adjust test for new return type of visitor, add download strategy parameter to config
|
2022-06-14 12:33:56 +02:00 |
|
Julius Unverfehrt
|
bb7e631f91
|
introduce flag to distinguish between server side tests and complete integration tests
|
2022-06-14 11:56:47 +02:00 |
|
Matthias Bisping
|
a69f613fe6
|
completed multi download to single response logic. but broke pipeline test again, maybe?
|
2022-06-14 11:08:03 +02:00 |
|
Matthias Bisping
|
14ab23b2cc
|
fixed bug in operation wrapper returning a tuple instead of an singleton-iterable with a tuple in one of the return-cases.
|
2022-06-13 15:36:17 +02:00 |
|
Matthias Bisping
|
8a64e5d868
|
narrowed down the pytest bug: n_items interacts with s3_backend: when n_items has more than one entry, s3_backend must not be the last decorator
|
2022-06-13 10:36:26 +02:00 |
|
Matthias Bisping
|
051cea3ded
|
found bug in pytest fixture setup causing serve_test to fail (s3 backend fixture function vs s3 backend decorator fixture)
|
2022-06-13 10:15:33 +02:00 |
|
Matthias Bisping
|
40bc8c2c2c
|
debugging of queue problem, when queue is not consumed by skipping a test configuration WIP
|
2022-06-13 09:49:10 +02:00 |
|
Matthias Bisping
|
9962651d88
|
download strategy WIP: added 1 -> n upload logic
|
2022-06-10 14:06:19 +02:00 |
|
Matthias Bisping
|
1940b974b1
|
added id setting to operatin mocks and thus removed need for random seed of randomly seeded hash in storage item identifier
|
2022-06-09 15:18:41 +02:00 |
|
Matthias Bisping
|
7e46a66698
|
fixed bug in stream_pages: return -> yield
|
2022-06-09 14:58:42 +02:00 |
|