Matthias Bisping
af88e2d7d4
file pattern matching WIP: using of different operations to select file pattern config section -- semi-hardcoded multi input file operation. next: refactor fixtures in order to de-hardcode
2022-06-22 11:18:55 +02:00
Julius Unverfehrt
bb7e631f91
introduce flag to distinguish between server side tests and complete integration tests
2022-06-14 11:56:47 +02:00
Matthias Bisping
a69f613fe6
completed multi download to single response logic. but broke pipeline test again, maybe?
2022-06-14 11:08:03 +02:00
Matthias Bisping
14ab23b2cc
fixed bug in operation wrapper returning a tuple instead of an singleton-iterable with a tuple in one of the return-cases.
2022-06-13 15:36:17 +02:00
Matthias Bisping
40bc8c2c2c
debugging of queue problem, when queue is not consumed by skipping a test configuration WIP
2022-06-13 09:49:10 +02:00
Matthias Bisping
1940b974b1
added id setting to operatin mocks and thus removed need for random seed of randomly seeded hash in storage item identifier
2022-06-09 15:18:41 +02:00
Matthias Bisping
7e46a66698
fixed bug in stream_pages: return -> yield
2022-06-09 14:58:42 +02:00
Matthias Bisping
9a47388017
adjusted param fixtures for serve test
2022-06-03 13:48:33 +02:00
Matthias Bisping
8a2b60a8f5
applied black
2022-06-03 13:42:52 +02:00
Matthias Bisping
9232385dea
modified core operations to return metadata for better classification mock test
2022-06-03 13:40:59 +02:00
Matthias Bisping
eb81e96400
added extraction test case (with page index) to serving test
2022-06-03 13:13:45 +02:00
Matthias Bisping
2d1ec16714
modified serve test to use components from fixtures; response file path depending on response metadata and request page index WIP
2022-05-25 16:56:08 +02:00
Matthias Bisping
7b998cdaf6
refactoring
2022-05-23 13:14:18 +02:00
Matthias Bisping
54ca81d577
moved parameter combination based test skipping into operation factory
2022-05-23 11:28:35 +02:00
Matthias Bisping
c97393f690
skipping undefined combinations for analysis_task
2022-05-23 10:07:28 +02:00
Matthias Bisping
7ff466e0ea
added test for empty return-data operation, like classification
2022-05-23 09:58:43 +02:00
Matthias Bisping
6cb13051eb
fixed following bugs:
...
- upper() did yield instead of return
- metdadata was not repeated when zipping with results generator
- since test metadata was empty dict, target data was therefore empty always, since results were zipped with {}
- hence added check for target lengths > 0
- fixed return value of queued stream function dispatcher; only returned first item of 1 -> n results
2022-05-17 21:48:16 +02:00
Matthias Bisping
9c262e7138
non-rest pipeline works again
2022-05-17 10:27:32 +02:00
Matthias Bisping
96bf831b00
refactoring: move
2022-05-16 13:31:11 +02:00
Matthias Bisping
5d2b71d647
target data fixture and test for flat stream buffer on different data
2022-05-16 13:21:31 +02:00
Matthias Bisping
7adbdefb4e
refactoring: skipping invalid parameter combinations
2022-05-16 12:18:04 +02:00
Matthias Bisping
a1c292a485
refactoring: pulled core operation taking only data out from operation taking data and metadata
2022-05-16 11:53:52 +02:00
Matthias Bisping
948575d199
renaming
2022-05-16 11:43:48 +02:00
Matthias Bisping
2070f300c9
refactoring: queued stream function returns first of generator
2022-05-16 10:28:53 +02:00
Matthias Bisping
092a0e2964
renaming
2022-05-13 17:16:27 +02:00
Matthias Bisping
40777ae609
refactoring: simplyfing lazy processor to queued function WIP
2022-05-13 16:59:50 +02:00
Matthias Bisping
9870aa38d1
renaming
2022-05-13 15:02:05 +02:00
Matthias Bisping
c09e5df23e
refactoring: introduced flat stream buffer class
2022-05-13 12:13:59 +02:00
Matthias Bisping
bfdce62ccf
refactoring; renaming
2022-05-12 19:46:29 +02:00
Matthias Bisping
8b0c2d4e07
refactoring: further simplified queue consuming function; added one -> many test fixture param
2022-05-12 17:55:45 +02:00
Matthias Bisping
e3793e5c7c
refactoring: split stream processor into two functions; moved queue streaming Nothing check from caller to stream function
2022-05-12 17:12:26 +02:00
Matthias Bisping
f29bd7d4d3
removed need for bufferize wrapper by composing with first(chunks(...)) and applying to on-demand processor execution chain; broke mock pipeline, fixing next
2022-05-09 11:06:14 +02:00
Matthias Bisping
f99d779c29
refactoring
2022-05-06 23:39:29 +02:00
Matthias Bisping
dca3eaaa54
fixed buffering and result streaming: all items are yielded individually and computed on demand now
2022-05-06 12:14:09 +02:00
Matthias Bisping
b58a9d11c3
refactoring: added processor class
2022-05-05 16:08:59 +02:00
Matthias Bisping
531ff8d3e0
refactoring; fixed sender
2022-05-04 16:57:08 +02:00
Matthias Bisping
cf0a877569
refactoring
2022-05-04 10:37:46 +02:00
Matthias Bisping
29fb0dda30
improved 1 -> n test and explanation
2022-05-03 15:43:30 +02:00
Matthias Bisping
ea9b405d2a
signature harminization for 1 -> 1 and 1 -> n completed
2022-05-02 15:50:14 +02:00
Matthias Bisping
3172a00aaa
refactoring
2022-04-29 17:08:34 +02:00
Matthias Bisping
501f0bd5fc
signature harminization for 1 -> 1 and 1 -> n WIP: batched and not batched working again
2022-04-29 16:46:25 +02:00
Matthias Bisping
fd57261631
signature harminization for 1 -> 1 and 1 -> n WIP
2022-04-29 15:43:20 +02:00
Matthias Bisping
00276dbcc7
added batching wrapper for internally batching functions
2022-04-29 10:35:55 +02:00
Matthias Bisping
bc0e9ed643
renaming
2022-04-29 09:51:41 +02:00
Matthias Bisping
940bc3a689
changed buffering function behaviour: applies function to buffer. function needs to be lifted from the outside if single items are to be processed.
2022-04-28 21:58:38 +02:00
Matthias Bisping
a999ce2c3b
refactoring
2022-04-28 21:36:42 +02:00
Matthias Bisping
e8315ffea9
signature of process_fn changed. Now returns {'data': ..., 'metadata'} instead of {'data': ...}
2022-04-28 21:12:25 +02:00
Matthias Bisping
3b0d0868b9
refactoring: method dispatch via peekable rather than special empty data request
2022-04-28 13:01:01 +02:00
Matthias Bisping
da84ff5112
buffer size constrained by assertion
2022-04-27 18:45:14 +02:00
Matthias Bisping
c9f26000d7
test parametrization for number of input items and buffer size
2022-04-27 18:38:43 +02:00
