Merge in RR/pyinfra from feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment to master
* commit '7a740403bb65db97c8e4cb54de00aac3536b2e4c':
update
update test config
update test config
add pytests to check if a configured bucket can be found
add submodule initialization
load different env vars for the variable depending on the set
Merge in RR/pyinfra from RED-5324 to master
* commit 'ffaa4a668b447fe3f4708d99ec6fccec14f85693':
RED-5324: Added missing storage-region to key to config and to minio-client creation so that storage access works on s3
Merge in RR/pyinfra from RED-5009-fix-ack-bug to master
Squashed commit of the following:
commit 7b00edf6fe1167345e774d658fcd2e60c01d05d5
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Wed Aug 24 14:52:57 2022 +0200
RED-5009 pyinfra now truly rejects messages that couldn't be processed by the callback (e.g. inobtainable storage file)
Merge in RR/pyinfra from RED-5009 to master
Squashed commit of the following:
commit 1f8114379bdeb3af8640c71c2edde2a672bb358c
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Aug 22 16:55:04 2022 +0200
RED-5009: Added the possibility for a callback to signal that a message should be declined/dead-lettered
commit be674c2915f6f149c581bc2fe2783217fe424df8
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Aug 19 16:26:38 2022 +0200
RED-5009: Changed callback to not process redelivered messages to prevent endless retries
Merge in RR/pyinfra from add-exists-to-storage to master
Squashed commit of the following:
commit 48d5e1c9e103702bfebfc115e811576514e115c3
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Fri Aug 12 13:32:40 2022 +0200
refactor
commit 711d2c8dbf7c78e26133e3ea3a57670fe829059b
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date: Fri Aug 12 11:45:42 2022 +0200
add method to check if objects exists for azure and s3
Merge in RR/pyinfra from RED-4653 to master
* commit '7e2cb20040a6be7510f5b06b0a522c1b044d5ee3':
RED-4653: Corrected if-operator
RED-4653: Added value to config to prevent writing the token as a default since that is only useful in a container
RED-4653: Implemented a startup probe for k8s
Merge in RR/pyinfra from RED-4653 to master
* commit '0efbd2c98cecaa1e33991473b1b120827df60ae9':
RED-4653: Removed unnecessary string formatting
RED-4653: Reordered code to prevent errors on application shutdown
RED-4653: Changed code to close only the connection instead of the channel & connection to see if that is sufficient for a clean shutdown
RED-4653: Added some debugging code to test if closing the connection needed
RED-4653: Corrected exception block to not swallow exceptions
RED-4653: Switch to closing channel instead of only cancelling subscription on shutdown.
RED-4653: Corrected signal handler by correctly handling passed params
Changed queue-consumption shutdown to close the channel before closing the connection, since only cancelling the consumers doesn't clean-up the channel correctly, which in turn can cause an error when closing the connection. Also reordered the code so that the connection and channel are only opened when queue-consumption starts.
Merge in RR/pyinfra from tidy_up to master
* commit 'db8f617aa78698760e5aaa198445d349755366a1':
clean up config hygiene; align queue manager and storage signature
Merge in RR/pyinfra from RED-4653 to master
Squashed commit of the following:
commit 14ed6d2ee79f9a6bc4bad187dc775f7476a05d97
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 11:08:16 2022 +0200
RED-4653: Disabled coverage check since there not tests at the moment
commit e926631b167d03e8cc0867db5b5c7d44d6612dcf
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 10:58:50 2022 +0200
RED-4653: Re-added test execution scripts
commit 94648cc449bbc392864197a1796f99f8953b7312
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 10:50:42 2022 +0200
RED-4653: Changed error case for processing messages to not requeue the message since that will be handled in DLQ logic
commit d77982dfedcec49482293d79818283c8d7a17dc7
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 10:46:32 2022 +0200
RED-4653: Removed unnecessary logging message
commit 8c00fd75bf04f8ecc0e9cda654f8e053d4cfb66f
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 10:03:35 2022 +0200
RED-4653: Re-added wrongly removed config
commit 759d72b3fa093b19f97e68d17bf53390cd5453c7
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 09:57:47 2022 +0200
RED-4653: Removed leftover Docker commands
commit 2ff5897ee38e39d6507278b6a82176be2450da16
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 09:48:08 2022 +0200
RED-4653: Removed leftover Docker config
commit 1074167aa98f9f59c0f0f534ba2f1ba09ffb0958
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Tue Jul 26 09:41:21 2022 +0200
RED-4653: Removed Docker build stage since it is not needed for a project that is used as a Python module
commit ec769c6cd74a74097d8ebe4800ea6e2ea86236cc
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Jul 25 16:11:50 2022 +0200
RED-4653: Renamed function for better clarity and consistency
commit 96e8ac4316ac57aac90066f35422d333c532513b
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Jul 25 15:07:40 2022 +0200
RED-4653: Added code to cancel the queue subscription on application exit to queue manager so that it can exit gracefully
commit 64d8e0bd15730898274c08d34f9c34fbac559422
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Jul 25 13:57:06 2022 +0200
RED-4653: Moved queue cancellation to a separate method so that it can be called on application exit
commit aff1d06364f5694c5922f37d961e401c12243221
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Jul 25 11:51:16 2022 +0200
RED-4653: Re-ordered message processing so that ack occurs after publishing the result, to prevent message loss
commit 9339186b86f2fe9653366c22fcdc9f7fc096b138
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 18:07:25 2022 +0200
RED-4653: RED-4653: Reordered code to acknowledge message before publishing a result message
commit 2d6fe1cbd95cd86832b086c6dfbcfa62b3ffa16f
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 17:00:04 2022 +0200
RED-4653: Hopefully corrected storage bucket env var name
commit 8f1ef0dd5532882cb12901721195d9acb336286c
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 16:37:27 2022 +0200
RED-4653: Switched to validating the connection url via a regex since the validators lib parses our endpoints incorrectly
commit 8d0234fcc5ff7ed1ae7695a17856c6af050065bd
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 15:02:54 2022 +0200
RED-4653: Corrected exception creation
commit 098a62335b3b695ee409363d429ac07284de7138
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 14:42:22 2022 +0200
RED-4653: Added a descriptive error message when the storage endpoint is nor a correct url
commit 379685f964a4de641ce6506713f1ea8914a3f5ab
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Fri Jul 22 14:11:48 2022 +0200
RED-4653: Removed variable re-use to make the code clearer
commit 4bf1a023453635568e16b1678ef5ad994c534045
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Thu Jul 21 17:41:55 2022 +0200
RED-4653: Added explicit conversion of the heartbeat config value to an int before passing it to pika
commit 8f2bc4e028aafdef893458d1433a05724f534fce
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date: Mon Jul 18 16:41:31 2022 +0200
RED-4653: Set heartbeat to lower value so that disconnects are detected more quickly
... and 6 more commits
