157 Commits

Author SHA1 Message Date
Julius Unverfehrt
3c4739ad8b Pull request #63: RED-6366 refactor
Merge in RR/pyinfra from RED-6366-refactor to master

Squashed commit of the following:

commit 8807cda514b5cc24b1be208173283275d87dcb97
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 13:15:15 2023 +0100

    enable docker-compose autouse for automatic tests

commit c4579581d3e9a885ef387ee97f3f3a5cf4731193
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 12:35:49 2023 +0100

    black

commit ac2b754c5624ef37ce310fce7196c9ea11bbca03
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 12:30:23 2023 +0100

    refactor storage url parsing

    - move parsing and validation to config where the connection url is
    actually read in
    - improve readability of parsing fn

commit 371802cc10b6d946c4939ff6839571002a2cb9f4
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 10:48:00 2023 +0100

    refactor

commit e8c381c29deebf663e665920752c2965d7abce16
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 09:57:34 2023 +0100

    rename

commit c8628a509316a651960dfa806d5fe6aacb7a91c1
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 09:37:01 2023 +0100

    renaming and refactoring

commit 4974d4f56fd73bc55bd76aa7a9bbb16babee19f4
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Fri Mar 10 08:53:09 2023 +0100

    refactor payload processor

    - limit make_uploader and make_downloader cache
    - partially apply them when the class is initialized with storage and
    bucket to make the logic and behaviour more comprehensive
    - renaming functional pipeline steps to be more expressive

commit f8d51bfcad2b815c8293ab27dd66b256255c5414
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Mar 9 15:30:32 2023 +0100

    remove monitor and rename Payload

commit 412ddaa207a08aff1229d7acd5d95402ac8cd578
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Mar 2 10:15:39 2023 +0100

    remove azure connection string and disable respective test for now for security reasons

commit 7922a2d9d325f3b9008ad4e3e56b241ba179f52c
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Mar 1 13:30:58 2023 +0100

    make payload formatting function names more expressive

commit 7517e544b0f5a434579cc9bada3a37e7ac04059f
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Mar 1 13:24:57 2023 +0100

    add some type hints

commit 095410d3009f2dcbd374680dd0f7b55de94c9e76
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Wed Mar 1 10:54:58 2023 +0100

    Refactoring

    - Renaming
    - Docstring adjustments

commit e992f0715fc2636eb13eb5ffc4de0bcc5d433fc8
Author: Matthias Bisping <matthias.bisping@axbit.com>
Date:   Wed Mar 1 09:43:26 2023 +0100

    Re-wording and typo fixes

commit 3c2d698f9bf980bc4b378a44dc20c2badc407b3e
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 28 14:59:59 2023 +0100

    enable auto startup for docker compose in tests

commit 55773b4fb0b624ca4745e5b8aeafa6f6a0ae6436
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 28 14:59:37 2023 +0100

    Extended tests for queue manager

commit 14f7f943f60b9bfb9fe77fa3cef99a1e7d094333
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 28 13:39:00 2023 +0100

    enable auto startup for docker compose in tests

commit 7caf354491c84c6e0b0e09ad4d41cb5dfbfdb225
Merge: 49d47ba d0277b8
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 28 13:32:52 2023 +0100

    Merge branch 'RED-6205-prometheus' of ssh://git.iqser.com:2222/rr/pyinfra into RED-6205-prometheus

commit 49d47baba8ccf11dee48a4c1cbddc3bbd12471e5
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 28 13:32:42 2023 +0100

    adjust Payload Processor signature

commit d0277b86bc54994b6032774bf0ec2d7b19d7f517
Merge: 5184a18 f6b35d6
Author: Christoph Schabert <christoph.schabert@iqser.com>
Date:   Tue Feb 28 11:07:16 2023 +0100

    Pull request #61: Change Sec Trigger to PR

    Merge in RR/pyinfra from cschabert/PlanSpecjava-1677578703647 to RED-6205-prometheus

    * commit 'f6b35d648c88ddbce1856445c3b887bce669265c':
      Change Sec Trigger to PR

commit f6b35d648c88ddbce1856445c3b887bce669265c
Author: Christoph Schabert <christoph.schabert@iqser.com>
Date:   Tue Feb 28 11:05:13 2023 +0100

    Change Sec Trigger to PR

... and 20 more commits
1.4.2
2023-03-13 15:11:25 +01:00
Julius Unverfehrt
46157031b5 Pull request #59: adjust response headers
Merge in RR/pyinfra from RED-6118-multi-tenancy-patch to master

Squashed commit of the following:

commit 02e471622e59baf5d2bb5c61980cea43ca1c6d61
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Feb 16 16:36:19 2023 +0100

    move acknowledgment function to outer scope

commit f9efffd8e6d90d5e371c66574b1afe361a1da146
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Feb 16 16:04:07 2023 +0100

    adjust response headers

    - change response formatting: only forward the
    request message headers instead of all properties
    - adjust build script to only increase patch
    version on master push
1.4.1
2023-02-16 16:37:57 +01:00
Julius Unverfehrt
c97ae3d2c2 Pull request #56: RED-6118 multi tenancy
Merge in RR/pyinfra from RED-6118-multi-tenancy to master

Squashed commit of the following:

commit 0a1301f9d7a12a1097e6bf9a1bb0a94025312d0a
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Feb 16 09:12:54 2023 +0100

    delete (for now) not needed exception module

commit 9b624f9c95c129bf186eaea8405a14d359ccb1ae
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Thu Feb 16 09:08:57 2023 +0100

    implement message properties forwarding

    - revert tenant validation logic since this functionality is not wanted
    - implement request message properties forwarding to response message.
    Thus, all message headers including x-tenant-id are present in the
    reponse.

commit ddac812d32eeec09d9434c32595875eb354767f8
Merge: ed4b495 6828c65
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Feb 15 17:00:54 2023 +0100

    Merge branch 'master' of ssh://git.iqser.com:2222/rr/pyinfra into RED-6118-multi-tenancy

commit ed4b4956c6cb6d201fc29b0318078dfb8fa99006
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Wed Feb 15 10:00:28 2023 +0100

    refactor

commit 970fd72aa73ace97d36f129031fb143209c5076b
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 14 17:22:54 2023 +0100

    RED-6118 make pyinfra multi-tenant ready

    - refactor message validation logic
    - add tenant validation step:
    	- messages without header/tenant id are accepted for now, until
    	  multi-tenancy is implemented in backend
    	- only valid tenant is 'redaction'

commit 0f04e799620e01b3346eeaf86f3e941830824202
Author: Julius Unverfehrt <julius.unverfehrt@iqser.com>
Date:   Tue Feb 14 15:42:28 2023 +0100

    add dev scripts

    - add scripts to ease pyinfra development by allowing to run pyinfra
    locally with callback mock and publishing script.
1.4.0
2023-02-16 09:44:43 +01:00
Francisco Schulz
6828c65396 Pull request #58: fix version conflict
Merge in RR/pyinfra from hotfix/version-conflict-during-build to master

* commit '73bfef686782112d448469ec14d84cab5965f318':
  increment version
1.3.0
2023-02-15 16:14:55 +01:00
Francisco Schulz
73bfef6867 increment version 1.2.0 2023-02-15 16:12:22 +01:00
Francisco Schulz
1af171bd3f Pull request #57: Bugfix/RED-5277 investigate missing heartbeat error
Merge in RR/pyinfra from bugfix/RED-5277-investigate-missing-heartbeat-error to master

Squashed commit of the following:

commit 9e139e79e46c52014986f9afb2c6534281b55c10
Author: Viktor Seifert <viktor.seifert@iqser.com>
Date:   Wed Feb 15 14:56:44 2023 +0100

    RED-5277: Moved async processing to its own functions

commit 244a941299dbf75b254adcad8b068b2917c6bf79
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 11:26:00 2023 +0100

    Revert "only set git tag on release and master branches"

    This reverts commit 9066856d223f0646723fa1c62c444e16a9bb3ce9.

commit adb35db6fa6daf4b79263a918716c34905e8b3bc
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 11:11:07 2023 +0100

    increment version

commit 9066856d223f0646723fa1c62c444e16a9bb3ce9
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 11:10:49 2023 +0100

    only set git tag on release and master branches

commit ee11e018efdbc63a740008e7fa2415cbb12476ae
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 10:18:08 2023 +0100

    configure root logger in `__init__.py`
    only set log levels for other loggers, inherit config

commit 776399912ddf1e936138cceb2af981f27d333823
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 10:16:57 2023 +0100

    update dependency via `poetry update`

commit 804a8d9fbd1ded3e154fe9b3cafa32428522ca0f
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Wed Feb 15 10:16:25 2023 +0100

    increment version

commit cf057daed23d5f5b0f6f3a1a31e956e015e86368
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:59:55 2023 +0100

    update

commit 51717d85fce592b8bf38a8b5235faa04379cce1a
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:48:51 2023 +0100

    define sonar source

commit ace57c211a61d8e473a700da161806f882b19dc6
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:46:24 2023 +0100

    update plan

commit 1fcc00eb18ed692e2646873b4a233a00b5f6d93b
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:46:13 2023 +0100

    fix typo

commit 20b59768a68d985e1bf2fe6f93a1e6283bac5cb0
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:43:39 2023 +0100

    increment version

commit 8e7b4bf302b5591b2c490ad89c8a01a87c5b4741
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 17:11:59 2023 +0100

    get rid of extra logger

commit 3fd3eb255c252d1e208b88b475ec8a07c521619d
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 16:45:56 2023 +0100

    increment version

commit b0b5e5ebd94554cdafed6cff333d73a9ba08bea1
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 16:40:22 2023 +0100

    update

commit b87b3c351722d6949833c397178bc0354c754d90
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 16:38:41 2023 +0100

    fix tag issue from build

commit 73f3dcb280b6f905eeef3c69123b1252e6c934b1
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 14:21:57 2023 +0100

    add comments & update logging

commit 72a9e2c51f5bf98fc9f0803183fc1d28aaea9e35
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 12:06:09 2023 +0100

    cleanup comments

commit 587814944921f0f148e4d3c4c76d4edffff55bba
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 11:16:17 2023 +0100

    use thread executor in a `with` statement

commit 9561a6b447d98d2f0d536f63c0946d7bf1e2ca7d
Author: Francisco Schulz <Francisco.Schulz@iqser.com>
Date:   Tue Feb 14 10:42:49 2023 +0100

    fix unbound issue `callback_result` & shutdown thread executor

... and 23 more commits
2023-02-15 16:02:17 +01:00
Francisco Schulz
61efbdaffd Pull request #55: add master to non-dev branches
Merge in RR/pyinfra from bugfix/add-master-to-non-dev-branches to master

* commit 'c94604cc666ec4a9d3803c949f228cbf4291aaf2':
  add master to non-dev branches
1.1.1a9
2022-11-16 10:44:43 +01:00
Francisco Schulz
c94604cc66 add master to non-dev branches 1.1.1a8 2022-11-16 10:35:33 +01:00
Francisco Schulz
edbe5fa4f0 Pull request #54: Feature/MLOPS-32 update pyinfra to use pypoetry.toml
Merge in RR/pyinfra from feature/MLOPS-32-update-pyinfra-to-use-pypoetry.toml to master

* commit '37d8ee49a22ab9ee81792217404ed0a7daea65c2': (34 commits)
  add convenience command for version updates
  testing version is ahead in project
  test equal version number
  echo latest git version tag
  update tag fetching
  rollback
  testing hardcoded
  remove specific planRepository
  remove parentheses
  change project key
  add planRepositories config
  fix typo: licence -> license
  ignore bamboo YAML configs
  switch back to bamboo Java config
  update version tag manually
  remove superfulous `then`
  isolate feature/bugfix/hotfix and dev tag setting
  fix script `echo` was missing
  add version update shortcut
  show pyproject.toml file
  ...
2022-11-15 16:03:59 +01:00
Francisco Schulz
37d8ee49a2 add convenience command for version updates 1.1.1a7 2022-11-15 15:56:54 +01:00
Francisco Schulz
7732e884c5 testing version is ahead in project 2022-11-15 15:56:35 +01:00
Francisco Schulz
40e516b4e8 test equal version number 1.1.1a6 2022-11-15 15:47:32 +01:00
Francisco Schulz
c8c0210945 echo latest git version tag 1.1.1a5 2022-11-15 15:41:12 +01:00
Francisco Schulz
280b14b4a0 update tag fetching 2022-11-15 15:13:19 +01:00
Kevin Tumma
203c0f669c rollback 2022-11-15 14:14:12 +01:00
Kevin Tumma
8227e18580 testing hardcoded 1.1.1a4 2022-11-15 14:07:23 +01:00
Francisco Schulz
73bb38f917 remove specific planRepository 2022-11-15 13:18:25 +01:00
Francisco Schulz
fa76003983 remove parentheses 2022-11-15 11:40:16 +01:00
Francisco Schulz
19540c7c08 change project key 2022-11-15 11:39:08 +01:00
Francisco Schulz
2fe4a75a57 add planRepositories config 2022-11-15 11:37:40 +01:00
Francisco Schulz
b5deb7b292 fix typo: licence -> license 2022-11-15 09:28:41 +01:00
Francisco Schulz
5cd30c08b3 ignore bamboo YAML configs 1.1.1a3 2022-11-15 09:00:46 +01:00
Francisco Schulz
17cbbeb620 switch back to bamboo Java config 2022-11-15 08:59:47 +01:00
Francisco Schulz
9c4cf3d220 update version tag manually 1.1.1a2 2022-11-14 17:04:30 +01:00
Francisco Schulz
3ccb1d2370 remove superfulous then 2022-11-14 16:26:19 +01:00
Francisco Schulz
398b1c271f isolate feature/bugfix/hotfix and dev tag setting 2022-11-14 16:20:16 +01:00
Francisco Schulz
05658784be fix script
`echo` was missing
1.1.1a1
2022-11-14 15:49:44 +01:00
Francisco Schulz
974df96bb9 add version update shortcut 2022-11-14 15:49:17 +01:00
Francisco Schulz
ca3f812527 show pyproject.toml file 2022-11-14 15:38:10 +01:00
Francisco Schulz
d78e6c45fb separate git-tag into own stage 2022-11-14 15:37:56 +01:00
Francisco Schulz
41220d3c80 increment version 2022-11-14 10:39:36 +01:00
Francisco Schulz
2d2e72c86e remove redundancies 2022-11-14 10:39:30 +01:00
Francisco Schulz
37b0280ab6 update tag logic 2022-11-10 17:38:18 +01:00
Francisco Schulz
4bd6ee867f fix circumflex formatting 1.1.1a0 2022-11-10 16:52:11 +01:00
Francisco Schulz
b0efed4007 update regex feature/MLOPS-32-update-pyinfra-to-use-pypoetry.toml_11 2022-11-10 16:45:32 +01:00
Francisco Schulz
28ee14e92f echo bamboo vars feature/MLOPS-32-update-pyinfra-to-use-pypoetry.toml_10 2022-11-10 16:04:38 +01:00
Francisco Schulz
fb3d4b5fc9 add sonar config feature/MLOPS-32-update-pyinfra-to-use-pypoetry.toml_9 2022-11-10 15:25:44 +01:00
Francisco Schulz
84351fd75c fix formatting issue 2022-11-10 13:24:51 +01:00
Francisco Schulz
244aaec470 use inline config-keys script opposed to file 2022-11-10 13:21:38 +01:00
Francisco Schulz
18d614f61c use bamboo config YAML 2022-11-10 13:17:01 +01:00
Francisco Schulz
7472939f21 ignore checks for bamboo.yml
otherwise check-yaml throws multi-file exception
2022-11-10 13:16:54 +01:00
Francisco Schulz
a819e60632 update 2022-11-10 13:09:09 +01:00
Francisco Schulz
05d5582479 convert into python package
- remove build specs
- move pytest.ini into pyproject.toml
- update readme
- add pre-commit config
- run formatters
- add Makefile
1.1.0
2022-11-03 16:10:12 +01:00
Francisco Schulz
64d6a8cec6 Pull request #53: Feature/MLOPS-23 pyinfra does not use the value of the storage azurecontainername environment
Merge in RR/pyinfra from feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment to master

* commit '7a740403bb65db97c8e4cb54de00aac3536b2e4c':
  update
  update test config
  update test config
  add pytests to check if a configured bucket can be found
  add submodule initialization
  load different env vars for the  variable depending on the set
1.0.9 feature/MLOPS-32-update-pyinfra-to-use-pypoetry.toml_2 master_75 release/3.6.x_2
2022-10-13 15:09:53 +02:00
Francisco Schulz
7a740403bb update feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment_7 2022-10-13 14:04:00 +02:00
Francisco Schulz
b2cd529519 update test config feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment_6 2022-10-13 13:53:53 +02:00
Francisco Schulz
1891519e19 update test config 2022-10-13 13:17:31 +02:00
Francisco Schulz
843d91c61a add pytests to check if a configured bucket can be found feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment_5 2022-10-13 11:26:58 +02:00
Francisco Schulz
5b948fdcc5 add submodule initialization feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment_4 2022-10-13 11:01:29 +02:00
Francisco Schulz
bb5b73e189 load different env vars for the variable depending on the set feature/MLOPS-23-pyinfra-does-not-use-the-value-of-the-storage_azurecontainername-environment_3 2022-10-13 10:29:57 +02:00