05d5582479
- remove build specs - move pytest.ini into pyproject.toml - update readme - add pre-commit config - run formatters - add Makefile
72 lines
3.9 KiB
Markdown
Executable File
72 lines
3.9 KiB
Markdown
Executable File
# Infrastructure to deploy Research Projects
|
|
|
|
The Infrastructure expects to be deployed in the same Pod / local environment as the analysis container and handles all outbound communication.
|
|
|
|
## Configuration
|
|
|
|
A configuration is located in `/config.yaml`. All relevant variables can be configured via exporting environment variables.
|
|
|
|
| Environment Variable | Default | Description |
|
|
| ----------------------------- | ------------------------------ | --------------------------------------------------------------------- |
|
|
| LOGGING_LEVEL_ROOT | DEBUG | Logging level for service logger |
|
|
| PROBING_WEBSERVER_HOST | "0.0.0.0" | Probe webserver address |
|
|
| PROBING_WEBSERVER_PORT | 8080 | Probe webserver port |
|
|
| PROBING_WEBSERVER_MODE | production | Webserver mode: {development, production} |
|
|
| RABBITMQ_HOST | localhost | RabbitMQ host address |
|
|
| RABBITMQ_PORT | 5672 | RabbitMQ host port |
|
|
| RABBITMQ_USERNAME | user | RabbitMQ username |
|
|
| RABBITMQ_PASSWORD | bitnami | RabbitMQ password |
|
|
| RABBITMQ_HEARTBEAT | 7200 | Controls AMQP heartbeat timeout in seconds |
|
|
| REQUEST_QUEUE | request_queue | Requests to service |
|
|
| RESPONSE_QUEUE | response_queue | Responses by service |
|
|
| DEAD_LETTER_QUEUE | dead_letter_queue | Messages that failed to process |
|
|
| ANALYSIS_ENDPOINT | "http://127.0.0.1:5000" | Endpoint for analysis container |
|
|
| STORAGE_BACKEND | s3 | The type of storage to use {s3, azure} |
|
|
| STORAGE_BUCKET | "redaction" | The bucket / container to pull files specified in queue requests from |
|
|
| STORAGE_ENDPOINT | "http://127.0.0.1:9000" | Endpoint for s3 storage |
|
|
| STORAGE_KEY | root | User for s3 storage |
|
|
| STORAGE_SECRET | password | Password for s3 storage |
|
|
| STORAGE_AZURECONNECTIONSTRING | "DefaultEndpointsProtocol=..." | Connection string for Azure storage |
|
|
| STORAGE_AZURECONTAINERNAME | "redaction" | AKS container |
|
|
|
|
## Response Format
|
|
|
|
### Expected AMQP input message:
|
|
|
|
```json
|
|
{
|
|
"dossierId": "",
|
|
"fileId": "",
|
|
"targetFileExtension": "",
|
|
"responseFileExtension": ""
|
|
}
|
|
```
|
|
|
|
Optionally, the input message can contain a field with the key `"operations"`.
|
|
|
|
### AMQP output message:
|
|
|
|
```json
|
|
{
|
|
"dossierId": "",
|
|
"fileId": "",
|
|
...
|
|
}
|
|
```
|
|
|
|
### Setup
|
|
|
|
Install project dependencies
|
|
|
|
```bash
|
|
make poetry
|
|
```
|
|
|
|
You don't have to install it independently in the project repo, just `import pyinfra` in any `.py`-file
|
|
|
|
or install form another project
|
|
|
|
```bash
|
|
poetry add git+ssh://git@git.iqser.com:2222/rr/pyinfra.git#TAG-NUMBER
|
|
```
|