From 030d71c7eeac13700fa8386cfcece19240165bd5 Mon Sep 17 00:00:00 2001 From: Julius Unverfehrt Date: Tue, 1 Mar 2022 13:21:54 +0100 Subject: [PATCH] Readme update WIP --- README.md | 84 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 61 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 26fc77a..287fad9 100755 --- a/README.md +++ b/README.md @@ -1,34 +1,72 @@ -# SETUP +# Infrastructure to deploy Research Projects -## 1. module +The Infrastructure expects to be deployed in the same Pod / local environment as the analysis container and handles all outbound communication. -```bash -pip install -e . -pip install -r requirements.txt -``` +## Configuration -## 1. docker image +A configuration is located in `/config.yaml`. All relevant variables can be configured via exporting environment variables. -```bash -docker build -f Dockerfile -t pyinfra . -``` +| Environment Variable | Default | Description | +|-------------------------------|--------------------------------|--------------------------------------------------------------------------------| +| _service_ | | | +| LOGGING_LEVEL_ROOT | DEBUG | Logging level for service logger | +| RESPONSE_FILE_EXTENSION | ".NER_ENTITIES.json.gz" | Extension to the file that stores the analyized response on storage | +| _probing_webserver_ | | | +| PROBING_WEBSERVER_HOST | "0.0.0.0" | Probe webserver address | +| PROBING_WEBSERVER_PORT | 8080 | Probe webserver port | +| PROBING_WEBSERVER_MODE | production | Webserver mode: {development, production} | +| _rabbitmq_ | | | +| RABBITMQ_HOST | localhost | RabbitMQ host address | +| RABBITMQ_PORT | 5672 | RabbitMQ host port | +| RABBITMQ_USERNAME | user | RabbitMQ username | +| RABBITMQ_PASSWORD | bitnami | RabbitMQ password | +| RABBITMQ_HEARTBEAT | 7200 | Controls AMQP heartbeat timeout in seconds | +| _queues_ | | | +| REQUEST_QUEUE | request_queue | Requests to service | +| RESPONSE_QUEUE | response_queue | Responses by service | +| DEAD_LETTER_QUEUE | dead_letter_queue | Messages that failed to process | +| _callback_ | | | +| RETRY | False | Toggles retry behaviour | +| MAX_ATTEMPTS | 3 | Number of times a message may fail before being published to dead letter queue | +| ANALYSIS_ENDPOINT | "http://127.0.0.1:5000" | | +| _storage_ | | | +| STORAGE_BACKEND | s3 | The type of storage to use {s3, azure} | +| STORAGE_BUCKET | "pyinfra-test-bucket" | The bucket / container to pull files specified in queue requests from | +| TARGET_FILE_EXTENSION | ".TEXT.json.gz" | Defines type of file to pull from storage: .TEXT.json.gz or .ORIGIN.pdf.gz | +| STORAGE_ENDPOINT | "http://127.0.0.1:9000" | | +| STORAGE_KEY | root | | +| STORAGE_SECRET | password | | +| STORAGE_AZURECONNECTIONSTRING | "DefaultEndpointsProtocol=..." | | -# RUN +## Development -## 2. rabbitmq & minio +### Local Setup -```bash -docker-compose up -``` +You can run the infrastructure either as module via. `src/serve.py` or as Dockercontainer simulating the kubernetes environment -## 3. module +1. Install module / build docker image -```bash -python src/serve.py -``` + ```bash + pip install -e . + pip install -r requirements.txt + ``` -## 3. docker container + ```bash + docker build -f Dockerfile -t pyinfra . + ``` -```bash -docker run --net=host pyinfra -``` +2. Run rabbitmq & minio + + ```bash + docker-compose up + ``` + +3. Run module / docker container + + ```bash + python src/serve.py + ``` + + ```bash + docker run --net=host pyinfra + ```