Marmelator/pyinfra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 36 Wiki Activity

feat(opentelemetry): add webserver tracing to default pipeline

Browse Source
This commit is contained in:
Julius Unverfehrt 2024-01-31 09:09:13 +01:00
parent 88cfb2b1c1
commit 7093e01925
3 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -11,7 +11,7 @@
Shared library for the research team, containing code related to infrastructure and communication with other services. Shared library for the research team, containing code related to infrastructure and communication with other services.
Offers a simple interface for processing data and sending responses via AMQP, monitoring via Prometheus and storage Offers a simple interface for processing data and sending responses via AMQP, monitoring via Prometheus and storage
access via S3 or Azure. access via S3 or Azure. Also export traces via OpenTelemetry for queue messages and webserver requests.
To start, see the [complete example](pyinfra/examples.py) which shows how to use all features of the service and can be To start, see the [complete example](pyinfra/examples.py) which shows how to use all features of the service and can be
imported and used directly for default research service pipelines (data ID in message, download data from storage, imported and used directly for default research service pipelines (data ID in message, download data from storage,
@ -66,7 +66,7 @@ configured
from environment variables, without additional work in the microservice app, although additional confiuration is from environment variables, without additional work in the microservice app, although additional confiuration is
possible. possible.
`TRACING_ENDPOINT` should typically be set `TRACING__OPENTELEMETRY__ENDPOINT` should typically be set
to `http://otel-collector-opentelemetry-collector.otel-collector:4318/v1/traces`. to `http://otel-collector-opentelemetry-collector.otel-collector:4318/v1/traces`.
## Queue Manager ## Queue Manager

3
pyinfra/examples.py
View File

@ -5,7 +5,7 @@ from kn_utils.logging import logger
from pyinfra.config.loader import get_pyinfra_validators, validate_settings from pyinfra.config.loader import get_pyinfra_validators, validate_settings
from pyinfra.queue.callback import Callback from pyinfra.queue.callback import Callback
from pyinfra.queue.manager import QueueManager from pyinfra.queue.manager import QueueManager
from pyinfra.utils.opentelemetry import instrument_pika, setup_trace from pyinfra.utils.opentelemetry import instrument_pika, setup_trace, instrument_app
from pyinfra.webserver.prometheus import ( from pyinfra.webserver.prometheus import (
add_prometheus_endpoint, add_prometheus_endpoint,
make_prometheus_processing_time_decorator_from_settings, make_prometheus_processing_time_decorator_from_settings,
@ -45,6 +45,7 @@ def start_standard_queue_consumer(
logger.info(f"OpenTelemetry tracing enabled.") logger.info(f"OpenTelemetry tracing enabled.")
setup_trace(settings) setup_trace(settings)
instrument_pika() instrument_pika()
instrument_app(app)
app = add_health_check_endpoint(app, queue_manager.is_ready) app = add_health_check_endpoint(app, queue_manager.is_ready)

4
pyinfra/utils/opentelemetry.py
View File

@ -69,5 +69,5 @@ def instrument_pika():
PikaInstrumentor().instrument() PikaInstrumentor().instrument()
def instrument_app(app: FastAPI): def instrument_app(app: FastAPI, excluded_urls: str = "/health,/ready,/prometheus"):
FastAPIInstrumentor().instrument_app(app) FastAPIInstrumentor().instrument_app(app, excluded_urls=excluded_urls)