diff --git a/docs/en/docs/advanced/opentelemetry.md b/docs/en/docs/advanced/opentelemetry.md new file mode 100644 index 0000000000..67310909cc --- /dev/null +++ b/docs/en/docs/advanced/opentelemetry.md @@ -0,0 +1,161 @@ +# OpenTelemetry { #opentelemetry } + +When your API is running, you might want to know how much traffic it receives, which requests are slow, and when errors happen. + +**Telemetry** is data about your application's behavior that helps you answer these questions. Common types include: + +- **Metrics**: measurements you can summarize over time, such as response times and the number of requests being handled. +- **Traces**: records of individual requests and the operations performed to handle them. Each timed operation is called a **span**. +- **Logs**: timestamped records of events, such as an application starting or an operation failing. + +[**OpenTelemetry**](https://opentelemetry.io/) is a set of standards and tools for collecting telemetry and sending it to a monitoring service, where you can explore it in dashboards. + +**FastAPI provides OpenTelemetry support by default** for HTTP request traces, metrics, and logs. WebSocket connections also provide traces and logs. To see that data, configure a monitoring service to receive it. + +## Install FastAPI { #install-fastapi } + +Install FastAPI with the `standard` extras, which include the packages for sending telemetry: + +
+ +```console +$ uv add "fastapi[standard]" +---> 100% +``` + +
+ +## Create the app { #create-the-app } + +Create a file `main.py`: + +{* ../../docs_src/opentelemetry/tutorial001_py310.py *} + +Notice that it all works by default, you don't need to write any custom code for telemetry to work. + +## FastAPI Cloud { #fastapi-cloud } + +When you deploy to [FastAPI Cloud](https://fastapicloud.com) with `fastapi[standard]`, metrics work automatically. You don't have to configure anything else. + +On Pro plans, you can view request counts, error rates, and response times in the [Metrics dashboard](https://fastapicloud.com/docs/monitoring-and-performance/metrics/). + +FastAPI Cloud Pro metrics dashboard with example data + +## Other monitoring services { #other-monitoring-services } + +To send telemetry to another monitoring service, configure an endpoint that accepts **OTLP**, the OpenTelemetry protocol for sending telemetry. Use the service's HTTP/protobuf base endpoint. + +Set these environment variables, replacing the example URL with your endpoint: + +```bash +export OTEL_SERVICE_NAME=my-api +export OTEL_EXPORTER_OTLP_ENDPOINT=https://collector.example.com +``` + +`OTEL_SERVICE_NAME` identifies your app in the monitoring service. The endpoint is the base URL for receiving data. Traces are sent to `/v1/traces`, metrics to `/v1/metrics`, and logs to `/v1/logs` under that URL. + +If your service requires authentication, set `OTEL_EXPORTER_OTLP_HEADERS` to the headers it specifies, for example `api-key=YOUR_API_KEY`. + +## Run the app { #run-the-app } + +Start the app in the same terminal: + +
+ +```console +$ uv run fastapi run +``` + +
+ +In another terminal, send a request: + +```console +$ curl http://127.0.0.1:8000/items/1 +{"item_id":1} +``` + +Open your monitoring service and find `my-api`. After the next export, you can see a trace with a `GET /items/{item_id}` span, along with metrics for request counts, response duration, and active requests. + +## Customize telemetry { #customize-telemetry } + +### Configure providers and exporters { #configure-providers-and-exporters } + +A **provider** supplies the objects that record traces, metrics, or logs. Its configuration controls how that data is processed and exported. + +Telemetry libraries can configure OpenTelemetry's global providers. Configure the library before the app starts, and FastAPI uses those providers automatically. + +When an OTLP endpoint is set in the environment, FastAPI adds an exporter for that destination to each enabled provider. Existing exporters continue sending data to their destinations. + +Configure each destination once. If another library already handles the environment destination, disable its environment export or turn off FastAPI's automatic setup: + +```python +app = FastAPI(telemetry={"auto_configure": False}) +``` + +You can also pass a provider directly in the `telemetry` dictionary. For example, this provider uses OpenTelemetry's console exporter to print request spans in your terminal: + +{* ../../docs_src/opentelemetry/tutorial002_py310.py hl[2:8] *} + +The **exporter** sends the spans to their destination. `BatchSpanProcessor` groups spans and sends them in the background. Replace the console exporter with one supplied by your monitoring library to use its destination. See [OpenTelemetry's Python instrumentation guide](https://opentelemetry.io/docs/languages/python/instrumentation/) for more configuration options. + +Use `meter_provider` or `logger_provider` in the same dictionary to supply a metrics or logs provider. The application or library creating a provider manages its shutdown. FastAPI manages the export components it adds. + +/// warning + +OpenTelemetry uses global providers by default. Independent telemetry configuration for [mounted sub-applications](sub-applications.md) is not guaranteed. + +/// + +### Trace request operations { #trace-request-operations } + +By default, request traces include spans for resolving dependencies, running your path operation function, serializing the response, and running each task in FastAPI's `BackgroundTasks`. These spans use the same provider and exporters. + +Background task spans remain part of the request's trace. They run after the HTTP response span ends, so they do not increase the measured response time. + +To record only the HTTP request span, set `operation_spans` to `False`: + +{* ../../docs_src/opentelemetry/tutorial003_py310.py hl[3] *} + + +### Trace WebSocket connections { #trace-websocket-connections } + +Each WebSocket connection has a span such as `WS /ws/{room}`, covering the handler and dependency cleanup. It uses the same providers and settings, including `operation_spans` for dependency resolution and endpoint execution. + +HTTP request metrics cover HTTP requests only. Normal WebSocket disconnects with codes `1000` or `1001` do not produce error logs. + +### Inspect errors { #inspect-errors } + +FastAPI records unhandled exceptions as OpenTelemetry logs, linked to the request's or connection's trace. Error logs are recorded even when the trace is not sampled. + +Exception logs include the exception's type, message, and stack trace. Messages and stack traces can contain sensitive information. Use your provider's log processors to filter or redact them, or set `logs` to `False` to disable these logs. + +FastAPI also records request validation failures as warning logs with the route and error count. These logs do not include the invalid input. + +## Choose what to record { #choose-what-to-record } + +The `telemetry` dictionary also accepts these settings: + +| Setting | Purpose | Default | +| --- | --- | --- | +| `tracing` | Record HTTP request and WebSocket connection spans | `True` | +| `metrics` | Record HTTP request metrics | `True` | +| `logs` | Record validation failures and unhandled exceptions | `True` | +| `operation_spans` | Add spans for request operations | `True` | +| `exclude` | Skip requests when a function receiving the ASGI scope returns `True` | `None` | +| `auto_configure` | Add exporters for endpoints set in environment variables | `True` | + +For example, to collect metrics while excluding health checks: + +```python +from fastapi import FastAPI + +app = FastAPI( + telemetry={ + "tracing": False, + "exclude": lambda scope: scope["path"] == "/health", + } +) +``` + +Set `auto_configure` to `False` when your application handles provider setup itself, such as inside its lifespan function. diff --git a/docs/en/docs/img/tutorial/opentelemetry/image01.png b/docs/en/docs/img/tutorial/opentelemetry/image01.png new file mode 100644 index 0000000000..1ce30bea36 Binary files /dev/null and b/docs/en/docs/img/tutorial/opentelemetry/image01.png differ diff --git a/docs/en/mkdocs.yml b/docs/en/mkdocs.yml index 9bcdbd652d..3ec657ac8f 100644 --- a/docs/en/mkdocs.yml +++ b/docs/en/mkdocs.yml @@ -159,6 +159,7 @@ nav: - advanced/templates.md - advanced/websockets.md - advanced/events.md + - advanced/opentelemetry.md - advanced/testing-websockets.md - advanced/testing-events.md - advanced/testing-dependencies.md diff --git a/docs_src/opentelemetry/tutorial001_py310.py b/docs_src/opentelemetry/tutorial001_py310.py new file mode 100644 index 0000000000..8272ad70d4 --- /dev/null +++ b/docs_src/opentelemetry/tutorial001_py310.py @@ -0,0 +1,8 @@ +from fastapi import FastAPI + +app = FastAPI() + + +@app.get("/items/{item_id}") +async def read_item(item_id: int): + return {"item_id": item_id} diff --git a/docs_src/opentelemetry/tutorial002_py310.py b/docs_src/opentelemetry/tutorial002_py310.py new file mode 100644 index 0000000000..b820cefb28 --- /dev/null +++ b/docs_src/opentelemetry/tutorial002_py310.py @@ -0,0 +1,13 @@ +from fastapi import FastAPI +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter + +tracer_provider = TracerProvider() +tracer_provider.add_span_processor(BatchSpanProcessor(ConsoleSpanExporter())) + +app = FastAPI(telemetry={"tracer_provider": tracer_provider}) + + +@app.get("/items/{item_id}") +async def read_item(item_id: int): + return {"item_id": item_id} diff --git a/docs_src/opentelemetry/tutorial003_py310.py b/docs_src/opentelemetry/tutorial003_py310.py new file mode 100644 index 0000000000..a5e65016e9 --- /dev/null +++ b/docs_src/opentelemetry/tutorial003_py310.py @@ -0,0 +1,8 @@ +from fastapi import FastAPI + +app = FastAPI(telemetry={"operation_spans": False}) + + +@app.get("/items/{item_id}") +async def read_item(item_id: int): + return {"item_id": item_id} diff --git a/fastapi/.agents/skills/fastapi/SKILL.md b/fastapi/.agents/skills/fastapi/SKILL.md index fc35b97ed9..164d645007 100644 --- a/fastapi/.agents/skills/fastapi/SKILL.md +++ b/fastapi/.agents/skills/fastapi/SKILL.md @@ -12,6 +12,7 @@ Official FastAPI skill to write code with best practices, keeping up to date wit * Serve frontend apps: use `app.frontend()` or `router.frontend()` for built frontend assets; see [Serve Frontend Apps](#serve-frontend-apps). * Server-Sent Events (SSE): use `response_class=EventSourceResponse` and `yield`; see [Streaming](#streaming-json-lines-sse-bytes) and [the streaming reference](references/streaming.md). * JSON Lines and byte streaming: see [the streaming reference](references/streaming.md). +* OpenTelemetry: use FastAPI's native traces, metrics, and logs. See [OpenTelemetry](#opentelemetry). * Dependencies: use `Annotated[..., Depends(...)]`; see [Dependency Injection](#dependency-injection) and [the dependency injection reference](references/dependencies.md) for `yield`, scopes, and class dependencies. * Response models: prefer return types; use `response_model` when the public response schema differs from the internal return value; see [the response reference](references/responses.md). * Pydantic models: do not use ellipsis or `RootModel`; see [the Pydantic reference](references/pydantic.md). @@ -203,6 +204,16 @@ app.include_router(router) `app.frontend()` and `router.frontend()` are low-priority routes: regular API routes are matched first, then frontend files and client-side routing fallbacks. Use this for single-page apps and built frontend assets instead of mounting `StaticFiles` manually. +## OpenTelemetry + +Prefer FastAPI's native OpenTelemetry support for request traces, metrics, and logs. + +Install `fastapi[standard]` to include the SDK and HTTP/protobuf exporters. Set `OTEL_SERVICE_NAME` to identify the app and `OTEL_EXPORTER_OTLP_ENDPOINT` to the collector's HTTP/protobuf base URL. Use `OTEL_EXPORTER_OTLP_HEADERS` when authentication is required. + +Use `FastAPI(telemetry={...})` for custom configuration, such as choosing signals or supplying providers. + +See the [OpenTelemetry tutorial](https://fastapi.tiangolo.com/advanced/opentelemetry/) for configuration details. + ## Dependency Injection Use dependencies when the logic can't be declared in Pydantic validation, depends on external resources, needs cleanup with `yield`, or is shared across endpoints. diff --git a/fastapi/applications.py b/fastapi/applications.py index b5fc76d160..689a828f3d 100644 --- a/fastapi/applications.py +++ b/fastapi/applications.py @@ -21,6 +21,12 @@ from fastapi.openapi.docs import ( ) from fastapi.openapi.utils import get_openapi from fastapi.params import Depends +from fastapi.telemetry import TelemetryConfig +from fastapi.telemetry._asgi import ( + ExceptionTelemetryMiddleware, + NativeTelemetry, + _legacy_otel, +) from fastapi.types import DecoratedCallable, IncEx from fastapi.utils import generate_unique_id from starlette.applications import Starlette @@ -58,6 +64,19 @@ class FastAPI(Starlette): def __init__( self: AppType, *, + telemetry: Annotated[ + TelemetryConfig | None, + Doc( + """ + Native OpenTelemetry configuration as a dictionary. Uses global + providers by default. Omitted options keep their defaults. + + ```python + app = FastAPI(telemetry={"tracing": False}) + ``` + """ + ), + ] = None, debug: Annotated[ bool, Doc( @@ -1011,6 +1030,20 @@ class FastAPI(Starlette): websocket_request_validation_exception_handler, # type: ignore[arg-type] ) # ty: ignore[no-matching-overload] + self._telemetry: TelemetryConfig = { + "tracer_provider": None, + "meter_provider": None, + "logger_provider": None, + "tracing": True, + "metrics": True, + "logs": True, + "operation_spans": True, + "auto_configure": True, + "exclude": None, + **(telemetry if telemetry is not None else {}), + } + self._native_telemetry = NativeTelemetry(self._telemetry) + self.user_middleware: list[Middleware] = ( [] if middleware is None else list(middleware) ) @@ -1031,7 +1064,10 @@ class FastAPI(Starlette): exception_handlers[key] = value middleware = ( - [Middleware(ServerErrorMiddleware, handler=error_handler, debug=debug)] + [ + Middleware(ServerErrorMiddleware, handler=error_handler, debug=debug), + Middleware(ExceptionTelemetryMiddleware), + ] + self.user_middleware + [ Middleware( @@ -1160,7 +1196,34 @@ class FastAPI(Starlette): async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: if self.root_path: scope["root_path"] = self.root_path - await super().__call__(scope, receive, send) + if scope["type"] == "lifespan": + from fastapi.telemetry._runtime import lifespan + + await lifespan( + config=self._telemetry, + app=super().__call__, + scope=scope, + receive=receive, + send=send, + ) + return + if ( + scope["type"] not in ("http", "websocket") + or "fastapi.telemetry" in scope + or not self._native_telemetry.enabled() + ): + await super().__call__(scope, receive, send) + return + if self.middleware_stack is None: + self.middleware_stack = self.build_middleware_stack() + scope["app"] = self + await self._native_telemetry( + app=super().__call__, + scope=scope, + receive=receive, + send=send, + legacy_otel=_legacy_otel(self.middleware_stack), + ) def add_api_route( self, diff --git a/fastapi/background.py b/fastapi/background.py index 7677058c43..37932a49e2 100644 --- a/fastapi/background.py +++ b/fastapi/background.py @@ -2,6 +2,7 @@ from collections.abc import Callable from typing import Annotated, Any from annotated_doc import Doc +from fastapi.telemetry._api import _operation from starlette.background import BackgroundTasks as StarletteBackgroundTasks from typing_extensions import ParamSpec @@ -59,3 +60,8 @@ class BackgroundTasks(StarletteBackgroundTasks): [FastAPI docs for Background Tasks](https://fastapi.tiangolo.com/tutorial/background-tasks/). """ return super().add_task(func, *args, **kwargs) + + async def __call__(self) -> None: + for task in self.tasks: + with _operation(name="background_task", function=task.func): + await task() diff --git a/fastapi/routing.py b/fastapi/routing.py index dc48a33d63..4ed655aa28 100644 --- a/fastapi/routing.py +++ b/fastapi/routing.py @@ -81,6 +81,13 @@ from fastapi.sse import ( ServerSentEvent, format_sse_event, ) +from fastapi.telemetry._api import ( + _operation, + _route_selected, + _run_sync_endpoint, + _validation_failed, + get_telemetry_data, +) from fastapi.types import DecoratedCallable, IncEx from fastapi.utils import ( create_model_field, @@ -349,9 +356,12 @@ async def run_endpoint_function( assert dependant.call is not None, "dependant.call must be a function" if is_coroutine: - return await dependant.call(**values) + with _operation(name="endpoint", function=dependant.call): + return await dependant.call(**values) else: - return await run_in_threadpool(dependant.call, **values) + return await run_in_threadpool( + _run_sync_endpoint, function=dependant.call, arguments=values + ) def _build_response_args( @@ -404,6 +414,9 @@ def get_request_handler( actual_strict_content_type = strict_content_type async def app(request: Request) -> Response: + telemetry_data = get_telemetry_data() + if telemetry_data is not None: + telemetry_data.request = request response: Response | None = None file_stack = request.scope.get("fastapi_middleware_astack") assert isinstance(file_stack, AsyncExitStack), ( @@ -462,6 +475,7 @@ def get_request_handler( body=e.doc, endpoint_ctx=endpoint_ctx, ) + _validation_failed(validation_error) raise validation_error from e except HTTPException: # If a middleware raises an HTTPException, it should be raised again @@ -472,20 +486,27 @@ def get_request_handler( ) raise http_error from e + if telemetry_data is not None: + telemetry_data.body = body + # Solve dependencies and run path operation function, auto-closing dependencies errors: list[Any] = [] async_exit_stack = request.scope.get("fastapi_inner_astack") assert isinstance(async_exit_stack, AsyncExitStack), ( "fastapi_inner_astack not found in request scope" ) - solved_result = await solve_dependencies( - request=request, - dependant=dependant, - body=cast(dict[str, Any] | FormData | bytes | None, body), - dependency_overrides_provider=dependency_overrides_provider, - async_exit_stack=async_exit_stack, - embed_body_fields=embed_body_fields, - ) + with _operation(name="dependencies", function=dependant.call): + solved_result = await solve_dependencies( + request=request, + dependant=dependant, + body=cast(dict[str, Any] | FormData | bytes | None, body), + dependency_overrides_provider=dependency_overrides_provider, + async_exit_stack=async_exit_stack, + embed_body_fields=embed_body_fields, + ) + if telemetry_data is not None: + telemetry_data.values = solved_result.values + telemetry_data.errors = solved_result.errors errors = solved_result.errors assert dependant.call # For types if not errors: @@ -724,19 +745,20 @@ def get_request_handler( use_dump_json = response_field is not None and isinstance( response_class, DefaultPlaceholder ) - content = await serialize_response( - field=response_field, - response_content=raw_response, - include=response_model_include, - exclude=response_model_exclude, - by_alias=response_model_by_alias, - exclude_unset=response_model_exclude_unset, - exclude_defaults=response_model_exclude_defaults, - exclude_none=response_model_exclude_none, - is_coroutine=is_coroutine, - endpoint_ctx=endpoint_ctx, - dump_json=use_dump_json, - ) + with _operation(name="serialization", function=dependant.call): + content = await serialize_response( + field=response_field, + response_content=raw_response, + include=response_model_include, + exclude=response_model_exclude, + by_alias=response_model_by_alias, + exclude_unset=response_model_exclude_unset, + exclude_defaults=response_model_exclude_defaults, + exclude_none=response_model_exclude_none, + is_coroutine=is_coroutine, + endpoint_ctx=endpoint_ctx, + dump_json=use_dump_json, + ) if use_dump_json: response = Response( content=content, @@ -752,6 +774,7 @@ def get_request_handler( validation_error = RequestValidationError( errors, body=body, endpoint_ctx=endpoint_ctx ) + _validation_failed(validation_error) raise validation_error # Return response @@ -767,6 +790,9 @@ def get_websocket_app( embed_body_fields: bool = False, ) -> Callable[[WebSocket], Coroutine[Any, Any, Any]]: async def app(websocket: WebSocket) -> None: + telemetry_data = get_telemetry_data() + if telemetry_data is not None: + telemetry_data.websocket = websocket endpoint_ctx = ( _extract_endpoint_context(dependant.call) if dependant.call @@ -780,20 +806,27 @@ def get_websocket_app( assert isinstance(async_exit_stack, AsyncExitStack), ( "fastapi_inner_astack not found in request scope" ) - solved_result = await solve_dependencies( - request=websocket, - dependant=dependant, - dependency_overrides_provider=dependency_overrides_provider, - async_exit_stack=async_exit_stack, - embed_body_fields=embed_body_fields, - ) + with _operation(name="dependencies", function=dependant.call): + solved_result = await solve_dependencies( + request=websocket, + dependant=dependant, + dependency_overrides_provider=dependency_overrides_provider, + async_exit_stack=async_exit_stack, + embed_body_fields=embed_body_fields, + ) + if telemetry_data is not None: + telemetry_data.values = solved_result.values + telemetry_data.errors = solved_result.errors if solved_result.errors: - raise WebSocketRequestValidationError( + validation_error = WebSocketRequestValidationError( solved_result.errors, endpoint_ctx=endpoint_ctx, ) + _validation_failed(validation_error) + raise validation_error assert dependant.call is not None, "dependant.call must be a function" - await dependant.call(**solved_result.values) + with _operation(name="endpoint", function=dependant.call): + await dependant.call(**solved_result.values) return app @@ -1791,6 +1824,12 @@ class _IncludedRouter(BaseRoute): await route.handle(scope, receive, send) return if effective_context is not None: + _route_selected( + scope=scope, + path=getattr(effective_context.starlette_route, "path_format", None) + or effective_context.path_format, + mount=isinstance(route, routing.Mount), + ) _get_fastapi_scope(scope)[_FASTAPI_EFFECTIVE_ROUTE_CONTEXT_KEY] = ( effective_context ) @@ -2189,6 +2228,10 @@ class _FrontendRouteGroup(BaseRoute): if match == Match.NONE or route is None: raise HTTPException(status_code=404) _update_scope(scope, child_scope) + _route_selected( + scope=scope, + path=_join_frontend_paths(prefix, route.path).rstrip("/") + "/{path}", + ) if match == Match.FULL and dependant and dependant.dependencies: async with self._solve_dependencies( scope, @@ -2731,6 +2774,11 @@ class APIRouter(routing.Router): match, child_scope = route.matches(scope) if match == Match.FULL: scope.update(child_scope) + _route_selected( + scope=scope, + path=getattr(route, "path_format", None), + mount=isinstance(route, routing.Mount), + ) await route.handle(scope, receive, send) return if match == Match.PARTIAL and partial is None: @@ -2739,6 +2787,11 @@ class APIRouter(routing.Router): if partial is not None: route, child_scope = partial scope.update(child_scope) + _route_selected( + scope=scope, + path=getattr(route, "path_format", None), + mount=isinstance(route, routing.Mount), + ) await route.handle(scope, receive, send) return @@ -2753,6 +2806,23 @@ class APIRouter(routing.Router): for route in self.routes: match, _ = route.matches(redirect_scope) if match != Match.NONE: + if scope.get("fastapi.telemetry") is not None: + telemetry_route: BaseRoute | _EffectiveRouteContext | None = ( + route + ) + while isinstance(telemetry_route, _IncludedRouter): + _, _, matched_route, matched_context = ( + telemetry_route._match(redirect_scope) + ) + telemetry_route = ( + matched_context.starlette_route or matched_context + if matched_context is not None + else matched_route + ) + _route_selected( + scope=scope, + path=getattr(telemetry_route, "path_format", None), + ) redirect_url = URL(scope=redirect_scope) response = RedirectResponse(url=str(redirect_url)) await response(scope, receive, send) @@ -2766,6 +2836,12 @@ class APIRouter(routing.Router): ) = self._match_low_priority(scope) if low_priority_match != Match.NONE and low_priority_route is not None: _update_scope(scope, low_priority_scope) + _route_selected( + scope=scope, + path=getattr( + low_priority_context or low_priority_route, "path_format", None + ), + ) if low_priority_context is not None: _get_fastapi_scope(scope)[_FASTAPI_EFFECTIVE_ROUTE_CONTEXT_KEY] = ( low_priority_context diff --git a/fastapi/telemetry/__init__.py b/fastapi/telemetry/__init__.py new file mode 100644 index 0000000000..d31e7d1638 --- /dev/null +++ b/fastapi/telemetry/__init__.py @@ -0,0 +1,3 @@ +from ._api import TelemetryConfig as TelemetryConfig +from ._api import TelemetryData as TelemetryData +from ._api import get_telemetry_data as get_telemetry_data diff --git a/fastapi/telemetry/_api.py b/fastapi/telemetry/_api.py new file mode 100644 index 0000000000..d9b581e285 --- /dev/null +++ b/fastapi/telemetry/_api.py @@ -0,0 +1,276 @@ +from collections.abc import Callable, Iterator, MutableMapping, Sequence +from contextlib import AbstractContextManager, contextmanager, nullcontext +from dataclasses import dataclass, field +from time import time_ns +from typing import Annotated, Any + +from annotated_doc import Doc +from fastapi.exceptions import RequestValidationError, WebSocketRequestValidationError +from opentelemetry import context as otel_context +from opentelemetry._logs import Logger, LoggerProvider, SeverityNumber +from opentelemetry.context import Context +from opentelemetry.metrics import MeterProvider +from opentelemetry.trace import Span, StatusCode, Tracer, TracerProvider +from starlette.exceptions import HTTPException, WebSocketException +from starlette.requests import Request +from starlette.websockets import WebSocket, WebSocketDisconnect +from typing_extensions import TypedDict + +_HTTP_METHODS = frozenset( + { + "CONNECT", + "DELETE", + "GET", + "HEAD", + "OPTIONS", + "PATCH", + "POST", + "PUT", + "QUERY", + "TRACE", + } +) + + +class TelemetryConfig(TypedDict, total=False): + """Optional settings for `FastAPI(telemetry={...})`. + + Omitted settings keep their defaults. FastAPI preserves existing providers + and their exporters. Environment setup can add exporters unless `auto_configure` + is `False`. Providers supplied by the application are never shut down by FastAPI. + """ + + tracer_provider: Annotated[ + TracerProvider | None, + Doc("Use this provider instead of the global tracer provider."), + ] + meter_provider: Annotated[ + MeterProvider | None, + Doc("Use this provider instead of the global meter provider."), + ] + logger_provider: Annotated[ + LoggerProvider | None, + Doc("Use this provider instead of the global logger provider."), + ] + tracing: Annotated[ + bool, + Doc("Enable HTTP request and WebSocket connection spans. Defaults to `True`."), + ] + metrics: Annotated[bool, Doc("Enable HTTP request metrics. Defaults to `True`.")] + logs: Annotated[ + bool, + Doc( + "Record validation failures and unhandled exceptions, including exception messages and stack traces. Defaults to `True`." + ), + ] + operation_spans: Annotated[ + bool, + Doc( + "Trace dependency resolution, endpoints, serialization, and background tasks. Defaults to `True`." + ), + ] + auto_configure: Annotated[ + bool, + Doc("Add OTLP exporters from environment variables. Defaults to `True`."), + ] + exclude: Annotated[ + Callable[[MutableMapping[str, Any]], bool] | None, + Doc( + "A function that receives the ASGI scope. Return `True` to skip FastAPI telemetry for that request or connection." + ), + ] + + +@dataclass(kw_only=True) +class TelemetryData: + """Local request or connection data for synchronous OpenTelemetry processors. + + FastAPI populates these fields as it handles the request or connection. + Integrations should treat the data as read-only and apply their capture and + redaction settings before exporting any of it. + """ + + request: Annotated[ + Request | None, Doc("The request, available when its route handler starts.") + ] = None + websocket: Annotated[ + WebSocket | None, + Doc("The WebSocket, available when its route handler starts."), + ] = None + body: Annotated[ + Any, Doc("The body read by FastAPI, before parameter validation.") + ] = None + values: Annotated[ + dict[str, Any] | None, + Doc("The original parsed arguments, available after dependency resolution."), + ] = None + errors: Annotated[ + Sequence[Any] | None, + Doc("Request validation errors, including the original input values."), + ] = None + + +@dataclass(kw_only=True) +class _RequestTelemetry: + span: Span | None + span_method: str + tracer: Tracer | None = None + logger: Logger | None = None + route: str | None = None + status_code: int | None = None + data: TelemetryData | None = field(default_factory=TelemetryData) + _mount_prefix: str = "" + _exceptions: list[BaseException] = field(default_factory=list, repr=False) + + +_REQUEST_TELEMETRY_KEY = otel_context.create_key("fastapi.request") +_NO_OPERATION = nullcontext() + + +def _get_request_telemetry(context: Context | None = None) -> _RequestTelemetry | None: + value = otel_context.get_value(_REQUEST_TELEMETRY_KEY, context) + return value if isinstance(value, _RequestTelemetry) else None + + +def get_telemetry_data(context: Context | None = None) -> TelemetryData | None: + """Read local FastAPI data from an OpenTelemetry context. + + Defaults to the current context. Log processors can pass `record.context`. + Read the data synchronously while FastAPI handles the request or connection. + After it finishes, this returns `None`, including for retained contexts. + This data is not automatically added to exported spans or logs. + """ + request_telemetry = _get_request_telemetry(context) + return request_telemetry.data if request_telemetry is not None else None + + +def _validation_failed( + exc: RequestValidationError | WebSocketRequestValidationError, +) -> None: + request_telemetry = _get_request_telemetry() + if request_telemetry is None: + return + if request_telemetry.data is not None: + if isinstance(exc, RequestValidationError): + request_telemetry.data.body = exc.body + request_telemetry.data.errors = exc.errors() + if request_telemetry.logger is not None: + attributes: dict[str, Any] = { + "fastapi.validation.error_count": len(exc.errors()) + } + if request_telemetry.route is not None: + attributes["http.route"] = request_telemetry.route + request_telemetry.logger.emit( + event_name="fastapi.validation.failed", + timestamp=time_ns(), + severity_number=SeverityNumber.WARN, + severity_text="WARN", + body="Request validation failed", + attributes=attributes, + ) + + +def _exception_type(exc: BaseException) -> str: + cls = type(exc) + return ( + f"{cls.__module__}.{cls.__qualname__}" + if cls.__module__ != "builtins" + else cls.__qualname__ + ) + + +def _normal_websocket_disconnect(exc: BaseException) -> bool: + return isinstance(exc, WebSocketDisconnect) and exc.code in (1000, 1001) + + +def _operation( + *, name: str, function: Callable[..., Any] | None = None +) -> AbstractContextManager[None]: + request_telemetry = _get_request_telemetry() + if request_telemetry is None or request_telemetry.tracer is None: + return _NO_OPERATION + return _traced_operation( + request_telemetry=request_telemetry, name=name, function=function + ) + + +@contextmanager +def _traced_operation( + *, + request_telemetry: _RequestTelemetry, + name: str, + function: Callable[..., Any] | None, +) -> Iterator[None]: + assert request_telemetry.tracer is not None + attributes = {} + if function is not None: + module = getattr(function, "__module__", type(function).__module__) + qualname = getattr(function, "__qualname__", type(function).__qualname__) + attributes["code.function.name"] = f"{module}.{qualname}" + with request_telemetry.tracer.start_as_current_span( + f"fastapi.{name}", + attributes=attributes, + record_exception=False, + set_status_on_exception=False, + ) as span: + try: + yield + except Exception as exc: + if name == "background_task" or ( + not isinstance( + exc, + ( + HTTPException, + RequestValidationError, + WebSocketException, + WebSocketRequestValidationError, + ), + ) + and not _normal_websocket_disconnect(exc) + ): + span.set_attribute("error.type", _exception_type(exc)) + span.set_status(StatusCode.ERROR) + raise + + +def _run_sync_endpoint( + *, function: Callable[..., Any], arguments: dict[str, Any] +) -> Any: + with _operation(name="endpoint", function=function): + return function(**arguments) + + +def _route_selected( + *, scope: MutableMapping[str, Any], path: str | None, mount: bool = False +) -> None: + request_telemetry = scope.get("fastapi.telemetry") + if not isinstance(request_telemetry, _RequestTelemetry) or path is None: + return + # Mount's path_format ends in /{path}. The child contributes its own path. + if mount: + path = path.removesuffix("/{path}") + request_telemetry._mount_prefix += path + request_telemetry.route = request_telemetry._mount_prefix + "/{path}" + else: + request_telemetry.route = request_telemetry._mount_prefix + path + if request_telemetry.span is not None: + request_telemetry.span.set_attribute("http.route", request_telemetry.route) + request_telemetry.span.update_name( + f"{request_telemetry.span_method} {request_telemetry.route}" + ) + + +_DEFERRED_PROVIDERS = frozenset( + { + ("opentelemetry.trace", "ProxyTracerProvider"), + ("opentelemetry.metrics._internal", "_ProxyMeterProvider"), + ("opentelemetry._logs._internal", "ProxyLoggerProvider"), + } +) + + +def _unconfigured(provider: Any) -> bool: + # Python's API has no public "is configured" query. Limit this bridge to its + # known deferred providers, never SDK implementations or vendor proxies. + cls = type(provider) + return (cls.__module__, cls.__name__) in _DEFERRED_PROVIDERS diff --git a/fastapi/telemetry/_asgi.py b/fastapi/telemetry/_asgi.py new file mode 100644 index 0000000000..758e3e3e61 --- /dev/null +++ b/fastapi/telemetry/_asgi.py @@ -0,0 +1,446 @@ +import os +from contextlib import nullcontext +from time import perf_counter, time_ns +from typing import Any +from urllib.parse import parse_qsl, urlencode, urlsplit + +from fastapi import __version__ +from fastapi.telemetry._api import ( + _HTTP_METHODS, + _REQUEST_TELEMETRY_KEY, + TelemetryConfig, + _exception_type, + _normal_websocket_disconnect, + _RequestTelemetry, + _unconfigured, +) +from opentelemetry import _logs, metrics, propagate, trace +from opentelemetry import context as otel_context +from opentelemetry._logs import SeverityNumber +from opentelemetry.metrics import Histogram, UpDownCounter +from opentelemetry.propagators.textmap import Getter +from opentelemetry.trace import SpanKind, StatusCode +from starlette.datastructures import Headers +from starlette.types import ASGIApp, Message, Receive, Scope, Send + +_DURATION_BUCKETS = ( + 0.005, + 0.01, + 0.025, + 0.05, + 0.075, + 0.1, + 0.25, + 0.5, + 0.75, + 1.0, + 2.5, + 5.0, + 7.5, + 10.0, +) +_SCHEMA_URL = "https://opentelemetry.io/schemas/1.44.0" +# https://opentelemetry.io/docs/specs/semconv/http/http-spans/#http-server-span +_SENSITIVE_QUERY_PARAMETERS = frozenset( + { + "X-Amz-Signature", + "X-Amz-Credential", + "X-Amz-Security-Token", + "sig", + "X-Goog-Signature", + } +) + + +class _HeadersGetter(Getter[Headers]): + def get(self, carrier: Headers, key: str) -> list[str] | None: + values = carrier.getlist(key) + if not values: + return None + if key.lower() == "baggage": + # The baggage propagator reads only the first value. + return [",".join(values)] + return values + + def keys(self, carrier: Headers) -> list[str]: + return carrier.keys() + + +_HEADERS_GETTER = _HeadersGetter() + + +def _legacy_otel(stack: Any) -> bool: + # Test the built stack, not contrib's flag: that flag also exists when its + # stack patch failed, or when instrument_app was called after stack creation. + seen: set[int] = set() + while stack is not None and id(stack) not in seen: + seen.add(id(stack)) + if ( + type(stack).__module__ == "opentelemetry.instrumentation.asgi" + and type(stack).__name__ == "OpenTelemetryMiddleware" + ): + return True + stack = getattr(stack, "app", None) + return False + + +def _exception(*, scope: Scope, exc: BaseException) -> None: + if scope["type"] == "websocket" and _normal_websocket_disconnect(exc): + return + request_telemetry = scope.get("fastapi.telemetry") + if isinstance(request_telemetry, _RequestTelemetry): + seen = request_telemetry._exceptions + if not any(previous is exc for previous in seen): + seen.append(exc) + if request_telemetry.logger is not None and isinstance(exc, Exception): + attributes = ( + {"http.route": request_telemetry.route} + if request_telemetry.route + else None + ) + request_telemetry.logger.emit( + exception=exc, + event_name="fastapi.websocket.exception" + if scope["type"] == "websocket" + else "http.server.request.exception", + timestamp=time_ns(), + severity_number=SeverityNumber.ERROR, + severity_text="ERROR", + body="Unhandled exception in FastAPI WebSocket connection" + if scope["type"] == "websocket" + else "Unhandled exception in FastAPI request", + attributes=attributes, + ) + + +def _server_attributes(scope: Scope) -> dict[str, Any]: + # ASGI exposes HTTP/2 and HTTP/3 authority as Host. Proxy normalization is + # handled by the server or middleware, as it is for the request URL. + authority = Headers(scope=scope).get("host") + if authority is not None: + try: + url = urlsplit("//" + authority) + address, port = url.hostname, url.port + except ValueError: + # Malformed authority must not prevent handling the request. + return {} + if url.username is not None or url.path or url.query or url.fragment: + return {} + if port is None: + port = {"http": 80, "https": 443, "ws": 80, "wss": 443}.get( + scope.get("scheme", "http") + ) + else: + address, port = scope.get("server") or (None, None) + if address is None: + return {} + attributes: dict[str, Any] = {"server.address": address} + if port is not None: + attributes["server.port"] = port + return attributes + + +class ExceptionTelemetryMiddleware: + """Observe exceptions before error handlers send and complete a response.""" + + def __init__(self, app: ASGIApp) -> None: + self.app = app + + async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: + try: + await self.app(scope, receive, send) + except Exception as exc: + _exception(scope=scope, exc=exc) + raise + + +class NativeTelemetry: + def __init__(self, config: TelemetryConfig) -> None: + self.config = config + known_methods = os.getenv("OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS", "").strip() + self._known_methods = ( + frozenset( + method.strip() for method in known_methods.split(",") if method.strip() + ) + if known_methods + else _HTTP_METHODS + ) + self._provider: metrics.MeterProvider | None = None + self._duration: Histogram | None = None + self._active: UpDownCounter | None = None + + def enabled(self) -> bool: + config = self.config + return bool( + ( + config["tracing"] + and ( + config["tracer_provider"] is not None + or not _unconfigured(trace.get_tracer_provider()) + ) + ) + or ( + config["metrics"] + and ( + config["meter_provider"] is not None + or not _unconfigured(metrics.get_meter_provider()) + ) + ) + or ( + config["logs"] + and ( + config["logger_provider"] is not None + or not _unconfigured(_logs.get_logger_provider()) + ) + ) + ) + + def _instruments(self) -> tuple[Histogram, UpDownCounter]: + provider = self.config["meter_provider"] or metrics.get_meter_provider() + if provider is not self._provider: + meter = provider.get_meter("fastapi", __version__, schema_url=_SCHEMA_URL) + self._duration = meter.create_histogram( + "http.server.request.duration", + unit="s", + description="Duration of HTTP server requests.", + explicit_bucket_boundaries_advisory=_DURATION_BUCKETS, + ) + self._active = meter.create_up_down_counter( + "http.server.active_requests", + unit="{request}", + description="Number of active HTTP server requests.", + ) + self._provider = provider + assert self._duration is not None and self._active is not None + return self._duration, self._active + + async def __call__( + self, + *, + app: ASGIApp, + scope: Scope, + receive: Receive, + send: Send, + legacy_otel: bool = False, + ) -> None: + config = self.config + if config["exclude"] is not None and config["exclude"](scope): + # Keep mounted FastAPI apps from observing this excluded request. + scope["fastapi.telemetry"] = None + try: + await app(scope, receive, send) + finally: + scope.pop("fastapi.telemetry", None) + return + tracing = config["tracing"] and not legacy_otel + logging = config["logs"] and not legacy_otel + if logging: + logger_provider = config["logger_provider"] or _logs.get_logger_provider() + logging = not ( + (config["logger_provider"] is None and _unconfigured(logger_provider)) + or isinstance(logger_provider, _logs.NoOpLoggerProvider) + ) + is_websocket = scope["type"] == "websocket" + metering = config["metrics"] and not legacy_otel and not is_websocket + if tracing: + provider = config["tracer_provider"] or trace.get_tracer_provider() + tracing = not ( + (config["tracer_provider"] is None and _unconfigured(provider)) + or isinstance(provider, trace.NoOpTracerProvider) + ) + if metering: + meter_provider = config["meter_provider"] or metrics.get_meter_provider() + metering = not ( + (config["meter_provider"] is None and _unconfigured(meter_provider)) + or isinstance(meter_provider, metrics.NoOpMeterProvider) + ) + if not tracing and not metering and not logging: + await app(scope, receive, send) + return + original_method = scope.get("method", "") + method = original_method if original_method in self._known_methods else "_OTHER" + span_method = "WS" if is_websocket else "HTTP" if method == "_OTHER" else method + attributes: dict[str, Any] = { + "url.scheme": scope.get("scheme", "ws" if is_websocket else "http"), + } + if is_websocket: + attributes["network.protocol.name"] = "websocket" + else: + attributes["http.request.method"] = method + if scope.get("http_version"): + attributes["network.protocol.version"] = scope["http_version"] + duration, active = self._instruments() if metering else (None, None) + active_attributes = { + k: v for k, v in attributes.items() if k != "network.protocol.version" + } + if active is not None: + active.add(1, active_attributes) + started = perf_counter() + span = None + tracer = None + parent_token = None + if tracing: + parent = propagate.extract(Headers(scope=scope), getter=_HEADERS_GETTER) + parent_token = otel_context.attach(parent) + tracer = trace.get_tracer( + "fastapi", + __version__, + config["tracer_provider"], + schema_url=_SCHEMA_URL, + ) + span_attributes = {**attributes, **_server_attributes(scope)} + if not is_websocket: + span_attributes["url.path"] = scope["path"] + if scope.get("query_string"): + span_attributes["url.query"] = urlencode( + [ + ( + key, + "REDACTED" + if key in _SENSITIVE_QUERY_PARAMETERS + else value, + ) + for key, value in parse_qsl( + scope["query_string"].decode("latin-1"), + keep_blank_values=True, + ) + ] + ) + if original_method != method: + span_attributes["http.request.method_original"] = original_method + span = tracer.start_span( + span_method, + context=parent, + kind=SpanKind.SERVER, + attributes=span_attributes, + ) + request_telemetry = _RequestTelemetry( + span=span, + span_method=span_method, + tracer=tracer if config["operation_spans"] else None, + logger=_logs.get_logger( + "fastapi", + __version__, + config["logger_provider"], + schema_url=_SCHEMA_URL, + ) + if logging + else None, + _mount_prefix=scope.get("app_root_path", scope.get("root_path", "")).rstrip( + "/" + ), + ) + scope["fastapi.telemetry"] = request_telemetry + token = otel_context.attach( + otel_context.set_value(_REQUEST_TELEMETRY_KEY, request_telemetry) + ) + finished = False + trailers = False + disconnected = False + + def finish(error: BaseException | None = None) -> None: + nonlocal finished + if finished: + return + finished = True + if request_telemetry.route is not None: + attributes["http.route"] = request_telemetry.route + if request_telemetry.status_code is not None: + attributes["http.response.status_code"] = request_telemetry.status_code + failed = ( + error is not None + or not is_websocket + and ( + request_telemetry.status_code is None + or request_telemetry.status_code >= 500 + ) + ) + if failed: + errors = request_telemetry._exceptions + failure = error or (errors[-1] if errors else None) + attributes["error.type"] = ( + _exception_type(failure) + if failure + else str(request_telemetry.status_code or "incomplete_response") + ) + if span is not None: + span.set_attributes(attributes) + if failed: + span.set_status(StatusCode.ERROR) + # Record while the span is current, so exemplars can correlate it. + if duration is not None: + duration.record(max(0, perf_counter() - started), attributes) + if active is not None: + active.add(-1, active_attributes) + if span is not None: + span.end() + + async def wrapped_send(message: Message) -> None: + nonlocal trailers + if message["type"] == "http.response.start": + request_telemetry.status_code = message["status"] + trailers = message.get("trailers", False) + await send(message) + if ( + ( + message["type"] == "http.response.body" + and not message.get("more_body", False) + and not trailers + ) + or ( + message["type"] == "http.response.trailers" + and not message.get("more_trailers", False) + ) + or message["type"] == "http.response.pathsend" + ): + finish() + + async def wrapped_receive() -> Message: + nonlocal disconnected + message = await receive() + if message["type"] == "http.disconnect": + disconnected = True + return message + + context = ( + trace.use_span( + span, + end_on_exit=False, + record_exception=False, + set_status_on_exception=False, + ) + if span is not None + else nullcontext() + ) + try: + with context: + try: + await app( + scope, + receive if is_websocket else wrapped_receive, + send if is_websocket else wrapped_send, + ) + except BaseException as exc: + _exception(scope=scope, exc=exc) + finish( + None + if is_websocket and _normal_websocket_disconnect(exc) + else exc + ) + raise + finally: + if not finished: + finish( + None + if is_websocket + else ConnectionError("Client disconnected") + if disconnected + else RuntimeError("Incomplete ASGI response") + ) + finally: + request_telemetry.data = None + request_telemetry._exceptions.clear() + otel_context.detach(token) + scope.pop("fastapi.telemetry", None) + if parent_token is not None: + otel_context.detach(parent_token) diff --git a/fastapi/telemetry/_runtime.py b/fastapi/telemetry/_runtime.py new file mode 100644 index 0000000000..641aea43c0 --- /dev/null +++ b/fastapi/telemetry/_runtime.py @@ -0,0 +1,239 @@ +import atexit +import os +import threading +from collections.abc import Callable +from typing import Any, cast +from urllib.parse import urlsplit + +from anyio.to_thread import run_sync +from fastapi.exceptions import FastAPIError +from fastapi.logger import logger +from fastapi.telemetry._api import TelemetryConfig, _unconfigured +from opentelemetry import _logs, metrics, trace +from starlette.types import ASGIApp, Message, Receive, Scope, Send + +_lock = threading.RLock() +_owned: list[Any] = [] +_configured: list[tuple[str, Any]] = [] + + +def _flush() -> None: + for component in tuple(_owned): + try: + component.force_flush() + except Exception: + logger.exception("FastAPI telemetry cleanup failed") + + +def _shutdown() -> None: + with _lock: + owned = tuple(_owned) + _owned.clear() + for component in owned: + try: + component.shutdown() + except Exception: + logger.exception("FastAPI telemetry cleanup failed") + + +atexit.register(_shutdown) + + +def _export_endpoint(signal: str) -> str | None: + exporter = (os.getenv(f"OTEL_{signal}_EXPORTER") or "otlp").strip().lower() + endpoint = os.getenv(f"OTEL_EXPORTER_OTLP_{signal}_ENDPOINT") + if not endpoint: + endpoint = os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT") + if endpoint: + endpoint = f"{endpoint.removesuffix('/')}/v1/{signal.lower()}" + if not endpoint or exporter == "none": + return None + if exporter != "otlp": + raise FastAPIError( + f"FastAPI automatic telemetry supports OTEL_{signal}_EXPORTER=otlp or none. " + "Configure other exporters explicitly and pass telemetry={'auto_configure': False} to FastAPI()." + ) + protocol = ( + os.getenv(f"OTEL_EXPORTER_OTLP_{signal}_PROTOCOL") + or os.getenv("OTEL_EXPORTER_OTLP_PROTOCOL") + or "http/protobuf" + ) + if protocol != "http/protobuf": + raise FastAPIError( + "FastAPI automatic telemetry requires the OTLP http/protobuf protocol. Configure other transports explicitly and pass telemetry={'auto_configure': False} to FastAPI()." + ) + parsed = urlsplit(endpoint) + if parsed.scheme not in {"http", "https"} or not parsed.hostname: + raise FastAPIError( + f"Invalid OTLP {signal.lower()} endpoint. Use an absolute HTTP or HTTPS URL." + ) + return endpoint + + +def _registration_provider(provider: Any) -> Any: + # Older Logfire meter wrappers do not expose add_metric_reader(). Attach + # the reader to their SDK provider. Measurements still use the original wrapper. + if ( + type(provider).__module__ == "logfire._internal.metrics" + and type(provider).__name__ == "ProxyMeterProvider" + and not hasattr(provider, "add_metric_reader") + ): + return provider.provider + return provider + + +def _configure_from_environment(config: TelemetryConfig) -> None: + """Add OTLP exporters from the environment to the selected providers. + + Called automatically before ASGI lifespan startup. Existing providers and + their exporters are preserved. FastAPI registers its own export components once + per provider. It does not inspect or deduplicate other components' exporters. + Pass `telemetry={"auto_configure": False}` to `FastAPI()` when another component + manages environment export. + """ + if ( + not config["auto_configure"] + or os.getenv("OTEL_SDK_DISABLED", "").lower() == "true" + ): + return + with _lock: + signals = [ + ("TRACES", config["tracing"], config["tracer_provider"]), + ("METRICS", config["metrics"], config["meter_provider"]), + ("LOGS", config["logs"], config["logger_provider"]), + ] + requested = [ + (signal, provider, endpoint) + for signal, enabled, provider in signals + if enabled and (endpoint := _export_endpoint(signal)) is not None + ] + if not requested: + return + try: + from opentelemetry.exporter.otlp.proto.http._log_exporter import ( + OTLPLogExporter, + ) + from opentelemetry.exporter.otlp.proto.http.metric_exporter import ( + OTLPMetricExporter, + ) + from opentelemetry.exporter.otlp.proto.http.trace_exporter import ( + OTLPSpanExporter, + ) + from opentelemetry.sdk._logs import LoggerProvider + from opentelemetry.sdk._logs.export import BatchLogRecordProcessor + from opentelemetry.sdk.metrics import MeterProvider + from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import BatchSpanProcessor + except ImportError as exc: + raise FastAPIError( + "Automatic OpenTelemetry export requires fastapi[opentelemetry] or fastapi[standard]. Install the extra, configure providers yourself, or pass telemetry={'auto_configure': False} to FastAPI()." + ) from exc + # The SDK's registration methods also allow vendor wrappers to forward + # additions. Keep a record of our own registrations, without examining + # another component's exporters or their destinations. + providers = { + "TRACES": ("tracer", trace.get_tracer_provider), + "METRICS": ("meter", metrics.get_meter_provider), + "LOGS": ("logger", _logs.get_logger_provider), + } + + def create_component(*, signal: str, endpoint: str) -> Any: + if signal == "TRACES": + return BatchSpanProcessor(OTLPSpanExporter(endpoint=endpoint)) + if signal == "METRICS": + return PeriodicExportingMetricReader( + OTLPMetricExporter(endpoint=endpoint) + ) + return BatchLogRecordProcessor(OTLPLogExporter(endpoint=endpoint)) + + for signal, explicit, endpoint in requested: + name, get_provider = providers[signal] + provider = _registration_provider( + explicit if explicit is not None else get_provider() + ) + if any(s == signal and p is provider for s, p in _configured): + continue + if explicit is None and _unconfigured(provider): + component = create_component(signal=signal, endpoint=endpoint) + if signal == "TRACES": + provider = TracerProvider(shutdown_on_exit=False) + provider.add_span_processor(component) + trace.set_tracer_provider(provider) + elif signal == "METRICS": + provider = MeterProvider( + metric_readers=[component], shutdown_on_exit=False + ) + metrics.set_meter_provider(provider) + else: + provider = LoggerProvider(shutdown_on_exit=False) + provider.add_log_record_processor(component) + _logs.set_logger_provider(provider) + current = get_provider() + if current is provider: + _configured.append((signal, provider)) + _owned.append(provider) + continue + # Another component configured the global provider concurrently. + # Close the unused pipeline and attach a new component to theirs. + provider.shutdown() + provider = _registration_provider(current) + # Vendor wrappers can expose the SDK registration methods without + # inheriting its provider classes. Check each method against its SDK + # class statically and its availability on the provider at runtime. + register: Callable[[Any], None] | None + try: + if signal == "TRACES": + register = cast(TracerProvider, provider).add_span_processor + elif signal == "METRICS": + register = cast(MeterProvider, provider).add_metric_reader + else: + register = cast(LoggerProvider, provider).add_log_record_processor + except AttributeError: + register = None + if not callable(register): + raise FastAPIError( + f"The configured OpenTelemetry {name} provider does not support " + "adding an OTLP exporter. Configure environment export through " + "that provider and pass telemetry={'auto_configure': False} to FastAPI()." + ) + component = create_component(signal=signal, endpoint=endpoint) + try: + register(component) + except Exception: + component.shutdown() + raise + _configured.append((signal, provider)) + _owned.append(component) + + +async def lifespan( + *, config: TelemetryConfig, app: ASGIApp, scope: Scope, receive: Receive, send: Send +) -> None: + # Consume startup only inside the wrapper, and preserve ASGI failure events. + # Initialization exceptions before app(scope) would otherwise look like a + # server's unsupported-lifespan fallback and silently disable telemetry. + async def wrapped_receive() -> Message: + message = await receive() + if message["type"] == "lifespan.startup": + try: + _configure_from_environment(config) + except Exception as exc: + await send({"type": "lifespan.startup.failed", "message": str(exc)}) + raise + return message + + async def wrapped_send(message: Message) -> None: + if ( + message["type"] + in { + "lifespan.shutdown.complete", + "lifespan.shutdown.failed", + "lifespan.startup.failed", + } + and _owned + ): + await run_sync(_flush) + await send(message) + + await app(scope, wrapped_receive, wrapped_send) diff --git a/pyproject.toml b/pyproject.toml index 43cde965c0..75d9d02675 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -47,6 +47,7 @@ dependencies = [ "typing-extensions>=4.8.0", "typing-inspection>=0.4.2", "annotated-doc>=0.0.2", + "opentelemetry-api>=1.44.0", ] [project.urls] @@ -57,7 +58,13 @@ Issues = "https://github.com/fastapi/fastapi/issues" Changelog = "https://fastapi.tiangolo.com/release-notes/" [project.optional-dependencies] +opentelemetry = [ + "opentelemetry-sdk>=1.44.0", + "opentelemetry-exporter-otlp-proto-http>=1.44.0", +] standard = [ + "opentelemetry-sdk>=1.44.0", + "opentelemetry-exporter-otlp-proto-http>=1.44.0", "fastapi-cli[standard] >=0.0.32", "fastar >= 0.9.0", # For the test client @@ -77,6 +84,8 @@ standard = [ ] standard-no-fastapi-cloud-cli = [ + "opentelemetry-sdk>=1.44.0", + "opentelemetry-exporter-otlp-proto-http>=1.44.0", "fastapi-cli[standard-no-fastapi-cloud-cli] >=0.0.32", # For the test client "httpx >=0.23.0,<1.0.0", @@ -95,6 +104,8 @@ standard-no-fastapi-cloud-cli = [ ] all = [ + "opentelemetry-sdk>=1.44.0", + "opentelemetry-exporter-otlp-proto-http>=1.44.0", "fastapi-cli[standard] >=0.0.32", # # For the test client "httpx >=0.23.0,<1.0.0", @@ -158,6 +169,12 @@ github-actions = [ "smokeshow >=0.5.0", ] tests = [ + "opentelemetry-instrumentation-fastapi>=0.65b0", + "logfire>=5.1.0", + "sentry-sdk>=2.70.0", + "opentelemetry-sdk>=1.44.0", + "opentelemetry-exporter-otlp-proto-http>=1.44.0", + { include-group = "docs-tests" }, "anyio[trio] >=3.2.1,<5.0.0", "coverage[toml] >=7.13,<8.0", @@ -233,6 +250,7 @@ filterwarnings = [ timeout = "20" [tool.coverage.run] +patch = ["subprocess", "_exit"] parallel = true data_file = "coverage/.coverage" source = [ diff --git a/tests/test_telemetry/__init__.py b/tests/test_telemetry/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/test_telemetry/_otlp.py b/tests/test_telemetry/_otlp.py new file mode 100644 index 0000000000..a537068a74 --- /dev/null +++ b/tests/test_telemetry/_otlp.py @@ -0,0 +1,36 @@ +import threading +from contextlib import contextmanager +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from socketserver import TCPServer + + +class _CollectorServer(ThreadingHTTPServer): + def server_bind(self): + TCPServer.server_bind(self) + self.server_name = str(self.server_address[0]) + self.server_port = self.server_address[1] + + +@contextmanager +def otlp_collector(): + received = [] + + class Handler(BaseHTTPRequestHandler): + def do_POST(self): + body = self.rfile.read(int(self.headers["Content-Length"])) + received.append((self.path, body, self.headers)) + self.send_response(200) + self.send_header("Content-Type", "application/x-protobuf") + self.end_headers() + + def log_message(self, format, *args): + pass + + with _CollectorServer(("127.0.0.1", 0), Handler) as server: + worker = threading.Thread(target=server.serve_forever, daemon=True) + worker.start() + try: + yield f"http://127.0.0.1:{server.server_port}", received + finally: + server.shutdown() + worker.join() diff --git a/tests/test_telemetry/_subprocess.py b/tests/test_telemetry/_subprocess.py new file mode 100644 index 0000000000..59675d5748 --- /dev/null +++ b/tests/test_telemetry/_subprocess.py @@ -0,0 +1,43 @@ +import json +import os +import subprocess +import sys +from functools import wraps +from importlib import import_module +from inspect import unwrap + + +def run_in_subprocess(function): + """Run a test in a fresh interpreter to isolate SDK globals and monkeypatches.""" + + @wraps(function) + def wrapper(**kwargs): + env = { + name: value + for name, value in os.environ.items() + if not name.startswith(("OTEL_", "LOGFIRE_", "SENTRY_")) + } + env["PYDANTIC_DISABLE_PLUGINS"] = "__all__" + result = subprocess.run( + [ + sys.executable, + "-m", + "tests.test_telemetry._subprocess", + function.__module__, + function.__name__, + json.dumps(kwargs), + ], + env=env, + capture_output=True, + text=True, + timeout=15, + ) + assert result.returncode == 0, result.stdout + result.stderr + + return wrapper + + +if __name__ == "__main__": + module_name, function_name, arguments = sys.argv[1:] + function = getattr(import_module(module_name), function_name) + unwrap(function)(**json.loads(arguments)) diff --git a/tests/test_telemetry/conftest.py b/tests/test_telemetry/conftest.py new file mode 100644 index 0000000000..f08ae0f273 --- /dev/null +++ b/tests/test_telemetry/conftest.py @@ -0,0 +1,69 @@ +import os + +import pytest +from opentelemetry.sdk._logs import LoggerProvider +from opentelemetry.sdk._logs.export import ( + InMemoryLogRecordExporter, + SimpleLogRecordProcessor, +) +from opentelemetry.sdk.metrics import MeterProvider +from opentelemetry.sdk.metrics.export import InMemoryMetricReader +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.fixture(autouse=True) +def clean_environment(monkeypatch): + remove_export_environment(monkeypatch) + + +def remove_export_environment(monkeypatch): + for name in os.environ: + if name.startswith(("OTEL_", "LOGFIRE_", "SENTRY_")): + monkeypatch.delenv(name) + + +@pytest.fixture +def telemetry(): + exporter = InMemorySpanExporter() + tracer = TracerProvider(shutdown_on_exit=False) + tracer.add_span_processor(SimpleSpanProcessor(exporter)) + reader = InMemoryMetricReader() + meter = MeterProvider(metric_readers=[reader], shutdown_on_exit=False) + yield {"tracer_provider": tracer, "meter_provider": meter}, exporter, reader + tracer.shutdown() + meter.shutdown() + + +@pytest.fixture +def logs(telemetry): + config, _, _ = telemetry + exporter = InMemoryLogRecordExporter() + provider = LoggerProvider(shutdown_on_exit=False) + provider.add_log_record_processor(SimpleLogRecordProcessor(exporter)) + config["logger_provider"] = provider + yield exporter + provider.shutdown() + + +def metric_points(*, reader, name="http.server.request.duration"): + data = reader.get_metrics_data() + if data is None: + return [] + return [ + point + for resource in data.resource_metrics + for scope in resource.scope_metrics + for metric in scope.metrics + if metric.name == name + for point in metric.data.data_points + ] + + +def server_spans(exporter): + from opentelemetry.trace import SpanKind + + return [ + span for span in exporter.get_finished_spans() if span.kind == SpanKind.SERVER + ] diff --git a/tests/test_telemetry/test_background.py b/tests/test_telemetry/test_background.py new file mode 100644 index 0000000000..99ff21d358 --- /dev/null +++ b/tests/test_telemetry/test_background.py @@ -0,0 +1,276 @@ +import threading +from functools import partial + +import anyio +import pytest +from fastapi import BackgroundTasks, Depends, FastAPI, HTTPException +from fastapi.responses import JSONResponse +from fastapi.telemetry import get_telemetry_data +from fastapi.testclient import TestClient +from opentelemetry import trace +from opentelemetry.sdk.trace import Span +from opentelemetry.sdk.trace.sampling import ALWAYS_OFF +from opentelemetry.trace import SpanKind, StatusCode +from starlette.background import BackgroundTask + +from ._subprocess import run_in_subprocess +from .conftest import metric_points, server_spans + + +@pytest.mark.parametrize("backend", ["asyncio", "trio"]) +@pytest.mark.parametrize("sync", [False, True]) +def test_background_task_spans(telemetry, backend, sync): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + executed = [] + endpoint_thread = [] + + def execute(number, *, value): + span = trace.get_current_span() + assert isinstance(span, Span) + assert span.is_recording() + assert span.name == "fastapi.background_task" + data = get_telemetry_data() + assert data is not None + assert data.request is not None + assert data.request.url.path == "/items/42" + assert len(server_spans(exporter)) == 1 + executed.append((number, value, threading.get_ident(), span.get_span_context())) + + async def async_task(number, *, value): + execute(number, value=value) + + task = execute if sync else async_task + + async def dependency(background: BackgroundTasks): + background.add_task(task, 1, value="private dependency value") + + @app.get("/items/{item_id}", dependencies=[Depends(dependency)]) + async def endpoint(background: BackgroundTasks, item_id: int): + endpoint_thread.append(threading.get_ident()) + background.add_task(task, 2, value="private endpoint value") + return {"item_id": item_id} + + assert TestClient(app, backend=backend).get("/items/42").json() == {"item_id": 42} + assert [(number, value) for number, value, _, _ in executed] == [ + (1, "private dependency value"), + (2, "private endpoint value"), + ] + spans = exporter.get_finished_spans() + (server,) = server_spans(exporter) + tasks = [span for span in spans if span.name == "fastapi.background_task"] + assert len(tasks) == 2 + for span, (_, _, thread_id, context) in zip(tasks, executed, strict=True): + assert span.kind == SpanKind.INTERNAL + assert span.parent.span_id == server.context.span_id + assert context.trace_id == server.context.trace_id + assert context.span_id == span.context.span_id + assert span.start_time >= server.end_time + assert span.status.status_code == StatusCode.UNSET + assert span.attributes == { + "code.function.name": f"{task.__module__}.{task.__qualname__}" + } + assert (thread_id != endpoint_thread[0]) == sync + assert metric_points(reader=reader)[0].count == 1 + assert get_telemetry_data() is None + + +def test_background_task_objects(telemetry): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + executed = [] + + def record(value): + span = trace.get_current_span() + assert isinstance(span, Span) + executed.append((value, span.name)) + + class AsyncTask: + async def __call__(self, *, value): + record(value) + + class CustomTask(BackgroundTask): + async def __call__(self): + record("before") + await super().__call__() + record("after") + + original = CustomTask(record, "custom") + background = BackgroundTasks([original]) + background.add_task(partial(record, "partial")) + background.add_task(AsyncTask(), value="async callable") + + @app.get("/") + async def endpoint(): + return JSONResponse("ok", background=background) + + assert TestClient(app).get("/").json() == "ok" + assert background.tasks[0] is original + assert executed == [ + (value, "fastapi.background_task") + for value in ("before", "custom", "after", "partial", "async callable") + ] + assert ( + len( + [ + s + for s in exporter.get_finished_spans() + if s.name == "fastapi.background_task" + ] + ) + == 3 + ) + + +@pytest.mark.parametrize("sync", [False, True]) +@pytest.mark.parametrize("http_error", [False, True]) +def test_background_task_failure(telemetry, logs, sync, http_error): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + error = HTTPException(503) if http_error else ValueError("background failure") + executed = [] + + def fail(): + executed.append("first") + raise error + + async def async_fail(): + fail() + + @app.get("/") + async def endpoint(background: BackgroundTasks): + background.add_task(fail if sync else async_fail) + background.add_task(executed.append, "second") + return "ok" + + assert TestClient(app, raise_server_exceptions=False).get("/").status_code == 200 + assert executed == ["first"] + (task,) = [ + s for s in exporter.get_finished_spans() if s.name == "fastapi.background_task" + ] + assert task.status.status_code == StatusCode.ERROR + assert task.attributes["error.type"] == ( + "fastapi.exceptions.HTTPException" if http_error else "ValueError" + ) + assert not task.events + (server,) = server_spans(exporter) + assert server.attributes["http.response.status_code"] == 200 + assert server.status.status_code == StatusCode.UNSET + (record,) = logs.get_finished_logs() + assert ( + record.log_record.trace_id == task.context.trace_id == server.context.trace_id + ) + assert "raise error" in record.log_record.attributes["exception.stacktrace"] + assert record.log_record.exception is not None + if http_error: + assert record.log_record.exception.__cause__ is error + else: + assert record.log_record.exception is error + + +@pytest.mark.parametrize( + "mode", ["operations_disabled", "tracing_disabled", "excluded", "unsampled"] +) +def test_background_task_settings(telemetry, mode): + config, exporter, _ = telemetry + if mode == "operations_disabled": + config["operation_spans"] = False + elif mode == "tracing_disabled": + config["tracing"] = False + elif mode == "excluded": + config["exclude"] = lambda scope: True + else: + config["tracer_provider"].sampler = ALWAYS_OFF + app = FastAPI(telemetry=config) + executed = [] + + @app.get("/") + async def endpoint(background: BackgroundTasks): + background.add_task(executed.append, "done") + return "ok" + + assert TestClient(app).get("/").json() == "ok" + assert executed == ["done"] + assert not any( + s.name == "fastapi.background_task" for s in exporter.get_finished_spans() + ) + + +def test_background_tasks_outside_request(): + executed = [] + background = BackgroundTasks() + background.add_task(executed.append, "done") + anyio.run(background) + assert executed == ["done"] + assert get_telemetry_data() is None + + +@pytest.mark.parametrize("integration", ["contrib", "logfire", "native_logfire"]) +@run_in_subprocess +def test_background_task_integrations(integration): + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + native = InMemorySpanExporter() + legacy = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(native)) + app = FastAPI( + telemetry={ + "tracer_provider": None if integration == "native_logfire" else provider + } + ) + executed = [] + + def sync_task(): + executed.append(trace.get_current_span().get_span_context()) + + async def async_task(): + executed.append(trace.get_current_span().get_span_context()) + + @app.get("/") + async def endpoint(background: BackgroundTasks): + background.add_task(sync_task) + background.add_task(async_task) + return "ok" + + if integration == "contrib": + from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor + + legacy_provider = TracerProvider() + legacy_provider.add_span_processor(SimpleSpanProcessor(legacy)) + FastAPIInstrumentor.instrument_app(app, tracer_provider=legacy_provider) + else: + import logfire + + logfire.configure( + send_to_logfire=False, + console=False, + metrics=False, + additional_span_processors=[SimpleSpanProcessor(legacy)], + ) + if integration == "logfire": + logfire.instrument_fastapi(app) + assert TestClient(app).get("/").json() == "ok" + assert not native.get_finished_spans() + tasks = [ + span + for span in legacy.get_finished_spans() + if ( + span.name.startswith("BackgroundTask ") + or span.name == "fastapi.background_task" + ) + and not ( + span.attributes + and span.attributes.get("logfire.span_type") == "pending_span" + ) + ] + assert [span.name for span in tasks] == ( + ["fastapi.background_task", "fastapi.background_task"] + if integration == "native_logfire" + else ["BackgroundTask sync_task", "BackgroundTask async_task"] + ) + assert [span.context for span in tasks] == executed diff --git a/tests/test_telemetry/test_data.py b/tests/test_telemetry/test_data.py new file mode 100644 index 0000000000..c6174ea3e6 --- /dev/null +++ b/tests/test_telemetry/test_data.py @@ -0,0 +1,286 @@ +import gc +import weakref + +import anyio +import pytest +from fastapi import Depends, FastAPI, HTTPException, Request, WebSocket +from fastapi.telemetry import get_telemetry_data +from fastapi.testclient import TestClient +from httpx import ASGITransport, AsyncClient +from opentelemetry import context +from opentelemetry._logs import SeverityNumber +from opentelemetry.sdk._logs import LoggerProvider, LogRecordProcessor +from opentelemetry.sdk._logs.export import ( + InMemoryLogRecordExporter, + SimpleLogRecordProcessor, +) +from pydantic import BaseModel +from starlette.websockets import WebSocketDisconnect + +from ._otlp import otlp_collector + + +@pytest.mark.anyio +async def test_data_is_local_to_each_request_and_cleared_afterwards(telemetry): + config, _, _ = telemetry + child = FastAPI(telemetry=config) + app = FastAPI(telemetry=config) + app.mount("/child", child) + saved_contexts = [] + saved_data = [] + entered = 0 + ready = anyio.Event() + service = object() + + class Item(BaseModel): + name: str + + def dependency(request: Request): + data = get_telemetry_data() + assert data is not None + assert data.request is request + assert data.values is None + assert data.body == {"name": request.path_params["name"]} + return service + + @child.post("/{name}") + async def endpoint( + *, name: str, item: Item, request: Request, value=Depends(dependency) + ): + nonlocal entered + data = get_telemetry_data() + assert data is not None + assert data.request is request + assert data.values is not None + assert data.values["item"] is item + assert data.values["value"] is service + entered += 1 + if entered == 2: + ready.set() + await ready.wait() + assert get_telemetry_data() is data + assert data.values["name"] == name + assert data.errors == [] + saved_contexts.append(context.get_current()) + saved_data.append(data) + return name + + assert get_telemetry_data() is None + async with AsyncClient( + transport=ASGITransport(app=app), base_url="http://test" + ) as client: + results = {} + + async def send(name): + results[name] = await client.post(f"/child/{name}", json={"name": name}) + + async with anyio.create_task_group() as tasks: + tasks.start_soon(send, "first") + tasks.start_soon(send, "second") + assert {name: response.json() for name, response in results.items()} == { + "first": "first", + "second": "second", + } + assert saved_data[0] is not saved_data[1] + assert all(get_telemetry_data(ctx) is None for ctx in saved_contexts) + assert get_telemetry_data() is None + + +def test_retained_context_does_not_keep_failed_request_alive(telemetry): + config, _, _ = telemetry + app = FastAPI(telemetry={**config, "tracing": False, "logs": False}) + saved_contexts = [] + request_refs = [] + + @app.post("/") + async def endpoint(request: Request): + await request.body() + saved_contexts.append(context.get_current()) + request_refs.append(weakref.ref(request)) + raise ValueError("endpoint failed") + + with TestClient(app, raise_server_exceptions=False) as client: + assert client.post("/", json={"name": "request data"}).status_code == 500 + + gc.collect() + assert get_telemetry_data(saved_contexts[0]) is None + assert request_refs[0]() is None + + +@pytest.mark.parametrize("invalid_json", [False, True]) +@pytest.mark.parametrize("logging", [False, True]) +def test_validation_data_without_tracing(invalid_json, logging): + exporter = InMemoryLogRecordExporter() + provider = LoggerProvider(shutdown_on_exit=False) + observed = [] + + class Validation(LogRecordProcessor): + def on_emit(self, log_record): + record = log_record.log_record + data = get_telemetry_data(record.context) + assert data is not None + assert data.request is not None + assert data.request.url.path == "/items" + assert data.errors is not None + assert data.errors[0]["type"] == ( + "json_invalid" if invalid_json else "int_parsing" + ) + assert data.body == ("{" if invalid_json else {"amount": "private-input"}) + observed.append(record.context) + + def shutdown(self): + pass + + def force_flush(self, timeout_millis=30000): + return True + + provider.add_log_record_processor(Validation()) + provider.add_log_record_processor(SimpleLogRecordProcessor(exporter)) + app = FastAPI( + telemetry={"logger_provider": provider, "tracing": False, "logs": logging} + ) + + @app.post("/items") + def endpoint(item: dict[str, int]): + return item # pragma: no cover + + try: + with TestClient(app) as client: + response = ( + client.post( + "/items", content="{", headers={"content-type": "application/json"} + ) + if invalid_json + else client.post("/items", json={"amount": "private-input"}) + ) + assert provider.force_flush() + assert response.status_code == 422 + if not logging: + assert not exporter.get_finished_logs() + assert observed == [] + return + (data,) = exporter.get_finished_logs() + record = data.log_record + assert record.severity_number == SeverityNumber.WARN + assert record.event_name == "fastapi.validation.failed" + assert record.exception is None + assert record.attributes == { + "http.route": "/items", + "fastapi.validation.error_count": 1, + } + assert record.body == "Request validation failed" + assert len(observed) == 1 + assert get_telemetry_data(observed[0]) is None + assert get_telemetry_data(record.context) is None + finally: + provider.shutdown() + + +def test_request_objects_are_not_exported(): + from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter + from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter + from opentelemetry.proto.collector.logs.v1.logs_service_pb2 import ( + ExportLogsServiceRequest, + ) + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + + with otlp_collector() as (base, received): + logger = LoggerProvider(shutdown_on_exit=False) + logger.add_log_record_processor( + SimpleLogRecordProcessor(OTLPLogExporter(endpoint=base + "/v1/logs")) + ) + tracer = TracerProvider(shutdown_on_exit=False) + tracer.add_span_processor( + SimpleSpanProcessor(OTLPSpanExporter(endpoint=base + "/v1/traces")) + ) + app = FastAPI(telemetry={"tracer_provider": tracer, "logger_provider": logger}) + service = object() + + def dependency(): + return service + + @app.post("/items") + def endpoint(*, item: dict[str, str], count: int, value=Depends(dependency)): + data = get_telemetry_data() + assert data is not None + assert data.values is not None + assert data.values["value"] is service + raise ValueError("endpoint failed") + + @app.get("/handled") + def handled(): + raise HTTPException(503, "private-handled-detail") + + @app.websocket("/ws") + async def websocket_endpoint(*, websocket: WebSocket, count: int): + data = get_telemetry_data() + assert data is not None + assert data.websocket is websocket + assert data.body is None + await websocket.accept() + await websocket.send_text(await websocket.receive_text()) + await websocket.close() + + try: + with TestClient(app, raise_server_exceptions=False) as client: + with client.websocket_connect( + "/ws?count=1&query=websocket-search", + headers={ + "authorization": "Bearer authorization-secret", + "cookie": "session=session-secret", + }, + ) as websocket: + websocket.send_text("private-websocket-message") + assert websocket.receive_text() == "private-websocket-message" + with pytest.raises(WebSocketDisconnect): + with client.websocket_connect("/ws?count=invalid-websocket-count"): + pass # pragma: no cover + assert client.get("/handled").status_code == 503 + for count, status in [("1", 500), ("invalid-count", 422)]: + response = client.post( + "/items", + params={"count": count, "query": "search-term"}, + headers={ + "authorization": "Bearer authorization-secret", + "cookie": "session=session-secret", + }, + json={"name": "private-body"}, + ) + assert response.status_code == status + assert {path for path, _, _ in received} == {"/v1/logs", "/v1/traces"} + for path, payload, _ in received: + if path == "/v1/logs": + assert b"invalid-count" not in payload + assert b"search-term" not in payload + for uncaptured_value in [ + b"websocket-search", + b"invalid-websocket-count", + b"authorization-secret", + b"session-secret", + b"private-body", + b"private-handled-detail", + b"private-websocket-message", + ]: + assert uncaptured_value not in payload + traces = b"".join( + payload for path, payload, _ in received if path == "/v1/traces" + ) + assert b"invalid-count" in traces + assert b"search-term" in traces + records = [ + record + for path, payload, _ in received + if path == "/v1/logs" + for resource in ExportLogsServiceRequest.FromString( + payload + ).resource_logs + for scope in resource.scope_logs + for record in scope.log_records + ] + assert len(records) == 3 + assert records[0].event_name == "fastapi.validation.failed" + assert records[2].event_name == "fastapi.validation.failed" + finally: + tracer.shutdown() + logger.shutdown() diff --git a/tests/test_telemetry/test_environment_integrations.py b/tests/test_telemetry/test_environment_integrations.py new file mode 100644 index 0000000000..7ba7e01879 --- /dev/null +++ b/tests/test_telemetry/test_environment_integrations.py @@ -0,0 +1,244 @@ +"""Environment export remains active alongside independently configured SDKs.""" + +import pytest + +from ._otlp import otlp_collector +from ._subprocess import run_in_subprocess + + +@pytest.mark.parametrize( + "mode", + [ + "global", + "explicit", + "sentry-first", + "fastapi-first", + "sentry-classic", + "logfire", + "logfire-opt-out", + ], +) +@run_in_subprocess +def test_environment_export_with_existing_integrations(mode): + import os + + from fastapi import FastAPI + from fastapi.telemetry import TelemetryConfig, _runtime + from fastapi.testclient import TestClient + from opentelemetry import _logs, metrics, trace + from opentelemetry.proto.collector.logs.v1.logs_service_pb2 import ( + ExportLogsServiceRequest, + ) + from opentelemetry.proto.collector.metrics.v1.metrics_service_pb2 import ( + ExportMetricsServiceRequest, + ) + from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ( + ExportTraceServiceRequest, + ) + from opentelemetry.proto.trace.v1.trace_pb2 import Span + from opentelemetry.sdk._logs import LoggerProvider + from opentelemetry.sdk._logs.export import ( + InMemoryLogRecordExporter, + SimpleLogRecordProcessor, + ) + from opentelemetry.sdk.metrics import MeterProvider + from opentelemetry.sdk.metrics.export import InMemoryMetricReader + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + with otlp_collector() as (base, received): + os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = base + "/environment" + os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-cloud=preserved" + os.environ["OTEL_BSP_SCHEDULE_DELAY"] = "60000" + os.environ["OTEL_BLRP_SCHEDULE_DELAY"] = "60000" + os.environ["OTEL_METRIC_EXPORT_INTERVAL"] = "60000" + settings: TelemetryConfig = {} + original_spans = InMemorySpanExporter() + original_logs = InMemoryLogRecordExporter() + original_metrics = InMemoryMetricReader() + if mode in ("global", "explicit"): + tp = TracerProvider(shutdown_on_exit=False) + tp.add_span_processor(SimpleSpanProcessor(original_spans)) + mp = MeterProvider( + metric_readers=[original_metrics], shutdown_on_exit=False + ) + lp = LoggerProvider(shutdown_on_exit=False) + lp.add_log_record_processor(SimpleLogRecordProcessor(original_logs)) + if mode == "global": + trace.set_tracer_provider(tp) + metrics.set_meter_provider(mp) + _logs.set_logger_provider(lp) + else: + settings = { + "tracer_provider": tp, + "meter_provider": mp, + "logger_provider": lp, + } + elif mode.startswith("logfire"): + import logfire + + logfire.configure( + send_to_logfire=False, + console=False, + additional_span_processors=[SimpleSpanProcessor(original_spans)], + metrics=logfire.MetricsOptions(additional_readers=[original_metrics]), + advanced=logfire.AdvancedOptions( + log_record_processors=[SimpleLogRecordProcessor(original_logs)] + ), + ) + settings["auto_configure"] = mode != "logfire-opt-out" + else: + import sentry_sdk + from sentry_sdk.integrations.fastapi import FastApiIntegration + from sentry_sdk.integrations.otlp import OTLPIntegration + from sentry_sdk.integrations.starlette import StarletteIntegration + from sentry_sdk.transport import Transport + + items = [] + + class LocalTransport(Transport): + def capture_envelope(self, envelope): + items.extend(item.type for item in envelope.items) + + if mode == "fastapi-first": + with TestClient(FastAPI()): + pass + integrations = ( + [StarletteIntegration(), FastApiIntegration()] + if mode == "sentry-classic" + else [ + OTLPIntegration( + collector_url=base + "/sentry/traces", setup_propagator=False + ) + ] + ) + sentry_sdk.init( + dsn="https://public@example.invalid/1", + transport=LocalTransport, + default_integrations=False, + auto_enabling_integrations=False, + integrations=integrations, + traces_sample_rate=1.0, + send_client_reports=False, + ) + + app = FastAPI(telemetry=settings) + + @app.get("/items/{item_id}") + def endpoint(item_id: int): + raise ValueError("environment export") + + try: + for _ in range(2): + with TestClient(app, raise_server_exceptions=False) as client: + assert client.get("/items/1").status_code == 500 + if mode.startswith("logfire"): + logfire.force_flush() + elif mode.startswith("sentry") or mode == "fastapi-first": + tracer_provider = trace.get_tracer_provider() + assert isinstance(tracer_provider, TracerProvider) + tracer_provider.force_flush() + sentry_sdk.flush() + + def spans_at(path): + requests = [ + ExportTraceServiceRequest.FromString(body) + for url, body, _ in received + if url == path + ] + return [ + span + for request in requests + for resource in request.resource_spans + for scope in resource.scope_spans + for span in scope.spans + ] + + environment_spans = spans_at("/environment/v1/traces") + multiplier = 2 if mode == "logfire" else 1 + servers = [ + span for span in environment_spans if span.kind == Span.SPAN_KIND_SERVER + ] + assert len(servers) == 2 * multiplier, (mode, len(servers)) + assert len({span.span_id for span in servers}) == 2 + assert all(span.name == "GET /items/{item_id}" for span in servers) + if mode in ("global", "explicit"): + assert sorted( + (span.context.trace_id, span.context.span_id, span.name) + for span in original_spans.get_finished_spans() + ) == sorted( + ( + int.from_bytes(span.trace_id, "big"), + int.from_bytes(span.span_id, "big"), + span.name, + ) + for span in environment_spans + ) + if mode in ("sentry-first", "fastapi-first"): + sentry_spans = spans_at("/sentry/traces") + assert {span.span_id for span in sentry_spans} == { + span.span_id for span in environment_spans + } + if mode == "sentry-classic": + assert items.count("event") == 2, items + assert items.count("transaction") == 2, items + + log_requests = [ + ExportLogsServiceRequest.FromString(body) + for url, body, _ in received + if url == "/environment/v1/logs" + ] + logs = [ + record + for request in log_requests + for resource in request.resource_logs + for scope in resource.scope_logs + for record in scope.log_records + ] + assert len(logs) == 2 * multiplier, (mode, len(logs)) + assert all( + record.trace_id in {span.trace_id for span in servers} + for record in logs + ) + metric_requests = [ + ExportMetricsServiceRequest.FromString(body) + for url, body, _ in received + if url == "/environment/v1/metrics" + ] + counts = [ + point.count + for request in metric_requests + for resource in request.resource_metrics + for scope in resource.scope_metrics + for metric in scope.metrics + if metric.name == "http.server.request.duration" + for point in getattr(metric, metric.WhichOneof("data")).data_points + ] + assert counts and counts[-1] == 2, counts + assert all( + headers.get("x-cloud") == "preserved" + for url, _, headers in received + if url.startswith("/environment/") + ) + assert len(_runtime._configured) == (0 if mode == "logfire-opt-out" else 3) + _runtime._shutdown() + if mode in ("global", "explicit"): + # FastAPI's cleanup closes its own exporters, leaving the provider and + # previously installed vendor components usable. + count = len(original_spans.get_finished_spans()) + with tp.get_tracer("vendor").start_as_current_span("still-running"): + pass + assert len(original_spans.get_finished_spans()) == count + 1 + lp.get_logger("vendor").emit(body="still-running") + assert len(original_logs.get_finished_logs()) == 3 + assert original_metrics.get_metrics_data() is not None + tp.shutdown() + mp.shutdown() + lp.shutdown() + finally: + _runtime._shutdown() + if mode.startswith("logfire"): + logfire.shutdown() diff --git a/tests/test_telemetry/test_exceptions.py b/tests/test_telemetry/test_exceptions.py new file mode 100644 index 0000000000..14f15a32ca --- /dev/null +++ b/tests/test_telemetry/test_exceptions.py @@ -0,0 +1,249 @@ +import pytest +from fastapi import FastAPI, HTTPException, Request +from fastapi.responses import JSONResponse, StreamingResponse +from fastapi.testclient import TestClient +from opentelemetry import trace +from opentelemetry._logs import NoOpLoggerProvider, SeverityNumber +from opentelemetry.sdk.trace.sampling import ALWAYS_OFF + +from .conftest import metric_points, server_spans + + +@pytest.mark.parametrize("sampled", [True, False]) +def test_exception_is_logged_once_with_trace_context(telemetry, logs, sampled): + config, spans, reader = telemetry + if not sampled: + config["tracer_provider"].sampler = ALWAYS_OFF + app = FastAPI(telemetry=config) + contexts = [] + error = ValueError("test failure") + + @app.get("/items/{item_id}") + def endpoint(item_id: int): + contexts.append(trace.get_current_span().get_span_context()) + raise error + + assert ( + TestClient(app, raise_server_exceptions=False).get("/items/1").status_code + == 500 + ) + (data,) = logs.get_finished_logs() + record = data.log_record + assert record.event_name == "http.server.request.exception" + assert record.timestamp is not None + assert record.timestamp <= record.observed_timestamp + assert ( + data.instrumentation_scope.schema_url + == "https://opentelemetry.io/schemas/1.44.0" + ) + assert record.exception is error + assert record.trace_id == contexts[0].trace_id + assert record.span_id != 0 + if sampled: + (server,) = server_spans(spans) + assert record.span_id == server.context.span_id + assert record.severity_number == SeverityNumber.ERROR + assert record.attributes["http.route"] == "/items/{item_id}" + assert record.attributes["exception.type"] == "ValueError" + assert record.attributes["exception.message"] == "test failure" + assert "raise error" in record.attributes["exception.stacktrace"] + assert len(server_spans(spans)) == int(sampled) + + +@pytest.mark.parametrize("raises", [False, True]) +def test_exclusion_applies_to_mounted_apps(telemetry, logs, raises): + config, spans, reader = telemetry + scopes = [] + child = FastAPI(telemetry=config) + + @child.get("/{name}") + def endpoint(*, request: Request, name: str): + scopes.append(request.scope) + if name == "excluded" and raises: + raise ValueError("excluded error") + return name + + parent = FastAPI( + telemetry={ + **config, + "exclude": lambda scope: scope["path"] == "/child/excluded", + } + ) + parent.mount("/child", child) + client = TestClient(parent, raise_server_exceptions=False) + assert client.get("/child/excluded").status_code == (500 if raises else 200) + assert not spans.get_finished_spans() + assert not logs.get_finished_logs() + assert not metric_points(reader=reader) + assert "fastapi.telemetry" not in scopes[0] + assert client.get("/child/included").json() == "included" + (span,) = server_spans(spans) + assert span.attributes["http.route"] == "/child/{name}" + assert metric_points(reader=reader)[0].count == 1 + + +def test_handled_exceptions_and_validation_are_not_error_logs(telemetry, logs): + config, spans, reader = telemetry + app = FastAPI(telemetry=config) + + @app.exception_handler(ValueError) + async def handled(request, exc): + return JSONResponse({"detail": "handled"}, status_code=400) + + @app.get("/items/{value}") + def endpoint(value: int): + if value == 1: + raise HTTPException(404) + raise ValueError("handled by application") + + client = TestClient(app) + assert client.get("/items/no").status_code == 422 + assert client.get("/items/1").status_code == 404 + assert client.get("/items/2").status_code == 400 + assert client.get("/missing").status_code == 404 + (validation,) = logs.get_finished_logs() + assert validation.log_record.severity_number == SeverityNumber.WARN + assert validation.log_record.event_name == "fastapi.validation.failed" + assert validation.log_record.timestamp is not None + assert validation.log_record.timestamp <= validation.log_record.observed_timestamp + assert validation.log_record.exception is None + + +@pytest.mark.parametrize("stage", ["stream", "cleanup"]) +def test_errors_after_response_started_are_logged(telemetry, logs, stage): + from fastapi import Depends + + config, spans, reader = telemetry + app = FastAPI(telemetry=config) + + def fail(): + raise ValueError("after response started") + + async def dependency(): + yield + if stage == "cleanup": + fail() + + @app.get("/", dependencies=[Depends(dependency)]) + async def endpoint(): + if stage == "stream": + + async def stream(): + yield "start" + fail() + + return StreamingResponse(stream()) + return "ok" + + assert TestClient(app, raise_server_exceptions=False).get("/").status_code == 200 + (data,) = logs.get_finished_logs() + (span,) = server_spans(spans) + assert data.log_record.trace_id == span.context.trace_id + assert ( + "after response started" in data.log_record.attributes["exception.stacktrace"] + ) + + +@pytest.mark.parametrize("mode", ["logs_only", "disabled", "noop", "excluded"]) +def test_independent_log_configuration(telemetry, logs, mode): + config, spans, reader = telemetry + if mode == "logs_only": + config.update(tracing=False, metrics=False) + elif mode == "disabled": + config["logs"] = False + elif mode == "noop": + config["logger_provider"] = NoOpLoggerProvider() + else: + config["exclude"] = lambda scope: True + app = FastAPI(telemetry=config) + + @app.get("/") + def endpoint(): + raise ValueError("test") + + assert TestClient(app, raise_server_exceptions=False).get("/").status_code == 500 + records = logs.get_finished_logs() + assert len(records) == int(mode == "logs_only") + if records: + assert records[0].log_record.trace_id == 0 + + +@pytest.mark.parametrize("status", [404, 503]) +@pytest.mark.parametrize("sampled", [False, True]) +def test_http_exception_records_status_without_exception_telemetry( + telemetry, logs, status, sampled +): + from opentelemetry.trace import StatusCode + + config, spans, reader = telemetry + if not sampled: + config["tracer_provider"].sampler = ALWAYS_OFF + app = FastAPI(telemetry=config) + + @app.get("/items/{item_id}") + def endpoint(item_id: int): + raise HTTPException(status, "private exception detail") + + response = TestClient(app).get("/items/1") + assert response.status_code == status + assert response.json() == {"detail": "private exception detail"} + assert not logs.get_finished_logs() + (point,) = metric_points(reader=reader) + assert point.count == 1 + assert point.attributes["http.response.status_code"] == status + assert point.attributes["http.route"] == "/items/{item_id}" + assert point.attributes.get("error.type") == ("503" if status == 503 else None) + finished = spans.get_finished_spans() + assert all(not span.events for span in finished) + if sampled: + (server,) = server_spans(spans) + assert server.attributes["http.response.status_code"] == status + assert server.status.status_code == ( + StatusCode.ERROR if status == 503 else StatusCode.UNSET + ) + else: + assert not finished + + +def test_exception_handler_failure_is_logged_once_as_unhandled(telemetry, logs): + config, _, _ = telemetry + failure = RuntimeError("handler failed") + app = FastAPI(telemetry=config) + + @app.exception_handler(HTTPException) + async def handler(request, exc): + raise failure + + @app.get("/") + async def endpoint(): + raise HTTPException(404, "original") + + assert TestClient(app, raise_server_exceptions=False).get("/").status_code == 500 + (data,) = logs.get_finished_logs() + assert data.log_record.exception is failure + assert data.log_record.body == "Unhandled exception in FastAPI request" + + +class ApplicationError(Exception): + pass + + +def test_exception_type_matches_spans_metrics_and_logs(telemetry, logs): + config, spans, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/") + async def endpoint(): + raise ApplicationError("failed") + + assert TestClient(app, raise_server_exceptions=False).get("/").status_code == 500 + (record,) = logs.get_finished_logs() + expected = f"{__name__}.ApplicationError" + assert record.log_record.attributes["exception.type"] == expected + failed_spans = [ + span for span in spans.get_finished_spans() if span.status.is_ok is False + ] + assert {span.name for span in failed_spans} == {"GET /", "fastapi.endpoint"} + assert all(span.attributes["error.type"] == expected for span in failed_spans) + (point,) = metric_points(reader=reader) + assert point.attributes["error.type"] == expected diff --git a/tests/test_telemetry/test_http.py b/tests/test_telemetry/test_http.py new file mode 100644 index 0000000000..3abeb06e24 --- /dev/null +++ b/tests/test_telemetry/test_http.py @@ -0,0 +1,695 @@ +import asyncio +from contextlib import asynccontextmanager + +import anyio +import pytest +from fastapi import ( + APIRouter, + BackgroundTasks, + Depends, + FastAPI, + HTTPException, + Request, +) +from fastapi.responses import JSONResponse, PlainTextResponse, StreamingResponse +from fastapi.testclient import TestClient +from opentelemetry import baggage, propagate, trace +from opentelemetry.propagators import textmap +from opentelemetry.sdk.trace.sampling import ALWAYS_OFF +from opentelemetry.trace import SpanKind, StatusCode +from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator +from starlette.middleware import Middleware +from starlette.types import Scope + +from ._subprocess import run_in_subprocess +from .conftest import metric_points, server_spans + + +def test_request_context_and_span_attributes(telemetry): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + seen = [] + + @app.get("/items/{item_id}") + async def endpoint(item_id: int): + seen.append(trace.get_current_span().get_span_context()) + return item_id + + client = TestClient(app) + parent_trace = "0123456789abcdef0123456789abcdef" + parent_span = "0123456789abcdef" + response = client.get( + "/items/123?page=2&sig=signature-secret", + headers={ + "traceparent": f"00-{parent_trace}-{parent_span}-01", + "authorization": "Bearer authorization-secret", + }, + ) + assert response.json() == 123 + (span,) = server_spans(exporter) + assert span.kind == SpanKind.SERVER + assert span.name == "GET /items/{item_id}" + endpoint_span = next( + s for s in exporter.get_finished_spans() if s.name == "fastapi.endpoint" + ) + assert endpoint_span.context == seen[0] + assert endpoint_span.parent.span_id == span.context.span_id + assert span.context.trace_id == int(parent_trace, 16) + assert span.parent.span_id == int(parent_span, 16) + assert span.status.status_code == StatusCode.UNSET + assert span.attributes["http.route"] == "/items/{item_id}" + assert span.attributes["http.response.status_code"] == 200 + assert "signature-secret" not in repr(span.attributes) + assert "authorization-secret" not in repr(span.attributes) + assert span.attributes["url.path"] == "/items/123" + assert span.attributes["url.query"] == "page=2&sig=REDACTED" + points = metric_points(reader=reader) + assert len(points) == 1 + assert points[0].count == 1 + assert points[0].sum > 0 + assert points[0].attributes["http.route"] == "/items/{item_id}" + assert points[0].explicit_bounds == ( + 0.005, + 0.01, + 0.025, + 0.05, + 0.075, + 0.1, + 0.25, + 0.5, + 0.75, + 1, + 2.5, + 5, + 7.5, + 10, + ) + assert ( + metric_points(reader=reader, name="http.server.active_requests")[0].value == 0 + ) + assert not trace.get_current_span().get_span_context().is_valid + + +@pytest.mark.parametrize("split_headers", [False, True]) +def test_propagation_headers(telemetry, split_headers): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + + @app.get("/") + async def endpoint(): + return dict(baggage.get_all()) + + trace_id = "0123456789abcdef0123456789abcdef" + parent_id = "0123456789abcdef" + headers = [("traceparent", f"00-{trace_id}-{parent_id}-01")] + for name, values in [ + ("tracestate", ["vendora=one", "vendorb=two"]), + ("baggage", ["first=one", "second=two"]), + ]: + headers.extend( + (name, value) for value in (values if split_headers else [",".join(values)]) + ) + response = TestClient(app).get("/", headers=headers) + assert response.json() == {"first": "one", "second": "two"} + (span,) = server_spans(exporter) + assert span.context.trace_id == int(trace_id, 16) + assert span.parent.span_id == int(parent_id, 16) + assert list(span.context.trace_state.items()) == [ + ("vendora", "one"), + ("vendorb", "two"), + ] + assert not baggage.get_all() + + +def test_custom_propagator_can_read_repeated_headers(telemetry, monkeypatch): + class HeaderPropagator(TraceContextTextMapPropagator): + def extract(self, carrier, context=None, getter=textmap.default_getter): + assert "x-custom-context" in getter.keys(carrier) + values = getter.get(carrier, "X-Custom-Context") + assert values is not None + assert values == ["one, two", "three"] + assert getter.get(carrier, "missing-header") is None + context = super().extract(carrier, context=context, getter=getter) + return baggage.set_baggage("custom", "|".join(values), context=context) + + monkeypatch.setattr(propagate, "get_global_textmap", lambda: HeaderPropagator()) + config, _, _ = telemetry + app = FastAPI(telemetry=config) + + @app.get("/") + async def endpoint(): + return baggage.get_baggage("custom") + + response = TestClient(app).get( + "/", headers=[("x-custom-context", "one, two"), ("x-custom-context", "three")] + ) + assert response.json() == "one, two|three" + assert not baggage.get_all() + + +@pytest.mark.parametrize( + "path,status,route,error", + [ + ("/ok", 200, "/ok", False), + ("/missing", 404, None, False), + ("/validation/no", 422, "/validation/{value}", False), + ("/validation/1", 200, "/validation/{value}", False), + ("/fail", 500, "/fail", True), + ("/handled", 503, "/handled", True), + ("/ok/", 307, "/ok", False), + ], +) +def test_http_statuses(telemetry, path, status, route, error): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/ok") + def ok(): + return "ok" + + @app.get("/validation/{value}") + def validation(value: int): + return value + + @app.get("/fail") + def fail(): + raise ValueError("endpoint failed") + + @app.get("/handled") + async def handled(): + raise HTTPException(503, "Unavailable") + + assert ( + TestClient(app, raise_server_exceptions=False) + .get(path, follow_redirects=False) + .status_code + == status + ) + (span,) = server_spans(exporter) + assert span.attributes["http.response.status_code"] == status + assert span.attributes.get("http.route") == route + assert (span.status.status_code == StatusCode.ERROR) == error + assert "endpoint failed" not in repr(span.attributes) + assert span.events == () + assert sum(p.count for p in metric_points(reader=reader)) == 1 + + +def test_custom_error_handler_sees_span(telemetry): + config, exporter, reader = telemetry + seen = [] + + async def error_handler(request, exc): + span = trace.get_current_span() + seen.append(span.is_recording()) + span.set_attribute("handled", True) + return JSONResponse({"error": "handled"}, status_code=500) + + app = FastAPI(telemetry=config, exception_handlers={500: error_handler}) + + @app.get("/") + def endpoint(): + raise ValueError("endpoint failed") + + assert TestClient(app, raise_server_exceptions=False).get("/").json() == { + "error": "handled" + } + (span,) = server_spans(exporter) + assert seen == [True] + assert span.attributes["handled"] is True + assert span.attributes["error.type"] == "ValueError" + + +def test_route_prefix_and_dynamic_mount(telemetry): + config, exporter, reader = telemetry + parent = FastAPI(telemetry=config, root_path="/proxy") + child = FastAPI() + router = APIRouter(prefix="/v1") + + @router.get("/items/{item_id}") + def endpoint(item_id: int): + return item_id + + child.include_router(router, prefix="/api") + parent.mount("/tenants/{tenant}", child) + response = TestClient(parent).get("/proxy/tenants/acme/api/v1/items/4") + assert response.json() == 4 + (span,) = server_spans(exporter) + assert ( + span.attributes["http.route"] + == "/proxy/tenants/{tenant}/api/v1/items/{item_id}" + ) + assert span.name == "GET /proxy/tenants/{tenant}/api/v1/items/{item_id}" + assert span.attributes["url.path"] == "/proxy/tenants/acme/api/v1/items/4" + assert "acme" not in repr(metric_points(reader=reader)[0].attributes) + assert sum(p.count for p in metric_points(reader=reader)) == 1 + + +@pytest.mark.parametrize("nested", [False, True]) +@pytest.mark.parametrize("route_type", ["fastapi", "starlette"]) +def test_included_router_redirect_template(telemetry, nested, route_type): + config, exporter, reader = telemetry + router = APIRouter() + + def endpoint(request: Request): + return PlainTextResponse("ok") + + if route_type == "fastapi": + router.add_api_route("/items/{item_id}/", endpoint) + else: + router.add_route("/items/{item_id}/", endpoint) + if nested: + outer = APIRouter() + outer.include_router(router, prefix="/v1") + router = outer + child = FastAPI() + child.include_router(router, prefix="/api") + app = FastAPI(telemetry=config) + app.mount("/tenants/{tenant}", child) + prefix = "/api" + ("/v1" if nested else "") + path = "/tenants/acme" + prefix + "/items/1" + template = "/tenants/{tenant}" + prefix + "/items/{item_id}/" + client = TestClient(app) + response = client.get(path, follow_redirects=False) + assert response.status_code == 307 + assert response.headers["location"] == "http://testserver" + path + "/" + (span,) = server_spans(exporter) + assert span.name == "GET " + template + assert span.attributes["http.route"] == template + (point,) = metric_points(reader=reader) + assert point.count == 1 + assert point.attributes["http.route"] == template + assert client.get(path + "/").text == "ok" + + +def test_method_not_allowed_and_docs(telemetry): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/items/{value}") + def endpoint(value: str): + return value + + client = TestClient(app) + assert client.post("/items/x").status_code == 405 + assert client.get("/docs").status_code == 200 + assert [s.attributes["http.route"] for s in exporter.get_finished_spans()] == [ + "/items/{value}", + "/docs", + ] + assert client.get("/items/x").json() == "x" + + +def test_stream_completes_before_background_and_cleanup(telemetry): + config, exporter, reader = telemetry + seen = [] + app = FastAPI(telemetry=config) + + async def dependency(): + yield + seen.append(("cleanup", len(server_spans(exporter)))) + + async def background(): + seen.append(("background", len(server_spans(exporter)))) + + @app.get("/", dependencies=[Depends(dependency)]) + async def endpoint(tasks: BackgroundTasks): + tasks.add_task(background) + + async def stream(): + yield "one" + assert not server_spans(exporter) + await anyio.sleep(0) + yield "two" + + return StreamingResponse(stream()) + + assert TestClient(app).get("/").text == "onetwo" + assert seen == [("background", 1), ("cleanup", 1)] + assert sum(p.count for p in metric_points(reader=reader)) == 1 + + +@pytest.mark.parametrize( + "tracing,metering", [(True, False), (False, True), (False, False)] +) +def test_independent_signals(telemetry, tracing, metering): + config, exporter, reader = telemetry + config["tracing"], config["metrics"] = tracing, metering + app = FastAPI(telemetry=config) + TestClient(app).get("/missing") + assert len(exporter.get_finished_spans()) == int(tracing) + assert sum(p.count for p in metric_points(reader=reader)) == int(metering) + + +def test_configuration_is_copied_for_each_app(telemetry): + config, exporter, reader = telemetry + original = config.copy() + enabled = FastAPI(telemetry=config) + assert config == original + + config["tracing"] = False + disabled = FastAPI(telemetry=config) + config["tracing"] = True + + assert TestClient(enabled).get("/").status_code == 404 + assert TestClient(disabled).get("/").status_code == 404 + assert len(server_spans(exporter)) == 1 + assert sum(point.count for point in metric_points(reader=reader)) == 2 + + +def test_unsampled_requests_still_record_metrics(telemetry): + config, exporter, reader = telemetry + config["tracer_provider"].sampler = ALWAYS_OFF + TestClient(FastAPI(telemetry=config)).get("/missing") + assert not exporter.get_finished_spans() + assert metric_points(reader=reader)[0].count == 1 + + +def test_exclusion(telemetry): + config, exporter, reader = telemetry + config["exclude"] = lambda scope: scope["path"] == "/health" + client = TestClient(FastAPI(telemetry=config)) + client.get("/health") + assert not exporter.get_finished_spans() + assert not metric_points(reader=reader) + client.get("/other") + assert len(exporter.get_finished_spans()) == 1 + + +@pytest.mark.parametrize("raises", [False, True]) +def test_middleware_response_or_error(telemetry, raises): + config, exporter, reader = telemetry + + class CustomMiddleware: + def __init__(self, app): + self.app = app + + async def __call__(self, scope, receive, send): + if raises: + raise RuntimeError("middleware failed") + await PlainTextResponse("middleware")(scope, receive, send) + + app = FastAPI(telemetry=config, middleware=[Middleware(CustomMiddleware)]) + response = TestClient(app, raise_server_exceptions=False).get("/") + assert response.status_code == (500 if raises else 200) + (span,) = server_spans(exporter) + assert "http.route" not in span.attributes + if raises: + assert span.attributes["error.type"] == "RuntimeError" + + +@pytest.mark.parametrize( + "kind", ["trailers", "disconnect", "cancel", "pathsend", "incomplete", "send_error"] +) +def test_asgi_lifecycle(telemetry, kind): + from fastapi.telemetry._asgi import NativeTelemetry + + config, exporter, reader = telemetry + sent = [] + scope: Scope = { + "type": "http", + "method": "GET", + "scheme": "http", + "path": "/", + "headers": [], + } + + async def receive(): + return {"type": "http.disconnect"} + + async def send(message): + if kind == "send_error": + raise OSError("closed") + sent.append(message) + + async def app(scope, receive, send): + if kind == "disconnect": + await receive() + return + if kind == "cancel": + raise asyncio.CancelledError() + if kind == "incomplete": + return + await send( + { + "type": "http.response.start", + "status": 200, + "trailers": kind == "trailers", + } + ) + if kind == "pathsend": + await send({"type": "http.response.pathsend", "path": "/tmp/example"}) + else: + await send({"type": "http.response.body", "body": b"ok"}) + if kind == "trailers": + assert not exporter.get_finished_spans() + await send( + {"type": "http.response.trailers", "headers": [], "more_trailers": True} + ) + assert not exporter.get_finished_spans() + await send({"type": "http.response.trailers", "headers": []}) + + async def run(): + await NativeTelemetry(FastAPI(telemetry=config)._telemetry)( + app=app, scope=scope, receive=receive, send=send + ) + + if kind == "cancel": + with pytest.raises(asyncio.CancelledError): + asyncio.run(run()) + elif kind == "send_error": + with pytest.raises(OSError): + asyncio.run(run()) + else: + asyncio.run(run()) + (span,) = server_spans(exporter) + assert (span.status.status_code == StatusCode.ERROR) == ( + kind in {"cancel", "disconnect", "incomplete", "send_error"} + ) + assert metric_points(reader=reader)[0].count == 1 + assert ( + metric_points(reader=reader, name="http.server.active_requests")[0].value == 0 + ) + assert "fastapi.telemetry" not in scope + + +def test_multiple_lifespans_borrowed_providers(telemetry): + config, exporter, reader = telemetry + lifecycle = [] + + @asynccontextmanager + async def lifespan(app): + lifecycle.append("start") + yield {"state": True} + lifecycle.append("stop") + + first = FastAPI(telemetry=config, lifespan=lifespan) + second = FastAPI(telemetry=config) + for app in [first, second, first]: + with TestClient(app) as client: + assert client.get("/").status_code == 404 + assert lifecycle == ["start", "stop", "start", "stop"] + assert len(exporter.get_finished_spans()) == 3 + assert sum(p.count for p in metric_points(reader=reader)) == 3 + + +@pytest.mark.parametrize( + "method", + [ + "CONNECT", + "DELETE", + "GET", + "HEAD", + "OPTIONS", + "PATCH", + "POST", + "PUT", + "QUERY", + "TRACE", + ], +) +def test_known_http_methods(telemetry, method): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.api_route("/items/{value}", methods=[method]) + def endpoint(value: str): + return value + + assert TestClient(app).request(method, "/items/1").status_code == 200 + (span,) = server_spans(exporter) + assert span.name == f"{method} /items/{{value}}" + assert span.attributes["http.request.method"] == method + assert "http.request.method_original" not in span.attributes + assert metric_points(reader=reader)[0].attributes["http.request.method"] == method + assert ( + metric_points(reader=reader, name="http.server.active_requests")[0].attributes[ + "http.request.method" + ] + == method + ) + + +@pytest.mark.parametrize( + "path,status,span_name", + [("/items/1", 405, "HTTP /items/{value}"), ("/missing", 404, "HTTP")], +) +def test_unknown_method_has_bounded_span_name(telemetry, path, status, span_name): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/items/{value}") + def endpoint(value: str): + return value + + assert TestClient(app).request("PRIVATE_METHOD", path).status_code == status + (span,) = server_spans(exporter) + assert span.name == span_name + assert span.attributes["http.request.method"] == "_OTHER" + assert span.attributes["http.request.method_original"] == "PRIVATE_METHOD" + for name in ("http.server.request.duration", "http.server.active_requests"): + attributes = metric_points(reader=reader, name=name)[0].attributes + assert attributes["http.request.method"] == "_OTHER" + assert "http.request.method_original" not in attributes + assert TestClient(app).get("/items/1").json() == "1" + + +@pytest.mark.parametrize("matched", [True, False]) +@pytest.mark.parametrize( + "known_methods,method,expected", + [ + (" QUERY , PROPFIND , ", "QUERY", "QUERY"), + (" QUERY , PROPFIND , ", "PROPFIND", "PROPFIND"), + ("QUERY,PROPFIND", "GET", "_OTHER"), + ("query", "QUERY", "_OTHER"), + ("CUSTOM", "CUSTOM", "CUSTOM"), + ("", "QUERY", "QUERY"), + (" ", "QUERY", "QUERY"), + ], +) +def test_known_http_methods_override( + telemetry, monkeypatch, known_methods, method, expected, matched +): + monkeypatch.setenv("OTEL_INSTRUMENTATION_HTTP_KNOWN_METHODS", known_methods) + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.api_route("/items/{value}", methods=[method]) + def endpoint(value: str): + return value + + path = "/items/1" if matched else "/missing" + assert TestClient(app).request(method, path).status_code == ( + 200 if matched else 404 + ) + (span,) = server_spans(exporter) + span_method = "HTTP" if expected == "_OTHER" else method + assert span.name == span_method + (" /items/{value}" if matched else "") + assert span.attributes["http.request.method"] == expected + if expected == "_OTHER": + assert span.attributes["http.request.method_original"] == method + else: + assert "http.request.method_original" not in span.attributes + for name in ("http.server.request.duration", "http.server.active_requests"): + attributes = metric_points(reader=reader, name=name)[0].attributes + assert attributes["http.request.method"] == expected + assert "http.request.method_original" not in attributes + + +def test_frontend_and_static_templates(telemetry, tmp_path): + from fastapi.staticfiles import StaticFiles + + config, exporter, reader = telemetry + (tmp_path / "index.html").write_text("index") + (tmp_path / "script.js").write_text("script") + app = FastAPI(telemetry=config) + app.mount("/static", StaticFiles(directory=tmp_path)) + router = APIRouter() + router.frontend("/ui", directory=tmp_path, fallback="index.html") + app.include_router(router, prefix="/app") + client = TestClient(app) + assert client.get("/static/script.js").text == "script" + assert ( + client.get("/app/ui/arbitrary/path", headers={"accept": "text/html"}).text + == "index" + ) + assert [s.attributes["http.route"] for s in exporter.get_finished_spans()] == [ + "/static/{path}", + "/app/ui/{path}", + ] + + +def test_included_mount_template(telemetry): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + child = FastAPI() + + @child.get("/items/{value}") + def endpoint(value: str): + return value + + router = APIRouter() + router.mount("/tenants/{tenant}", child) + app.include_router(router, prefix="/api") + assert TestClient(app).get("/api/tenants/acme/items/1").json() == "1" + (span,) = server_spans(exporter) + assert span.attributes["http.route"] == "/api/tenants/{tenant}/items/{value}" + + +@pytest.mark.parametrize("explicit_proxy", [False, True]) +@run_in_subprocess +def test_late_global_providers_enable_existing_app(explicit_proxy): + from fastapi import FastAPI + from fastapi.testclient import TestClient + from opentelemetry import metrics, trace + from opentelemetry.sdk.metrics import MeterProvider + from opentelemetry.sdk.metrics.export import InMemoryMetricReader + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + app = FastAPI( + telemetry={ + "tracer_provider": trace.get_tracer_provider(), + "meter_provider": metrics.get_meter_provider(), + } + if explicit_proxy + else None + ) + client = TestClient(app) + assert client.get("/").status_code == 404 + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + reader = InMemoryMetricReader() + trace.set_tracer_provider(provider) + metrics.set_meter_provider(MeterProvider(metric_readers=[reader])) + assert client.get("/").status_code == 404 + assert len(exporter.get_finished_spans()) == 1 + assert reader.get_metrics_data() is not None + + +def test_base_http_middleware_preserves_context_and_route(telemetry): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + seen = [] + + @app.middleware("http") + async def middleware(request, call_next): + seen.append(trace.get_current_span().get_span_context().span_id) + return await call_next(request) + + @app.get("/items/{item_id}") + async def endpoint(item_id: int): + seen.append(trace.get_current_span().get_span_context().span_id) + return item_id + + assert TestClient(app).get("/items/1").json() == 1 + (span,) = server_spans(exporter) + endpoint_span = next( + s for s in exporter.get_finished_spans() if s.name == "fastapi.endpoint" + ) + assert seen == [span.context.span_id, endpoint_span.context.span_id] + assert endpoint_span.parent.span_id == span.context.span_id + assert span.attributes["http.route"] == "/items/{item_id}" diff --git a/tests/test_telemetry/test_integrations.py b/tests/test_telemetry/test_integrations.py new file mode 100644 index 0000000000..ee37db0b34 --- /dev/null +++ b/tests/test_telemetry/test_integrations.py @@ -0,0 +1,351 @@ +"""Published integration compatibility, isolated because SDKs patch globals.""" + +import os + +import pytest + +from ._subprocess import run_in_subprocess + + +@pytest.mark.parametrize("mode", ["app", "global", "late", "uninstrument"]) +@run_in_subprocess +def test_current_contrib(mode): + import os + + os.environ.update({"OTEL_SEMCONV_STABILITY_OPT_IN": "http"}) + + import fastapi + from fastapi.testclient import TestClient + from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor + from opentelemetry.sdk.metrics import MeterProvider + from opentelemetry.sdk.metrics.export import InMemoryMetricReader + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + from opentelemetry.trace import SpanKind + + native = InMemorySpanExporter() + legacy = InMemorySpanExporter() + p1, p2 = TracerProvider(), TracerProvider() + p1.add_span_processor(SimpleSpanProcessor(native)) + p2.add_span_processor(SimpleSpanProcessor(legacy)) + r1, r2 = InMemoryMetricReader(), InMemoryMetricReader() + m1, m2 = MeterProvider(metric_readers=[r1]), MeterProvider(metric_readers=[r2]) + hooks = [] + + def server_request_hook(span, scope): + hooks.append(scope["path"]) + + if mode == "global": + FastAPIInstrumentor().instrument( + tracer_provider=p2, + meter_provider=m2, + server_request_hook=server_request_hook, + exclude_spans=["send", "receive"], + ) + app = fastapi.FastAPI(telemetry={"tracer_provider": p1, "meter_provider": m1}) + + @app.get("/items/{value}") + def endpoint(value: int): + return value + + client = TestClient(app) + if mode == "late": + assert client.get("/items/1").json() == 1 + if mode != "global": + FastAPIInstrumentor.instrument_app( + app, + tracer_provider=p2, + meter_provider=m2, + server_request_hook=server_request_hook, + exclude_spans=["send", "receive"], + ) + assert client.get("/items/2").json() == 2 + if mode == "late": + # Current contrib cannot replace an already-built stack. Native remains live. + assert len(native.get_finished_spans()) == 8 + assert not legacy.get_finished_spans() + else: + assert not native.get_finished_spans() + assert ( + len([s for s in legacy.get_finished_spans() if s.kind == SpanKind.SERVER]) + == 1 + ) + assert hooks == ["/items/2"] + assert r1.get_metrics_data() is None + if mode == "uninstrument": + FastAPIInstrumentor.uninstrument_app(app) + assert client.get("/items/3").json() == 3 + assert len(native.get_finished_spans()) == 4 + + @app.websocket("/ws") + async def websocket_endpoint(websocket: fastapi.WebSocket): + await websocket.accept() + await websocket.close() + + native.clear() + legacy.clear() + with client.websocket_connect("/ws"): + pass + if mode in ("late", "uninstrument"): + assert len(native.get_finished_spans()) == 3 + assert not legacy.get_finished_spans() + else: + assert not native.get_finished_spans() + assert ( + len( + [ + span + for span in legacy.get_finished_spans() + if span.kind == SpanKind.SERVER + ] + ) + == 1 + ) + assert hooks[-1] == "/ws" + + +@pytest.mark.parametrize("order", ["before", "after"]) +@run_in_subprocess +def test_current_logfire(order): + import logfire + from fastapi import FastAPI, WebSocket + from fastapi.testclient import TestClient + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + from opentelemetry.trace import SpanKind + + legacy, native = InMemorySpanExporter(), InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(native)) + + def configure(): + logfire.configure( + send_to_logfire=False, + console=False, + metrics=False, + additional_span_processors=[SimpleSpanProcessor(legacy)], + ) + + if order == "before": + configure() + app = FastAPI(telemetry={"tracer_provider": provider}) + if order == "after": + configure() + + @app.get("/items/{value}") + def endpoint(value: int): + return value + + mapped = [] + + def mapper(request, attributes): + mapped.append(request.url.path) + return attributes + + logfire.instrument_fastapi(app, request_attributes_mapper=mapper, extra_spans=True) + assert TestClient(app).get("/items/5").json() == 5 + spans = legacy.get_finished_spans() + assert len([s for s in spans if s.kind == SpanKind.SERVER]) == 1, spans + assert any("arguments" in s.name for s in spans) + assert any( + s.attributes is not None + and s.attributes.get("code.function") + in (endpoint.__name__, endpoint.__qualname__) + for s in spans + ) + assert mapped == ["/items/5"] + assert not native.get_finished_spans() + + @app.websocket("/ws/{value}") + async def websocket_endpoint(*, websocket: WebSocket, value: int): + await websocket.accept() + await websocket.close() + + legacy.clear() + with TestClient(app).websocket_connect("/ws/5"): + pass + assert not native.get_finished_spans() + assert ( + len( + [ + span + for span in legacy.get_finished_spans() + if span.kind == SpanKind.SERVER + ] + ) + == 1 + ) + assert mapped[-1] == "/ws/5" + + +@pytest.mark.parametrize("sampling", [0.0, 1.0]) +@run_in_subprocess +def test_current_sentry(sampling): + import sentry_sdk + from fastapi import FastAPI, HTTPException, WebSocket + from fastapi.testclient import TestClient + from opentelemetry.sdk._logs import LoggerProvider + from opentelemetry.sdk._logs.export import ( + InMemoryLogRecordExporter, + SimpleLogRecordProcessor, + ) + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + from sentry_sdk.integrations.fastapi import FastApiIntegration + from sentry_sdk.integrations.starlette import StarletteIntegration + from sentry_sdk.transport import Transport + + items = [] + + class LocalTransport(Transport): + def capture_envelope(self, envelope): + items.extend(item.type for item in envelope.items) + + sentry_sdk.init( + dsn="https://public@example.invalid/1", + transport=LocalTransport, + default_integrations=False, + auto_enabling_integrations=False, + integrations=[StarletteIntegration(), FastApiIntegration()], + traces_sample_rate=sampling, + send_client_reports=False, + ) + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + logs = InMemoryLogRecordExporter() + logger = LoggerProvider() + logger.add_log_record_processor(SimpleLogRecordProcessor(logs)) + app = FastAPI(telemetry={"tracer_provider": provider, "logger_provider": logger}) + + @app.get("/items/{value}") + def endpoint(value: int): + if value == 2: + raise ValueError("test") + if value == 3: + raise HTTPException(503, "handled") + return value + + client = TestClient(app, raise_server_exceptions=False) + assert client.get("/items/1").status_code == 200 + assert client.get("/items/2").status_code == 500 + assert client.get("/items/3").status_code == 503 + sentry_sdk.flush() + assert items.count("event") == 2, items + (unhandled,) = logs.get_finished_logs() + assert isinstance(unhandled.log_record.exception, ValueError) + assert items.count("transaction") == (3 if sampling else 0), items + assert len(exporter.get_finished_spans()) == 10 + + @app.websocket("/ws") + async def websocket_endpoint(websocket: WebSocket): + await websocket.accept() + raise ValueError("websocket failed") + + with pytest.raises(ValueError, match="websocket failed"): + with client.websocket_connect("/ws"): + pass # pragma: no cover + sentry_sdk.flush() + assert items.count("event") == 3, items + assert items.count("transaction") == (4 if sampling else 0), items + assert len(exporter.get_finished_spans()) == 13 + assert len(logs.get_finished_logs()) == 2 + + +@run_in_subprocess +def test_api_only_and_no_implicit_sdk_import(): + import sys + from importlib.abc import MetaPathFinder + + class BlockSDK(MetaPathFinder): + def find_spec(self, fullname, path=None, target=None): + if fullname.startswith(("opentelemetry.sdk", "opentelemetry.exporter")): + raise AssertionError( + f"Unexpected optional import: {fullname}" + ) # pragma: no cover + + sys.meta_path.insert(0, BlockSDK()) + from fastapi import FastAPI, WebSocket + from fastapi.testclient import TestClient + + app = FastAPI() + + @app.websocket("/ws") + async def websocket_endpoint(websocket: WebSocket): + await websocket.accept() + await websocket.close() + + with TestClient(app) as client: + assert client.get("/").status_code == 404 + with client.websocket_connect("/ws"): + pass + assert not any(name.startswith("opentelemetry.sdk") for name in sys.modules) + + +def test_inactive_sentry_does_not_disable_native(telemetry): + import sentry_sdk + from fastapi import FastAPI + from fastapi.testclient import TestClient + + config, exporter, reader = telemetry + assert sentry_sdk.get_client().get_integration("fastapi") is None + assert TestClient(FastAPI(telemetry=config)).get("/").status_code == 404 + assert len(exporter.get_finished_spans()) == 1 + + +@run_in_subprocess +def test_logfire_global_provider_without_fastapi_instrumentor(): + import logfire + from fastapi import FastAPI + from fastapi.testclient import TestClient + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + exporter = InMemorySpanExporter() + logfire.configure( + send_to_logfire=False, + console=False, + metrics=False, + additional_span_processors=[SimpleSpanProcessor(exporter)], + ) + app = FastAPI() + + @app.get("/") + def endpoint(): + return "ok" + + assert TestClient(app).get("/").json() == "ok" + spans = exporter.get_finished_spans() + assert {span.name for span in spans} == { + "GET /", + "fastapi.dependencies", + "fastapi.endpoint", + "fastapi.serialization", + } + assert all( + span.instrumentation_scope is not None + and span.instrumentation_scope.name == "fastapi" + for span in spans + ) + + +def test_environment_isolation_removes_export_credentials(monkeypatch): + from .conftest import remove_export_environment + + for name in ["OTEL_EXPORTER_OTLP_HEADERS", "LOGFIRE_TOKEN", "SENTRY_DSN"]: + monkeypatch.setenv(name, "test-only") + remove_export_environment(monkeypatch) + assert not any( + name.startswith(("OTEL_", "LOGFIRE_", "SENTRY_")) for name in os.environ + ) diff --git a/tests/test_telemetry/test_native_integrations.py b/tests/test_telemetry/test_native_integrations.py new file mode 100644 index 0000000000..e54d0d859d --- /dev/null +++ b/tests/test_telemetry/test_native_integrations.py @@ -0,0 +1,316 @@ +"""Integration prototypes using standard OpenTelemetry SDK extension points.""" + +import pytest + +from ._subprocess import run_in_subprocess + + +@run_in_subprocess +def test_logfire_native_spans_and_exception_logs(): + import logfire + from fastapi import Depends, FastAPI, WebSocket, routing + from fastapi.telemetry import get_telemetry_data + from fastapi.testclient import TestClient + from opentelemetry import trace + from opentelemetry.sdk._logs import LogRecordProcessor + from opentelemetry.sdk._logs.export import ( + InMemoryLogRecordExporter, + SimpleLogRecordProcessor, + ) + from opentelemetry.sdk.trace import SpanProcessor + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + original = ( + routing.get_websocket_app, + routing.get_request_handler, + routing.run_endpoint_function, + routing.solve_dependencies, + ) + spans = InMemorySpanExporter() + logs = InMemoryLogRecordExporter() + dependency_value = object() + observed = [] + + class Arguments(SpanProcessor): + def on_start(self, span, parent_context=None): + if span.name != "fastapi.endpoint": + return + data = get_telemetry_data(parent_context) + assert data is not None + connection = data.request or data.websocket + assert connection is not None + assert data.values is not None + if connection.url.path in ("/error", "/ws/error"): + return + assert data.values["service"] is dependency_value + assert data.values["item_id"] == 42 + observed.append(connection.url.path) + span.set_attribute("test.item_id", data.values["item_id"]) + + class Validation(LogRecordProcessor): + def on_emit(self, log_record): + record = log_record.log_record + if record.event_name != "fastapi.validation.failed": + return + data = get_telemetry_data(record.context) + assert data is not None + connection = data.request or data.websocket + assert connection is not None + assert data.errors is not None + assert data.errors[0]["input"] == "invalid-item-id" + observed.append(connection.url.path) + record.attributes["test.error_types"] = tuple( + error["type"] for error in data.errors + ) + + def shutdown(self): + pass + + def force_flush(self, timeout_millis=30000): + return True + + logfire.configure( + send_to_logfire=False, + console=False, + metrics=False, + additional_span_processors=[Arguments(), SimpleSpanProcessor(spans)], + advanced=logfire.AdvancedOptions( + log_record_processors=[Validation(), SimpleLogRecordProcessor(logs)] + ), + ) + app = FastAPI() + contexts = [] + + def dependency(): + assert trace.get_current_span().get_span_context().is_valid + return dependency_value + + @app.get("/sync") + def sync_endpoint(*, item_id: int, service=Depends(dependency)): + return "ok" + + @app.get("/async") + async def async_endpoint(*, item_id: int, service=Depends(dependency)): + return "ok" + + @app.get("/error") + async def error_endpoint(): + contexts.append(trace.get_current_span().get_span_context()) + raise ValueError("native error") + + @app.websocket("/ws") + async def websocket_endpoint( + *, websocket: WebSocket, item_id: int, service=Depends(dependency) + ): + await websocket.accept() + await websocket.send_text("ok") + await websocket.close() + + @app.websocket("/ws/error") + async def websocket_error(websocket: WebSocket): + contexts.append(trace.get_current_span().get_span_context()) + await websocket.accept() + raise ValueError("native error") + + client = TestClient(app, raise_server_exceptions=False) + assert client.get("/sync?item_id=42").json() == "ok" + assert client.get("/async?item_id=42").json() == "ok" + assert client.get("/error").status_code == 500 + assert client.get("/async?item_id=invalid-item-id").status_code == 422 + from starlette.websockets import WebSocketDisconnect + + with client.websocket_connect("/ws?item_id=42") as websocket: + assert websocket.receive_text() == "ok" + with pytest.raises(ValueError, match="native error"): + with client.websocket_connect("/ws/error"): + pass # pragma: no cover + with pytest.raises(WebSocketDisconnect) as caught: + with client.websocket_connect("/ws?item_id=invalid-item-id"): + pass # pragma: no cover + assert caught.value.code == 1008 + logfire.force_flush() + finished = spans.get_finished_spans() + assert len([s for s in finished if s.kind == trace.SpanKind.SERVER]) == 7 + assert ( + len( + [ + s + for s in finished + if s.name == "fastapi.endpoint" + and s.attributes is not None + and s.attributes.get("logfire.span_type") != "pending_span" + ] + ) + == 5 + ) + data, validation, websocket_error_log, websocket_validation = ( + logs.get_finished_logs() + ) + assert websocket_error_log.log_record.exception is not None + assert websocket_error_log.log_record.trace_id == contexts[1].trace_id + assert websocket_validation.log_record.event_name == "fastapi.validation.failed" + assert websocket_validation.log_record.attributes is not None + assert "test.error_types" in websocket_validation.log_record.attributes + assert len([span for span in finished if span.name == "WS /ws"]) == 2 + assert validation.log_record.event_name == "fastapi.validation.failed" + assert validation.log_record.attributes is not None + assert "test.error_types" in validation.log_record.attributes + assert observed == ["/sync", "/async", "/async", "/ws", "/ws"] + arguments = [s for s in finished if s.attributes and "test.item_id" in s.attributes] + assert arguments + assert all(s.attributes and s.attributes["test.item_id"] == 42 for s in arguments) + assert get_telemetry_data() is None + + assert data.log_record.attributes is not None + assert data.log_record.attributes["exception.message"] == "native error" + assert data.log_record.trace_id == contexts[0].trace_id + assert original == ( + routing.get_websocket_app, + routing.get_request_handler, + routing.run_endpoint_function, + routing.solve_dependencies, + ) + + +@pytest.mark.parametrize("sampled", [True, False]) +@run_in_subprocess +def test_sentry_standard_log_processor(sampled): + import sentry_sdk + from fastapi import FastAPI, WebSocket, routing + from fastapi.telemetry import get_telemetry_data + from fastapi.testclient import TestClient + from opentelemetry import trace + from opentelemetry.sdk._logs import LoggerProvider, LogRecordProcessor + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + from opentelemetry.sdk.trace.sampling import ALWAYS_OFF, ALWAYS_ON + from sentry_sdk.integrations.otlp import OTLPIntegration + from sentry_sdk.transport import Transport + + original = ( + routing.get_websocket_app, + routing.get_request_handler, + routing.run_endpoint_function, + routing.solve_dependencies, + ) + items = [] + + class LocalTransport(Transport): + def capture_envelope(self, envelope): + items.extend( + item.payload.json for item in envelope.items if item.type == "event" + ) + + class SentryErrors(LogRecordProcessor): + def on_emit(self, log_record): + if ( + log_record.instrumentation_scope.name == "fastapi" + and log_record.log_record.exception is not None + ): + data = get_telemetry_data(log_record.log_record.context) + assert data is not None + connection = data.request or data.websocket + assert connection is not None + assert data.values is not None + + def enrich(event, hint): + event["request"] = { + "url": str(connection.url.replace(query=None)), + "query_string": str(connection.query_params), + "data": data.body, + } + if data.request is not None: + event["request"]["method"] = data.request.method + return event + + with sentry_sdk.new_scope() as scope: + scope.add_event_processor(enrich) + sentry_sdk.capture_exception(log_record.log_record.exception) + + def shutdown(self): + pass + + def force_flush(self, timeout_millis=30000): + return True + + sentry_sdk.init( + dsn="https://public@example.invalid/1", + transport=LocalTransport, + default_integrations=False, + auto_enabling_integrations=False, + integrations=[ + OTLPIntegration(setup_otlp_traces_exporter=False, setup_propagator=False) + ], + send_client_reports=False, + ) + logger = LoggerProvider() + logger.add_log_record_processor(SentryErrors()) + tracer = TracerProvider(sampler=ALWAYS_ON if sampled else ALWAYS_OFF) + exporter = InMemorySpanExporter() + tracer.add_span_processor(SimpleSpanProcessor(exporter)) + app = FastAPI(telemetry={"tracer_provider": tracer, "logger_provider": logger}) + contexts = [] + + @app.post("/sync") + def sync_endpoint(payload: dict): + contexts.append(trace.get_current_span().get_span_context()) + raise ValueError("sync error") + + @app.post("/async") + async def async_endpoint(payload: dict): + contexts.append(trace.get_current_span().get_span_context()) + raise ValueError("async error") + + @app.websocket("/ws") + async def websocket_endpoint(*, websocket: WebSocket, item_id: int): + contexts.append(trace.get_current_span().get_span_context()) + await websocket.accept() + raise ValueError("websocket error") + + client = TestClient(app, raise_server_exceptions=False) + assert client.post("/sync?item_id=1", json={"value": "sync"}).status_code == 500 + assert client.post("/async?item_id=2", json={"value": "async"}).status_code == 500 + with pytest.raises(ValueError, match="websocket error"): + with client.websocket_connect("/ws?item_id=3"): + pass # pragma: no cover + assert logger.force_flush() + sentry_sdk.flush() + assert len(items) == 3 + assert [item["request"] for item in items] == [ + { + "method": "POST", + "url": "http://testserver/sync", + "query_string": "item_id=1", + "data": {"value": "sync"}, + }, + { + "method": "POST", + "url": "http://testserver/async", + "query_string": "item_id=2", + "data": {"value": "async"}, + }, + { + "url": "ws://testserver/ws", + "query_string": "item_id=3", + "data": None, + }, + ] + assert [item["contexts"]["trace"]["trace_id"] for item in items] == [ + format(ctx.trace_id, "032x") for ctx in contexts + ] + assert all(item["exception"]["values"][0]["stacktrace"]["frames"] for item in items) + assert bool(exporter.get_finished_spans()) == sampled + assert original == ( + routing.get_websocket_app, + routing.get_request_handler, + routing.run_endpoint_function, + routing.solve_dependencies, + ) + logger.shutdown() + tracer.shutdown() diff --git a/tests/test_telemetry/test_operations.py b/tests/test_telemetry/test_operations.py new file mode 100644 index 0000000000..a9112ca22e --- /dev/null +++ b/tests/test_telemetry/test_operations.py @@ -0,0 +1,93 @@ +import threading +from typing import Annotated + +import pytest +from fastapi import Depends, FastAPI, HTTPException +from fastapi.testclient import TestClient +from opentelemetry import baggage, trace +from opentelemetry.sdk.trace import SpanProcessor +from opentelemetry.trace import SpanKind, StatusCode + + +@pytest.mark.parametrize("sync", [False, True]) +def test_native_operations_and_worker_context(telemetry, sync): + config, exporter, reader = telemetry + started = {} + executed = {} + + class Processor(SpanProcessor): + def on_start(self, span, parent_context=None): + started[span.name] = (threading.get_ident(), span.get_span_context()) + + config["tracer_provider"].add_span_processor(Processor()) + app = FastAPI(telemetry=config) + + def dependency(): + executed["dependency"] = trace.get_current_span().get_span_context() + return "private argument" + + def implementation(value: Annotated[str, Depends(dependency)]): + executed["endpoint"] = ( + threading.get_ident(), + trace.get_current_span().get_span_context(), + ) + assert value == "private argument" + return baggage.get_baggage("example") + + async def async_endpoint(value: Annotated[str, Depends(dependency)]): + return implementation(value) + + app.get("/")(implementation if sync else async_endpoint) + assert ( + TestClient(app).get("/", headers={"baggage": "example=value"}).json() == "value" + ) + assert started["fastapi.endpoint"] == executed["endpoint"] + assert started["fastapi.dependencies"][1] == executed["dependency"] + assert ( + started["fastapi.endpoint"][0] != started["fastapi.dependencies"][0] + ) == sync + spans = exporter.get_finished_spans() + server = next(s for s in spans if s.kind == SpanKind.SERVER) + children = [s for s in spans if s is not server] + assert {s.name for s in children} == { + "fastapi.dependencies", + "fastapi.endpoint", + "fastapi.serialization", + } + assert all(s.parent.span_id == server.context.span_id for s in children) + assert all(s.context.trace_id == server.context.trace_id for s in children) + assert all("code.function.name" in s.attributes for s in children) + assert "private argument" not in repr([s.attributes for s in spans]) + assert baggage.get_baggage("example") is None + assert not trace.get_current_span().get_span_context().is_valid + + +@pytest.mark.parametrize( + "failure", ["dependency", "endpoint", "serialization", "handled"] +) +def test_operation_errors(telemetry, failure): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + async def dependency(): + if failure == "dependency": + raise ValueError("dependency failure") + + @app.get("/", dependencies=[Depends(dependency)], response_model=int) + async def endpoint(): + if failure == "endpoint": + raise ValueError("endpoint failure") + if failure == "handled": + raise HTTPException(400) + return "not an integer" + + response = TestClient(app, raise_server_exceptions=False).get("/") + assert response.status_code == (400 if failure == "handled" else 500) + spans = exporter.get_finished_spans() + if failure == "handled": + assert all(s.status.status_code == StatusCode.UNSET for s in spans) + else: + name = "dependencies" if failure == "dependency" else failure + failed = next(s for s in spans if s.name == f"fastapi.{name}") + assert failed.status.status_code == StatusCode.ERROR + assert all(not s.events for s in spans) diff --git a/tests/test_telemetry/test_runtime.py b/tests/test_telemetry/test_runtime.py new file mode 100644 index 0000000000..5adccf0bfe --- /dev/null +++ b/tests/test_telemetry/test_runtime.py @@ -0,0 +1,613 @@ +import asyncio +import threading +from unittest.mock import Mock + +import pytest +from fastapi import FastAPI +from fastapi.exceptions import FastAPIError +from fastapi.telemetry import _runtime as runtime +from fastapi.testclient import TestClient + +from ._otlp import otlp_collector +from ._subprocess import run_in_subprocess + + +def test_otlp_collector_does_not_resolve_hostname(monkeypatch): + # Reverse DNS during HTTPServer binding caused macOS CI timeouts. + lookup = Mock() + monkeypatch.setattr("socket.getfqdn", lookup) + with otlp_collector(): + pass + lookup.assert_not_called() + + +@pytest.mark.parametrize("base_path", ["", "/collector/"]) +@run_in_subprocess +def test_real_otlp_export_and_repeated_lifespans(base_path): + import os + from contextlib import asynccontextmanager + + from fastapi import FastAPI + from fastapi.telemetry import _runtime + from fastapi.testclient import TestClient + from opentelemetry import _logs, trace + from opentelemetry.proto.collector.logs.v1.logs_service_pb2 import ( + ExportLogsServiceRequest, + ) + from opentelemetry.proto.collector.metrics.v1.metrics_service_pb2 import ( + ExportMetricsServiceRequest, + ) + from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ( + ExportTraceServiceRequest, + ) + from opentelemetry.proto.trace.v1.trace_pb2 import Span + + with otlp_collector() as (base, received): + prefix = base_path.rstrip("/") + os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = base + base_path + os.environ["OTEL_SERVICE_NAME"] = "native-test" + os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "x-test=value" + os.environ["OTEL_RESOURCE_ATTRIBUTES"] = "test.resource=example" + os.environ["OTEL_BSP_SCHEDULE_DELAY"] = "60000" + os.environ["OTEL_BLRP_SCHEDULE_DELAY"] = "60000" + os.environ["OTEL_METRIC_EXPORT_INTERVAL"] = "60000" + seen = [] + + @asynccontextmanager + async def lifespan(app): + seen.append(trace.get_tracer_provider()) + yield + + app = FastAPI(lifespan=lifespan) + + @app.get("/items/{value}") + def endpoint(value: int): + _logs.get_logger("test").emit(body="request processed") + return value + + try: + for _ in range(2): + with TestClient(app) as client: + assert client.get("/items/3").json() == 3 + assert len(_runtime._owned) == 3 + assert seen[0] is seen[1] + assert all(headers.get("x-test") == "value" for _, _, headers in received) + traces = [ + ExportTraceServiceRequest.FromString(body) + for path, body, _ in received + if path == prefix + "/v1/traces" + ] + spans = [ + span + for request in traces + for resource in request.resource_spans + for scope in resource.scope_spans + for span in scope.spans + ] + assert len(spans) == 8 + servers = [span for span in spans if span.kind == Span.SPAN_KIND_SERVER] + assert len(servers) == 2 + assert all(span.name == "GET /items/{value}" for span in servers) + metric_requests = [ + ExportMetricsServiceRequest.FromString(body) + for path, body, _ in received + if path == prefix + "/v1/metrics" + ] + assert metric_requests + resources = metric_requests[-1].resource_metrics + resource_attributes = { + a.key: a.value.string_value for a in resources[0].resource.attributes + } + assert resource_attributes["service.name"] == "native-test" + assert resource_attributes["test.resource"] == "example" + histograms = [ + metric.histogram + for resource in resources + for scope in resource.scope_metrics + for metric in scope.metrics + if metric.name == "http.server.request.duration" + ] + assert histograms[0].data_points[0].count == 2 + log_requests = [ + ExportLogsServiceRequest.FromString(body) + for path, body, _ in received + if path == prefix + "/v1/logs" + ] + records = [ + record + for request in log_requests + for resource in request.resource_logs + for scope in resource.scope_logs + for record in scope.log_records + ] + assert len(records) == 2 + assert all( + record.body.string_value == "request processed" for record in records + ) + finally: + _runtime._shutdown() + + +@pytest.mark.parametrize( + "env", + [ + { + "OTEL_TRACES_EXPORTER": "", + "OTEL_METRICS_EXPORTER": "", + "OTEL_LOGS_EXPORTER": "", + "OTEL_EXPORTER_OTLP_PROTOCOL": "", + "OTEL_EXPORTER_OTLP_TRACES_PROTOCOL": "", + "OTEL_EXPORTER_OTLP_METRICS_PROTOCOL": "", + "OTEL_EXPORTER_OTLP_LOGS_PROTOCOL": "", + }, + { + "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT": "", + "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT": "", + "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT": "", + }, + ], +) +def test_empty_environment_uses_defaults(monkeypatch, env): + monkeypatch.setenv( + "OTEL_EXPORTER_OTLP_ENDPOINT", "http://127.0.0.1:4318/collector/" + ) + for name, value in env.items(): + monkeypatch.setenv(name, value) + for signal in ("TRACES", "METRICS", "LOGS"): + assert runtime._export_endpoint(signal) == ( + f"http://127.0.0.1:4318/collector/v1/{signal.lower()}" + ) + + +@pytest.mark.parametrize("protocol", ["http/protobuf", "grpc"]) +def test_empty_signal_protocol_uses_general_protocol(monkeypatch, protocol): + monkeypatch.setenv( + "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "http://127.0.0.1:4318/v1/traces" + ) + monkeypatch.setenv("OTEL_EXPORTER_OTLP_PROTOCOL", protocol) + monkeypatch.setenv("OTEL_EXPORTER_OTLP_TRACES_PROTOCOL", "") + if protocol == "http/protobuf": + assert runtime._export_endpoint("TRACES") == "http://127.0.0.1:4318/v1/traces" + else: + with pytest.raises(FastAPIError, match="http/protobuf"): + runtime._export_endpoint("TRACES") + + +@pytest.mark.parametrize("signal", ["TRACES", "METRICS", "LOGS"]) +def test_signal_endpoint_overrides_general_endpoint(monkeypatch, signal): + endpoint = "http://127.0.0.1:4318/custom/" + monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "invalid") + monkeypatch.setenv(f"OTEL_EXPORTER_OTLP_{signal}_ENDPOINT", endpoint) + assert runtime._export_endpoint(signal) == endpoint + + +@pytest.mark.parametrize( + "env,message", + [ + ({"OTEL_EXPORTER_OTLP_ENDPOINT": "not-a-url"}, "absolute HTTP"), + ( + { + "OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:1", + "OTEL_EXPORTER_OTLP_PROTOCOL": "grpc", + }, + "http/protobuf", + ), + ( + { + "OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:1", + "OTEL_TRACES_EXPORTER": "console", + }, + "otlp or none", + ), + ], +) +@run_in_subprocess +def test_invalid_configuration_reports_startup_failure(env, message): + import os + + os.environ.update(env) + + import asyncio + + from fastapi import FastAPI + from fastapi.exceptions import FastAPIError + + messages = [] + + async def receive(): + return {"type": "lifespan.startup"} + + async def send(message): + messages.append(message) + + async def run(): + with pytest.raises(FastAPIError) as exc_info: + await FastAPI()({"type": "lifespan", "state": {}}, receive, send) + assert message in str(exc_info.value) + + asyncio.run(run()) + assert len(messages) == 1, messages + assert messages[0]["type"] == "lifespan.startup.failed" + + +@pytest.mark.parametrize( + "env", + [ + {}, + {"OTEL_SERVICE_NAME": "no-endpoint"}, + {"OTEL_TRACES_EXPORTER": "otlp", "OTEL_METRICS_EXPORTER": "otlp"}, + { + "OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:1", + "OTEL_TRACES_EXPORTER": "none", + "OTEL_METRICS_EXPORTER": "none", + "OTEL_LOGS_EXPORTER": "none", + }, + { + "OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:1", + "OTEL_SDK_DISABLED": "true", + }, + ], +) +@run_in_subprocess +def test_no_implicit_export(env): + import os + + os.environ.update(env) + + from fastapi import FastAPI + from fastapi.telemetry import _runtime + from fastapi.testclient import TestClient + + with TestClient(FastAPI()) as client: + assert client.get("/").status_code == 404 + assert _runtime._owned == [] + + +@run_in_subprocess +def test_missing_sdk_diagnostic(): + import os + + os.environ.update( + {"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT": "http://127.0.0.1:1/metrics"} + ) + + import sys + from importlib.abc import MetaPathFinder + + class BlockSDK(MetaPathFinder): + def find_spec(self, fullname, path=None, target=None): + if fullname.startswith(("opentelemetry.sdk", "opentelemetry.exporter")): + raise ImportError("SDK absent") + + sys.meta_path.insert(0, BlockSDK()) + from fastapi import FastAPI + from fastapi.exceptions import FastAPIError + from fastapi.testclient import TestClient + + with pytest.raises(FastAPIError) as exc_info: + with TestClient(FastAPI()): + pass # pragma: no cover + assert "fastapi[opentelemetry]" in str(exc_info.value) + + +@run_in_subprocess +def test_external_globals_are_unchanged_when_auto_configuration_is_disabled(): + import os + + os.environ.update({"OTEL_EXPORTER_OTLP_ENDPOINT": "http://127.0.0.1:1"}) + + from fastapi import FastAPI + from fastapi.telemetry import _runtime + from fastapi.testclient import TestClient + from opentelemetry import _logs, metrics, trace + from opentelemetry.sdk._logs import LoggerProvider + from opentelemetry.sdk.metrics import MeterProvider + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + exporter = InMemorySpanExporter() + tp = TracerProvider() + tp.add_span_processor(SimpleSpanProcessor(exporter)) + mp = MeterProvider() + lp = LoggerProvider() + _logs.set_logger_provider(lp) + trace.set_tracer_provider(tp) + metrics.set_meter_provider(mp) + for _ in range(2): + with TestClient(FastAPI(telemetry={"auto_configure": False})) as client: + client.get("/") + assert trace.get_tracer_provider() is tp + assert metrics.get_meter_provider() is mp + assert _logs.get_logger_provider() is lp + assert not _runtime._owned + assert len(exporter.get_finished_spans()) == 2 + + +def test_auto_configuration_opt_out(monkeypatch): + monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "invalid") + with TestClient(FastAPI(telemetry={"auto_configure": False})) as client: + assert client.get("/").status_code == 404 + + +def test_owned_flush_failure_does_not_break_shutdown(monkeypatch, caplog): + calls = [] + + class FailingProvider: + def force_flush(self): + raise RuntimeError("flush failed") + + def shutdown(self): + raise RuntimeError("shutdown failed") + + class Provider: + def force_flush(self): + calls.append("flush") + + def shutdown(self): + calls.append("shutdown") + + monkeypatch.setattr(runtime, "_owned", [FailingProvider(), Provider()]) + with TestClient(FastAPI()) as client: + assert client.get("/").status_code == 404 + runtime._shutdown() + runtime._shutdown() + assert calls == ["flush", "shutdown"] + assert caplog.text.count("FastAPI telemetry cleanup failed") == 2 + + +@pytest.mark.parametrize( + "message_type", + [ + "lifespan.startup.failed", + "lifespan.shutdown.failed", + "lifespan.shutdown.complete", + ], +) +def test_lifespan_flush_runs_off_event_loop(monkeypatch, message_type): + calls = [] + flush_threads = [] + event_loop_thread = threading.get_ident() + + class Provider: + def force_flush(self): + flush_threads.append(threading.get_ident()) + calls.append("flush") + + def shutdown(self): + calls.append("shutdown") + + monkeypatch.setattr(runtime, "_owned", [Provider()]) + + async def app(scope, receive, send): + assert await receive() == {"type": "lifespan.startup"} + await send({"type": message_type}) + + async def receive(): + return {"type": "lifespan.startup"} + + async def send(message): + calls.append(message["type"]) + + asyncio.run( + runtime.lifespan( + config=FastAPI()._telemetry, app=app, scope={}, receive=receive, send=send + ) + ) + assert calls == ["flush", message_type] + assert len(flush_threads) == 1 + assert flush_threads[0] != event_loop_thread + runtime._shutdown() + runtime._shutdown() + assert calls == ["flush", message_type, "shutdown"] + + +def test_registration_provider_prefers_public_metric_reader(monkeypatch): + from logfire._internal.metrics import ProxyMeterProvider + from opentelemetry.metrics import NoOpMeterProvider + + provider = NoOpMeterProvider() + proxy = ProxyMeterProvider(provider=provider) + assert runtime._registration_provider(proxy) is provider + + monkeypatch.setattr(proxy, "add_metric_reader", lambda reader: None, raising=False) + assert runtime._registration_provider(proxy) is proxy + + +@pytest.mark.parametrize("wrapped_meter", [False, True]) +def test_concurrent_provider_owner_wins(monkeypatch, wrapped_meter): + from opentelemetry import _logs, metrics, trace + from opentelemetry.sdk import _logs as sdk_logs + from opentelemetry.sdk import metrics as sdk_metrics + from opentelemetry.sdk import trace as sdk_trace + + stopped = [] + winners = [] + for name, api, sdk, cls_name in [ + ("tracer", trace, sdk_trace, "TracerProvider"), + ("meter", metrics, sdk_metrics, "MeterProvider"), + ("logger", _logs, sdk_logs, "LoggerProvider"), + ]: + original = getattr(sdk, cls_name) + winner = original(shutdown_on_exit=False) + winners.append(winner) + + class Provider(original): + def shutdown(self, _name=name): + stopped.append(_name) + super().shutdown() + + if wrapped_meter and name == "meter": + from logfire._internal.metrics import ProxyMeterProvider + + winner = ProxyMeterProvider(winner) + current = [getattr(api, f"get_{name}_provider")()] + monkeypatch.setattr(sdk, cls_name, Provider) + monkeypatch.setattr(api, f"get_{name}_provider", lambda c=current: c[0]) + monkeypatch.setattr( + api, + f"set_{name}_provider", + lambda provider, c=current, w=winner: c.__setitem__(0, w), + ) + monkeypatch.setattr(runtime, "_owned", []) + monkeypatch.setattr(runtime, "_configured", []) + monkeypatch.setenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://127.0.0.1:1") + runtime._configure_from_environment(FastAPI()._telemetry) + assert stopped == ["tracer", "meter", "logger"] + assert [provider for _, provider in runtime._configured] == winners + assert len(runtime._owned) == 3 + assert all(component not in winners for component in runtime._owned) + runtime._shutdown() + for winner in winners: + winner.shutdown() + + +@pytest.mark.skipif( + not hasattr(__import__("os"), "fork"), reason="POSIX worker lifecycle" +) +@run_in_subprocess +def test_environment_export_initializes_after_fork(): + import os + import threading + from http.server import BaseHTTPRequestHandler + + from fastapi import FastAPI + from fastapi.telemetry import _runtime + from fastapi.testclient import TestClient + + from tests.test_telemetry._otlp import _CollectorServer + + received = [] + + class Handler(BaseHTTPRequestHandler): + def do_POST(self): + self.rfile.read(int(self.headers["Content-Length"])) + received.append(self.path) + self.send_response(200) + self.end_headers() + + def log_message(self, format, *args): + pass + + server = _CollectorServer(("127.0.0.1", 0), Handler) + os.environ["OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"] = ( + f"http://127.0.0.1:{server.server_port}/traces" + ) + # Importing and constructing before fork must not create providers. + app = FastAPI() + assert not _runtime._owned + pid = os.fork() + if pid == 0: + try: + with TestClient(app) as client: + assert client.get("/").status_code == 404 + assert len(_runtime._owned) == 1 + _runtime._shutdown() + except BaseException: # pragma: no cover + os._exit(1) + os._exit(0) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + _, status = os.waitpid(pid, 0) + assert status == 0, status + assert received == ["/traces"], received + assert not _runtime._owned + server.shutdown() + server.server_close() + + +@run_in_subprocess +def test_real_otlp_exception_export_without_traces_or_metrics(): + import os + + from fastapi import FastAPI + from fastapi.telemetry import _runtime + from fastapi.testclient import TestClient + from opentelemetry.proto.collector.logs.v1.logs_service_pb2 import ( + ExportLogsServiceRequest, + ) + + with otlp_collector() as (base, received): + os.environ["OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"] = base + "/logs" + os.environ["OTEL_SERVICE_NAME"] = "errors-only" + app = FastAPI() + + @app.get("/items/{item_id}") + def endpoint(item_id: int): + raise ValueError("exported exception") + + try: + for _ in range(2): + with TestClient(app, raise_server_exceptions=False) as client: + assert client.get("/items/1").status_code == 500 + assert len(_runtime._owned) == 1 + assert all(path == "/logs" for path, _, _ in received) + requests = [ + ExportLogsServiceRequest.FromString(body) for _, body, _ in received + ] + records = [ + record + for request in requests + for resource in request.resource_logs + for scope in resource.scope_logs + for record in scope.log_records + ] + assert len(records) == 2 + for record in records: + attributes = {a.key: a.value.string_value for a in record.attributes} + assert attributes["exception.type"] == "ValueError" + assert attributes["exception.message"] == "exported exception" + assert attributes["http.route"] == "/items/{item_id}" + assert "in endpoint" in attributes["exception.stacktrace"] + assert ( + "ValueError: exported exception" + in attributes["exception.stacktrace"] + ) + finally: + _runtime._shutdown() + + +@pytest.mark.parametrize("name", ["tracer", "meter", "logger"]) +def test_unsupported_provider_reports_configuration_error(monkeypatch, name): + signal = {"tracer": "TRACES", "meter": "METRICS", "logger": "LOGS"}[name] + monkeypatch.setenv(f"OTEL_EXPORTER_OTLP_{signal}_ENDPOINT", "http://127.0.0.1:1") + app = FastAPI(telemetry={f"{name}_provider": object()}) + with pytest.raises(FastAPIError, match="does not support.*auto_configure"): + with TestClient(app): + pass # pragma: no cover + + +@pytest.mark.parametrize("error_type", [ValueError, AttributeError]) +def test_failed_registration_closes_new_exporter(monkeypatch, error_type): + from opentelemetry.exporter.otlp.proto.http import trace_exporter + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + stopped = [] + + class Exporter(InMemorySpanExporter): + def __init__(self, *, endpoint): + super().__init__() + + def shutdown(self): + stopped.append(True) + super().shutdown() + + provider = TracerProvider(shutdown_on_exit=False) + + def fail(component): + raise error_type("registration failed") + + monkeypatch.setattr(provider, "add_span_processor", fail) + monkeypatch.setattr(trace_exporter, "OTLPSpanExporter", Exporter) + monkeypatch.setenv("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "http://127.0.0.1:1") + app = FastAPI(telemetry={"tracer_provider": provider}) + with pytest.raises(error_type, match="registration failed"): + with TestClient(app): + pass # pragma: no cover + assert stopped == [True] + provider.shutdown() diff --git a/tests/test_telemetry/test_semantic_conventions.py b/tests/test_telemetry/test_semantic_conventions.py new file mode 100644 index 0000000000..a687e4c725 --- /dev/null +++ b/tests/test_telemetry/test_semantic_conventions.py @@ -0,0 +1,163 @@ +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient + +from .conftest import metric_points, server_spans + + +@pytest.mark.parametrize( + "scheme,host,expected", + [ + ( + "http", + "public.example", + {"server.address": "public.example", "server.port": 80}, + ), + ( + "https", + "public.example", + {"server.address": "public.example", "server.port": 443}, + ), + ( + "http", + "public.example:8443", + {"server.address": "public.example", "server.port": 8443}, + ), + ( + "http", + "[2001:db8::1]:8443", + {"server.address": "2001:db8::1", "server.port": 8443}, + ), + ("http", "[2001:db8::1]", {"server.address": "2001:db8::1", "server.port": 80}), + ("http", "public.example:invalid", {}), + ("http", "[invalid", {}), + ("http", "public.example:99999", {}), + ("http", "user:password@public.example", {}), + ("http", "public.example/path", {}), + ("http", "", {}), + ("custom", "public.example", {"server.address": "public.example"}), + ], +) +def test_request_authority_is_only_on_spans(telemetry, scheme, host, expected): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/") + async def endpoint(): + return "ok" + + async def asgi(scope, receive, send): + scope["scheme"] = scheme + await app(scope, receive, send) + + client = TestClient(asgi, base_url="http://listener:8000") + assert client.get("/", headers={"host": host}).json() == "ok" + (span,) = server_spans(exporter) + assert { + key: value + for key, value in span.attributes.items() + if key.startswith("server.") + } == expected + for name in ("http.server.request.duration", "http.server.active_requests"): + (point,) = metric_points(reader=reader, name=name) + assert not any(key.startswith("server.") for key in point.attributes) + + +def test_server_fallback_without_host_header(telemetry): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + + async def asgi(scope, receive, send): + scope["headers"] = [ + (name, value) for name, value in scope["headers"] if name != b"host" + ] + await app(scope, receive, send) + + assert TestClient(asgi, base_url="http://listener:8000").get("/").status_code == 404 + (span,) = server_spans(exporter) + assert span.attributes["server.address"] == "listener" + assert span.attributes["server.port"] == 8000 + + +@pytest.mark.parametrize( + "query,expected", + [ + ("", None), + ( + "?tag=one&tag=two&empty=&flag", + "tag=one&tag=two&empty=&flag=", + ), + ( + "?X-Amz-Signature=secret&X-Amz-Credential=secret&X-Amz-Security-Token=secret&sig=secret&X-Goog-Signature=secret", + "X-Amz-Signature=REDACTED&X-Amz-Credential=REDACTED&X-Amz-Security-Token=REDACTED&sig=REDACTED&X-Goog-Signature=REDACTED", + ), + ( + "?%70age=2&q=fastapi%20tutorial&sort%20by=name", + "page=2&q=fastapi+tutorial&sort+by=name", + ), + ( + "?%73ig=secret&q=hello%26world", + "sig=REDACTED&q=hello%26world", + ), + ( + "?x-amz-signature=keep&x-amz-credential=keep&x-amz-security-token=keep&SIG=keep&x-goog-signature=keep", + "x-amz-signature=keep&x-amz-credential=keep&x-amz-security-token=keep&SIG=keep&x-goog-signature=keep", + ), + ( + "?sig=one&sig=two&sig=&q=hello+world", + "sig=REDACTED&sig=REDACTED&sig=REDACTED&q=hello+world", + ), + ], +) +def test_url_attributes_do_not_split_metrics(telemetry, query, expected): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.get("/items/{item_id}") + async def endpoint(item_id: int): + return item_id + + client = TestClient(app) + for item_id in (1, 2): + assert ( + client.get( + f"/items/{item_id}{query}", headers={"host": f"host-{item_id}.example"} + ).json() + == item_id + ) + first, second = server_spans(exporter) + assert first.attributes["url.path"] == "/items/1" + assert second.attributes["url.path"] == "/items/2" + assert first.attributes.get("url.query") == expected + assert second.attributes.get("url.query") == expected + (point,) = metric_points(reader=reader) + assert point.count == 2 + assert "url.path" not in point.attributes + assert "url.query" not in point.attributes + assert point.attributes["http.route"] == "/items/{item_id}" + + +@pytest.mark.parametrize("metrics", [None, False, True]) +def test_active_requests_follow_metrics_setting(telemetry, metrics): + config, _, reader = telemetry + if metrics is not None: + config["metrics"] = metrics + app = FastAPI(telemetry=config) + seen = [] + + @app.get("/") + async def endpoint(): + seen.extend( + point.value + for point in metric_points( + reader=reader, name="http.server.active_requests" + ) + ) + return "ok" + + assert TestClient(app).get("/").json() == "ok" + active = metrics is not False + assert seen == ([1] if active else []) + points = metric_points(reader=reader, name="http.server.active_requests") + assert [point.value for point in points] == ([0] if active else []) + assert bool(metric_points(reader=reader)) == active diff --git a/tests/test_telemetry/test_websockets.py b/tests/test_telemetry/test_websockets.py new file mode 100644 index 0000000000..8b92b7e085 --- /dev/null +++ b/tests/test_telemetry/test_websockets.py @@ -0,0 +1,292 @@ +import pytest +from fastapi import APIRouter, Depends, FastAPI, WebSocket, WebSocketException +from fastapi.telemetry import get_telemetry_data +from fastapi.testclient import TestClient +from opentelemetry import context +from opentelemetry._logs import SeverityNumber +from opentelemetry.sdk._logs import LogRecordProcessor +from opentelemetry.sdk.trace.sampling import ALWAYS_OFF +from opentelemetry.trace import StatusCode +from starlette.websockets import WebSocketDisconnect + +from .conftest import server_spans + + +@pytest.mark.parametrize("operation_spans", [False, True]) +def test_connection_context_route_and_data(telemetry, operation_spans): + config, exporter, reader = telemetry + config["operation_spans"] = operation_spans + app = FastAPI(telemetry=config) + child = FastAPI(telemetry=config) + router = APIRouter() + service = object() + saved = [] + + def dependency(websocket: WebSocket): + data = get_telemetry_data() + assert data is not None + assert data.websocket is websocket + assert data.request is None + assert data.body is None + assert data.values is None + return service + + @router.websocket("/rooms/{room}") + async def endpoint(*, websocket: WebSocket, room: int, value=Depends(dependency)): + data = get_telemetry_data() + assert data is not None + assert data.values is not None + assert data.values["value"] is service + assert data.values["room"] == room + assert data.errors == [] + saved.append(context.get_current()) + await websocket.accept() + for _ in range(2): + await websocket.send_text(await websocket.receive_text()) + await websocket.close() + + child.include_router(router, prefix="/api") + app.mount("/tenants/{tenant}", child) + with TestClient(app).websocket_connect( + "/tenants/acme/api/rooms/42", + headers={"traceparent": "00-" + "1" * 32 + "-" + "2" * 16 + "-01"}, + ) as websocket: + websocket.send_text("private payload") + assert websocket.receive_text() == "private payload" + assert not server_spans(exporter) + websocket.send_text("second message") + assert websocket.receive_text() == "second message" + (span,) = server_spans(exporter) + assert span.name == "WS /tenants/{tenant}/api/rooms/{room}" + assert span.parent.span_id == int("2" * 16, 16) + assert span.context.trace_id == int("1" * 32, 16) + assert span.attributes["network.protocol.name"] == "websocket" + assert span.attributes["url.scheme"] == "ws" + assert span.status.status_code == StatusCode.UNSET + assert not any(key.startswith("http.request.") for key in span.attributes) + assert "http.response.status_code" not in span.attributes + assert "network.protocol.version" not in span.attributes + assert reader.get_metrics_data() is None + assert len(exporter.get_finished_spans()) == (3 if operation_spans else 1) + assert "private payload" not in repr(exporter.get_finished_spans()) + assert all(get_telemetry_data(saved_context) is None for saved_context in saved) + assert get_telemetry_data() is None + + +@pytest.mark.parametrize("code", [1000, 1001, 1006]) +def test_client_disconnect(telemetry, logs, code): + config, exporter, reader = telemetry + app = FastAPI(telemetry=config) + + @app.websocket("/ws") + async def endpoint(websocket: WebSocket): + await websocket.accept() + await websocket.receive_text() + + with pytest.raises(WebSocketDisconnect) as caught: + with TestClient(app).websocket_connect("/ws") as websocket: + websocket.close(code=code) + assert caught.value.code == code + (server,) = server_spans(exporter) + assert server.status.status_code == ( + StatusCode.ERROR if code == 1006 else StatusCode.UNSET + ) + assert len(logs.get_finished_logs()) == int(code == 1006) + for span in exporter.get_finished_spans(): + assert not span.events + assert span.status.status_code == ( + StatusCode.ERROR + if code == 1006 and span.name != "fastapi.dependencies" + else StatusCode.UNSET + ) + assert reader.get_metrics_data() is None + + +@pytest.mark.parametrize("mode", ["sampled", "unsampled", "logs_only", "logs_disabled"]) +@pytest.mark.parametrize("stage", ["dependency", "endpoint", "cleanup"]) +def test_unexpected_exception(telemetry, logs, mode, stage): + config, exporter, reader = telemetry + if mode == "unsampled": + config["tracer_provider"].sampler = ALWAYS_OFF + elif mode == "logs_only": + config["tracing"] = False + elif mode == "logs_disabled": + config["logs"] = False + app = FastAPI(telemetry=config) + failure = ValueError("websocket failed") + saved = [] + + async def dependency(): + saved.append(context.get_current()) + if stage == "dependency": + raise failure + yield + if stage == "cleanup": + raise failure + + @app.websocket("/ws", dependencies=[Depends(dependency)]) + async def endpoint(websocket: WebSocket): + await websocket.accept() + if stage == "endpoint": + raise failure + await websocket.close() + + with pytest.raises(ValueError, match="websocket failed"): + with TestClient(app).websocket_connect("/ws"): + pass # pragma: no cover + records = logs.get_finished_logs() + assert len(records) == int(mode != "logs_disabled") + if records: + record = records[0].log_record + assert record.event_name == "fastapi.websocket.exception" + assert record.timestamp is not None + assert record.timestamp <= record.observed_timestamp + assert record.exception is failure + assert record.severity_number == SeverityNumber.ERROR + assert record.body == "Unhandled exception in FastAPI WebSocket connection" + assert record.attributes["http.route"] == "/ws" + assert bool(record.trace_id) == (mode != "logs_only") + assert get_telemetry_data(record.context) is None + if mode in ("sampled", "logs_disabled"): + (server,) = server_spans(exporter) + assert server.status.status_code == StatusCode.ERROR + assert server.attributes["error.type"] == "ValueError" + if records: + assert records[0].log_record.span_id == server.context.span_id + else: + assert not exporter.get_finished_spans() + assert reader.get_metrics_data() is None + assert all(get_telemetry_data(saved_context) is None for saved_context in saved) + + +@pytest.mark.parametrize("tracing", [False, True]) +def test_validation_data(telemetry, logs, tracing): + config, exporter, reader = telemetry + config["tracing"] = tracing + app = FastAPI(telemetry=config) + observed = [] + + class Observe(LogRecordProcessor): + def on_emit(self, log_record): + data = get_telemetry_data(log_record.log_record.context) + assert data is not None + assert data.websocket is not None + assert data.request is None + assert data.body is None + assert data.errors is not None + assert data.errors[0]["input"] == "private-invalid-input" + observed.append(log_record.log_record.context) + + def shutdown(self): + pass + + def force_flush(self, timeout_millis=30000): + return True + + config["logger_provider"].add_log_record_processor(Observe()) + + @app.websocket("/ws/{value}") + async def endpoint(*, websocket: WebSocket, value: int): + pytest.fail( + "Validation should fail before the endpoint runs" + ) # pragma: no cover + + with pytest.raises(WebSocketDisconnect) as caught: + with TestClient(app).websocket_connect("/ws/private-invalid-input"): + pass # pragma: no cover + assert caught.value.code == 1008 + assert config["logger_provider"].force_flush() + (record,) = logs.get_finished_logs() + assert record.log_record.event_name == "fastapi.validation.failed" + assert record.log_record.severity_number == SeverityNumber.WARN + assert record.log_record.attributes == { + "http.route": "/ws/{value}", + "fastapi.validation.error_count": 1, + } + assert record.log_record.exception is None + assert "private-invalid-input" not in repr(record.log_record.attributes) + assert len(observed) == 1 + assert get_telemetry_data(observed[0]) is None + if tracing: + assert all( + span.status.status_code == StatusCode.UNSET + for span in exporter.get_finished_spans() + ) + assert reader.get_metrics_data() is None + + +@pytest.mark.parametrize("excluded", [False, True]) +def test_disabled_and_excluded_connections(telemetry, logs, excluded): + config, exporter, reader = telemetry + if excluded: + config["exclude"] = lambda scope: scope["type"] == "websocket" + else: + config.update(tracing=False, logs=False) + app = FastAPI(telemetry=config) + child = FastAPI(telemetry=config) + app.mount("/child", child) + + @child.websocket("/ws") + async def endpoint(websocket: WebSocket): + assert get_telemetry_data() is None + await websocket.accept() + await websocket.close() + + with TestClient(app).websocket_connect("/child/ws"): + pass + assert not exporter.get_finished_spans() + assert not logs.get_finished_logs() + assert reader.get_metrics_data() is None + + +def test_handled_websocket_exception(telemetry, logs): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + + @app.websocket("/ws") + async def endpoint(websocket: WebSocket): + raise WebSocketException(code=1008) + + with pytest.raises(WebSocketDisconnect) as caught: + with TestClient(app).websocket_connect("/ws"): + pass # pragma: no cover + assert caught.value.code == 1008 + assert len(server_spans(exporter)) == 1 + assert not logs.get_finished_logs() + assert all( + span.status.status_code == StatusCode.UNSET + for span in exporter.get_finished_spans() + ) + + +def test_concurrent_connections_keep_local_data_separate(telemetry): + config, exporter, _ = telemetry + app = FastAPI(telemetry=config) + saved = [] + observed = [] + + @app.websocket("/ws/{name}") + async def endpoint(*, websocket: WebSocket, name: str): + data = get_telemetry_data() + assert data is not None + await websocket.accept() + await websocket.receive_text() + assert get_telemetry_data() is data + assert data.values is not None + assert data.values["name"] == name + assert data.websocket is websocket + saved.append(context.get_current()) + observed.append(data) + await websocket.send_text(name) + await websocket.close() + + with TestClient(app) as client: + with client.websocket_connect("/ws/first") as first: + with client.websocket_connect("/ws/second") as second: + second.send_text("go") + assert second.receive_text() == "second" + first.send_text("go") + assert first.receive_text() == "first" + assert observed[0] is not observed[1] + assert len(server_spans(exporter)) == 2 + assert all(get_telemetry_data(saved_context) is None for saved_context in saved) diff --git a/tests/test_tutorial/test_opentelemetry/__init__.py b/tests/test_tutorial/test_opentelemetry/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/test_tutorial/test_opentelemetry/conftest.py b/tests/test_tutorial/test_opentelemetry/conftest.py new file mode 100644 index 0000000000..5cbd06ff3c --- /dev/null +++ b/tests/test_tutorial/test_opentelemetry/conftest.py @@ -0,0 +1,8 @@ +import pytest + +from tests.test_telemetry.conftest import remove_export_environment + + +@pytest.fixture(autouse=True) +def clean_environment(monkeypatch): + remove_export_environment(monkeypatch) diff --git a/tests/test_tutorial/test_opentelemetry/test_tutorial001.py b/tests/test_tutorial/test_opentelemetry/test_tutorial001.py new file mode 100644 index 0000000000..724220f5e3 --- /dev/null +++ b/tests/test_tutorial/test_opentelemetry/test_tutorial001.py @@ -0,0 +1,8 @@ +from fastapi.testclient import TestClient + + +def test_default_example(): + from docs_src.opentelemetry.tutorial001_py310 import app + + with TestClient(app) as client: + assert client.get("/items/1").json() == {"item_id": 1} diff --git a/tests/test_tutorial/test_opentelemetry/test_tutorial002.py b/tests/test_tutorial/test_opentelemetry/test_tutorial002.py new file mode 100644 index 0000000000..6bcbf62559 --- /dev/null +++ b/tests/test_tutorial/test_opentelemetry/test_tutorial002.py @@ -0,0 +1,28 @@ +import json +from io import StringIO + +from fastapi.testclient import TestClient + + +def test_console_provider_example(monkeypatch): + from opentelemetry.sdk.trace import export + + output = StringIO() + exporter = export.ConsoleSpanExporter( + out=output, formatter=lambda span: span.to_json(indent=None) + "\n" + ) + monkeypatch.setattr(export, "ConsoleSpanExporter", lambda: exporter) + from docs_src.opentelemetry.tutorial002_py310 import app, tracer_provider + + try: + with TestClient(app) as client: + assert client.get("/items/2").json() == {"item_id": 2} + assert tracer_provider.force_flush() + spans = [json.loads(line) for line in output.getvalue().splitlines()] + assert len(spans) == 4 + span = next(span for span in spans if span["kind"] == "SpanKind.SERVER") + assert span["name"] == "GET /items/{item_id}" + assert span["kind"] == "SpanKind.SERVER" + assert span["attributes"]["http.route"] == "/items/{item_id}" + finally: + tracer_provider.shutdown() diff --git a/tests/test_tutorial/test_opentelemetry/test_tutorial003.py b/tests/test_tutorial/test_opentelemetry/test_tutorial003.py new file mode 100644 index 0000000000..dd161fa27b --- /dev/null +++ b/tests/test_tutorial/test_opentelemetry/test_tutorial003.py @@ -0,0 +1,22 @@ +from fastapi.testclient import TestClient +from opentelemetry import trace +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +def test_disable_operation_spans_example(monkeypatch): + exporter = InMemorySpanExporter() + provider = TracerProvider(shutdown_on_exit=False) + provider.add_span_processor(SimpleSpanProcessor(exporter)) + monkeypatch.setattr(trace, "get_tracer_provider", lambda: provider) + from docs_src.opentelemetry.tutorial003_py310 import app + + try: + with TestClient(app) as client: + assert client.get("/items/3").json() == {"item_id": 3} + (span,) = exporter.get_finished_spans() + assert span.kind == trace.SpanKind.SERVER + assert span.name == "GET /items/{item_id}" + finally: + provider.shutdown() diff --git a/uv.lock b/uv.lock index 2feffec443..75792782aa 100644 --- a/uv.lock +++ b/uv.lock @@ -144,6 +144,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ee/82/82745642d3c46e7cea25e1885b014b033f4693346ce46b7f47483cf5d448/argon2_cffi_bindings-25.1.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:da0c79c23a63723aa5d782250fbf51b768abca630285262fb5144ba5ae01e520", size = 29187, upload-time = "2025-07-30T10:02:03.674Z" }, ] +[[package]] +name = "asgiref" +version = "3.12.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e6/26/3b59f2bdae5f640389becb1f673cded775287f5fc4f816309d9ca9a3f93d/asgiref-3.12.1.tar.gz", hash = "sha256:59dcb51c272ad209d59bed5708a64a333083e86017d7fcdd67498eeab7784340", size = 42378, upload-time = "2026-07-14T09:56:18.087Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c0/1b/54f4ad77cd8a584fa70746c47df988e002cf1ee1eba43364d46f87803647/asgiref-3.12.1-py3-none-any.whl", hash = "sha256:fe386d1c2bff7259ea95929266d12a8cf9a8b5a1c2598402967d8792e7a7c094", size = 25478, upload-time = "2026-07-14T09:56:16.926Z" }, +] + [[package]] name = "ast-serialize" version = "0.6.0" @@ -851,6 +863,7 @@ name = "fastapi" source = { editable = "." } dependencies = [ { name = "annotated-doc" }, + { name = "opentelemetry-api" }, { name = "pydantic" }, { name = "starlette" }, { name = "typing-extensions" }, @@ -864,18 +877,26 @@ all = [ { name = "httpx" }, { name = "itsdangerous" }, { name = "jinja2" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-sdk" }, { name = "pydantic-extra-types" }, { name = "pydantic-settings" }, { name = "python-multipart" }, { name = "pyyaml" }, { name = "uvicorn", extra = ["standard"] }, ] +opentelemetry = [ + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-sdk" }, +] standard = [ { name = "email-validator" }, { name = "fastapi-cli", extra = ["standard"] }, { name = "fastar" }, { name = "httpx" }, { name = "jinja2" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-sdk" }, { name = "pydantic-extra-types" }, { name = "pydantic-settings" }, { name = "python-multipart" }, @@ -886,6 +907,8 @@ standard-no-fastapi-cloud-cli = [ { name = "fastapi-cli", extra = ["standard-no-fastapi-cloud-cli"] }, { name = "httpx" }, { name = "jinja2" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-sdk" }, { name = "pydantic-extra-types" }, { name = "pydantic-settings" }, { name = "python-multipart" }, @@ -908,10 +931,14 @@ dev = [ { name = "httpx2" }, { name = "inline-snapshot" }, { name = "jieba" }, + { name = "logfire" }, { name = "markdown-include-variants" }, { name = "mdx-include" }, { name = "mkdocstrings", extra = ["python"] }, { name = "mypy" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-instrumentation-fastapi" }, + { name = "opentelemetry-sdk" }, { name = "pillow" }, { name = "playwright" }, { name = "prek" }, @@ -928,6 +955,7 @@ dev = [ { name = "python-slugify" }, { name = "pyyaml" }, { name = "ruff" }, + { name = "sentry-sdk" }, { name = "sqlmodel" }, { name = "strawberry-graphql" }, { name = "ty" }, @@ -975,7 +1003,11 @@ tests = [ { name = "httpx" }, { name = "httpx2" }, { name = "inline-snapshot" }, + { name = "logfire" }, { name = "mypy" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-instrumentation-fastapi" }, + { name = "opentelemetry-sdk" }, { name = "pwdlib", extra = ["argon2"] }, { name = "pyjwt" }, { name = "pytest" }, @@ -986,6 +1018,7 @@ tests = [ { name = "pytest-xdist", extra = ["psutil"] }, { name = "pyyaml" }, { name = "ruff" }, + { name = "sentry-sdk" }, { name = "sqlmodel" }, { name = "strawberry-graphql" }, { name = "ty" }, @@ -1014,6 +1047,15 @@ requires-dist = [ { name = "jinja2", marker = "extra == 'all'", specifier = ">=3.1.5" }, { name = "jinja2", marker = "extra == 'standard'", specifier = ">=3.1.5" }, { name = "jinja2", marker = "extra == 'standard-no-fastapi-cloud-cli'", specifier = ">=3.1.5" }, + { name = "opentelemetry-api", specifier = ">=1.44.0" }, + { name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'all'", specifier = ">=1.44.0" }, + { name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'opentelemetry'", specifier = ">=1.44.0" }, + { name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'standard'", specifier = ">=1.44.0" }, + { name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'standard-no-fastapi-cloud-cli'", specifier = ">=1.44.0" }, + { name = "opentelemetry-sdk", marker = "extra == 'all'", specifier = ">=1.44.0" }, + { name = "opentelemetry-sdk", marker = "extra == 'opentelemetry'", specifier = ">=1.44.0" }, + { name = "opentelemetry-sdk", marker = "extra == 'standard'", specifier = ">=1.44.0" }, + { name = "opentelemetry-sdk", marker = "extra == 'standard-no-fastapi-cloud-cli'", specifier = ">=1.44.0" }, { name = "pydantic", specifier = ">=2.9.0" }, { name = "pydantic-extra-types", marker = "extra == 'all'", specifier = ">=2.0.0" }, { name = "pydantic-extra-types", marker = "extra == 'standard'", specifier = ">=2.0.0" }, @@ -1032,7 +1074,7 @@ requires-dist = [ { name = "uvicorn", extras = ["standard"], marker = "extra == 'standard'", specifier = ">=0.12.0" }, { name = "uvicorn", extras = ["standard"], marker = "extra == 'standard-no-fastapi-cloud-cli'", specifier = ">=0.12.0" }, ] -provides-extras = ["standard", "standard-no-fastapi-cloud-cli", "all"] +provides-extras = ["opentelemetry", "standard", "standard-no-fastapi-cloud-cli", "all"] [package.metadata.requires-dev] dev = [ @@ -1050,10 +1092,14 @@ dev = [ { name = "httpx2", specifier = ">=2.0.0" }, { name = "inline-snapshot", specifier = ">=0.21.1" }, { name = "jieba", specifier = ">=0.42.1" }, + { name = "logfire", specifier = ">=5.1.0" }, { name = "markdown-include-variants", specifier = ">=0.0.8" }, { name = "mdx-include", specifier = ">=1.4.1,<2.0.0" }, { name = "mkdocstrings", extras = ["python"], specifier = ">=1.0.3" }, { name = "mypy", specifier = ">=1.14.1" }, + { name = "opentelemetry-exporter-otlp-proto-http", specifier = ">=1.44.0" }, + { name = "opentelemetry-instrumentation-fastapi", specifier = ">=0.65b0" }, + { name = "opentelemetry-sdk", specifier = ">=1.44.0" }, { name = "pillow", specifier = ">=11.3.0" }, { name = "playwright", specifier = ">=1.57.0" }, { name = "prek", specifier = ">=0.2.22" }, @@ -1070,6 +1116,7 @@ dev = [ { name = "python-slugify", specifier = ">=8.0.4" }, { name = "pyyaml", specifier = ">=5.3.1,<7.0.0" }, { name = "ruff", specifier = ">=0.14.14,<0.16.0" }, + { name = "sentry-sdk", specifier = ">=2.70.0" }, { name = "sqlmodel", specifier = ">=0.0.31" }, { name = "strawberry-graphql", specifier = ">=0.200.0,<1.0.0" }, { name = "ty", specifier = ">=0.0.25" }, @@ -1118,7 +1165,11 @@ tests = [ { name = "httpx", specifier = ">=0.23.0,<1.0.0" }, { name = "httpx2", specifier = ">=2.0.0" }, { name = "inline-snapshot", specifier = ">=0.21.1" }, + { name = "logfire", specifier = ">=5.1.0" }, { name = "mypy", specifier = ">=1.14.1" }, + { name = "opentelemetry-exporter-otlp-proto-http", specifier = ">=1.44.0" }, + { name = "opentelemetry-instrumentation-fastapi", specifier = ">=0.65b0" }, + { name = "opentelemetry-sdk", specifier = ">=1.44.0" }, { name = "pwdlib", extras = ["argon2"], specifier = ">=0.2.1" }, { name = "pyjwt", specifier = ">=2.9.0" }, { name = "pytest", specifier = ">=9.0.0" }, @@ -1129,6 +1180,7 @@ tests = [ { name = "pytest-xdist", extras = ["psutil"], specifier = ">=3.0.2" }, { name = "pyyaml", specifier = ">=5.3.1,<7.0.0" }, { name = "ruff", specifier = ">=0.14.14,<0.16.0" }, + { name = "sentry-sdk", specifier = ">=2.70.0" }, { name = "sqlmodel", specifier = ">=0.0.31" }, { name = "strawberry-graphql", specifier = ">=0.200.0,<1.0.0" }, { name = "ty", specifier = ">=0.0.25" }, @@ -1605,8 +1657,8 @@ name = "httpcore2" version = "2.12.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "h11" }, - { name = "truststore" }, + { name = "h11", marker = "python_full_version < '3.12' or python_full_version >= '3.14' or sys_platform != 'emscripten'" }, + { name = "truststore", marker = "python_full_version < '3.12' or python_full_version >= '3.14' or sys_platform != 'emscripten'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/be/ad/f4f0e57345f1870f3e8cb624e058d7eca6e5a27d33bcc3311d9b618734cd/httpcore2-2.12.0.tar.gz", hash = "sha256:9293522bba0aa7c4c8e9e3f040c16575bd8868e155a77fa30c7a9085a5eae648", size = 67548, upload-time = "2026-08-18T13:22:08.211Z" } wheels = [ @@ -1720,7 +1772,7 @@ name = "importlib-metadata" version = "8.7.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "zipp" }, + { name = "zipp", marker = "(python_full_version < '3.14' and sys_platform != 'emscripten') or (python_full_version < '3.12' and sys_platform == 'emscripten')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/f3/49/3b30cad09e7771a4982d9975a8cbf64f00d4a1ececb53297f1d9a7be1b10/importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb", size = 57107, upload-time = "2025-12-21T10:00:19.278Z" } wheels = [ @@ -2070,7 +2122,7 @@ wheels = [ [[package]] name = "logfire" -version = "4.41.0" +version = "5.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "executing" }, @@ -2082,9 +2134,9 @@ dependencies = [ { name = "tomli", marker = "python_full_version < '3.11'" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/64/1a/529f5fd3d0b72eca62737e07b290d38737f104f31891d23e5ed47a8ec7a0/logfire-4.41.0.tar.gz", hash = "sha256:3806fba60389d57c38a12a88135a7c7bf9d0fca09325094517e976b29b5b9d33", size = 1302531, upload-time = "2026-08-20T17:42:23.037Z" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/da/0c73a4447dffe01e1737580556f5f061ffda8f85e46fbf35ac36f5b64d27/logfire-5.1.0.tar.gz", hash = "sha256:edf6c7d33c00214e11c2cdc567eb53051be36289747091e8bb910527e5970bea", size = 1359738, upload-time = "2026-09-11T10:25:11.674Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/fc/e1/ee33bf0e3f85c00a4235c8c9c4e23f3955154d842f15f0377936461ec6a3/logfire-4.41.0-py3-none-any.whl", hash = "sha256:5bae36637aef81eeee6bfa5d764bf3cff0755af613a4888fd0ae4a656cd2451e", size = 426654, upload-time = "2026-08-20T17:42:20.092Z" }, + { url = "https://files.pythonhosted.org/packages/cd/02/b32c3d5c298e4a9010694d261fb82c2c672747c0bf2b47bffda484aef349/logfire-5.1.0-py3-none-any.whl", hash = "sha256:c1d54eabf39f4c8f5e18e71f35da13375c8adb144c263f3076e650ba85929108", size = 474841, upload-time = "2026-09-11T10:25:08.131Z" }, ] [package.optional-dependencies] @@ -2524,6 +2576,38 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/40/7b/85eab1215f72adf0e68d3dc4a679b9bff993fa679ff34cd8dd378e2659fd/opentelemetry_instrumentation-0.65b0-py3-none-any.whl", hash = "sha256:ea967a72b9939b5fcfdad572753b4306c59dcb99e3f382d95dae04286805e137", size = 36717, upload-time = "2026-07-16T15:24:51.424Z" }, ] +[[package]] +name = "opentelemetry-instrumentation-asgi" +version = "0.65b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "asgiref" }, + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "opentelemetry-util-http" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/17/83/8e8e83b7ac285281687c7be2fd305213ccccbb8c0a2dd4fb45a8ccaf12c7/opentelemetry_instrumentation_asgi-0.65b0.tar.gz", hash = "sha256:892bca67c56522ffa85a8a83cf934d7b50b3be2132e45cbee705825f0a5ba426", size = 26140, upload-time = "2026-07-16T15:25:54.544Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/9c/376962840b619d2d55fe8ee2285f8c70971c090e5fff614516fc654a6f3a/opentelemetry_instrumentation_asgi-0.65b0-py3-none-any.whl", hash = "sha256:3a845a8ebd1c4ef0d8263401e6545f5b219b2feee612090d50f578a87e71fd65", size = 15903, upload-time = "2026-07-16T15:24:57.198Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation-fastapi" +version = "0.65b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-instrumentation-asgi" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "opentelemetry-util-http" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/30/23/b057f8196d06efdc1b50e3ff11fbc499a7d96b35c87f217eb7885542f4ea/opentelemetry_instrumentation_fastapi-0.65b0.tar.gz", hash = "sha256:10a3a95486036230413a58fe4fdf4a83fa6bba46918407e527476994bd92bd97", size = 26236, upload-time = "2026-07-16T15:26:05.954Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fa/b0/c9b0300d33349ecc3dfd2362516eaffc44877e90970e6a52178ff953fec3/opentelemetry_instrumentation_fastapi-0.65b0-py3-none-any.whl", hash = "sha256:cda2610a0ec1b22d19886f33e4d861e9f5dbb886aeaa3a1263b47aff82c36943", size = 13261, upload-time = "2026-07-16T15:25:12.429Z" }, +] + [[package]] name = "opentelemetry-instrumentation-httpx" version = "0.65b0" @@ -4027,8 +4111,8 @@ name = "secretstorage" version = "3.5.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cryptography" }, - { name = "jeepney" }, + { name = "cryptography", marker = "python_full_version < '3.12' or python_full_version >= '3.14' or sys_platform != 'emscripten'" }, + { name = "jeepney", marker = "python_full_version < '3.12' or python_full_version >= '3.14' or sys_platform != 'emscripten'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" } wheels = [ @@ -4037,15 +4121,15 @@ wheels = [ [[package]] name = "sentry-sdk" -version = "2.52.0" +version = "2.70.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "certifi" }, { name = "urllib3" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/59/eb/1b497650eb564701f9a7b8a95c51b2abe9347ed2c0b290ba78f027ebe4ea/sentry_sdk-2.52.0.tar.gz", hash = "sha256:fa0bec872cfec0302970b2996825723d67390cdd5f0229fb9efed93bd5384899", size = 410273, upload-time = "2026-02-04T15:03:54.706Z" } +sdist = { url = "https://files.pythonhosted.org/packages/52/0a/e37553f3106d0f7f0d4b8ad2c7b92410d131929786ffd116833d3b8164df/sentry_sdk-2.70.0.tar.gz", hash = "sha256:27e4b512f47be93136645dbe8f02d430473ee73a389ce430b2b7104baa66cf75", size = 1047234, upload-time = "2026-09-22T09:40:11.721Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ca/63/2c6daf59d86b1c30600bff679d039f57fd1932af82c43c0bde1cbc55e8d4/sentry_sdk-2.52.0-py2.py3-none-any.whl", hash = "sha256:931c8f86169fc6f2752cb5c4e6480f0d516112e78750c312e081ababecbaf2ed", size = 435547, upload-time = "2026-02-04T15:03:51.567Z" }, + { url = "https://files.pythonhosted.org/packages/eb/ff/b1e56f009bfb9b8023910b533d5ebf4ec5650da49891f2b8c7f3d4815401/sentry_sdk-2.70.0-py3-none-any.whl", hash = "sha256:89600e8bb5fc5239cad5c4434217f453204f2207ad29399ea58947460b37a62e", size = 531752, upload-time = "2026-09-22T09:40:10.063Z" }, ] [[package]]