Tracing

Use built-in workflow traces and custom spans to inspect job execution, task duration, and bottlenecks.

Tilebox traces workflow jobs automatically. Job submission creates a root trace, runners continue that trace across machines, and every task execution creates a span.

Built-in traces connect task order, dependencies, parallel execution, task duration, task status, runner identity, service identity, and logs emitted while a span was active.

Add custom spans

Use context.tracer inside a task to add spans around meaningful parts of your own code.

Python

from tilebox.workflows import ExecutionContext, Task

class ProcessScene(Task):
    scene_id: str

    def execute(self, context: ExecutionContext) -> None:
        with context.tracer.span("download-scene") as span:
            span.set_attribute("scene_id", self.scene_id)
            # download input data

        with context.tracer.span("compute-index"):
            # perform expensive computation
            pass

Custom spans are nested under the current task span. Logs emitted inside the span are correlated with its trace_id and span_id.

Span status and exceptions

If a task raises an exception, Tilebox records the exception on the task span and marks the span as failed before the task is retried or marked failed.

For finer-grained error reporting, record errors on your custom spans before re-raising them.

Python

class ProcessScene(Task):
    scene_id: str

    def execute(self, context: ExecutionContext) -> None:
        with context.tracer.span("publish-output") as span:
            try:
                # publish output
                pass
            except Exception as error:
                span.record_exception(error)
                raise

Query spans

You can retrieve spans for a job through the jobs client. Python results can also be converted to a pandas DataFrame.

Python

from tilebox.workflows import Client

client = Client()
job = client.jobs().submit("process-scene", ProcessScene(scene_id="S2A_001"))

spans = client.jobs().query_spans(job)
for span in spans:
    print(span.name, span.status_code, span.duration)

df = spans.to_pandas()

See Query telemetry for the log and span query APIs.

Export to another backend

Tilebox stores traces by default. To export spans to your own observability backend as well, configure an OpenTelemetry or Axiom integration when the runner process starts.