Troubleshoot the blob size limit error
The BlobSizeLimitError is an error that occurs when the size of a blob (payloads including Workflow context and each Workflow and Activity argument and return value) exceeds the set limit in Temporal.
- The max payload for a single request is 2 MB.
- The max size limit for any given Event History transaction is 4 MB.
Why does this error occur?
This error occurs when the size of the blob exceeds the maximum size allowed by Temporal.
This limit helps ensure that the Temporal Service prevents excessive resource usage and potential performance issues when handling large payloads.
How do I resolve this error?
To resolve this error, reduce the size of the blob so that it is within the 4 MB limit.
There are multiple strategies you can use to avoid this error:
-
Use External Storage to offload large payloads to an object store like S3. The Temporal SDKs support this natively through the claim check pattern: when a payload exceeds a size threshold, a storage driver uploads it to your external store and replaces it with a small reference token in the Event History. Your Workflow and Activity code doesn't need to change. Even if your payloads are within the limit today, consider implementing External Storage if their size could grow over time.
For SDK-specific guides, see:
-
Use compression with a custom Payload Codec for large payloads. This addresses the immediate issue, but if payload sizes continue to grow, the problem can arise again.
-
Break larger batches of commands into smaller batch sizes:
- Workflow-level batching:
- Change the Workflow to process Activities or Child Workflows into smaller batches.
- Iterate through each batch, waiting for completion before moving to the next.
- Workflow Task-level batching:
- Execute Activities in smaller batches within a single Workflow Task.
- Introduce brief pauses or sleeps between batches.
- Workflow-level batching:
Workflow termination due to oversized response
When a Workflow Task response exceeds the 4 MB gRPC message size limit, Temporal automatically terminates the Workflow Execution. This is a non-recoverable error. The Workflow can't progress if it generates a response that's too large, so retrying won't help.
This typically happens when a Workflow schedules too many Activities, Child Workflows, or other commands in a single Workflow Task. The total size of all commands generated by the Workflow Task must fit within the 4 MB limit.
If your Workflow was terminated for this reason, you'll see a WorkflowExecutionTerminated event in the Event History with the cause WORKFLOW_TASK_FAILED_CAUSE_GRPC_MESSAGE_TOO_LARGE.
To prevent this, use the batching strategies described above to split work across multiple Workflow Tasks instead of scheduling everything at once.
See the gRPC Message Too Large error reference for more details.