Eventarc triggers for Google Workflow end up with http 400 errors

Question

I've created a eventarc trigger that triggers on a Firestore change and it'll call a Google Workflow. The Google Workflow publishes a message to a PubSub topic. This works fine for thousands of messages per hour. But recently I have a single one that is stuck in the pubsub subscription that is created by Eventarc to communicate with the Workflow.

I don't see any errors in the Google Workflow logs. In the Eventarc trigger I see: url_4xx_error_400. I also see this message in the Pubsub subscription that pushes to the Google Workflow.

Because all other messages are published fine, I don't think it's a service account issue. But I'm unable to find any log entries.

I tried to recreate the eventarc trigger and workflow, but the problem returned.

Any clues on how to debug this?