CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-pubsubsplunk
dumb_coder
dumb_coder

Reputation: 325

Dataflow template - Splunk not reading data from topic

I am using the Dataflow template that streams data from pubsub topic to splunk. I have followed the steps below from pubsub side

  1. Create a topic
  2. create a subscription
  3. create a sink that exports log to the topic
  4. give the service account permissions of publisher

For the Dataflow template

  1. give topic subscription name
  2. HEC endpoint
  3. HEC token base64 encoded
  4. create a service account that has the role/worker data, role/pubsub.reader, role/project viewer
  5. Disable SSL -> true
  6. vpc network
  7. subnetwork name
  8. create job

The worker machine spins up. The workflow is running but splunk is not pulling data from the topic. Is there anything particular I should check?

Thank you in advance!!

Upvotes: 1

Views: 252

Answers (1)

dumb_coder
dumb_coder

Reputation: 325

So what I found after hours or wrecking my brain is that the Dataflow template requires internet access probably to download some files when it boots up. If you are behind a private vpc and internet is not accessible the please make sure to create a route with the tag "dataflow". It should then work

Upvotes: 1

Related Questions