CODEWITHSUNDEEP
azureazure-stream-analytics
GANdalf85
GANdalf85

Reputation: 129

Stream Analytics query hits size limit

I'm new to Azure Stream Analytics. I have an Event hub as input source and now I'm trying to execute a simple query on this stream. An example query is like this:

SELECT
count(*)
INTO [output1]
FROM
[input1] TIMESTAMP BY Time
GROUP BY TumblingWindow(second, 10)

So I want to count the events which arrived within a certain time frame.

When executing this query, I always get the following error:

Request exceeded maximum allowed size limit

As I already narrowed down the checked time window and I'm certain that the amount of events within this time frame is not very big (at most several 100) I'm not sure how to avoid this error.

Do you have a hint?

Thanks!

Upvotes: 2

Views: 1593

Answers (1)

Jay Gong
Jay Gong

Reputation: 23782

Request exceeded maximum allowed size limit

This error(i believe it should be more explicit) indicates that you violated the azure stream analytic resource and object limits.

enter image description here

It's not just about quantity, it's also about size.Please check your source inputs' size or try to reduce the windowsize and test again.

1.Does the record size of the source query mean that one event can only have 64 KB or does this parameter mean 64 K events?

It means the size of one event should below 64KB.

  1. Is there a possibility to use Stream Analytics to select only certain subfields of the event or is the only way to reduce the event size before it is sent to the event hub?

As i know,ASA only collects data for processing it,so the size is all depends on the source side and your query sql. Since you need to use COUNT, i'm afraid that you have to do something on the eventhub side.Please refer to my thoughts:

Use Event Hub Azure Function Trigger, when an event streams into event hub,trigger the function and pick only partial key-values and save it into another event hub namespace.(Just in order to reduce the size of source event) Anyway you only need to COUNT records, i think it works for you.

Upvotes: 2

Related Questions