How to force an empty output file with Azure Stream Analytics

Question

I have configured a Stream Analytics Jobs so that input data goes to an Azure Data Lake repository every hour.

Sometimes there is no event to track, so no output. But my Data Factory goes in error because the file doesn't exist.

I wonder if exist a way to force empty file out from Stream Analytics?

Many thanks!

Answer 1

Find the way to do it...

I had an activity using the data lake analytics, what I do is to run an U-SQL than read data with no transformation and write it to the output with headers.

In that way the activity always write an output file!

Very easy!

Answer 2

Are you using ADF v2?

I didn't find anything inbuilt in ADF to come up with it. But I can see few workarounds - starting from simplest one:

1. In your ASA query, you can use WITH statement and union your input with a fake empty message. - Then there will be always output
2. As a second output in ASA job you can store in some DB info whenever a file was produced. Then in ADF you can check whenever there are files and run copy conditionally.
3. In ADF run web activity e.g. LogicApp/FunctionApp to get info whenever files in container exist.

Answer 3

You can look at our common query patterns here. In particular I think you can use the one named "fill missing values" to generate some events regularly, even when there is no input. Let me know if it works for you.

Thanks!

JS