Streaming process output in Azure Function response

Question

I have a working HTTP trigger Function that kicks off a new Process. What I would like to do is take the standard error/output and send those to the web client, but I haven't figured out the correct combination of process handling and streaming to make it work...

Usually, everything runs but no output seems to be sent to the client. If possible I would like to flush each line, though I know the web server may prevent that. Currently I'm testing locally through the AF webjob host. I'm guessing it's something to do with object lifetime, or maybe it's just not possible.

I've tried various things, but this is the code I have currently, which runs and the process completes successfully, output gets logged, but no output makes it to Postman (not even a 200 OK):

[FunctionName("Function1")]
public static async Task<HttpResponseMessage> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]HttpRequestMessage req,
    CancellationToken cancellationToken,
    TraceWriter log
)
{
    using (var memoryStream = new MemoryStream())
    using (var streamWriter = new StreamWriter(memoryStream, Encoding.UTF8, 1, true))
    //using (var streamWriter = new StreamWriter(memoryStream))
    {
        streamWriter.WriteLine($"Starting execution.");

        using (var proc = new Process())
        {
            proc.StartInfo.FileName = @"D:\Runner.exe";
            proc.StartInfo.RedirectStandardError = true;
            proc.StartInfo.RedirectStandardOutput = true;
            proc.StartInfo.UseShellExecute = false;
            //proc.OutputDataReceived += (object sender, DataReceivedEventArgs e) =>
            //{
            //    log.Info(e.Data);
            //    streamWriter.WriteLine(e.Data);
            //};
            //proc.ErrorDataReceived += (object sender, DataReceivedEventArgs e) =>
            //{
            //    log.Error(e.Data);
            //    streamWriter.WriteLine(e.Data);
            //};
            proc.Start();

            /// Need to drain the output otherwise the process will
            /// hang and not stop until the computer is shut down.
            /// It can be killed by Administrator but will remain as
            /// a process until restarted.
            /// taskkill /f /im runner.exe
            /// tasklist /fi "imagename eq runner.exe"
            while (!proc.StandardOutput.EndOfStream)
            {
                log.Info(proc.StandardOutput.ReadLine());
                streamWriter.Flush();
            }

            //proc.WaitForExit();
            streamWriter.Flush();

            var response = new HttpResponseMessage
            {
                StatusCode = HttpStatusCode.OK,
                Content = new StreamContent(memoryStream)
            };

            return response;
        }
    }
}

Answer 1

My example is a little newer than the question, but here is how you do it.

I don't have runner.exe and I think using CliWrap is a much better option to using System.Diagnostics.Process. I've tried to use a solution which didn't require Synchronous IO, but if you do need to do something which does require that, you can set "FUNCTIONS_V2_COMPATIBILITY_MODE": true in the function host environment variables to make it work.

using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using CliWrap;

namespace AdamCoulter
{
    public static class StreamBack
    {
        [FunctionName("StreamBack")]
        public static HttpResponseMessage Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequestMessage request)
        {
            var response = new HttpResponseMessage
            {
                RequestMessage = request,
                Content = new PushStreamContent(new Func<Stream, HttpContent, TransportContext, Task>(RunTerraform), "text/event-stream")
            };
            response.Headers.TransferEncodingChunked = true;

            return response;
        }

        public static async Task RunTerraform(Stream stream, HttpContent content, TransportContext context)
        {
            _ = await Cli.Wrap("terraform")
                .WithArguments("init")
                .WithWorkingDirectory("/Volumes/Code/tftest/")
                .WithStandardOutputPipe(PipeTarget.ToStream(stream))
                .ExecuteAsync();
            stream.Close();
        }
    }
}

GET /api/StreamBack HTTP/1.1
Host: localhost:7071

Incrementally streamed output in chrome (shows up bit by bit as the terminal output is generated):

[0m[1mInitializing the backend...[0m

[0m[1mInitializing provider plugins...[0m
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Using previously-installed hashicorp/azurerm v2.58.0

[0m[1m[32mTerraform has been successfully initialized![0m[32m[0m
[0m[32m
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.[0m