Problem after migrating from .NET 6 to .NET 8: Queue message is automatically converted to byte[] instead of string

Question

I am experiencing an issue after migrating from .NET 6 to .NET 8 in my Azure Functions project. The problem appeared during the processing of queue messages.

In .NET 6, the messages sent to the queue were being read as a string (string compressedMessageBase64), and I was able to base64-decode the message properly. However, after migrating to .NET 8, it seems that the queue message is being automatically converted to a byte[] when received, even though the sending code has not changed.

Here is the code for sending the message:

var message = JsonSerializer.Serialize(batch);
var compressedMessage = Compression.CompressMessage(message);

if (compressedMessage.Length > 64 * 1024)
{
    _logger.LogError("Compressed message size exceeds Azure Queue limit.");
    throw new InvalidOperationException("Message size exceeds Azure Queue limit after compression.");
}

await queueClient.SendMessageAsync(Convert.ToBase64String(compressedMessage));
_logger.LogInformation($"Batch with {batch.Count} items pushed to queue.");

The issue is specifically happening with the receive function. In .NET 6, I had the following receive function (string), which worked fine:

public async Task ProcessBatchFromQueue(
    [QueueTrigger("test-queue", Connection = "AzureWebJobsStorage")] string compressedMessageBase64,
    ILogger log)
{
    try
    {
        log.LogInformation("Processing batch from queue...");
        var compressedMessage = Convert.FromBase64String(compressedMessageBase64);
        var decompressedMessage = Compression.DecompressMessage(compressedMessage);
        var batch = JsonSerializer.Deserialize<IReadOnlyCollection<InventoryProduct>>(decompressedMessage);
    }
    catch (Exception ex)
    {
        log.LogError($"Error: {ex.Message}");
    }
}

However, after migrating to .NET 8, the function signature changed to:

[Function("ProcessBatchFromQueue")]
public async Task ProcessBatchFromQueue([QueueTrigger("test-queue", Connection = "AzureWebJobsStorage")] byte[] compressedMessageBase64)
{
    try
    {
        _logger.LogInformation("Processing batch from queue...");

        string decompressedMessage = Compression.DecompressMessage(compressedMessageBase64);

        var batch = JsonSerializer.Deserialize<IReadOnlyCollection<InventoryProduct>>(decompressedMessage);
    }
    catch (Exception ex)
    {
        _logger.LogError($"Error: {ex.Message}");
    }
}

Here's the compression and decompression logic that I'm using:

using System.IO.Compression;

public static class Compression
{
    /// <summary>
    /// Compresses a string using GZip.
    /// </summary>
    public static byte[] CompressMessage(string message)
    {
        using (var memoryStream = new MemoryStream())
        {
            using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress))
            using (var writer = new StreamWriter(gzipStream))
            {
                writer.Write(message);
                writer.Flush();
            }

            return memoryStream.ToArray();
        }
    }

    /// <summary>
    /// Decompresses a GZip-compressed string.
    /// </summary>
    public static string DecompressMessage(byte[] compressedMessage)
    {
        using (var memoryStream = new MemoryStream(compressedMessage))
        using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
        using (var reader = new StreamReader(gzipStream))
        {
            return reader.ReadToEnd();
        }
    }
}

The problem: after migration, the message in the queue is automatically passed as a byte[] in the function. The issue is that the message is being base64-decoded and then passed as raw bytes, while it was previously passed as a string. The message still gets processed, but I receive warnings about the message's body being non-UTF-8.

[2025-02-28T01:37:19.025Z] Batch with 200 items pushed to queue.
[2025-02-28T01:37:19.115Z] Queue message's body cannot be converted to string because it contains non-UTF8 bytes, message id=d8887de8-f2d8-40cf-8990-555ss7b280d6
[2025-02-28T01:37:19.131Z] Queue message's body cannot be converted to string because it contains non-UTF8 bytes, message id=d8887de8-f2d8-40cf-8990-555ss7b280d6
[2025-02-28T01:37:19.135Z] Queue message's body cannot be converted to string because it contains non-UTF8 bytes, message id=d8887de8-f2d8-40cf-8990-555ss7b280d6
[2025-02-28T01:37:19.137Z] Executing 'Functions.ProcessBatchFromQueue' (Reason='New queue message detected on 'test-queue'.', Id=a148187e-9f70-4bb9-b9df-8446d1b337d7)
[2025-02-28T01:37:19.139Z] Trigger Details: MessageId: d8887de8-f2d8-40cf-8990-555ss7b280d6, DequeueCount: 1, InsertedOn: 2025-02-28T01:37:19.000+00:00

Additionally, if I try to keep the parameter as string in the function, I get the following error:

2025-02-28T02:09:56.143Z] Trigger Details: MessageId: 27ad4e9b-86a1-4e9b-9596-7f697424e58a, DequeueCount: 4, InsertedOn: 2025-02-28T02:09:53.000+00:00
[2025-02-28T02:09:56.148Z] Executed 'Functions.ProcessBatchFromQueue' (Failed, Id=4bb49420-5b7c-43ad-adc4-2870d3a3e382, Duration=7ms)
[2025-02-28T02:09:56.150Z] System.Private.CoreLib: Exception while executing function: Functions.ProcessBatchFromQueue. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'compressedMessageBase64'. System.Private.CoreLib: Unable to translate bytes [8B] at index 1 from specified code page to Unicode.

This error occurs because the body of the message is being treated as non-UTF-8 data, which is expected since it’s compressed and base64 encoded.

My question: is there any configuration or setting in .NET 8 or Azure Functions that causes this behavior? Can I explicitly configure the queue trigger to receive the message as a string instead of a byte array? How can I handle this migration from .NET 6 to .NET 8 without manually managing the byte-to-string conversion?

Expected outcome: I would like to know if there's a way to handle this change without needing to modify the sending logic, and to prevent receiving the queue message as raw byte[]. A solution where the queue message could remain as a string (or be properly decoded) in .NET 8 would be appreciated.

Answer 1

If you don't want to manually managing the byte-to-string conversion, One option is to explicitly configure serialization:

1. Modify your host.json file to enforce string-based serialization:

json:

{
  "extensions": {
    "queues": {
      "messageEncoding": "base64"
    }
  }
}

2. Other option is Explicitly Bind the Queue Message as a String, Modify your function signature:

code:

[Function("ProcessBatchFromQueue")]
public async Task ProcessBatchFromQueue([QueueTrigger("test-queue", Connection = "AzureWebJobsStorage")] string compressedMessageBase64)
{ 
  your code 
}

3. If the above method still results in byte[], Try manually convert it inside your function:

code:

public async Task ProcessBatchFromQueue([QueueTrigger("test-queue", Connection = "AzureWebJobsStorage")] byte[] messageBytes)
{
    string compressedMessageBase64 = Encoding.UTF8.GetString(messageBytes);
    var decodedMessage = Encoding.UTF8.GetString(Convert.FromBase64String(compressedMessageBase64));
}