C# - FormUrlEncodedContent Encode space into '%20' instead of '+'

Question

I'm trying to connect to Discord's OAuth endpoint using Client credential grant (https://discord.com/developers/docs/topics/oauth2#client-credentials-grant)

Discord is expecting the scope to be send as a urlencoded string: identify%20email%20guilds

By default the C# HttpClient seems to convert spaces into + instead of %20.

Following code

  var scopeasStr = string.Join(" ", opts.Scopes);
            //scopeasStr = HttpUtility.UrlEncode(scopeasStr);
            //scopeasStr = Uri.EscapeDataString(scopeasStr);
       
            var nvc = new List<KeyValuePair<string, string>>();
            nvc.Add(new KeyValuePair<string, string>("grant_type", opts.GrantType));
            nvc.Add(new KeyValuePair<string, string>("scope", scopeasStr));

            var content = new FormUrlEncodedContent(nvc);
            var requestMessage = new HttpRequestMessage(HttpMethod.Post, $"{apiUrl}/oauth2/token");
            requestMessage.Content = content;


            var response = await externalHttpClient.SendAsync(requestMessage);

Generates following request

POST https://discord.com/api/v10/oauth2/token HTTP/1.1
Host: discord.com
Authorization: Basic VerySecret
Content-Type: application/x-www-form-urlencoded
Content-Length: 57

grant_type=client_credentials&scope=identify+guilds+email

Which returns a 400 Bad Request

{"error": "invalid_scope", "error_description": "The requested scope is invalid, unknown, or malformed."}

I've tried using scopeasStr = Uri.EscapeDataString(scopeasStr); to encode the value. But then the %20 is encoded to %2520 by the httpClient

POST https://discord.com/api/v10/oauth2/token HTTP/1.1
Host: discord.com
Authorization: Basic VerySecret
Content-Type: application/x-www-form-urlencoded
Content-Length: 65

grant_type=client_credentials&scope=identify%2520guilds%2520email

• When sending only one scope, the requests works. So it's definitly the space in scopes that is causing the issue. I've confirmed with Discord support that they are only accepting %20 for seperating the scopes.
• How am I supposed to correctly encode this using the HttpClient?

full C# code:

public static async Task Authenticate(this HttpClient client, AuthenticateOptions opts, bool forceNew = false)
        {
            if (client.DefaultRequestHeaders.Authorization == null || forceNew)
            {
                var externalHttpClient = new HttpClient();

                var apiUrl = opts.EndPointUrl;
                var clientId = opts.ClientId;
                var secret = opts.ClientSecret;
                var scopeasStr = string.Join(" ", opts.Scopes);
                //scopeasStr = HttpUtility.UrlEncode(scopeasStr);
                //scopeasStr = Uri.EscapeDataString(scopeasStr);
                externalHttpClient.BaseAddress = new Uri(apiUrl);
                
                var nvc = new List<KeyValuePair<string, string>>();
                nvc.Add(new KeyValuePair<string, string>("grant_type", opts.GrantType));

                nvc.Add(new KeyValuePair<string, string>("scope", scopeasStr));
                var content = new FormUrlEncodedContent(nvc);
                var requestMessage = new HttpRequestMessage(HttpMethod.Post, $"{apiUrl}/oauth2/token");
                requestMessage.Content = content;
                var authenticationString = $"{clientId}:{secret}";
                var base64EncodedAuthenticationString = Convert.ToBase64String(Encoding.ASCII.GetBytes(authenticationString));
                requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Basic", base64EncodedAuthenticationString);

                var response = await externalHttpClient.SendAsync(requestMessage);
                if (response.IsSuccessStatusCode)
                {
                    var value = await response.ParseResponse();
                    var json = JObject.Parse(value);
                    json.TryGetValue("access_token", out var v);
                    var accessToken = v.Value<string>();
                    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("bearer", accessToken);

                }
                else
                {
                    var value = await response.ParseResponse();
                    Console.Write($"{response.StatusCode} - {value}");
                    Assert.Fail();
                }

            }
        }

Answer 1

Found that there's a Replace which swaps out %20 for + in FormUrlEncodedContent Ended up making my own version of ByteArrayContent/FormUrlEncodedContent

public class CustomFormUrlEncodedContent : ByteArrayContent
{
        public CustomFormUrlEncodedContent(
            IEnumerable<KeyValuePair<string, string>> nameValueCollection)
            : base(GetContentByteArray(nameValueCollection))
        {
            Headers.ContentType = new MediaTypeHeaderValue("application/x-www-form-urlencoded");
        }

        private static byte[] GetContentByteArray(IEnumerable<KeyValuePair<string?, string?>> nameValueCollection)
        {
            if (nameValueCollection == null)
            {
                throw new ArgumentNullException(nameof(nameValueCollection));
            }

            // Encode and concatenate data
            StringBuilder builder = new StringBuilder();
            foreach (KeyValuePair<string?, string?> pair in nameValueCollection)
            {
                if (builder.Length > 0)
                {
                    builder.Append('&');
                }

                builder.Append(Encode(pair.Key));
                builder.Append('=');
                builder.Append(Encode(pair.Value));
            }

            return Encoding.GetEncoding(28591).GetBytes(builder.ToString());
        }

        private static string Encode(string? data)
        {
            if (string.IsNullOrEmpty(data))
            {
                return string.Empty;
            }
        // Escape spaces as '+'.
        return Uri.EscapeDataString(data);//.Replace("%20", "+");
        }
}

Answer 2

Basically all url encoding does is encoding some characters in a string to other predefined constants. It is basic substitution. I'm sure you can create your own urlencoding method, and instead of encoding to a plus, encode it to %20. Because %20 is indeed the correct urlencoded equivalent of a space.