C# HTTPS GET and POST with COOKIE

Question

I want to use some REST API shared for my client where it's required two requests in one shot or something like that. I can't get documentation from client so I need to resolve this problem by myself.

So, what I want to achive: if I send request by Insomnia (something like postman), I'm getting stacktrace like below

* Preparing request to https://api.censored_api.com/api/test
* Current time is 2023-02-10T21:32:05.491Z
* Enable automatic URL encoding
* Using default HTTP version
* Enable timeout of 30000ms
* Enable SSL validation
* Enable cookie sending with jar of 2 cookies
*   Trying 111.111.111.88:443...
* Connected to api.censored_api.com (111.111.111.88) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
CN=www.censored_api.com
*  start date: Dec  6 00:00:00 2022 GMT
*  expire date: Jul 14 23:59:59 2023 GMT
*  subjectAltName: host "api.censored_api.com" matched cert's "api.censored_api.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x1d5fa281de0)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):

> GET /api/test HTTP/2
> Host: api.censored_api.com
> user-agent: insomnia/2022.7.5
> accept: */*

* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):

< HTTP/2 200 
< content-type: application/json; charset=utf-8
< server: Kestrel
< lang: en
< srctype: default
< rid: 64c619b5-0e3f-4ac9-9a69-3d6b2657014e
< gid: 102
< x-edgeconnect-midmile-rtt: 5
< x-edgeconnect-origin-mex-latency: 192
< expires: Fri, 10 Feb 2023 21:32:06 GMT
< cache-control: max-age=0, no-cache, no-store
< pragma: no-cache
< date: Fri, 10 Feb 2023 21:32:06 GMT
< strict-transport-security: max-age=86400
< set-cookie: bmtf=2B19763A78567D483337DEEFB95CEEEF~000000000000000000000000000000~YAAQrIfdWBS4DDiGAQAA3Bw+PRI3JD7HGcbuLAJM36p53BaHdmMTjHNHIMvdFBAZ3dKYeeQPonmJ0OrCqK4G8pQhdkfnpEY8zWZi/fEhsiYkICnaRAk3linKO82YMdggBDZsyEONtwKmmIPH2PHyb/wJvTasPxIblGSb5QTEG03tcJuzMPwlxEbKTGdSvrqp1o+tk36qsn/aCKMBA1zsaTeKI2uPgTDBgFW67h+nxZKP0iGFfiQ4w+Vg1J485FUEAQ1RgHpkXtWUF1CwbFUkoESPawW81Euygm4+ExLWgPQa9tOCNG9xW2RsvENZzPDMXWfq8/DfSCrdJ89GVAC8e0kZI9cSZ7jVFtdvI5GoDCoRWwyp4L5aLhr1K03gurLu6owTf1XpmjpcP1s=; Domain=.censored_api.com; Path=/; Expires=Fri, 10 Feb 2023 23:32:05 GMT; Max-Age=7199; HttpOnly


* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Received 3.2 KB chunk
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Received 4 KB chunk

So what I made is:

var baseAddress = new Uri("https://api.censored_api.com");
        var cookieContainer = new CookieContainer();
        using (var handler = new HttpClientHandler() { CookieContainer = cookieContainer })
        using (var client = new HttpClient(handler) { BaseAddress = baseAddress })
        {
            var result = await client.GetAsync("/api/test");
            result.EnsureSuccessStatusCode();
        }

But after 100s I'm getting timeout and I don't know what can I do other.

What's funny, I made the same request in python and it's working perfectly, but I need this in c#.

import http.client

conn = http.client.HTTPSConnection("api.censored_api.com")

payload = ""

headers = { 'cookie': "" }

conn.request("GET", "/api/test", payload, headers)

res = conn.getresponse()
data = res.read()

f = open("result.txt", "a")
f.write(data.decode("utf-8"))
f.close()

Do you have some idea how to implement this in C#?

Wireshark result: (under white block I hide my IP, under orange block is api IP)

Answer 1

Its probably because you aren't supplying a user agent based on your other answer and the lack of one in your question.

HttpClient client = new HttpClient();

client.DefaultRequestHeaders.Add("User-Agent", "SomeUserAgentYouCanSetThisToAnything");

or

client.DefaultRequestHeaders.UserAgent.ParseAdd("SomeUserAgentYouCanSetThisToAnything");

Combined with my other suggestion to move HttpClient out of the method this might work.

Your wireshark image shows that the server is just ignoring you len=0 after completing the handshake, So some firewall is probably blocking the default user-agent (or lack of one) or something like that.

If WebRequest works there is no reason not to use that.

Answer 2

It's intersting, I use WebRequest and when I set UserAgent like postman it start working.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.KeepAlive = false;
request.UserAgent = "PostmanRuntime/7.29.2";
request.Accept = "*/*";
request.Timeout = 30 * 1000;  // 60 second timeout

HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string responseText = reader.ReadToEnd();