About the HTTP Protocol

Question

I'm quite new to the HTTP Protocol, but I'm working on it in order to be able to automatically thought a C# Console Application create a new account on Twitter. What I'm doing, is first doing a GET request in order to gather some data on the page (like the authenticity token for example) and then I'm posting my data to a specific URL designed to it. The URL I'm using for the GET operation is "https://www.twitter.com/signup/"

Then I do my POST operation, giving all the required headers (except the cookie, I don't know if the cookie is important or not for the process) and the formatted data of the account I want to create.

The problem is : when I start my program, first it doesn't create any account, and second the server answers me redirecting me back to www.twitter.com/account/new, it feels like I don't fill up the correct fields but I can't get any clue of what is actually really happening.

Here's my main code:

static void Main(string[] args)
        {
            // Set up data info
            string[] infos = SetUpAccountInfo();
            string parsedData= FormDataParsing(infos);

            StreamWriter myWriter = null;
            StreamReader myReader = null;
            CookieContainer cookies = new CookieContainer();

            HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(POSTSignUpURL);
            objRequest.Method = "POST";
            objRequest.Host = "twitter.com";
            objRequest.ProtocolVersion = HttpVersion.Version11;
            objRequest.ContentLength = machin.Length;
            objRequest.ContentType = "application/x-www-form-urlencoded";
            objRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
            objRequest.Headers.Add("accept-encoding", "gzip,deflate,sdch");
            objRequest.Headers.Add("accept-language", "fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4");
            objRequest.Headers.Add("cache-control", "max-age=0");
            objRequest.Headers.Add("origin", "https://twitter.com");
            objRequest.Referer = "https://twitter.com/signup";
            objRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36";
            System.Net.ServicePointManager.Expect100Continue = false;
            objRequest.CookieContainer = cookies;

            try
            {
                myWriter = new StreamWriter(objRequest.GetRequestStream());
                myWriter.Write(parsedData);


            }
            catch (Exception e)
            {
                // return e.Message;
            }
            finally
            {
                myWriter.Close();

            }

            HttpWebResponse response = objRequest.GetResponse() as HttpWebResponse;
            using (StreamReader sr =
                 new StreamReader(response.GetResponseStream()))
            {
                string result = sr.ReadToEnd();
                HtmlAgilityPack.HtmlDocument document = new HtmlAgilityPack.HtmlDocument();
                document.LoadHtml(result);

                // Close and clean up the StreamReader
                sr.Close();
            }



        }

Answer 1

Use Fiddler to observe what you are sending and what comes back. Compare that to what happens when you successfully register using a web browser.

You will find a difference. Twitter cannot tell what client program you are using. The problem can only be caused by what you are sending.