Transferring Cookies from WebBrowser to WebClient

I have prepared many web scrapping application and crawlers. Few of them were developed using WebClient and others were with WebBrowser control of .NET (see my previous article to Get and Set Cookies .NET WebBrowser Control). Recently I got stuck in a situation where I need features of both tools. That is WebClient and WebBrowser.

So my idea was to use WebBrowser control and pass on the session to WebClient when WebClient features are required.

For this my first task was to make WebClient aware of cookies and behave accordingly. SO After searching for few minutes I got this SO link which gave me a class to make cookie aware WebClient and after modifying it I got this final class CookieAwareWebClient

public class CookieAwareWebClient : WebClient
{
	public CookieContainer CookieContainer { get; set; }
	public Uri Uri { get; set; }
	
	public CookieAwareWebClient() : this(new CookieContainer())
	{
	
	}

	public CookieAwareWebClient(CookieContainer cookies)
	{
		this.CookieContainer = cookies;
	}

	protected override WebRequest GetWebRequest(Uri address)
	{
		WebRequest request = base.GetWebRequest(address);
		if (request is HttpWebRequest)
		{
			(request as HttpWebRequest).CookieContainer = this.CookieContainer;
		}
		
		HttpWebRequest httpRequest = (HttpWebRequest)request;
		httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
		return httpRequest;
	}

	protected override WebResponse GetWebResponse(WebRequest request)
	{
		WebResponse response = base.GetWebResponse(request);
		String setCookieHeader = response.Headers[HttpResponseHeader.SetCookie];

		if (setCookieHeader != null)
		{
			//do something if needed to parse out the cookie.
			if (setCookieHeader != null)
			{
				Cookie cookie = new Cookie(); //create cookie
				this.CookieContainer.Add(cookie);
			}
		}
		return response;
	}
}

Now I have to use this class in place of WebClient because it supports cookies for by browsing.

Next thing that I have to do is pass all the cookies from my WebBrowser to this cookie aware web client. Here is how I did this:

CookieContainer cookieJar = new CookieContainer();
string[] cookies = browser.Document.Cookie.Split(new char[] { '=' });
cookieJar.Add(new Cookie(cookies[0], cookies[1], "/", "edunyte.com"));

In above code browser is my WebBrowser control object and because I have only one cookie value associated with it so I parsed (cookie key and value) it and assigned it to my cookie container. If you have multiple cookies then parse each of them and associate with cookie container.

Now because all cookies which were associated with WebBrowser browsing session is not assigned to my Cookie container so I can create CookieAwareWebClient object using this cookie container.

CookieAwareWebClient client = new CookieAwareWebClient(cookieJar);

Now we are ready with our cookie aware client with required cookies assigned to it so we can start sending request using this object and it will behave like it is cookie owner.

Leave a Reply

Your email address will not be published.