Download contents of a Web Page through HTTP

CSDownloadURL is a class, which has two functions - SetURL and DownloadURL. The set URL sets the current URL and GetDownload downloads the URL contents and returns its contents in a string.

Note: This class was written in Beta 1. In recent .NET versions, the WebRequestFactory class is no longer valid. This is now updated to RC3.

You can use the WebRequest class to download the contents of a web page. The WebRequest uses HTTP protocol to download the contents. You use Create method of WebRequest to create a WebRequest object by passing a URL as a parameter.

After that you can call GetResponse and GetResponseStream methods. The GetResponseStream method returns data in a Stream object.

Imports System
Imports System.Text
Imports System.IO
Imports System.Net
Namespace Project5
' <summary>
' Summary description for Class1.
' </summary>
Public Class CSDownloadURL
Private m_strURL As String
Public Sub setURL1(strURL As String)
m_strURL = strURL
End Sub 'setURL1
Public Sub DownloadURL(ByRef strContents As String)
Dim req As WebRequest = WebRequest.Create(m_strURL)
Dim res As WebResponse = req.GetResponse()
Dim iTotalBuff As Integer = 0
Dim Buffer(128) As [Byte]
Dim stream As Stream = res.GetResponseStream()
iTotalBuff = stream.Read(Buffer, 0, 128)
Dim strRes As New StringBuilder("")
While iTotalBuff <> 0
strRes.Append(Encoding.ASCII.GetString(Buffer, 0, iTotalBuff))
iTotalBuff = stream.Read(Buffer, 0, 128)
End While
strContents = strRes.ToString()
End Sub 'DownloadURL
End Class 'CSDownloadURL
End Namespace 'Project5

Use the CSDownloadURL

Class TestUrl
Public Shared Sub Main()
Dim strOut As String
im web As New CSDownloadURL()
web.setURL1(http://www.mindcracker.com)
web.DownloadURL(strOut)
Console.WriteLine(strOut)
End Sub 'Main
End Class 'TestUrl