(PowerShell) HTTP GET -- Read Response from Stream

Demonstrates how to send an HTTP GET for a web page, and then read the response from a Stream. The purpose is to fulfill a situation such as the following: "I have a URL and just want the <head> </head> portion of the HTML. This would avoid having to download a potentially enormous web page just to get at header information, such as the <styles>."

Chilkat .NET Downloads

Chilkat .NET Assemblies

Add-Type -Path "C:\chilkat\ChilkatDotNet47-9.5.0-x64\ChilkatDotNet47.dll"

# This example requires the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

$rest = New-Object Chilkat.Rest

# In this example, we'll get the web page at http://www.nytimes.com/pages/business/index.html
# The domain is "www.nytimes.com", and the path is "/pages/business/index.html"

# If we have only the full URL to begin with, it can be loaded into the Chilkat URL object to
# access the parts:
$url = New-Object Chilkat.Url
$url.ParseUrl("http://www.nytimes.com/pages/business/index.html")

$bAutoReconnect = $true
$success = $rest.Connect($url.Host,$url.Port,$url.Ssl,$bAutoReconnect)

# Send the GET request (This sends the GET request, but does not read the response.)
$success = $rest.SendReqNoBody("GET",$url.Path)
if ($success -ne $true) {
    $($rest.LastErrorText)
    exit
}

# Read the response header.
$responseStatusCode = $rest.ReadResponseHeader()
if ($responseStatusCode -lt 0) {
    $($rest.LastErrorText)
    exit
}

$("Response status code = " + $responseStatusCode)

# We expect a 200 response status.
if ($responseStatusCode -ne 200) {

    # If the response status code is not 200, we could check for a redirect status code and 
    # then follow it, read the entire response (as shown here), or just call rest.Disconnect
    $errResponse = $rest.ReadRespBodyString()
    if ($rest.LastMethodSuccess -ne $true) {
        $($rest.LastErrorText)
    }
    else {
        $($errResponse)
    }

    exit
}

$bodyStream = New-Object Chilkat.Stream
# Set a 10 second read timeout for the stream. 
# (Give up if no data arrives within 10 seconds after calling a read method.)
$bodyStream.ReadTimeoutMs = 10000

# Create a background thread task to read the response body (which feeds
# it to the bodyStream object.)
$readResponseBodyTask = $rest.ReadRespBodyStreamAsync($bodyStream,$true)

# Start the task. 
$success = $readResponseBodyTask.Run()

# Read the HTTP response body until the "</head>" is seen, or until
# the end-of-stream is reached.

$sbBody = New-Object Chilkat.StringBuilder
$exitLoop = $false
while (!$exitLoop -and ($bodyStream.EndOfStream -ne $true)) {

    $bodyText = $bodyStream.ReadString()
    if ($bodyStream.LastMethodSuccess -eq $true) {
        $sbBody.Append($bodyText)
        if ($sbBody.Contains("</head>",$false)) {
            $exitLoop = $true
        }

    }
    else {
        $exitLoop = $true
    }

}

# Cancel the remainder of the task...
$readResponseBodyTask.Cancel()

# Ensure we're disconnected from the server.
$maxWaitMs = 50
$rest.Disconnect($maxWaitMs)

$("----")
$($sbBody.GetAsString())
$("----")
$("Successfully received the body up to the desired point.")