(Ruby) HTTP GET -- Read Response from Stream

Demonstrates how to send an HTTP GET for a web page, and then read the response from a Stream. The purpose is to fulfill a situation such as the following: "I have a URL and just want the <head> </head> portion of the HTML. This would avoid having to download a potentially enormous web page just to get at header information, such as the <styles>."

Chilkat Ruby Downloads

install from rubygems.org
gem install chilkat
or download... Ruby Library for Windows, MacOS, Linux, Alpine Linux

require 'chilkat'

# This example requires the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

rest = Chilkat::CkRest.new()

# In this example, we'll get the web page at http://www.nytimes.com/pages/business/index.html
# The domain is "www.nytimes.com", and the path is "/pages/business/index.html"

# If we have only the full URL to begin with, it can be loaded into the Chilkat URL object to
# access the parts:
url = Chilkat::CkUrl.new()
url.ParseUrl("http://www.nytimes.com/pages/business/index.html")

bAutoReconnect = true
success = rest.Connect(url.host(),url.get_Port(),url.get_Ssl(),bAutoReconnect)

# Send the GET request (This sends the GET request, but does not read the response.)
success = rest.SendReqNoBody("GET",url.path())
if (success != true)
    print rest.lastErrorText() + "\n";
    exit
end

# Read the response header.
responseStatusCode = rest.ReadResponseHeader()
if (responseStatusCode < 0)
    print rest.lastErrorText() + "\n";
    exit
end

print "Response status code = " + responseStatusCode.to_s() + "\n";

# We expect a 200 response status.
if (responseStatusCode != 200)

    # If the response status code is not 200, we could check for a redirect status code and 
    # then follow it, read the entire response (as shown here), or just call rest.Disconnect
    errResponse = rest.readRespBodyString()
    if (rest.get_LastMethodSuccess() != true)
        print rest.lastErrorText() + "\n";
    else
        print errResponse + "\n";
    end

    exit
end

bodyStream = Chilkat::CkStream.new()
# Set a 10 second read timeout for the stream. 
# (Give up if no data arrives within 10 seconds after calling a read method.)
bodyStream.put_ReadTimeoutMs(10000)

# Create a background thread task to read the response body (which feeds
# it to the bodyStream object.)
# readResponseBodyTask is a CkTask
readResponseBodyTask = rest.ReadRespBodyStreamAsync(bodyStream,true)

# Start the task. 
success = readResponseBodyTask.Run()

# Read the HTTP response body until the "</head>" is seen, or until
# the end-of-stream is reached.

sbBody = Chilkat::CkStringBuilder.new()
exitLoop = false
while !exitLoop and (bodyStream.get_EndOfStream() != true)

    bodyText = bodyStream.readString()
    if (bodyStream.get_LastMethodSuccess() == true)
        sbBody.Append(bodyText)
        if (sbBody.Contains("</head>",false))
            exitLoop = true
        end

    else
        exitLoop = true
    end

end

# Cancel the remainder of the task...
readResponseBodyTask.Cancel()

# Ensure we're disconnected from the server.
maxWaitMs = 50
rest.Disconnect(maxWaitMs)

print "----" + "\n";
print sbBody.getAsString() + "\n";
print "----" + "\n";
print "Successfully received the body up to the desired point." + "\n";