(CkPython) Socket Select for Reading

Demonstrates how the Chilkat socket object can become a "socket set" that contains other connected socket objects, and this can be used to "select" on multiple sockets for reading. The SelectForReading method waits until one or more sockets in the set have incoming data ready and available to read.

Chilkat Python Downloads install with pip pip3 install chilkat or download... Python Module for Windows, MacOS, Linux, Alpine Linux, Solaris

import sys
import chilkat

# This example requires the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

# This will be the socket object that will contain other 
# connected sockets.
socketSet = chilkat.CkSocket()

# Before we begin, create a few StringArray objects holding
# domain names, URLs, and output filenames to be used
# in this example.
saDomains = chilkat.CkStringArray()
saUrls = chilkat.CkStringArray()
saFilenames = chilkat.CkStringArray()

saDomains.Append("www.chilkatsoft.com")
saDomains.Append("www.cknotes.com")
saDomains.Append("www.google.com")
saDomains.Append("www.example-code.com")
saDomains.Append("www.yahoo.com")

saUrls.Append("http://www.chilkatsoft.com/")
saUrls.Append("http://www.cknotes.com/")
saUrls.Append("http://www.google.com/")
saUrls.Append("http://www.example-code.com/")
saUrls.Append("http://www.yahoo.com/")

saFilenames.Append("chilkatsoft.out")
saFilenames.Append("cknotes.out")
saFilenames.Append("google.out")
saFilenames.Append("example-code.out")
saFilenames.Append("yahoo.out")

numConnections = 5

# This example will connect to 5 different web servers,
# send an HTTP request to each, and read the HTTP response
# form each.  It will call the SelectForReading method to wait
# until data arrives from any of the connected sockets.

# NOTE: As you'll see later, communicating with HTTP servers
# is better left to the Chilkat HTTP component/library because
# of many HTTP protocol issues such as redirects, GZIP responses,
# chunked responses, etc.  We're only communicating with
# HTTP servers here as a convenient way of demonstrating
# the technique of waiting for data to arrive from a set of 
# connected sockets.

# First, connect to each of the web servers:

useSsl = False
maxWaitMs = 5000
port = 80

for i in range(0,numConnections):
    sock = chilkat.CkSocket()

    domain = saDomains.getString(i)

    success = sock.Connect(domain,port,useSsl,maxWaitMs)
    if (success != True):
        print(sock.lastErrorText())
        sys.exit()

    # Save a filename in the socket object's UserData.  We'll use this
    # later when saving the data read in the HTTP response.
    sock.put_UserData(saFilenames.getString(i))

    # Display the remote IP address of the connected socket:
    print("connected to " + sock.remoteIpAddress())

    # Build the HTTP GET request to be sent to the web server.
    getReq = sock.buildHttpGetRequest(saUrls.getString(i))

    # Send the HTTP GET request to the web server.
    success = sock.SendString(getReq)
    if (success != True):
        print(sock.lastErrorText())
        sys.exit()

    # Tell the socketSet object to take ownership of the connected socket.
    # This adds the connection to the internal set of sockets contained
    # within socketSet.   
    success = socketSet.TakeSocket(sock)
    if (success != True):
        # The TakeSocket method shouldn't fail.  The only reason it might
        # is if we try to give it an unconnected socket -- but we already 
        # know it's connected at this point..
        print(socketSet.lastErrorText())
        sys.exit()

# At this point we have 5 TCP/IP connections contained within the socketSet object.
# Each of the web servers have received a GET request and will be
# sending a response.  We now want to write a loop that waits for data
# to arrive from any of the 5 connections, reads incoming data, and
# saves the data to files.

# We'll use the StartTiming method and ElapsedSeconds property
# to wait a maximum of 10 seconds.  The reason we do this is because
# different web servers will behave differently w.r.t. idle connections.
# Some web servers close the connections quickly while others do not.
# The socket component is not "HTTP aware", meaning that it is not parsing
# the HTTP response and therefore does not know when it has received
# the full response.  Because of this, we might receive the full response
# and then wait a long time before the server decides to close its side of
# the connection.  
# The ElapsedSeconds property will return the number of seconds that
# have elapsed since the last call to StartTiming (on the same object instance).
socketSet.StartTiming()

while ((socketSet.get_NumSocketsInSet() > 0) and (socketSet.get_ElapsedSeconds() < 10)) :

    maxWaitMs = 300

    # Wait for data to arrive on any of the sockets.  Waits a maximum
    # of 300 milliseconds.  If maxWaitMs = 0, then it is effectively a poll.
    # If no sockets have data available for reading, then a value of 0 is 
    # returned.  A value of -1 indicates an error condition.
    # Note: when the remote peer (in this case the web server) disconnects,
    # the socket will appear as if it has data available.  A "ready" socket
    # is one where either data is available for reading or the socket has
    # become disconnected.
    numReady = socketSet.SelectForReading(maxWaitMs)
    if (numReady < 0):
        print(socketSet.lastErrorText())
        sys.exit()

    # Consume data from each of the ready sockets, and show those
    # that become disconnected.   
    for i in range(0,numReady):

        # To make method calls on the "ready" socket, or to get/set property
        # values, set the SelectorReadIndex.  
        socketSet.put_SelectorReadIndex(i)

        # Print information that identifies the socket that is ready for reading:
        print("ready: IP address = " + socketSet.remoteIpAddress() + " filename = " + socketSet.userData())

        # Read the selected socket and send the data directly to a file (appending
        # to the file).
        success = socketSet.ReceiveBytesToFile(socketSet.userData())

        # The success/failure of a socket read should normally be checked.
        # In this case it is not necessary.  If the socket became ready for
        # reading because the remote peer disconnected and there is no
        # data available, then nothing will be read.  The socketSet object
        # will automatically remove a disconnected socket from its internal set of sockets
        # the next time SelectForReading or SelectForWriting is called.
        # We can check to see if the socket is still connected by examining
        # the IsConnected property:
        if (socketSet.get_IsConnected() != True):
            print("disconnected: " + socketSet.userData())

    # Set the SelectorReadIndex = -1 so that method calls
    # and property gets/sets are not routed to one of the socketSet's
    # contained sockets.  This is necessary because we want to
    # check the number of elapsed seconds from when we called
    # StartTiming.  
    socketSet.put_SelectorReadIndex(-1)

print("Finished.")

# Some final notes:
# If you examine the output files, you'll see that the HTTP protocol is a little
# more complex than you may have originally thought.
# 1) HTTP responses are MIME -- there is a MIME header followed by a MIME body.
#     In most cases, the MIME body is the HTML document itself.
# 2) Some HTTP responses return gzip compressed data (which is binary).  It must
#      be decompressed to get the HTML.
# 3) Some HTTP responses arrive in "chunked" form.  You'll see lines containing nothing but
#      a hexidecimal number indicating the size of the chunk to follow.  It ends with
#      a 0-sized chunk.
# 4) You'll find that some HTTP responses might be redirects to other URL's (such as a 302 redirect).

# For the above reasons, and many more, you would typically use the Chilkat HTTP component
# to handle the complexities of HTTP.  

# The reason HTTP was chosen for this example is because:
# 1) It is easy to establish multiple connections with existing servers that are not typically not blocked by firewalls.
# 2) It is easy to send a request and receive a response.
# 3) There is variability  in the amount of time before a server decides to disconnect from an idle client