PureBasic

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

PureBasic

IncludeFile "CkRest.pb"
IncludeFile "CkStream.pb"
IncludeFile "CkAuthAws.pb"
IncludeFile "CkCrypt2.pb"

Procedure ChilkatExample()

    success.i = 0

    ; This example requires the Chilkat API to have been previously unlocked.
    ; See Global Unlock Sample for sample code.

    ; This first part is optional.  AWS authentication requires
    ; the SHA-256 hash of the request body (i.e. the contents of the file
    ; to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
    ; and then provide it to the authenticator object.  This way, if the file is
    ; extremely large, it never needs to completely reside in memory.
    crypt.i = CkCrypt2::ckCreate()
    If crypt.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    CkCrypt2::setCkEncodingMode(crypt, "hex")
    CkCrypt2::setCkHashAlgorithm(crypt, "sha-256")
    fileToUploadPath.s = "qa_data/xml/hamlet.xml"
    hashStr.s = CkCrypt2::ckHashFileENC(crypt,fileToUploadPath)

    rest.i = CkRest::ckCreate()
    If rest.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    ; Connect to the Amazon AWS REST server.
    bTls.i = 1
    port.i = 443
    bAutoReconnect.i = 1
    success = CkRest::ckConnect(rest,"s3.amazonaws.com",port,bTls,bAutoReconnect)

    ; ----------------------------------------------------------------------------
    ; Important: For buckets created in regions outside us-east-1,
    ; there are three important changes that need to be made.
    ; See Working with S3 Buckets in Non-us-east-1 Regions for the details.
    ; ----------------------------------------------------------------------------

    ; Provide AWS credentials for the REST call.
    authAws.i = CkAuthAws::ckCreate()
    If authAws.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    CkAuthAws::setCkAccessKey(authAws, "AWS_ACCESS_KEY")
    CkAuthAws::setCkSecretKey(authAws, "AWS_SECRET_KEY")
    CkAuthAws::setCkServiceName(authAws, "s3")
    ; Provide the pre-computed SHA-256 here:
    CkAuthAws::setCkPrecomputedSha256(authAws, hashStr)

    success = CkRest::ckSetAuthAws(rest,authAws)

    ; Set the bucket name via the HOST header.
    ; In this case, the bucket name is "chilkat100".
    CkRest::setCkHost(rest, "chilkat100.s3.amazonaws.com")

    fileStream.i = CkStream::ckCreate()
    If fileStream.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    CkStream::setCkSourceFile(fileStream, fileToUploadPath)

    ; Upload to the S3 Storage service.
    ; If the application provided the SHA-256 hash of the file contents (as shown above)
    ; then file is streamed and never has to completely reside in memory.
    ; If the application did NOT provide the SHA-256, then Chilkat will (internally) 
    ; load the entire file into memory, calculate the SHA-256, and then upload.
    responseStr.s = CkRest::ckFullRequestStream(rest,"PUT","/hamlet.xml",fileStream)
    If CkRest::ckLastMethodSuccess(rest) <> 1
        Debug CkRest::ckLastErrorText(rest)
        CkCrypt2::ckDispose(crypt)
        CkRest::ckDispose(rest)
        CkAuthAws::ckDispose(authAws)
        CkStream::ckDispose(fileStream)
        ProcedureReturn
    EndIf

    ; When successful, the S3 Storage service will respond with a 200 response code,
    ; with an XML body.  
    If CkRest::ckResponseStatusCode(rest) = 200
        Debug responseStr
        Debug "File uploaded."
    Else
        ; Examine the request/response to see what happened.
        Debug "response status code = " + Str(CkRest::ckResponseStatusCode(rest))
        Debug "response status text = " + CkRest::ckResponseStatusText(rest)
        Debug "response header: " + CkRest::ckResponseHeader(rest)
        Debug "response body: " + responseStr
        Debug "---"
        Debug "LastRequestStartLine: " + CkRest::ckLastRequestStartLine(rest)
        Debug "LastRequestHeader: " + CkRest::ckLastRequestHeader(rest)
    EndIf



    CkCrypt2::ckDispose(crypt)
    CkRest::ckDispose(rest)
    CkAuthAws::ckDispose(authAws)
    CkStream::ckDispose(fileStream)


    ProcedureReturn
EndProcedure