PowerShell

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

PowerShell

Add-Type -Path "C:\chilkat\ChilkatDotNet47-x64\ChilkatDotNet47.dll"

$success = $false

# This example requires the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

# This first part is optional.  AWS authentication requires
# the SHA-256 hash of the request body (i.e. the contents of the file
# to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
# and then provide it to the authenticator object.  This way, if the file is
# extremely large, it never needs to completely reside in memory.
$crypt = New-Object Chilkat.Crypt2
$crypt.EncodingMode = "hex"
$crypt.HashAlgorithm = "sha-256"
$fileToUploadPath = "qa_data/xml/hamlet.xml"
$hashStr = $crypt.HashFileENC($fileToUploadPath)

$rest = New-Object Chilkat.Rest

# Connect to the Amazon AWS REST server.
$bTls = $true
$port = 443
$bAutoReconnect = $true
$success = $rest.Connect("s3.amazonaws.com",$port,$bTls,$bAutoReconnect)

# ----------------------------------------------------------------------------
# Important: For buckets created in regions outside us-east-1,
# there are three important changes that need to be made.
# See Working with S3 Buckets in Non-us-east-1 Regions for the details.
# ----------------------------------------------------------------------------

# Provide AWS credentials for the REST call.
$authAws = New-Object Chilkat.AuthAws
$authAws.AccessKey = "AWS_ACCESS_KEY"
$authAws.SecretKey = "AWS_SECRET_KEY"
$authAws.ServiceName = "s3"
# Provide the pre-computed SHA-256 here:
$authAws.PrecomputedSha256 = $hashStr

$success = $rest.SetAuthAws($authAws)

# Set the bucket name via the HOST header.
# In this case, the bucket name is "chilkat100".
$rest.Host = "chilkat100.s3.amazonaws.com"

$fileStream = New-Object Chilkat.Stream
$fileStream.SourceFile = $fileToUploadPath

# Upload to the S3 Storage service.
# If the application provided the SHA-256 hash of the file contents (as shown above)
# then file is streamed and never has to completely reside in memory.
# If the application did NOT provide the SHA-256, then Chilkat will (internally) 
# load the entire file into memory, calculate the SHA-256, and then upload.
$responseStr = $rest.FullRequestStream("PUT","/hamlet.xml",$fileStream)
if ($rest.LastMethodSuccess -ne $true) {
    $($rest.LastErrorText)
    exit
}

# When successful, the S3 Storage service will respond with a 200 response code,
# with an XML body.  
if ($rest.ResponseStatusCode -eq 200) {
    $($responseStr)
    $("File uploaded.")
}
else {
    # Examine the request/response to see what happened.
    $("response status code = " + $rest.ResponseStatusCode)
    $("response status text = " + $rest.ResponseStatusText)
    $("response header: " + $rest.ResponseHeader)
    $("response body: " + $responseStr)
    $("---")
    $("LastRequestStartLine: " + $rest.LastRequestStartLine)
    $("LastRequestHeader: " + $rest.LastRequestHeader)
}