(C#) AWS S3 File Streaming Upload

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Chilkat .NET Downloads Chilkat .NET Framework Chilkat for .NET Core

// This example requires the Chilkat API to have been previously unlocked.
// See Global Unlock Sample for sample code.

// This first part is optional.  AWS authentication requires
// the SHA-256 hash of the request body (i.e. the contents of the file
// to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
// and then provide it to the authenticator object.  This way, if the file is
// extremely large, it never needs to completely reside in memory.
Chilkat.Crypt2 crypt = new Chilkat.Crypt2();
crypt.EncodingMode = "hex";
crypt.HashAlgorithm = "sha-256";
string fileToUploadPath = "qa_data/xml/hamlet.xml";
string hashStr = crypt.HashFileENC(fileToUploadPath);

Chilkat.Rest rest = new Chilkat.Rest();

// Connect to the Amazon AWS REST server.
bool bTls = true;
int port = 443;
bool bAutoReconnect = true;
bool success = rest.Connect("s3.amazonaws.com",port,bTls,bAutoReconnect);

// ----------------------------------------------------------------------------
// Important: For buckets created in regions outside us-east-1,
// there are three important changes that need to be made.
// See Working with S3 Buckets in Non-us-east-1 Regions for the details.
// ----------------------------------------------------------------------------

// Provide AWS credentials for the REST call.
Chilkat.AuthAws authAws = new Chilkat.AuthAws();
authAws.AccessKey = "AWS_ACCESS_KEY";
authAws.SecretKey = "AWS_SECRET_KEY";
authAws.ServiceName = "s3";
// Provide the pre-computed SHA-256 here:
authAws.PrecomputedSha256 = hashStr;

success = rest.SetAuthAws(authAws);

// Set the bucket name via the HOST header.
// In this case, the bucket name is "chilkat100".
rest.Host = "chilkat100.s3.amazonaws.com";

Chilkat.Stream fileStream = new Chilkat.Stream();
fileStream.SourceFile = fileToUploadPath;

// Upload to the S3 Storage service.
// If the application provided the SHA-256 hash of the file contents (as shown above)
// then file is streamed and never has to completely reside in memory.
// If the application did NOT provide the SHA-256, then Chilkat will (internally) 
// load the entire file into memory, calculate the SHA-256, and then upload.
string responseStr = rest.FullRequestStream("PUT","/hamlet.xml",fileStream);
if (rest.LastMethodSuccess != true) {
    Debug.WriteLine(rest.LastErrorText);
    return;
}

// When successful, the S3 Storage service will respond with a 200 response code,
// with an XML body.  
if (rest.ResponseStatusCode == 200) {
    Debug.WriteLine(responseStr);
    Debug.WriteLine("File uploaded.");
}
else {
    // Examine the request/response to see what happened.
    Debug.WriteLine("response status code = " + Convert.ToString(rest.ResponseStatusCode));
    Debug.WriteLine("response status text = " + rest.ResponseStatusText);
    Debug.WriteLine("response header: " + rest.ResponseHeader);
    Debug.WriteLine("response body: " + responseStr);
    Debug.WriteLine("---");
    Debug.WriteLine("LastRequestStartLine: " + rest.LastRequestStartLine);
    Debug.WriteLine("LastRequestHeader: " + rest.LastRequestHeader);
}