Rust

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Rust

// This example requires the Chilkat API to have been previously unlocked.
// See Global Unlock Sample for sample code.

// This first part is optional.  AWS authentication requires
// the SHA-256 hash of the request body (i.e. the contents of the file
// to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
// and then provide it to the authenticator object.  This way, if the file is
// extremely large, it never needs to completely reside in memory.
let crypt = chilkat::Crypt2::new();
crypt.set_encoding_mode("hex");
crypt.set_hash_algorithm("sha-256");
let file_to_upload_path = "qa_data/xml/hamlet.xml".to_string();
let hash_str = crypt.hash_file_enc(&file_to_upload_path).unwrap_or_default();

let rest = chilkat::Rest::new();

// Connect to the Amazon AWS REST server.
let b_tls = true;
let port = 443;
let b_auto_reconnect = true;
let _ = rest.connect("s3.amazonaws.com", port, b_tls, b_auto_reconnect).is_ok();

// ----------------------------------------------------------------------------
// Important: For buckets created in regions outside us-east-1,
// there are three important changes that need to be made.
// See Working with S3 Buckets in Non-us-east-1 Regions for the details.
// ----------------------------------------------------------------------------

// Provide AWS credentials for the REST call.
let auth_aws = chilkat::AuthAws::new();
auth_aws.set_access_key("AWS_ACCESS_KEY");
auth_aws.set_secret_key("AWS_SECRET_KEY");
auth_aws.set_service_name("s3");
// Provide the pre-computed SHA-256 here:
auth_aws.set_precomputed_sha256(&hash_str);

let _ = rest.set_auth_aws(&auth_aws).is_ok();

// Set the bucket name via the HOST header.
// In this case, the bucket name is "chilkat100".
rest.set_host("chilkat100.s3.amazonaws.com");

let file_stream = chilkat::Stream::new();
file_stream.set_source_file(&file_to_upload_path);

// Upload to the S3 Storage service.
// If the application provided the SHA-256 hash of the file contents (as shown above)
// then file is streamed and never has to completely reside in memory.
// If the application did NOT provide the SHA-256, then Chilkat will (internally) 
// load the entire file into memory, calculate the SHA-256, and then upload.
let Ok(response_str) = rest.full_request_stream("PUT", "/hamlet.xml", &file_stream) else {
    println!("{}", rest.last_error_text());
    return;
};

// When successful, the S3 Storage service will respond with a 200 response code,
// with an XML body.  
if rest.response_status_code() == 200 {
    println!("{}", response_str);
    println!("File uploaded.");
} else {
    // Examine the request/response to see what happened.
    println!("response status code = {}", rest.response_status_code());
    println!("response status text = {}", rest.response_status_text());
    println!("response header: {}", rest.response_header());
    println!("response body: {}", response_str);
    println!("---");
    println!("LastRequestStartLine: {}", rest.last_request_start_line());
    println!("LastRequestHeader: {}", rest.last_request_header());
}