Zig

AWS S3 File Streaming Upload

See more Amazon S3 (new) Examples

Demonstrates how to do a streaming upload from a file to the AWS S3 storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Zig

const std = @import("std");
const chilkat = @import("chilkat");

pub fn main(init: std.process.Init) !void {
    const alloc = init.arena.allocator();

    // This example requires the Chilkat API to have been previously unlocked.
    // See Global Unlock Sample for sample code.

    // This first part is optional.  AWS authentication requires
    // the SHA-256 hash of the request body (i.e. the contents of the file
    // to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
    // and then provide it to the authenticator object.  This way, if the file is
    // extremely large, it never needs to completely reside in memory.
    const crypt = try chilkat.Crypt2.init();
    defer crypt.deinit();
    crypt.setEncodingMode("hex");
    crypt.setHashAlgorithm("sha-256");
    const file_to_upload_path = "qa_data/xml/hamlet.xml";
    const hash_str = try crypt.hashFileENC(alloc, file_to_upload_path);

    const rest = try chilkat.Rest.init();
    defer rest.deinit();

    // Connect to the Amazon AWS REST server.
    const b_tls = true;
    const port = 443;
    const b_auto_reconnect = true;
    try rest.connect("s3.amazonaws.com", port, b_tls, b_auto_reconnect);

    // ----------------------------------------------------------------------------
    // Important: For buckets created in regions outside us-east-1,
    // there are three important changes that need to be made.
    // See Working with S3 Buckets in Non-us-east-1 Regions for the details.
    // ----------------------------------------------------------------------------

    // Provide AWS credentials for the REST call.
    const auth_aws = try chilkat.AuthAws.init();
    defer auth_aws.deinit();
    auth_aws.setAccessKey("AWS_ACCESS_KEY");
    auth_aws.setSecretKey("AWS_SECRET_KEY");
    auth_aws.setServiceName("s3");
    // Provide the pre-computed SHA-256 here:
    auth_aws.setPrecomputedSha256(hash_str);

    try rest.setAuthAws(auth_aws);

    // Set the bucket name via the HOST header.
    // In this case, the bucket name is "chilkat100".
    rest.setHost("chilkat100.s3.amazonaws.com");

    const file_stream = try chilkat.Stream.init();
    defer file_stream.deinit();
    file_stream.setSourceFile(file_to_upload_path);

    // Upload to the S3 Storage service.
    // If the application provided the SHA-256 hash of the file contents (as shown above)
    // then file is streamed and never has to completely reside in memory.
    // If the application did NOT provide the SHA-256, then Chilkat will (internally)
    // load the entire file into memory, calculate the SHA-256, and then upload.
    const response_str = rest.fullRequestStream(alloc, "PUT", "/hamlet.xml", file_stream) catch {
        std.debug.print("{s}\n", .{try rest.getLastErrorText(alloc)});
        return;
    };

    // When successful, the S3 Storage service will respond with a 200 response code,
    // with an XML body.
    if (rest.getResponseStatusCode() == 200) {
        std.debug.print("{s}\n", .{response_str});
        std.debug.print("File uploaded.\n", .{});
    } else {
        // Examine the request/response to see what happened.
        std.debug.print("response status code = {d}\n", .{rest.getResponseStatusCode()});
        std.debug.print("response status text = {s}\n", .{try rest.getResponseStatusText(alloc)});
        std.debug.print("response header: {s}\n", .{try rest.getResponseHeader(alloc)});
        std.debug.print("response body: {s}\n", .{response_str});
        std.debug.print("---\n", .{});
        std.debug.print("LastRequestStartLine: {s}\n", .{try rest.getLastRequestStartLine(alloc)});
        std.debug.print("LastRequestHeader: {s}\n", .{try rest.getLastRequestHeader(alloc)});
    }
}