(Go) Wasabi File Streaming Upload

See more Wasabi Examples

Demonstrates how to do a streaming upload from a file to the Wasabi storage service. The AWS authorization presents some difficulties when the REST request body is to be streamed from a file (or from some other source). The issue is that the SHA-256 hash of the file data must be calculated. There are only two possible ways to do this: (1) stream the file into memory in its entirety and calculate the SHA-256 hash prior to uploading, or (2) pre-calculate the SHA-256 in a streaming fashion, and then provide it to the AWS authentication object.

If the application does NOT pre-compute the SHA-256, then Chilkat (internally) is forced to stream into memory, calculate the SHA-256, and then upload from the in-memory copy of the file.

Chilkat Go Downloads Go Package for Windows, MacOS, Linux, Alpine Linux, Solaris

    // This example requires the Chilkat API to have been previously unlocked.
    // See Global Unlock Sample for sample code.

    // This first part is optional.  AWS authentication requires
    // the SHA-256 hash of the request body (i.e. the contents of the file
    // to be uploaded).  We can choose to pre-calculate the SHA-256 in a streaming fashion
    // and then provide it to the authenticator object.  This way, if the file is
    // extremely large, it never needs to completely reside in memory.
    crypt := chilkat.NewCrypt2()
    // ----------------------------------------------------
    // Important: Wasabi requires a lowercase hex format.
    // ----------------------------------------------------
    crypt.SetEncodingMode("hex-lower")
    crypt.SetHashAlgorithm("sha-256")
    fileToUploadPath := "qa_data/xml/hamlet.xml"
    hashStr := crypt.HashFileENC(fileToUploadPath)

    rest := chilkat.NewRest()

    // Connect to the Amazon AWS REST server.
    bTls := true
    port := 443
    bAutoReconnect := true
    // This example will demonstrate uploading to a bucket in the eu-west-2 region
    success := rest.Connect("s3.eu-west-2.wasabisys.com",port,bTls,bAutoReconnect)

    // Provide AWS credentials for the REST call.
    authAws := chilkat.NewAuthAws()
    authAws.SetAccessKey("access-key")
    authAws.SetSecretKey("secret-key")
    authAws.SetServiceName("s3")
    // Provide the pre-computed SHA-256 here:
    authAws.SetPrecomputedSha256(*hashStr)

    success = rest.SetAuthAws(authAws)

    rest.AddHeader("Expect","100-continue")
    rest.AddHeader("Content-Type","application/xml")

    // Set the bucket name via the HOST header.
    // In this case, the bucket name is "chilkat100".
    rest.SetHost("chilkat100.s3.eu-west-2.wasabisys.com")

    fileStream := chilkat.NewStream()
    fileStream.SetSourceFile(fileToUploadPath)

    // Upload to the Wasabi Storage service.
    // If the application provided the SHA-256 hash of the file contents (as shown above)
    // then file is streamed and never has to completely reside in memory.
    // If the application did NOT provide the SHA-256, then Chilkat will (internally) 
    // load the entire file into memory, calculate the SHA-256, and then upload.
    responseStr := rest.FullRequestStream("PUT","/hamlet.xml",fileStream)
    if rest.LastMethodSuccess() != true {
        fmt.Println(rest.LastErrorText())
        crypt.DisposeCrypt2()
        rest.DisposeRest()
        authAws.DisposeAuthAws()
        fileStream.DisposeStream()
        return
    }

    // When successful, the Wasabi Storage service will respond with a 200 response code,
    // with an XML body.  
    if rest.ResponseStatusCode() == 200 {
        fmt.Println(*responseStr)
        fmt.Println("File uploaded.")
    } else {
        // Examine the request/response to see what happened.
        fmt.Println("response status code = ", rest.ResponseStatusCode())
        fmt.Println("response status text = ", rest.ResponseStatusText())
        fmt.Println("response header: ", rest.ResponseHeader())
        fmt.Println("response body: ", *responseStr)
        fmt.Println("---")
        fmt.Println("LastRequestStartLine: ", rest.LastRequestStartLine())
        fmt.Println("LastRequestHeader: ", rest.LastRequestHeader())
    }


    crypt.DisposeCrypt2()
    rest.DisposeRest()
    authAws.DisposeAuthAws()
    fileStream.DisposeStream()