(Ruby) SFTP Upload Large File in Parts

See more SFTP Examples

Demonstrates how to split a large file into N parts and then upload each part separately. This technique can be used to potentially avoid limits a SFTP server might impose on upload file size.

Chilkat Ruby Downloads install from rubygems.org gem install chilkat or download... Ruby Library for Windows, MacOS, Linux, Alpine Linux

require 'chilkat'

# This example assumes the Chilkat API to have been previously unlocked.
# See Global Unlock Sample for sample code.

sftp = Chilkat::CkSFtp.new()

# Set some timeouts, in milliseconds:
sftp.put_ConnectTimeoutMs(5000)
sftp.put_IdleTimeoutMs(10000)

# Connect to the SSH server.  
# The standard SSH port = 22
# The hostname may be a hostname or IP address.
hostname = "sftp.example.com"
port = 22
success = sftp.Connect(hostname,port)
if (success != true)
    print sftp.lastErrorText() + "\n";
    exit
end

# Authenticate with the SSH server.  Chilkat SFTP supports
# both password-based authenication as well as public-key
# authentication.  This example uses password authenication.
success = sftp.AuthenticatePw("myLogin","myPassword")
if (success != true)
    print sftp.lastErrorText() + "\n";
    exit
end

# After authenticating, the SFTP subsystem must be initialized:
success = sftp.InitializeSftp()
if (success != true)
    print sftp.lastErrorText() + "\n";
    exit
end

# Open a remote file for writing on the SSH server.
# If the file already exists, it is overwritten.
# (Specify "createNew" instead of "createTruncate" to
# prevent overwriting existing files.)
handle = sftp.openFile("big.zip","writeOnly","createTruncate")
if (sftp.get_LastMethodSuccess() != true)
    print sftp.lastErrorText() + "\n";
    exit
end

# We're going to upload the local file "qa_data/zips/big.zip" (a relative local path from our current working directory)
# to the SFTP server.
localFilePath = "qa_data/zips/big.zip"

# Rather than uploading in one shot, we'll read the local file in chunks
# and upload a chunk at a time.
fac = Chilkat::CkFileAccess.new()

success = fac.OpenForRead(localFilePath)
if (success == false)
    sftp.CloseHandle(handle)
    print fac.lastErrorText() + "\n";
    exit
end

# If we were to split the local file into 128K chunks, how many chunks (blocks) would
# we have, including the last partial block?
blockSize = 131072
numBlocks = fac.GetNumBlocks(blockSize)
print "Number of blocks = " + numBlocks.to_s() + "\n";

bd = Chilkat::CkBinData.new()
i = 0
while i < numBlocks
    # Read the Nth block, which gets appended to the bd.  (which means
    # we must clear the contents of bd before the next iteration)
    success = fac.ReadBlockBd(i,blockSize,bd)
    if (success == false)
        sftp.CloseHandle(handle)
        print fac.lastErrorText() + "\n";
        exit
    end

    # Upload this block to the open remote file.
    success = sftp.WriteFileBd(handle,bd)
    if (success == false)
        print sftp.lastErrorText() + "\n";
        exit
    end

    # Clear the contents for the next iteration.
    bd.Clear()

    i = i + 1
end

# Close the local file.
fac.FileClose()

# -----------------------------------------------------------------------------
# This is important.  You must close the handle on the server.
# Otherwise open handles will accumulate on the server until eventually a limit
# is reached and the server will fail on a call to OpenFile.
# -----------------------------------------------------------------------------

# Close the remote file handle on the server.
success = sftp.CloseHandle(handle)
if (success != true)
    print sftp.lastErrorText() + "\n";
    exit
end

print "Success." + "\n";