(PureBasic) Stream a Large ZIP Entry While Uncompressing Using ZipEntry.UnzipToStreamAsync

See more Zip Examples

This example demonstrates how to use ZipEntry.UnzipToStreamAsync to asynchronously unzip a ZIP entry directly to a stream while simultaneously reading the uncompressed data from the foreground thread.

The example is especially useful for large ZIP entries because the entire uncompressed file does not need to be held in memory at once. Instead, the uncompressed data is processed incrementally in chunks as it becomes available.

The example performs the following steps:

• Opens an existing ZIP archive
• Gets a ZIP entry
• Starts an asynchronous unzip operation to a Stream
• Reads the uncompressed data chunk-by-chunk from the stream
• Writes each chunk directly to an output file

This approach is useful when:

• Processing very large ZIP entries
• Avoiding large memory allocations
• Streaming uncompressed data to another destination
• Performing incremental processing while decompression occurs

The example demonstrates a producer/consumer pattern where:

• The background thread inflates the ZIP entry into the Stream
• The foreground thread reads the uncompressed bytes from the Stream as they become available

The uncompressed bytes are written incrementally to:

qa_output/out.dat

Note: The standard Zip.Unzip* methods that extract directly to filesystem files already perform decompression in streaming mode internally within Chilkat. Chilkat does not inflate the entire file into memory before writing the output file.

Therefore, it is not necessary to use a more complex streaming approach such as this example merely to efficiently unzip large files to disk.

The purpose of this example is instead to demonstrate how an application can directly access and process the uncompressed data incrementally in chunks as the unzip operation is occurring.

This technique is ideal when the application needs to inspect, analyze, transform, transmit, or otherwise process the uncompressed bytes while they are being produced.

Note: This example requires Chilkat v11.0.0 or greater.

Chilkat PureBasic Module Download Chilkat PureBasic Module

IncludeFile "CkBinData.pb"
IncludeFile "CkFileAccess.pb"
IncludeFile "CkZip.pb"
IncludeFile "CkStream.pb"
IncludeFile "CkTask.pb"
IncludeFile "CkZipEntry.pb"

Procedure ChilkatExample()

    success.i = 0

    success = 0

    zip.i = CkZip::ckCreate()
    If zip.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    ; Open an existing ZIP archive.
    success = CkZip::ckOpenZip(zip,"qa_data/zips/big.zip")
    If success = 0
        Debug CkZip::ckLastErrorText(zip)
        CkZip::ckDispose(zip)
        ProcedureReturn
    EndIf

    ; Get the ZIP entry to be unzipped.
    ; 
    ; This example uses the first entry in the ZIP archive.
    ; The entry may contain a large file, so we will unzip it
    ; to a stream instead of loading the entire file into memory.
    entry.i = CkZipEntry::ckCreate()
    If entry.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    success = CkZip::ckEntryAt(zip,0,entry)
    If success = 0
        Debug CkZip::ckLastErrorText(zip)
        CkZip::ckDispose(zip)
        CkZipEntry::ckDispose(entry)
        ProcedureReturn
    EndIf

    ; Create the stream that will receive the uncompressed data.
    dataStream.i = CkStream::ckCreate()
    If dataStream.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    ; Start an asynchronous unzip operation.
    ; 
    ; UnzipToStreamAsync writes the uncompressed entry data to dataStream
    ; from a background thread. The foreground thread can read from the
    ; stream as data becomes available.
    unzipTask.i = CkZipEntry::ckUnzipToStreamAsync(entry,dataStream)
    If CkZipEntry::ckLastMethodSuccess(entry) = 0
        Debug CkZipEntry::ckLastErrorText(entry)
        CkZip::ckDispose(zip)
        CkZipEntry::ckDispose(entry)
        CkStream::ckDispose(dataStream)
        ProcedureReturn
    EndIf

    ; Start the background unzip thread.
    CkTask::ckRun(unzipTask)

    ; Open the output file that will receive the uncompressed data.
    fac.i = CkFileAccess::ckCreate()
    If fac.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    success = CkFileAccess::ckOpenForWrite(fac,"c:/temp/qa_output/out.dat")
    If success = 0
        Debug CkFileAccess::ckLastErrorText(fac)
        CkZip::ckDispose(zip)
        CkZipEntry::ckDispose(entry)
        CkStream::ckDispose(dataStream)
        CkFileAccess::ckDispose(fac)
        ProcedureReturn
    EndIf

    ; Read the uncompressed data from the stream in chunks.
    ; 
    ; This avoids holding the entire uncompressed file in memory.
    ; Each chunk is written immediately to the output file.
    bd.i = CkBinData::ckCreate()
    If bd.i = 0
        Debug "Failed to create object."
        ProcedureReturn
    EndIf

    While (CkStream::ckEndOfStream(dataStream) <> 1)

        ; Read the next available chunk of uncompressed bytes.
        CkStream::ckReadBd(dataStream,bd)

        ; Write this chunk to the output file.
        CkFileAccess::ckFileWriteBd(fac,bd,0,0)

        ; Clear the BinData object so it can be reused for the next chunk.
        CkBinData::ckClear(bd)
    Wend

    ; Close the output file.
    CkFileAccess::ckFileClose(fac)

    ; The background unzip task has completed when the stream reaches EOF.
    ; Delete the task object when no longer needed.
    CkTask::ckDispose(unzipTask)

    ; Close the ZIP archive.
    CkZip::ckCloseZip(zip)

    Debug "Success."


    CkZip::ckDispose(zip)
    CkZipEntry::ckDispose(entry)
    CkStream::ckDispose(dataStream)
    CkFileAccess::ckDispose(fac)
    CkBinData::ckDispose(bd)


    ProcedureReturn
EndProcedure