DataFlex

Stream a Large ZIP Entry While Uncompressing Using ZipEntry.UnzipToStreamAsync

See more Zip Examples

This example demonstrates how to use ZipEntry.UnzipToStreamAsync to asynchronously unzip a ZIP entry directly to a stream while simultaneously reading the uncompressed data from the foreground thread.

The example is especially useful for large ZIP entries because the entire uncompressed file does not need to be held in memory at once. Instead, the uncompressed data is processed incrementally in chunks as it becomes available.

The example performs the following steps:

• Opens an existing ZIP archive
• Gets a ZIP entry
• Starts an asynchronous unzip operation to a Stream
• Reads the uncompressed data chunk-by-chunk from the stream
• Writes each chunk directly to an output file

This approach is useful when:

• Processing very large ZIP entries
• Avoiding large memory allocations
• Streaming uncompressed data to another destination
• Performing incremental processing while decompression occurs

The example demonstrates a producer/consumer pattern where:

• The background thread inflates the ZIP entry into the Stream
• The foreground thread reads the uncompressed bytes from the Stream as they become available

The uncompressed bytes are written incrementally to:

qa_output/out.dat

Note: The standard Zip.Unzip* methods that extract directly to filesystem files already perform decompression in streaming mode internally within Chilkat. Chilkat does not inflate the entire file into memory before writing the output file.

Therefore, it is not necessary to use a more complex streaming approach such as this example merely to efficiently unzip large files to disk.

The purpose of this example is instead to demonstrate how an application can directly access and process the uncompressed data incrementally in chunks as the unzip operation is occurring.

This technique is ideal when the application needs to inspect, analyze, transform, transmit, or otherwise process the uncompressed bytes while they are being produced.

DataFlex

Use ChilkatAx-win32.pkg

Procedure Test
    Boolean iSuccess
    Handle hoZip
    Variant vEntry
    Handle hoEntry
    Variant vDataStream
    Handle hoDataStream
    Variant vUnzipTask
    Handle hoUnzipTask
    Handle hoFac
    Variant vBd
    Handle hoBd
    String sTemp1
    Boolean bTemp1

    Move False To iSuccess

    Move False To iSuccess

    Get Create (RefClass(cComChilkatZip)) To hoZip
    If (Not(IsComObjectCreated(hoZip))) Begin
        Send CreateComObject of hoZip
    End

    // Open an existing ZIP archive.
    Get ComOpenZip Of hoZip "qa_data/zips/big.zip" To iSuccess
    If (iSuccess = False) Begin
        Get ComLastErrorText Of hoZip To sTemp1
        Showln sTemp1
        Procedure_Return
    End

    // Get the ZIP entry to be unzipped.
    // 
    // This example uses the first entry in the ZIP archive.
    // The entry may contain a large file, so we will unzip it
    // to a stream instead of loading the entire file into memory.
    Get Create (RefClass(cComChilkatZipEntry)) To hoEntry
    If (Not(IsComObjectCreated(hoEntry))) Begin
        Send CreateComObject of hoEntry
    End

    Get pvComObject of hoEntry to vEntry
    Get ComEntryAt Of hoZip 0 vEntry To iSuccess
    If (iSuccess = False) Begin
        Get ComLastErrorText Of hoZip To sTemp1
        Showln sTemp1
        Procedure_Return
    End

    // Create the stream that will receive the uncompressed data.
    Get Create (RefClass(cComChilkatStream)) To hoDataStream
    If (Not(IsComObjectCreated(hoDataStream))) Begin
        Send CreateComObject of hoDataStream
    End

    // Start an asynchronous unzip operation.
    // 
    // UnzipToStreamAsync writes the uncompressed entry data to dataStream
    // from a background thread. The foreground thread can read from the
    // stream as data becomes available.
    Get pvComObject of hoDataStream to vDataStream
    Get ComUnzipToStreamAsync Of hoEntry vDataStream To vUnzipTask
    If (IsComObject(vUnzipTask)) Begin
        Get Create (RefClass(cComChilkatTask)) To hoUnzipTask
        Set pvComObject Of hoUnzipTask To vUnzipTask
    End
    Get ComLastMethodSuccess Of hoEntry To bTemp1
    If (bTemp1 = False) Begin
        Get ComLastErrorText Of hoEntry To sTemp1
        Showln sTemp1
        Procedure_Return
    End

    // Start the background unzip thread.
    Get ComRun Of hoUnzipTask To iSuccess

    // Open the output file that will receive the uncompressed data.
    Get Create (RefClass(cComCkFileAccess)) To hoFac
    If (Not(IsComObjectCreated(hoFac))) Begin
        Send CreateComObject of hoFac
    End

    Get ComOpenForWrite Of hoFac "c:/temp/qa_output/out.dat" To iSuccess
    If (iSuccess = False) Begin
        Get ComLastErrorText Of hoFac To sTemp1
        Showln sTemp1
        Procedure_Return
    End

    // Read the uncompressed data from the stream in chunks.
    // 
    // This avoids holding the entire uncompressed file in memory.
    // Each chunk is written immediately to the output file.
    Get Create (RefClass(cComChilkatBinData)) To hoBd
    If (Not(IsComObjectCreated(hoBd))) Begin
        Send CreateComObject of hoBd
    End

    While ((ComEndOfStream(hoDataStream)) <> True)

        // Read the next available chunk of uncompressed bytes.
        Get pvComObject of hoBd to vBd
        Get ComReadBd Of hoDataStream vBd To iSuccess

        // Write this chunk to the output file.
        Get pvComObject of hoBd to vBd
        Get ComFileWriteBd Of hoFac vBd 0 0 To iSuccess

        // Clear the BinData object so it can be reused for the next chunk.
        Get ComClear Of hoBd To iSuccess
    Loop

    // Close the output file.
    Send ComFileClose To hoFac

    // The background unzip task has completed when the stream reaches EOF.
    // Delete the task object when no longer needed.
    Send Destroy of hoUnzipTask

    // Close the ZIP archive.
    Send ComCloseZip To hoZip

    Showln "Success."


End_Procedure