(SQL Server) Duplicate OpensSSL to Encrypt a Signed File

This example duplicates the following:

openssl smime -encrypt –in SIGNED.P7M –outform der –binary –des3 -out ENCRYPTED.ENC OTHERPARTYCERTIFICATE.PEM

Note: Although "smime" is the OpenSSL command, it's not actually producing S/MIME. The arguments "-outform der -binary" indicates that the output is binary DER (i.e. the PKCS7 binary signature). The input file (in this case) is a .p7m opaque signature, but it can be any type of file. There is nothing special about the fact that the input is .p7m. It is treated exactly the same as any other type of file. From the standpoint of encryption, the input file is just a bunch of bytes that needs to be encrypted and emitted to a PKCS7 binary DER output.

Chilkat ActiveX Downloads ActiveX for 32-bit and 64-bit Windows

-- Important: See this note about string length limitations for strings returned by sp_OAMethod calls.
--
CREATE PROCEDURE ChilkatSample
AS
BEGIN
    DECLARE @hr int
    -- Important: Do not use nvarchar(max).  See the warning about using nvarchar(max).
    DECLARE @sTmp0 nvarchar(4000)
    -- This example requires the Chilkat API to have been previously unlocked.
    -- See Global Unlock Sample for sample code.

    DECLARE @crypt int
    -- Use "Chilkat_9_5_0.Crypt2" for versions of Chilkat < 10.0.0
    EXEC @hr = sp_OACreate 'Chilkat.Crypt2', @crypt OUT
    IF @hr <> 0
    BEGIN
        PRINT 'Failed to create ActiveX component'
        RETURN
    END

    -- PKCS7 encryption only requires the certificate (which contains the public key).
    -- Load the certificate..

    -- The cert.LoadFromFile method detects the file format.  You may load any type of file
    -- containing a certificate..
    DECLARE @cert int
    -- Use "Chilkat_9_5_0.Cert" for versions of Chilkat < 10.0.0
    EXEC @hr = sp_OACreate 'Chilkat.Cert', @cert OUT

    DECLARE @success int
    EXEC sp_OAMethod @cert, 'LoadFromFile', @success OUT, 'qa_data/certs/otherPartyCert.pem'
    IF @success <> 1
      BEGIN
        EXEC sp_OAGetProperty @cert, 'LastErrorText', @sTmp0 OUT
        PRINT @sTmp0
        EXEC @hr = sp_OADestroy @crypt
        EXEC @hr = sp_OADestroy @cert
        RETURN
      END

    -- Indicate that we want Public Key Infrastructure (PKI) encryption.
    EXEC sp_OASetProperty @crypt, 'CryptAlgorithm', 'pki'

    -- The desired encryption is "-des3", so choose "3des" for the underlying PKCS7 symmetric encryption:
    EXEC sp_OASetProperty @crypt, 'Pkcs7CryptAlg', '3des'

    -- Indicate the cert/public key to use for encryption:
    EXEC sp_OAMethod @crypt, 'SetEncryptCert', @success OUT, @cert
    IF @success <> 1
      BEGIN
        EXEC sp_OAGetProperty @crypt, 'LastErrorText', @sTmp0 OUT
        PRINT @sTmp0
        EXEC @hr = sp_OADestroy @crypt
        EXEC @hr = sp_OADestroy @cert
        RETURN
      END

    -- Call a crypt.Encrypt* methods to encrypt to/from memory sources (i.e. strings or bytes)
    -- or.. call CkEncryptFile to do file-to-file encryption.
    -- In this case the file to be encrypted is "signed.p7m", but it could be any type of file..
    EXEC sp_OAMethod @crypt, 'CkEncryptFile', @success OUT, 'somedir/signed.p7m', 'qa_output/encrypted.enc'
    IF @success <> 1
      BEGIN
        EXEC sp_OAGetProperty @crypt, 'LastErrorText', @sTmp0 OUT
        PRINT @sTmp0
        EXEC @hr = sp_OADestroy @crypt
        EXEC @hr = sp_OADestroy @cert
        RETURN
      END


    PRINT 'Success.'

    EXEC @hr = sp_OADestroy @crypt
    EXEC @hr = sp_OADestroy @cert


END
GO