SQL Server

Setting a Maximum Response Size

See more Spider Examples

The MaxResponseSize property protects your spider from downloading a page that is too large. By default, MaxResponseSize = 0, which indicates that there is no maximum. You may set it to a number indicating the maximum number of bytes to download. URLs with response sizes larger than this will be skipped. SQL Server

-- Important: See this note about string length limitations for strings returned by sp_OAMethod calls.
--
CREATE PROCEDURE ChilkatSample
AS
BEGIN
    DECLARE @hr int
    DECLARE @spider int
    EXEC @hr = sp_OACreate 'Chilkat.Spider', @spider OUT
    IF @hr <> 0
    BEGIN
        PRINT 'Failed to create ActiveX component'
        RETURN
    END

    EXEC sp_OAMethod @spider, 'Initialize', NULL, 'www.chilkatsoft.com'

    -- Add the 1st URL:
    EXEC sp_OAMethod @spider, 'AddUnspidered', NULL, 'http://www.chilkatsoft.com/'

    -- This example demonstrates setting the MaxResponseSize property
    -- Do not download anything with a response size greater than 100,000 bytes.
    EXEC sp_OASetProperty @spider, 'MaxResponseSize', 100000

    EXEC @hr = sp_OADestroy @spider


END
GO