(Unicode C) Setting a Maximum Response Size
The MaxResponseSize property protects your spider from downloading a page that is too large. By default, MaxResponseSize = 0, which indicates that there is no maximum. You may set it to a number indicating the maximum number of bytes to download. URLs with response sizes larger than this will be skipped.
#include <C_CkSpiderW.h>
void ChilkatSample(void)
{
HCkSpiderW spider;
spider = CkSpiderW_Create();
CkSpiderW_Initialize(spider,L"www.chilkatsoft.com");
// Add the 1st URL:
CkSpiderW_AddUnspidered(spider,L"http://www.chilkatsoft.com/");
// This example demonstrates setting the MaxResponseSize property
// Do not download anything with a response size greater than 100,000 bytes.
CkSpiderW_putMaxResponseSize(spider,100000);
CkSpiderW_Dispose(spider);
}
|