PowerBuilder

Setting a Maximum Response Size

See more Spider Examples

The MaxResponseSize property protects your spider from downloading a page that is too large. By default, MaxResponseSize = 0, which indicates that there is no maximum. You may set it to a number indicating the maximum number of bytes to download. URLs with response sizes larger than this will be skipped. PowerBuilder

integer li_rc
oleobject loo_Spider

loo_Spider = create oleobject
li_rc = loo_Spider.ConnectToNewObject("Chilkat.Spider")
if li_rc < 0 then
    destroy loo_Spider
    MessageBox("Error","Connecting to COM object failed")
    return
end if

loo_Spider.Initialize("www.chilkatsoft.com")

// Add the 1st URL:
loo_Spider.AddUnspidered("http://www.chilkatsoft.com/")

// This example demonstrates setting the MaxResponseSize property
// Do not download anything with a response size greater than 100,000 bytes.
loo_Spider.MaxResponseSize = 100000


destroy loo_Spider