Chilkat Examples

ChilkatHOME.NET Core C#Android™AutoItCC#C++Chilkat2-PythonCkPythonClassic ASPDataFlexDelphi ActiveXDelphi DLLGoJavaLianjaMono C#Node.jsObjective-CPHP ActiveXPHP ExtensionPerlPowerBuilderPowerShellPureBasicRubySQL ServerSwift 2Swift 3,4,5...TclUnicode CUnicode C++VB.NETVBScriptVisual Basic 6.0Visual FoxProXojo Plugin

.NET Core C# Examples

Web API Categories

ASN.1
AWS KMS
AWS Misc
Amazon EC2
Amazon Glacier
Amazon S3
Amazon S3 (new)
Amazon SES
Amazon SNS
Amazon SQS
Async
Azure Cloud Storage
Azure Key Vault
Azure Service Bus
Azure Table Service
Base64
Bounced Email
Box
CAdES
CSR
CSV
Certificates
Cloud Signature CSC
Code Signing
Compression
DKIM / DomainKey
DNS
DSA
Diffie-Hellman
Digital Signatures
Dropbox
Dynamics CRM
EBICS
ECC
Ed25519
Email Object
Encryption
FTP
FileAccess
Firebase
GMail REST API
GMail SMTP/IMAP/POP
Geolocation
Google APIs
Google Calendar
Google Cloud SQL
Google Cloud Storage
Google Drive
Google Photos
Google Sheets
Google Tasks
Gzip
HTML-to-XML/Text
HTTP

HTTP Misc
IMAP
JSON
JSON Web Encryption (JWE)
JSON Web Signatures (JWS)
JSON Web Token (JWT)
Java KeyStore (JKS)
MHT / HTML Email
MIME
MS Storage Providers
Microsoft Graph
Misc
NTLM
OAuth1
OAuth2
OIDC
Office365
OneDrive
OpenSSL
Outlook
Outlook Calendar
Outlook Contact
PDF Signatures
PEM
PFX/P12
PKCS11
POP3
PRNG
REST
REST Misc
RSA
SCP
SCard
SFTP
SMTP
SSH
SSH Key
SSH Tunnel
ScMinidriver
SharePoint
SharePoint Online
Signing in the Cloud
Socket/SSL/TLS
Spider
Stream
Tar Archive
ULID/UUID
Upload
WebSocket
XAdES
XML
XML Digital Signatures
XMP
Zip
curl
uncategorized

 

 

 

(.NET Core C#) Crawling the Web

If the Chilkat Spider component only crawls a single site, how do you crawl the Web? The answer is simple: as you crawl a site, the spider collects outbound links and makes them accessible to you. You may then instantiate an instance of the Spider object for each site, and crawl it. The task of keeping track of what sites you've already crawled is left to you. This example retrieves the home page of http://www.joelonsoftware.com/ and displays the outbound links.

Chilkat .NET Downloads

Chilkat .NET Assemblies

Chilkat for .NET Core

Chilkat for Mono

Chilkat.Spider spider = new Chilkat.Spider();

// The Initialize method may be called with just the domain name,
// such as "www.joelonsoftware.com" or a full URL.  If you pass only
// the domain name, you must add URLs to the unspidered list by calling
// AddUnspidered.  Otherwise, the URL you pass to Initialize is the 1st
// URL in the unspidered list.
spider.Initialize("www.joelonsoftware.com");

spider.AddUnspidered("http://www.joelonsoftware.com/");

bool success;
success = spider.CrawlNext();

int i;
for (i = 0; i <= spider.NumOutboundLinks - 1; i++) {
    Debug.WriteLine(spider.GetOutboundLink(i));
}


 

© 2000-2024 Chilkat Software, Inc. All Rights Reserved.