(Android™) Crawling the Web

If the Chilkat Spider component only crawls a single site, how do you crawl the Web? The answer is simple: as you crawl a site, the spider collects outbound links and makes them accessible to you. You may then instantiate an instance of the Spider object for each site, and crawl it. The task of keeping track of what sites you've already crawled is left to you. This example retrieves the home page of http://www.joelonsoftware.com/ and displays the outbound links.

Chilkat Android™ Downloads Android™ Java Libraries Android C/C++ Libraries

// Important: Don't forget to include the call to System.loadLibrary
// as shown at the bottom of this code sample.
package com.test;

import android.app.Activity;
import com.chilkatsoft.*;

import android.widget.TextView;
import android.os.Bundle;

public class SimpleActivity extends Activity {

  private static final String TAG = "Chilkat";

  // Called when the activity is first created.
  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    CkSpider spider = new CkSpider();

    // The Initialize method may be called with just the domain name,
    // such as "www.joelonsoftware.com" or a full URL.  If you pass only
    // the domain name, you must add URLs to the unspidered list by calling
    // AddUnspidered.  Otherwise, the URL you pass to Initialize is the 1st
    // URL in the unspidered list.
    spider.Initialize("www.joelonsoftware.com");

    spider.AddUnspidered("http://www.joelonsoftware.com/");

    boolean success;
    success = spider.CrawlNext();

    int i;
    for (i = 0; i <= spider.get_NumOutboundLinks() - 1; i++) {
        Log.i(TAG, spider.getOutboundLink(i));
        }


  }

  static {
      System.loadLibrary("chilkat");

      // Note: If the incorrect library name is passed to System.loadLibrary,
      // then you will see the following error message at application startup:
      //"The application <your-application-name> has stopped unexpectedly. Please try again."
  }
}