Share Email Print

Proceedings Paper

Web object collection: here or there?
Author(s): Ruth Kurniawati; Jesse S. Jin; John A. Shepherd
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Although the advantages to performing the collection of web objects at the site where the objects reside seem obvious, most of the popular search engines use their own centralized or coarsely distributed collectors (robots). Sending each object to the collector is almost always the worst option with respect to resource usage. An alternative is to distribute the collection process by sending the collector to the source site, which has the obvious advantage of distributing the significant computational load involved in cataloguing, as well as giving opportunities for summarizing and compression. In this paper, we propose a system for distributed object cataloguing over the world wide web via lightweight collector agents. This approach differs to previous approaches such as Harvest in using small Java- based collectors that can be very easily deployed on the site being indexed, thus allowing much finer grained distribution of the collection task.

Paper Details

Date Published: 5 October 1998
PDF: 12 pages
Proc. SPIE 3527, Multimedia Storage and Archiving Systems III, (5 October 1998); doi: 10.1117/12.325835
Show Author Affiliations
Ruth Kurniawati, Univ. of New South Wales (Australia)
Jesse S. Jin, Univ. of New South Wales (Australia)
John A. Shepherd, Univ. of New South Wales (Australia)

Published in SPIE Proceedings Vol. 3527:
Multimedia Storage and Archiving Systems III
C.-C. Jay Kuo; Shih-Fu Chang; Sethuraman Panchanathan, Editor(s)

© SPIE. Terms of Use
Back to Top