Internet Resources Discovery (IRD)

1. Internet Resources Discovery (IRD) Shopping Agents T.Sharon-A.Frank

2. ShopBot • ShopBot is a softbot that carries out comparison shopping at Web vendors on a person’s behalf. • It autonomously learns to extract product information from the Web vendors. • ShopBot learns how to query a store’s searchable product catalog. T.Sharon-A.Frank

3. Example Search Form T.Sharon-A.Frank

4. Shopbot Assumptions • Navigation simplicity • virtual stores have simple - user-friendly interfaces. • Uniformity • product lists have similar structure. • Separation tenure • description fields in a list of products will be separated • every product will have a separate line. T.Sharon-A.Frank

5. ShopBot Implementation • ShopBot consists of two major components: • Learner: • Gets as input the domain description and site URL. • Creates the vendor description for use byShopper. • Moderately time-consuming; works off-line. • Shopper: • On-line agent, shops for requested products using vendors descriptions created by Learner. T.Sharon-A.Frank

6. ShopBot Learner for each vendor search for indices for each potential index for each sample product query on attributes accumulate responses analyze Domain Description URLs of possible vendors ShopBot Learner Online vendors Vendor Description T.Sharon-A.Frank

7. Learner Domain Description • Example products: P1, P2, …, Pn • Attributes of the products:manufacturer(P1) = “Digital Village”, name(P1) = “Starship Titanic”, etc… • Shopbot learner uses domain-specific heuristics. T.Sharon-A.Frank

8. Learner Vendor Description • URL of a searchable index form. • A function that maps product attributes to the form fields. • Functions for extracting data from pages returned: • A function that recognizes failure pages (“product not found”). • A function that strips header and trailer from successful pages. • A function that extracts a set of product descriptions from the remaining text on a successful page. T.Sharon-A.Frank

9. Learner Strategy • Identify an appropriate search form. • Finds all forms, discards inappropriate forms using simple heuristics. • Determine how to fill in the forms. • Identify product description formats: • Fills in the form with dummy product to identify failure page. • Identifies header, trailer and the body for success page. • Parse body to a number of product descriptions using domain-specific heuristics. T.Sharon-A.Frank

10. ShopBot Buyer get user request for each vendor go to index fill in form parse results sort display Domain Description VendorDescription ShopBot Buyer Online Vendors GUI T.Sharon-A.Frank

11. 1. The Client: --Submits “that book by Bill Gates” 4. The Client: --Queries sources --Filters results --Presents reports The Jango Architecture http://www.jango.com 2. Jango Server: --interprets query Netbot Server 3. Multiple Sources Queried Simultaneously--Amazon.com, --American Library Association... Client T.Sharon-A.Frank

12. Jango - Example T.Sharon-A.Frank

13. Jango -- example T.Sharon-A.Frank

14. Jango -- example T.Sharon-A.Frank

15. Jango Shopping Agent T.Sharon-A.Frank

16. Shopping Results T.Sharon-A.Frank

17. Another Shopping Example T.Sharon-A.Frank

18. Softbots - Summary 1. A meta-service that leverages existing services and collates their results. 2. It enables a human user to state what he or she wants to accomplish. 3. It attempts to disambiguate the request and to dynamically determine how and where to satisfy it. 4. It utilizes automatic planning technology to dynamically generate the results T.Sharon-A.Frank