How would you program a large comparison shopping website?

[hwarden]

This is a question for those of you who know SEO alongside programming.

I am trying to get my head around technical issues from development point of view, your feedback will be highly appreciated.

Let's say there is a large product comparison site I want to build running off merchant APIs to display products/comparison within my site. Assume all pages created on my site will be based on the products available within the merchant feed.

Q1. What will happen when I refresh my merchant feed, I know that if new products are found in merchant API then new pages will be created on my site. What will happen to the older items as those pages on my site would have been indexed by now but the merchant does not offer the product anymore? Are we going to delete those older product pages/URLs?

Q2. If we are going to keep the pages, then would it not explode the size of the sitemap at some point, as bear in mind that I intend to make a site comparing thousands of products and chances are that hundreds of products might go missing from API on a weekly basis.

Would appreciate feedback from anyone who has practically dealt with such issues in the past.

Thanks.

 

[hwarden]

I have been waiting to see any experienced webmaster to answer this question. Your response will be highly appreciated.

 

[hwarden]

I would have highly appreciated a reply form anyone experienced with tweaking large sites. I have made some discoveries such as using dynamic sitemaps with sitemap type defined as 'index' but I am still not sure what to with the bulk of products that might go missing on a weekly basis?