“Adsterra”/ “AdsKeeper”/
Welcome to Our Community
Wanting to join the rest of our members? Feel free to sign up today.

Ask Me Anything Setting Up Data Feeds and Algorithms for ?

Discussion in 'Guides, Case Studies and Tutorials' started by Graybeard, Nov 20, 2020 at 7:19 AM.

Can you follow this

This poll will close on Jan 8, 2021 at 7:19 AM.
  1. WTF is this?

    0 vote(s)
    0.0%
  2. I want to make dymamic content with datafeeds and will learn.

    1 vote(s)
    100.0%
  3. When you develop a tool --email me!

    0 vote(s)
    0.0%
Multiple votes are allowed.
  1. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    I am setting up a 140K item datafeed from a shareasale vendor
    the objective is to create a MySQL database and an algorithm to find;
    • the best discounts from the MSRP (full list price) to the selling price
    • to find the newest product updates and their in stock status
    • to get set up for the ecommerce year end holiday selling season

    I am not going to be using tools and WordPress
    Is there any interest in this and learning how to code this and set it up for advertising media and web site use --or is it really too advanced?

    Last Saturday: apply to shareasale.com affiliate network --wait a few days for the approval
    Account Approved, sign up to tomtop.com affiliate program, download datafeed this Wednesday, study feed format, feed details

    Yesterday: make a new database and start to design the table for this 140K line datafeed
    this is what we are talking about;
    upload_2020-11-20_6-56-3.png

    how i got here
    upload_2020-11-20_7-2-2.png

    1. wc -L finds the longest line in the datafeed
    2. you make the first table entry with that line and let the 'tool' MySQL decide the field type i.e. varchar(33) and NOT NULL or NULL if there is no data
    the delimiter in the datafeed is | (pipe symbol)
    you need to escape the URLs and : = & encode to the MySQL does to freak out and error
    upload_2020-11-20_7-13-0.png
    create the first table (Mother of all Tables from hell) in the new database I made `share_a_sale` last night ...
    can anyone follow this and what i have done so far --or is it all advanced geek-speak to you?
     
    EuronGreyjoy likes this.
  2. Voluum
  3. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    All sorts of problems yesterday Uuugh !!!
    this was the biggest problem!
    upload_2020-11-21_8-20-4.png

    There were 'illegal' non utf8mb4 characters in the data feed and the loading of the datafeed kept failing for this reason initially.
    After hours for searching and reading all the complaints of this MySQL issue I finally read it is a bug that the MySQL developers have tried to gloss over for a long time ...
    --the 'illegal' non utf8mb4 were probably some Chinese characters and ^@ <null bit binary code is what is supposed to be an ANSI UTF file! NEVER DO THAT! I had to waste a lot of time fixing that error ...


    after hours of searching and failed experiments
    Viola! this fixed the issue
    Code:
    fix binary
    
      tr '[\000-\011\013-\037\177-\377]' '.' < tomtop-loadfile-2020-11-20.csv |cat >newfile
    
    I found this on page 2 of a search rarely used posted

    That resolved, then I had to deal with the fact that the vendor added a column and that caused 600,000 Warnings on the file upload to the MySQL table I created using the Shareasale 'standard' (and apparently ignored) headings for each vendor's datafeed Uuugh

    So, I added a dummy column to my table to keep the order correct with the datafile (that was fkd up)
    upload_2020-11-21_9-4-8.png

    1/2
     
    Last edited: Nov 21, 2020 at 9:26 AM
  4. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    2/2
    had to split this too complex to edit --will add to this later in the day (or this weekend) it's 9:00 in the morning --time to work
    here is a teaser for now
    upload_2020-11-21_9-13-42.png

    descriptions are too long to post here as an image but you get the idea :p
    upload_2020-11-21_9-21-11.png

    indexing is the next issue towards creating dynamic content or ads :p
     
    Last edited: Nov 21, 2020 at 9:22 AM
    EuronGreyjoy likes this.
  5. EuronGreyjoy

    EuronGreyjoy Affiliate affiliate

    769
    719
    93
    1. You obviously genius
    2. This is a very good idea
    3. Don't understand this level of code
     
  6. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Not a genuis -- still learning after 20 yrs exp :p
    tr man page
    this can be applied to any feed basically

    One thing I would want to point out --this feed with all of it's inconsistencies would fail if any common 'tool' were used.
    So it has some exclusivity.
    Deducing an effective algorithm will not be easy as the 'bestselling' field is 0 so scratch that idea.
    So I will have to think up some way to find the best values for x-mas gifts without doing monkey work with search engines and browsers :p

    Haven't got time --the googler app is blocked now --I used to be able to do automated text searches and extract the data and use it. Fuck Google! Have to find another way.
    upload_2020-11-21_10-32-5.png

    USELESS upload_2020-11-21_10-28-55.png
     
    Last edited: Nov 21, 2020 at 10:39 AM
    EuronGreyjoy likes this.
  7. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Looking for a gift for your wife, GF ?

    upload_2020-11-21_17-47-45.png

    Code:
    +-------------+----------+-------------+----------+------------------------------------------------------------------------------------------------+
    | Subcategory | Category | RetailPrice | Status   | BigImage                                                                                       |
    +-------------+----------+-------------+----------+------------------------------------------------------------------------------------------------+
    | Lingerie    | Fashion  |        8.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI307B-2XL/GI307B-2XL-1-0eb5-Rwux.jpg      |
    | Lingerie    | Fashion  |        8.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI307BL-2XL/GI307BL-2XL-1-d797-M0k7.jpg    |
    | Lingerie    | Fashion  |        8.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI307GR-2XL/GI307GR-2XL-1-3384-gr3i.jpg    |
    | Lingerie    | Fashion  |        8.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI307LB-2XL/GI307LB-2XL-1-0aec-zcsj.jpg    |
    | Lingerie    | Fashion  |        8.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI307PU-2XL/GI307PU-2XL-1-e8c5-7zOx.jpg    |
    | Lingerie    | Fashion  |        8.65 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/M/GI123B-M/GI123B-M-1-8f95-bKeq.jpg          |
    | Lingerie    | Fashion  |        8.73 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI144B-L/GI144B-L-1-bee6-Rkhr.jpg          |
    | Lingerie    | Fashion  |        8.93 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI297R-2XL/GI297R-2XL-1-1ac7-9lQa.jpg      |
    | Lingerie    | Fashion  |        9.49 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/B/G1496B38B/G1496B38B-1-0b08-LLiR.jpg        |
    | Lingerie    | Fashion  |        9.85 | In stock | https://img.tttcdn.com/product/xy/500/500/p/tt/2/0/20110808_img_025689.jpg                     |
    | Lingerie    | Fashion  |       10.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI302R-2XL/GI302R-2XL-1-c59a-Wo5u.jpg      |
    | Lingerie    | Fashion  |       10.81 | In stock | https://img.tttcdn.com/product/xy/500/500/p/tt/h/2/h2246r-5-426f.jpg                           |
    | Lingerie    | Fashion  |       11.34 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/B/ALG1931037B/ALG1931037B-1-80fe-uQZu.jpg    |
    | Lingerie    | Fashion  |       12.30 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI218B-L/GI218B-L-1-e7d8-b49A.jpg          |
    | Lingerie    | Fashion  |       12.30 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI218W-XL/GI218W-XL-1-d70d-e30Q.jpg        |
    | Lingerie    | Fashion  |       13.82 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI106B-L/GI106B-L-1-3b8c-1UuI.jpg          |
    | Lingerie    | Fashion  |       13.82 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI106W-L/GI106W-L-1-59d7-IWHR.jpg          |
    | Lingerie    | Fashion  |       13.82 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/G/L/GI263B-L/GI263B-L-1-3ccf-rnQq.jpg          |
     
    EuronGreyjoy likes this.
  8. EuronGreyjoy

    EuronGreyjoy Affiliate affiliate

    769
    719
    93
    @Graybeard if someone wants to understand it what programming languages should learn?
     
  9. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Well, what I am showing here is mostly SQL subcategory MySQL 8
    But I have used a Bash shell (.sh) a lot here in a terminal to fix these files.

    for instance:
    ~$ sed -e 's/^/<img src="/g' -e '/$/">/g'
    will do the HTML code for the images is bash
    Or, the database can be queried in PHP and the HTML code be added also. to create web pages or a catalog --very common use scenario.
     
    EuronGreyjoy likes this.
  10. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Need a present that is a game?

    upload_2020-11-21_18-25-1.png
    Code:
    mysql>  SELECT DISTINCT
    ->   Subcategory
    ->   , Category
    ->   , RetailPrice
    ->   , Status
    ->   ,REPLACE(BigImage, '%3A',':') as `BigImage`
    ->   FROM tomtop_feed
    ->   WHERE Category RLIKE'Toys'
    ->    AND ShortDescription RLIKE 'game'
    ->    AND RetailPrice BETWEEN 8 AND 22
    -> ORDER BY RetailPrice;
    +-------------+------------+-------------+----------+-------------------------------------------------------------------------------------------+
    | Subcategory | Category   | RetailPrice | Status   | BigImage                                                                                  |
    +-------------+------------+-------------+----------+-------------------------------------------------------------------------------------------+
    | Video       | Toys/Games |        8.01 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/B/F2062B/F2062B-1-8823-yPtL.jpg         |
    | Video       | Toys/Games |        8.01 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/L/F2062BL/F2062BL-1-0417-kYAD.jpg       |
    | Video       | Toys/Games |        8.01 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/R/F2062R/F2062R-1-2310-EzTv.jpg         |
    | Video       | Toys/Games |        8.23 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/2/ALF2279712/ALF2279712-1-148a-jiQW.jpg |
    | Video       | Toys/Games |        8.32 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/F1470-1/F1470-1-8-6b5a.jpg              |
    | Video       | Toys/Games |        8.32 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/F1470-4/F1470-4-2-4d55.jpg              |
    | Electronic  | Toys/Games |        8.89 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/R/L/RM11954BL/RM11954BL-1-f8fb-jrOG.jpg   |
    | Electronic  | Toys/Games |        8.89 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/R/P/RM11954P/RM11954P-1-3c3b-u0kI.jpg     |
    | Electronic  | Toys/Games |        8.89 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/R/Y/RM11954Y/RM11954Y-1-e402-zLhY.jpg     |
    | Video       | Toys/Games |        9.71 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/B/F1729B/F1729B-1-0d7f-nI8D.jpg         |
    | Video       | Toys/Games |        9.71 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/W/F1729W/F1729W-1-fa50-8mqj.jpg         |
    | Video       | Toys/Games |        9.82 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/3/ALF2432893/ALF2432893-1-6a33-FRAA.jpg |
    | Musical     | Toys/Games |       10.55 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/I/4/I4414/I4414-1-de73-ty3I.jpg           |
    | Video       | Toys/Games |       10.73 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/2/F2022/F2022-1-431e-tBJL.jpg           |
    | Video       | Toys/Games |       10.79 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/Y/F1930DGY/F1930DGY-1-046c-bGRM.jpg     |
    | Video       | Toys/Games |       11.55 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/8/ALF2288208/ALF2288208-1-0dde-LGpi.jpg |
    | Video       | Toys/Games |       11.55 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/9/ALF2288209/ALF2288209-1-e25a-t9j7.jpg |
    | Musical     | Toys/Games |       11.69 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/I/5/I5955/I5955-1-371a-Zt8z.jpg           |
    | Video       | Toys/Games |       12.35 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/3/ALF2435743/ALF2435743-1-0d33-bL4o.jpg |
    | Video       | Toys/Games |       12.93 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/L/F1985BL/F1985BL-1-7fa6-jUHA.jpg       |
    | Video       | Toys/Games |       12.93 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/R/F1985R/F1985R-1-9718-fOUI.jpg         |
    | Video       | Toys/Games |       12.93 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/Y/F1985Y/F1985Y-1-acc1-stHF.jpg         |
    | Video       | Toys/Games |       12.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/0/F2020/F2020-1-7775-hBBr.jpg           |
    | Musical     | Toys/Games |       13.19 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/I/6/I5956/I5956-1-a089-eDy1.jpg           |
    | Video       | Toys/Games |       13.75 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/6/ALF2432906/ALF2432906-1-d823-Ar8R.jpg |
    | Video       | Toys/Games |       13.75 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/7/ALF2432907/ALF2432907-1-4fc7-idhD.jpg |
    | Video       | Toys/Games |       14.49 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/7/F2047/F2047-1-51aa-OTyD.jpg           |
    | Video       | Toys/Games |       14.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/L/F1759BL/F1759BL-1-d430-2bSA.jpg       |
    | Video       | Toys/Games |       15.39 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/B/F1942B/F1942B-1-9a83-ecNw.jpg         |
    | Video       | Toys/Games |       15.39 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/L/F1942BL/F1942BL-1-4cb6-UqzG.jpg       |
    | Video       | Toys/Games |       15.39 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/R/F1942R/F1942R-1-7dd5-0e4y.jpg         |
    | Video       | Toys/Games |       15.39 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/W/F1942W/F1942W-1-4645-Ydp1.jpg         |
    | Video       | Toys/Games |       15.39 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/Y/F1942Y/F1942Y-1-3728-Dsvj.jpg         |
    | Video       | Toys/Games |       15.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/1/PAG0075-1/PAG0075-1-1-1106-NXby.jpg   |
    | Video       | Toys/Games |       15.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/2/PAG0075-2/PAG0075-2-1-92e2-a9If.jpg   |
    | Video       | Toys/Games |       17.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/1/F1670-1/F1670-1-1-2d57-V9po.jpg       |
    | Video       | Toys/Games |       17.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/5/F2035/F2035-1-98a8-vetU.jpg           |
    | Video       | Toys/Games |       17.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/L/F2031BL/F2031BL-1-a4e9-c50x.jpg       |
    | Video       | Toys/Games |       17.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/P/F2031P/F2031P-1-4837-VbSW.jpg         |
    | Video       | Toys/Games |       17.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/Y/F2031Y/F2031Y-1-2ff7-G6Dy.jpg         |
    | Video       | Toys/Games |       18.23 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/6/F1706/F1706-1-717d-rcR7.jpg           |
    | Musical     | Toys/Games |       18.33 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/1/PAA2380-1/PAA2380-1-1-dd07-uGVY.jpg   |
    | Musical     | Toys/Games |       18.33 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/2/PAA2380-2/PAA2380-2-1-afab-8b1L.jpg   |
    | Video       | Toys/Games |       18.38 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/A/6/ALF2279716/ALF2279716-1-de65-7ZOw.jpg |
    | Video       | Toys/Games |       18.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/1/F1669-1/F1669-1-1-d953-9niV.jpg       |
    | Video       | Toys/Games |       18.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/B/F1892B/F1892B-1-3fc8-KDhv.jpg         |
    | Video       | Toys/Games |       18.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/R/F1892R/F1892R-1-f8eb-Fsyk.jpg         |
    | Video       | Toys/Games |       18.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/W/F1892W/F1892W-1-e32b-27hI.jpg         |
    | Video       | Toys/Games |       20.54 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/L/PAG0094BL/PAG0094BL-1-fef5-mEDV.jpg   |
    | Video       | Toys/Games |       20.54 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/R/PAG0094GR/PAG0094GR-1-d970-YM1h.jpg   |
    | Video       | Toys/Games |       20.54 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/P/Y/PAG0094Y/PAG0094Y-1-9e3d-eR5t.jpg     |
    | Video       | Toys/Games |       20.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/3/F2073/F2073-1-05ce-W7nG.jpg           |
    | Video       | Toys/Games |       20.99 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/4/F2034/F2034-1-bec2-HWTv.jpg           |
    | Video       | Toys/Games |       21.59 | In stock | https://img.tttcdn.com/product/xy/500/500/p/gu1/F/U/F1926EU/F1926EU-1-4028-QDpR.jpg       |
    +-------------+------------+-------------+----------+-------------------------------------------------------------------------------------------+
    ex
    https://img.tttcdn.com/product/xy/500/500/p/gu1/F/R/F1942R/F1942R-1-7dd5-0e4y.jpg
     
    EuronGreyjoy likes this.
  11. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Had to fix the errors on the inital LOAD INFILE to the database.tomtop_feed table. Something went wrong and since this feed is so specific to this vendor with all the COLLATE=utf8mb4_0900_ai_ci errors, well China ya know :p


    upload_2020-11-23_14-12-3.png

    Against share A Sales advice; I am going to use the
    PRIMARY KEY (`id`), <<<the row numbers as the primary key
    KEY `ProductID` (`ProductID`) <<<my index key
    this command will create indexes
    this is the slow but sure way. it took 0.12 to 0.32 seconds to find and update each row; or about 5 hrs for the 107,208 rows :eek:

    Code:
    ALTER TABLE `tomtop_feed` ADD INDEX `ProductID` (`ProductID`);
    upload_2020-11-23_14-16-24.png
    as so
    here is why

    upload_2020-11-23_14-28-27.png

    id | ProductID | link |
    +--------+------------+------------------------------------------------------------------------------------------+
    | 60813 | 970339645 |//////////////////////////?merchantID=27868&userID=2621106&productID=970339645

    as you can see my ProductID(index) matches the &productID= of the link so that will be a good index for the data
    the problem with using the id column is that that id my be deleted or possible changed for some reason (usually a revision of some kind and a new row insert statement).

    there are sql (reserved) control characters in the URLs and as I said above there were errors "I made" so the repetitive REPLACE nested part of the above statement fixes the mess and returns real urls.
    Nested like HTML tags
    <div>
    <div>
    <div>
    </div></div></div>
    same idea
     

    Attached Files:

    EuronGreyjoy likes this.
  12. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    So now we can query this
    upload_2020-11-23_15-0-39.png

    As you can see there are unique ProductID s and duplicate ShortDescription s
    why maybe color variations I shortened the descriptions to 120 char for possible edited page titles ...
     
  13. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    Now these are my Category/Subcategory possibilities to find products to meet demands
    upload_2020-11-23_15-11-58.png

    if these are the 'short description' what are the 'full' descriptions? some of this is transator gibberish?
    upload_2020-11-23_15-27-55.png
    Code:
    +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | ProductID  | REPLACE( ShortDescription, 'Features%3A', '')                                                                                                                                                                                                                   |
    +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | 648714668  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 676230880  | Adjustable padded shoulder strap for comfortable fit.Adjustable waist belt with quick release buckles.Four magazine pouches with fastener tape closure.Two side pouches to store additional gears.Padded back cushion to protect your back from b               |
    | 685319720  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685320719  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685912337  | Made of polyester, very strong and durable.Water, rot and mould resistant.Mesh net reinforcement throughout the net.Perfect for concealment, sunshade, decoration and hunting etc.                                                                              |
    | 685916832  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685919062  | Made of polyester, very strong and durable.Water, rot and mould resistant.Mesh net reinforcement throughout the net.Perfect for concealment, sunshade, decoration and hunting etc.                                                                              |
    | 685925961  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685926619  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685929500  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685930550  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685934823  | Made by high density nylon material.With two root magic stick band to fix gun.2 zippered and 2 velcro small pouches for carrying magazine or small gears.With 2 hand carry straps and adjustable padded shoulder straps.Photo/name card holder on               |
    | 685937571  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685939540  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685943365  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685945835  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685949347  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685952284  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685955037  | Camouflage color will help you hide and not easy to be found.Ideal for car covering, bird watching, camping, hunting, CS, etc.                                                                                                                                  |
    | 685957465  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685958651  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685959425  | Size is optional%3A M(CN)%3DChest 45.3" / L(CN)%3DChest 47.2" / XL(CN)%3DChest 49.2" / XXL(CN)%3DChest 51.2".Polyester fiber, breathable, water-resistant and windproof.Inner fleece jacket, warm, soft and comfortable to wear.Fastener tape adjustable cuf    |
    | 685959989  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685965302  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 685965880  | Size is optional%3A M(CN)%3DWaist 32.7 / L(CN)%3DWaist 34.6 / XL(CN)%3DWaist 36.6 / XXL(CN)%3DWaist 38.6.Polyester fiber, breathable, water-resistant and windproof.Fleece lining, warm, soft and comfortable to wear.Zipper fly design, cool and stylish.T     |
    | 708939476  | Strong and durable tactical utility belt with reinforced threads and stitches.2 wide belt is adjustable to fit for 80-105cm waist circumference.Secure and quick-release buckle.Include multiple pouches and holsters, such as flashlight pouch, m              |
    | 793536374  | Lixada 36" Outdoor Padded Barrel Carrying Bag with Shoulder Sling Strap                                                                                                                                                                                         |
    | 793544847  | Made with high quality 1000D nylon and well padded in all pressure areas, breathable mesh backfor comfortable fit.Quick release buckles for easy put on and off while providing extra security.Both side of the vest equipped with molle straps for             |
    | 793560706  | Easy and convenient to use.With laser point aiming to see the target that one wishes to measure distance.Wide measurement range and high sensitivity and accuracy.Lightweight and easy for you to carryMeasure distance from 0.6m to 18m.Automat                |
    | 793586715  | Made with high quality 1000D nylon and well padded in all pressure areas, breathable mesh backfor comfortable fit.Quick release buckles for easy put on and off while providing extra security.Both side of the vest equipped with molle straps for             |
    | 797635552  | Perfect for outdoor sport likes golf, hiking.Energy save, no batteries required.Eco-friendly by using solar energy.Can be individually adjusted to different head sizes.Can face the sunshine directly and enjoy your outdoor life.                             |
    | 797635994  | Perfect for outdoor sport likes golf, hiking.Energy save, no batteries required.Eco-friendly by using solar energy.Can be individually adjusted to different head sizes.Can face the sunshine directly and enjoy your outdoor life.                             |
    | 797639958  | Perfect for outdoor sport likes golf, hiking.Energy save, no batteries required.Eco-friendly by using solar energy.Can be individually adjusted to different head sizes.Can face the sunshine directly and enjoy your outdoor life.                             |
    | 914372940  | Dog Training Bungee Leash Elastic Dog Leads Rope                                                                                                                                                                                                                |
    | 914376429  | QUALITY MATERIAL -- Made from nylon material.Features in durable, lightweight, and breathable design.MULTIFUNCTIONAL - Classical cummerbund designed with hinging system on the outside.Convenient for holding keys, accessories and tools.QUICK TO             |
    | 914377033  | Constructed from high-quality brass and iron, durable, sturdy and solid to use.Adjustable tension control.Can be used with all types of bow string serving materials.Suitable for compound bow and recurve bow.                                                 |
    | 914377433  | Low energy consumption, not only a mosquito trap, but also a night light and decoration in your family and office.No toxicant, no chemical and no odor whatsoever.Healthier than traditional pest repellent or bug zapper, spray and mosquito-repel             |
    | 914378660  | Dog Training Bungee Leash Elastic Dog Leads Rope                                                                                                                                                                                                                |
    | 914379808  | QUALITY MATERIAL -- Made from nylon material.Features in durable, lightweight, and breathable design.MULTIFUNCTIONAL - Classical cummerbund designed with hinging system on the outside.Convenient for holding keys, accessories and tools.QUICK TO             |
    | 914386416  | Dog Training Bungee Leash Elastic Dog Leads Rope                                                                                                                                                                                                                |
    | 914387821  | Soft EVA foam for good protection.12 slots for arrows.Securely keep your arrows in place.                                                                                                                                                                       |
    | 914388901  | Soft EVA foam for good protection.12 slots for arrows.Securely keep your arrows in place.                                                                                                                                                                       |
    | 914390170  | QUALITY MATERIAL -- Made from nylon material.Features in durable, lightweight, and breathable design.MULTIFUNCTIONAL - Classical cummerbund designed with hinging system on the outside.Convenient for holding keys, accessories and tools.QUICK TO             |
    | 914393201  | QUALITY MATERIAL -- Made from nylon material.Features in durable, lightweight, and breathable design.MULTIFUNCTIONAL - Classical cummerbund designed with hinging system on the outside.Convenient for holding keys, accessories and tools.QUICK TO             |
    | 956952728  | Multi pockets with fastener tape, convenient to keep more items in placeOxford fabric ensures maximum durability and water repellent, and lint collar to protect your neck.Adjust the waist to fit your body.Zipper open for easy and quick on                  |
    | 956953443  | Archery Single Side V Bar Connector for Compound Bow Recurve Bow                                                                                                                                                                                                |
    | 956961011  | 1PCS Archery Protector 3 Finger Tab Glove with Wrist Strap                                                                                                                                                                                                      |
    | 956961080  | Phone Bow Mount Phone Holder Bracket                                                                                                                                                                                                                            |
    | 956962299  | Constructed from oxford fabric, water-resistant and tear-resistant.Spacious main compartment with mesh interior pocket.Front pocket, rear pocket, mesh side pocket, and flap pocket for storing small items.Molle webbing for hanging additional i              |
    | 956962953  | Archery Arrow Case Carrier                                                                                                                                                                                                                                      |
    | 956964326  | Archery Double Side V Bar Connector for Compound Bow Recurve Bow                                                                                                                                                                                                |
    | 956967687  | Universal Magnetic Flashlight Mount Magnet Holder                                                                                                                                                                                                               |
    | 956968588  | Superior Image and Video Captures%3A The trail and game camera captures 8 megapixel crisp, brightly colored photos in daytime (black and white at night) and 720P video with sound. Adjustable resolutions are available for both still images and vide         |
    | 956969482  | Archery Double Side V Bar Connector for Compound Bow Recurve Bow                                                                                                                                                                                                |
    | 956970689  | Archery Arrow Case Carrier                                                                                                                                                                                                                                      |
    | 956971751  | QUALITY MATERIAL -- Classical outdoor nylon belt with reliable durability, lightweight, breathable and quick to dry, the best choice for outdoor activities.QUICK TO RELEASE -- Quick-release buckle design for convenient using, especially for out            |
    | 970339669  | Universal Magnetic Buckle Belt Side                                                                                                                                                                                                                             |
    | 970340067  | Universal Magnetic Buckle Belt Side                                                                                                                                                                                                                             |
    | 970341266  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 970341311  | Men Women Belts 47.2 Inch                                                                                                                                                                                                                                       |
    | 970341320  | Men Women Belts 49.2 Inch                                                                                                                                                                                                                                       |
    | 970341344  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970341615  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970341660  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 970342103  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970342135  | Men Women Belts 49.2 Inch                                                                                                                                                                                                                                       |
    | 970342148  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970342268  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970342270  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970342694  | Men Women Belts 49.2 Inch                                                                                                                                                                                                                                       |
    | 970343037  | Men Women Belts 47.2 Inch                                                                                                                                                                                                                                       |
    | 970343078  | Men Women Belts 49.2 Inch                                                                                                                                                                                                                                       |
    | 970343275  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970343753  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970343870  | 2 Pack Men Women Belts                                                                                                                                                                                                                                          |
    | 970344072  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 970344280  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 970344297  | Men Women Belts 47.2 Inch                                                                                                                                                                                                                                       |
    | 970344402  | Men Women Belts 49.2 Inch                                                                                                                                                                                                                                       |
    | 970344427  | Men Women Belts 47.2 Inch                                                                                                                                                                                                                                       |
    | 970344753  | Men Women Belts 47.2 Inch                                                                                                                                                                                                                                       |
    | 970344832  | Mainly constructed of aluminum, anti-rust and can provides long service time.You can use it to paste feathers at the end of the arrow, help save time and money by repairing your arrows by yourself.Gears adjustable, suitable for most kinds of a             |
    | 970344952  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 970345422  | Adjustable Webbing Belt Men Women Belts                                                                                                                                                                                                                         |
    | 988361180  | 33IN Outdoor Padded Gear Bag Combat                                                                                                                                                                                                                             |
    | 988361884  | 33IN Outdoor Padded Gear Bag Combat                                                                                                                                                                                                                             |
    | 988366121  | Archery Quiver 3 Tubes Arrow                                                                                                                                                                                                                                    |
    | 1161213015 | HaleBor 12X50 Compass Dual Focus Monoculars Telescope                                                                                                                                                                                                           |
    | 1161216125 | Keep you away from danger while enjoying multi sport activities. Its important to have the helmet for protection.The helmet would make a bit compression to the body for the safty reason, the resonable could allow the body to hold the right posi            |
    | 1161218883 | 16x52mm objective diameter. Field of view%3A 66m/8000m.All optical glass lens deliver high and sharp image quality.Non-slip adjustable wheel rotated the precise and suitable lens.Multicoated optics with BAK4 roof prism guarantee superior light             |
    | 1161224393 | Keep you away from danger while enjoying multi sport activities. Its important to have the helmet for protection.The helmet would make a bit compression to the body for the safty reason, the resonable could allow the body to hold the right posi            |
    | 1161224474 | Keep you away from danger while enjoying multi sport activities. Its important to have the helmet for protection.The helmet would make a bit compression to the body for the safty reason, the resonable could allow the body to hold the right posi            |
    | 1161226536 | Keep you away from danger while enjoying multi sport activities. Its important to have the helmet for protection.The helmet would make a bit compression to the body for the safty reason, the resonable could allow the body to hold the right posi            |
    | 1161229097 | HaleBor 12X50 Compass Dual Focus Monoculars Telescope                                                                                                                                                                                                           |
    | 1161229318 | So small and portable, great for enjoying scenery, camping, bird viewing and other many outdoor activities.40X magnification and 60mm large optic lens, high clarity image effects.Supporting installing the compass to connect your phone for view             |
    | 1189089871 | Superior Image and Video Captures%3A The trail and game camera captures 12 megapixel crisp, brightly colored photos in daytime (black and white at night) and 1080P video with sound.Super Fast Trigger Speed%3A With 120 super wide lens angle, 110 PIR        |
    | 1189089872 | HD 1080P images and videos, so you will see pictures clearly.Support PIR induction sensitivity adjustment.IP56 protections grade, effectively prevents rain, dust and insects from entering the camera.Great for home, warehouse, apiary, tree far              |
    | 1189089874 | features a highly sensitive PIR sensor that can be triggered by detecting any movement of objects or animals in an impressively fast speed of 0.8 seconds. It takes clear 5MP pictures and 480P video clips, perfect choice for outdoor scouting games and wild |
    | 1189089875 | The PIR sensors with 110-degree wide detection angle, wide lens angle up to 120-degree and up to 15 meters night sensor distance, 0.8s quite quick trigger speed, customizable up to 3 pictures shots each triggerSupported memory card (not include            |
    | 1189089878 | The PIR sensors with 110-degree wide detection angle, wide lens angle up to 120-degree and up to 15 meters night sensor distance, 0.8s quite quick trigger speed, customizable up to 3 pictures shots each triggerSupported memory card (not include            |
    +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    100 rows in set (0.72 sec)
     
    EuronGreyjoy likes this.
  14. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    this caught my eye --I want a monocular to use on the shooting target range and the 16x52mm is very high power to see your target at 100+ yards!
    upload_2020-11-23_15-45-14.png

    upload_2020-11-23_15-48-39.png
    https://www.shareasale.com/m-pr.cfm?merchantID=27868&userID=2621106&productID=1161218883
    seems like to good deal but the niche is very narrow.
    is the optics quality decent? if it is that's a real value ...

    so that is how the index works --look at the query statement
    -> WHERE ProductID =1161218883; <<<my added KEY or INDEX
     
    EuronGreyjoy likes this.
  15. Graybeard

    Graybeard Well-Known Member affiliate

    7,361
    4,358
    113
    using Apify App
    they have a google scraper that works :) not that expensive (oh well)
    upload_2020-11-23_16-49-3.png

    json data in a text file
    next step is to automate the page scraping to get some size, type and price data ...

    Monoculars for sale | In Stock | eBay
    prices all over the place (WTF) i expected this actually ;)
     

    Attached Files:

    EuronGreyjoy likes this.
MI