“webinar”/ “webinar”/
Welcome to Our Community
Wanting to join the rest of our members? Feel free to sign up today.

How can I check if an IP is a bot or not?

Discussion in 'General Affiliate Marketing Forum' started by essaid, Jun 13, 2019.

  1. essaid

    essaid Affiliate affiliate

    12
    5
    3
    Hi,

    My prelanders and funnels are hosted on my VPS, and I have access to the server log.
    I can get the IPs coming for each prelander, I want to check by myself if those IPs are bot or not, is it possible?

    Regards
     
  2. Voluum
  3. Certified
    CPAHub

    CPAHub Active Member Certified Vendor Affiliate Manager affiliate

    281
    86
    28
    First step would be to check if it's a proxy/datacenter IP or a residential IP.
    I can recommend this website and API for that: Proxy & VPN detection API - IPHub.info
    Works pretty good in my experience.
     
    essaid likes this.
  4. Graybeard

    Graybeard Well-Known Member affiliate

    4,158
    2,069
    113
    Take a list of all the IP's hitting your website and analyze them all.
    Or, you can hire 100 monkeys to type them into the form on some website ...
    Do you have access to your webserver logs where the traffic is logged? Start there.
    How much traffic? a random sample of 2,000 IPs would be enough to make a pretty statistically accurate assessment.
    in ssh cut and shuf -n2000 >list

    Learn what a reverse lookup is and what it will tell you. hostnames that come back are tell alls
    do you understand ssh and server administration at all?
    make a bash script to loop and do a lookup on the /24 CIDR of every IP sorted to sort -u (without duplication) |unique -c in a sort will reveal a lot -- many bots will use similar IP C and D blocks (usually belonging to datacenters) 1.2.1-254.1-254 (IPv4)
    IPv6 is a real can of worms
     
    Last edited: Jun 14, 2019
    essaid likes this.
  5. essaid

    essaid Affiliate affiliate

    12
    5
    3
    I didn't test it yet, but looks good, thank you
     
  6. essaid

    essaid Affiliate affiliate

    12
    5
    3
    Thank you Graybeard.
    Yes, I have a root access to my server and have all server logs.
    Also I know about ssh cmd little bit, I'm not good on networkside, but I got what you said.
    Good idea, thank you again
     
  7. Andre Houston

    Andre Houston Service Manager Service Manager affiliate

    19
    4
    3
    Hmm. You need to check the IP address against known proxy,bot database. IP2Location

    Also you would want to check the HTTP Request Header and check the User-Agent. that would give some details about what made the click (Web browser, Search Engine Spider..etc)
     
  8. Graybeard

    Graybeard Well-Known Member affiliate

    4,158
    2,069
    113
    It is so damn easy to forge the user-agent LMAO.
    From my experience with PCI-DSS billing servers these commercial data bases are not that effective -- plenty slip through the cracks trying to, and sometimes succeeding, with fraudulent payment transactions. Data bases we paid near $400/mo for.

    I penetrated near 100% of the websites I connected with >>>Ask Me Anything - Private proxy-cloaker FREE! (<LOL>) Now, I made no attempts to hack or defraud some way -- but give me your ad links and I'll try and we can see if it's counted :p Dare Ya >:=D
     
    Andre Houston and essaid like this.
  9. essaid

    essaid Affiliate affiliate

    12
    5
    3
    IP2Location is too expensive, I'll do it manually as said Graybeard.
     
    Andre Houston likes this.
  10. Graybeard

    Graybeard Well-Known Member affiliate

    4,158
    2,069
    113
    I has occasion to match the MaxMind Geo-City-Light (free distrubution) data base against my own proprietary ban list by AS networks assigned to known data centers (bots and servers). My list is by no means exhaustive however -- of 140 million CIDR assignments in the MaxMind Geo-City-Light my lists found 40 million banned CIDRs.

    The remaining 100 Million were thought to be normal user IPs (residential IPs)
    I spot checked manually and some were hereto unknown possible 'bougies' (suspects).

    What does this all mean you ask? I means maybe 1/3 of the IP distribution in the US is to be suspect.

    I can just white list my own server's IPs and the IP of the services I want to do business with or communicate with.
     
MI