Free solutions for detecting proxies

  • Thread starter Thread starter Jarfi
  • Start date Start date
Click For Summary

Discussion Overview

The discussion revolves around methods for detecting proxy usage on a website, particularly in the context of security concerns related to suspicious visitors. Participants explore various techniques for identifying proxies, bots, and legitimate users, while also sharing experiences with specific IP addresses and their associated behaviors.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant notes repeated visits from specific IPs associated with major organizations and expresses concern about differentiating between legitimate web spiders and potential malicious users.
  • Another participant suggests that detecting all proxies is fundamentally impossible, using a thought experiment to illustrate the limitations of tracing user actions through proxies.
  • It is mentioned that lists of known Tor exit nodes can provide clues for identifying proxy usage, but complete detection remains elusive.
  • One participant provides specific examples of IP addresses and their associated network organizations, suggesting methods such as checking headers, proxy lists, and common proxy ports for detection.
  • A participant describes encountering referrer spam, explaining how bots can manipulate referrer fields to mislead website owners into visiting spam sites.
  • Another participant shares detailed IP lookup results for a suspicious site, indicating potential risks associated with clicking unknown links.

Areas of Agreement / Disagreement

Participants generally agree that detecting proxies is challenging and that no method guarantees complete accuracy. Multiple competing views exist regarding the effectiveness of various detection strategies, and the discussion remains unresolved on the best approach.

Contextual Notes

Participants highlight limitations in detection methods, including the dependence on external databases and the inherent difficulty in distinguishing between legitimate and malicious traffic.

Jarfi
Messages
384
Reaction score
12
For security reasons, There is an IP-logger on my website, also logging reference page. What I've noticed is repeated visitors from Beijing, Microsoft or Google INC. Also getting visits from websites like: "http://hvd-store.com/".

How can I detect a proxy without paying for some service? I've been estimating proxies from Network Organization information and by googling the IP, but I don't have any automatic mechanism that I can use to display threat-info or into the page for other admins to see, something like "detected proxy"/"Individual".

And is there a way to detect and differentiate bots, search-spiders and proxies? it's hard to be sure if the ones from Microsoft and Google INC are web-spiders or someone with bad intents.

An example of a suspicious visit logged:

Network Organization: AS16276 OVH SAS
Ref-page: http://hvd-store.com/
IP: 176.31.182.218
Location: Paris
Browser: Google Chrome

A bot that hangs around,

Network Organization: AS55967 Beijing Baidu Netcom Science and Technology Co., Ltd.
Ref-page: no ref-page
IP: 180.76.15.34
Location: Beijing
Browser: Unknown

I thought this was a spider, but it has a browser so I am led to believe it's something different:

Network Organization: AS15169 Google Inc.
Ref-page: no ref-page
IP: 66.249.93.252
Location: Mountain View
Browser: Mozilla Firefox

-Thanks in advance.
 
Last edited by a moderator:
Technology news on Phys.org
It's a probability game. For instance you can find lists of Tor exit node IPs, that will give you some clue. But I can tell you from a pure computer science perspective, its just not possible to detect all proxies, even if you pay for a service. As a thought experiment, imagine someone A who calls a friend B and tells them to visit your page, and read what they find there. How do you detect anything about A from B? You don't, and digital versions of the same process will reveal nothing about A either. Download Teamviewer, and launch a browser on a remote computer to view your site. How can you tell that its being remotely invoked by Teamviewer and seen by another computer? You can't. Its really something you need to set aside to accomplish your security goals.
 
176.31.182.218 is listed as a TOR exit node.

180-76-15-34 has hostname baiduspider-180-76-15-34.crawl.baidu.com

66.249.93.252 has hostname google-proxy-66-249-93-252.google.com

If I was going to try to do this I would check headers, proxy lists, tor exits, and check for open common proxy ports and do an automated search on google like +"66.249.93.252" proxy

But Foolality is right. Can't get them all. And some proxy accesses might be legit (didn't all AOL accesses come through a proxy? don't remember).

Logging what people throw at your computer is a great way to collect exploits. I would think one could look for suspicious requests and list those IP's (but maybe that's impractical -- I'm not a security expert)
 
"hvd-store.com" turned up in my website log file today:

89.105.194.71 - - [14/Aug/2015:00:47:13 -0400] "GET / HTTP/1.1" 301 230 "http://hvd-store.com/" [/PLAIN] "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"
89.105.194.71 - - [14/Aug/2015:00:47:14 -0400] "GET / HTTP/1.1" 200 3095 "http://hvd-store.com/" [/PLAIN] "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"

These are requests for my home page (GET /) from the IP address 89.105.194.71, which appear to be the result of clicking on a link on the home page at http://hvd-store.com/. I haven't gone to hvd-store.com to look, but I would wager strong odds that there is not actually a link to my site there. It would probably at best try to sell me something, or at worst try to infect my computer with malware. This is called "referrer spam", which tries to trick curious website owners into following the links to see who is apparently linking to their sites. It's created by bots which fetch pages from your site, inserting the spam site URL into the referrer field of the requests.

Whenever I see a new referrer in my log file, and it doesn't seem to be related to the topic of my site, I do a Google search on it to try to find out something about it, before deciding whether to click on the link. In this case I didn't find anything for "hvd-store.com" which gave any indication of what this site is actually about, which is why I didn't go there. This thread turned up on the first page of that search. :smile:

http://whatismyipaddress.com/ip-lookup gives the following information about the originating IP address:

IP: 89.105.194.71
Decimal: 1500103239
Hostname: tor-exit-readme.as24875.net
ASN: 24875
ISP: Avira B.V.
Organization: Avira B.V.
Services: http://whatismyipaddress.com/ip-services
http://whatismyipaddress.com/ip-services
Recently reported forum spam source. (83)

http://whatismyipaddress.com/hostname-ip gives me the following IP addresses for hvd-store.com:

Lookup Hostname: hvd-store.com
Lookup IPv4 Address: http://whatismyipaddress.com/ip/208.73.210.217
Lookup IPv4 Address: http://whatismyipaddress.com/ip/208.73.211.178
Lookup IPv4 Address: http://whatismyipaddress.com/ip/208.73.210.200
Lookup IPv4 Address: http://whatismyipaddress.com/ip/208.73.210.214

Plugging the first address back into the ip-lookup tool gives me

General IP Information
IP: 208.73.210.217
Decimal: 3494499033
Hostname: 208.73.210.217
ASN: 40034
ISP: Oversee.net
Organization: Confluence Networks
Services: None detected
Type: http://whatismyipaddress.com/broadband
Assignment: http://whatismyipaddress.com/dynamic-static
Blacklist:
Geolocation Information
Continent: North America
Country: United States
us.png

State/Region: California
City: Los Angeles
Latitude: 34.0533 (34° 3′ 11.88″ N)
Longitude: -118.2549 (118° 15′ 17.64″ W)
Postal Code: 90071
 
Last edited by a moderator:

Similar threads

Anonimity: Time to take the internet back.
  • · Replies 3 ·
Replies
3
Views
4K