* Proxy checkers such as Charon can filter by IP ranges.
* The AccessDiver proxy checker has an option to remove CoDeeN/PlanetLab proxies using the port.
(Also, IP / Proxy List Formatter | CoDeeN Filter | Batch IP to Country Lookup - [IPMaster.org] filters CoDeeN (probably by port or IP).

These methods seem blunt and insufficient.
* CoDeeN IP's are probably changed from time to time (added,removed).
* Some non-CoDeeN proxies use some CoDeeN ports.
* There are proxy "nasties" other than CoDeeN (DansGuardian, SurfControl, NetSweeper, Barracuda, etc.)

AccessDiver has an option to test proxies against a site (Charon also?). In theory, this seems to be a better way. However, a simple test run against amkingdom (for which I assumed CoDeeN,etc. would disallow access) was completely disappointing, not appearing to filter much, if anything.

(amkingdom was chosen because it is fast and popular - proxies that do filtering will almost certainly filter this).

I then wrote a little test program to independently test this concept: cycle through ~2000 proxies leached by FPL, executing http connections (HEAD, 12s timeout) to amkingdom, and checking the response to see if the target has been reached. This only returned about 10 "valid" proxies. This can't be right, can it?

Yet another method, used by "On Cracking Rampage" and CyberWaper, is to test the response against "proxy block phrases" such as "CoDeeN" and "DansGuardian".

These methods appear to be sound.
Or am I overlooking something?
Is there any working implementation of these methods for filtering proxies?
If not, is there any source code available (preferably java) for proxy checking that can be adapted?


References:
G-Lock Software • View topic - Planetlab/Codeen Exclusion List
http://forum.my-proxy.com/suggestion...s-t256.15.html
http://forum.my-proxy.com/proxy-ques...s-t3381.0.html

You ask to many questions within your questions and within your professed knowledge about this subject which appears to be very limited. It's confusing to say the least.

AD (AccessDiver) should never be used to test proxies. It has never been any good at this, except when some of the tools we have today weren't available. It's author, Jean Fages, and I go way back.

Charon can filter nearly all Planetlab/CoDeeN proxies very simply. You eliminate proxy ports 3124, 3127, 3128 and 8888. That's it, no more problem. Do you omit a few good proxies also using some of these ports? Yes, mostly on 3128, but most of those will be educational facilities, which depending on your intentions might not be ideal anyway.

To filter these out in any other way requires some work and some knowledge. In our Proxy Tools section I keep an updated Planetlab/CoDeen Exclusion List with complete instructions on how to use it in most of the leading proxy analyzing programs.

Those three reference sources you mentioned are either mine or those I basically took over. Katmando is an alias.

Actually you need to do more reading about what Charon can do, because you make it much more limiting than it really is.

Wrong, they actually learned how to do this from me. Again, I refer you to my post in our Proxy Tools section.

Brilliant deduction, and the reason why I regularly update the Planetlab/CoDeeN Exclusion List in our Proxy Tools section. It cost nothing to leave those P/C listings in your filter list which appear to be dead now. They could always be reactivated.

Your questions regarding cracking/hacking porn sites and the like go way beyond the scope of this site, and although could be answered, won't be here.

__________________
I tell it like it is and that's offensive to those professing knowledge they don't have.

It may be more than it first appears...

The reason I use AD from time to time is because of the two step testing.
I only do the first test, the check for valid proxies, skipping the anonymity test.
(I seem recall an early version of Charon also used to allow this. Perhaps 0.3?)
That's because I "utilize" these proxies over public wifi, so that provides the required anonymity. Adding non-anonymous proxies to the mix seems to greatly expand the pool of available proxies. Do you see anything technically wrong with this?

Yes, I saw this. Along with the your recently added "More proxies to exclude" thread.
Many thanks.
But will you be providing this service forever?

Of course. I read them all. (Some food fight with Xploitz.)
And that's the primary reason I posted here. You appear to be the guru in this field.

Can Charon filter based on keywords?
That was the point I was awkwardly trying to make.
Let's say I do IP range and port filtering using Charon with your lists.
What about the other "nasties"? Are there IP exclusion lists for DansGuardian, SurfControl, NetSweeper, Barracuda, etc.? (there's one that always redirects to the New York Times. Just hate constantly seeing that.)

Or should we think about doing keyword based filtering?
Or should we only be concerned about whether a given proxy allows access to a given site?

Hmmm, since Charon (actually originally called Calamity) was created by Rhino at my suggestion, and I served as one the his lead beta testers as long as he was actively revising the program, I don't recall this being the case at all, but hey maybe I've forgotten something.

Forever? Doubtful! But what's the point, I already told you how to eliminate them (Planetlab/CoDeeN proxies ) in total.

Yes, that has been said and it is probably accurate!

If you looked at nothing other than my Planetlab/CoDeeN Exclusion List, you would know that it can and does.

I'm sorry, I must be missing something here. Aren't we talking about an entirely different animal here?

__________________
I tell it like it is and that's offensive to those professing knowledge they don't have.

This is the point I've been (unsuccessfully) trying to make from the start.
Let me explain with a simple example, and you can tell me where I'm going wrong.

Say Bob has a server that runs a proxy service, and he's getting real tired of people wasting his bandwidth on "inappropriate" websites. So he installs DansGuardian (or something similar) to deny access to those sites. How well can exclusion lists do in these types of cases?

Note that Bob is not a member of the CoDeeN network, so you can't filter his proxy by IP address, or by doing a reverse DNS lookup and checking the hostname for "planetlab" or ".edu".

Anyone who utilizes proxies for "testing" purposes knows that even after removal of the CoDeeN variety, there are a number of these other "nasties" (what I've been calling them) that tend to muck up the works.

And that also goes to the point I was trying to make when I referenced the "cracking" tools - the logic of looking in the response headers for certain keywords that indicate that web filtering software is installed.

Since it's probably been 4-5 years since I last used Charon, I went back and reviewed the tutorial, and I saw something interesting that I had forgotten about. Namely, the option of using an internal proxy judge. Do you know of any proxy judges written in Java?

You have given any thought whatsoever to your question? Do you understand anything about which you are asking? Do you know how DansGuardian works? If so, how could an exclusion list work with it?

As they say in the 4th grade, no shit Dick Tracey!

[quote]Anyone who utilizes proxies for "testing" purposes knows that even after removal of the CoDeeN variety, there are a number of these other "nasties" (what I've been calling them) that tend to muck up the works.[/qupte]

Provide a list of them ... the proxies.

And again I will point out to you that these type questions go way beyond the scope and purpose of this site. Continue on this theme and I will ban you. Got it?

Yes, it always helps to read about programs we are talking about ... something I believe I pointed out to you from the beginning! By the way, the internal judge option does not always work for everyone! In fact, more times than not it doesn't.

__________________
I tell it like it is and that's offensive to those professing knowledge they don't have.