Home › Forums › Development

Somewhere to start (code)

Thu, 2006-12-21 12:11 — knayil

Hopefully this time someone will actually listen to this idea to get you started...

A project exist that has an anti-spam solution available under the GNU GPL. This project is no longer being maintained, as the developer is ill. Now before I tell you where, let me tell you why this is what you need to look at.

First off, your not going anywhere right now. End of story, you can't seem to get started on your own so you need a starting point. This will give you something to start with and build from.

Secondly, it has most of what you need/want already made. No it isn't everything you want, but it gets you going and if you start to show work, more developers may come to your aid.

Lastly, it already has a documented API that can be used to add more features by more people. This would give you some great benefits down the road, especially if my other suggestion is followed.

The project in question already has a lot of features for filtering spam and marking it. It will work with most any mail client out there and it already has a P2P code available.
This means most of your work is done for you, all you need is the reporting and opt-out.

Now my suggestion for this is to make the reporting a plug-in and the opt-out a separate plug-ins. The reason for this is it gives the user flexibility, it lets them decide if they want to opt-out, report spam or just filter it. I am sure some of you will hate this idea, but if you go this route you will probably find more supporters willing to write code for the main tree and other plug-ins who do not agree with opt-out campaigns. The more willing to work on the main code, the more you can devote time to making the opt-out better. This will also give you a much larger user base to work from, which means more developers and assistance.

The code base I am suggesting is http://www.spampal.org. I am sure some of you have heard of this before. But below is a quick run down on the features it has already built in:

DNSBL lists
White list
local blacklist
People and mailservers with whom you correspond frequently are Automatically Whitelisted
multiple POP3 servers support
Supports APOP authentication
Can filter IMAP4
Unobtrusive
Multi-language support
Subject Line Tagging
Plug-ins Support

The choice is yours really, but I think this is a good way for you guys to get started.

What is the deal? ›
Thu, 2007-07-19 07:10 — HeaDCase

Re: Somewhere to start (code)

OK, I looked at the SpamPal site (not the code, yet). It's written in C and has plugins. Meaning that it can be used as a fast prototyping platform for Okopipi - just write plugins with clearly defined, limited specific functionality, test them with spampal, and modify/add new ones until satisfied. At some point we should be confident enough of the functionality and security to refactor the C plugins into a standalone crossplatform client, but we'll see about that later.

At this early phase we should get away with using central servers for testing, so I propose a first plugin and invite you to think of more:
URL analysis plugin:
takes a known SPAM email, rips out all the URL domains from the message body (so from http://some.site.com/malwaredispenser.php?email=joe@ISP.com&WeGotALiveOne=yes it would take only site.com) and sends them to a simple server side script that tallies them up.

I will write this first plugin plus the server side script, and commit it to the SVN (I do C, php, Javascript and UNIX shell scripts). It would be nice if somebody could host the script.

[edit] It just occurred to me how the creation of scripts for the clients can be put in the hands of the users: PureTest, a free website testing tool, has a "recording proxy" component that lets one record and save a "scenario" of a web session and then customize it, add variables, etc. I used it in the past and I found it both intuitive and very user-friendly. It's written in java :)

Tue, 2007-08-07 22:58 — Serenefrog

Re: Somewhere to start (code)

Hi guys - this sounds interesting - I'm sorry to segue the thread, but can someone point me in the direction of the Okopipi white-paper? I can't seem to find any links to it using the Search function.
I want to be up-to-date with the master-plan before I try to think about how all of this is going to work.

Thu, 2006-12-28 23:26 — scrapiron

This sounds like a real idea

Good lord finally finally this is starting to make scence now get the ball rolling, they'll be "things" to iron out but start fleshing this out. . . .it will come together once down on paper and being tested. . . Onward through the Fog. . .
Scrapiron

Sun, 2006-12-24 02:21 — koko

nice!!

I like the idea of finally getting started!! As said in the post, it doesn't matter whether this client hasn't ALL the features you wish, but it gives uw a start. In the future, you can write some kind of plugins to get more things included in the software, as long as the program has the main functionalities.

Take a look at firefox! In the beginning you could only browse with it, now there are so many plugins you can almost do anything with it....

Sun, 2006-12-24 01:39 — Pharmacy

Yah its a good idea, But its

Yah its a good idea, But its single platform and the big question is, is it verbose enough that it could reconize and do the OPT-OUT part that blue frog did. A Spam Filter is nice, but the spam filters are not working any more, we need a more proactive approch.
---
Yes the pen may be mightier than the sword, But if you have millions of swords running toward you it might be time to fight back with something a little stronger.

Sun, 2006-12-24 17:38 — knayil

Opt-Out

Actually it's not single platform, the code is a port of a Unix tool to windows with some other additions made for a proxy. The Unix world has much better stuff and an opt-out tool could be written for them.

The developers (okopipi) stated awhile back they were more concerned with Windows than making it cross platform anyway(one of the reasons I withdrew as a developer).

As for the opt-out portion, if they do it as a plug-in then it would have the ability just like bluefrog. The opt-out portion of bluefrog wasn't anything special. It simply executed some specially designed scripts. This would be trival to code in as a plug-in for spampal as would one for reporting. Some minor additions to the P2P plug-in and you have a way to submit spam to reviewers to write the scripts for the opt-out.

As for filters not working any longer I don't see that as the case. I get roughly 500-600 spam e-mails a day, 0 of them make it to my inbox. I have filters that catch every bit of spam I get, even the image spam. I get no false positives either, so all my valid e-mails get through as they should. But I report 100% of the spam I get to every place that will accept it. Filters do work, but I do not think that is the final solution. It only mask the problem, I think it is going to take a combination of methods to be truly effective.

Filter as your first line of defense, catch it all and have a tool such as this deal with it so users don't have to.

Second line should be reporting to all the law enforcement agencies and private anti-spam groups out there (i.e. SpamCop, spamhaus, etc).

Last line is the opt-out campaigns and complaints to supporters of spammers as well as the ISPs they are abusing.

But make all 3 able to function separate from the others. This would keep the system running no matter what happens. The other benefit of this model is that users could pick and choose how they want to help fight spam. They could choose just one option, any combination of the 3 or even all 3. Something for everyone.

--
Most failure is due to giving up, not realizing how close to success you were - Thomas Edison

Wed, 2006-12-27 18:27 — Poltergeist

Complexity

Quote:
The opt-out portion of bluefrog wasn't anything special. It simply executed some specially designed scripts. This would be trival to code in as a plug-in for spampal as would one for reporting.

Indeed you could implement basic opt-out functionality with trivial code. The problem is that then the spammers can, with trivial ease, turn your network into a botnet, and/or you have to make your network depend on one or a few publicly known Central Authorities, which the spammers can then easily DDOS and/or take over and use to control your frog network for their own purposes.

We had quite a lot of discussion about this. I think my solution is the simplest so far, and it's certainly not trivial. If you have a simpler solution that would really be immune against the spammers, everyone here will be very happy indeed, me included. I'd love to see some competition.

Thu, 2006-12-28 10:00 — knayil

I was referring to the

I was referring to the script execution system, not delivery. While I do not know what you suggested, a simple solution for delivery would be to use libgpg to sign all updates. Simple error check if ( code != signed ); break (of course little more than that but you get the idea). Only the core developers get the private key, and you change it ever so often (30-90 days?) to make sure it doesn't get compromised.

The program links against that lib, gets the public key from a public key server and if any script or update isn't signed by the key it will not execute. You could even go a step further and have it shut down the program and alert the user and maybe even the other nodes something was fishy with X script. PGP is well established, your key is distributed from multiple points and the updates can be passed out via torrent. Now you have strong security and no central point to attack.

They could theoretically take down the key servers, but I can't see them cracking the key as long as you use a strong password to sign with and a secure machine where the private key is located. What good would it do to take down the key servers? They would have to keep it down for say 30-90 days to keep it from getting the new keys. You could even add an extra layer and use SSL in with all the communications between nodes.

--
Most failure is due to giving up, not realizing how close to success you were - Thomas Edison

Sat, 2006-12-23 22:55 — pspri92229

Great!

This sounds like a real idea.