NetWizard's OLD Blog

Moving on...

2004-02-12T01:33:00.000-05:00

I finally got around to switching my blog software to Movable Type from the hosted Blogger.com. Bear with me while I work out the kinks. Also, I was able to get moblogging going with my Treo, so I am really looking forward to this.

The new blog location is at shaftek.org/blog/.

Our Arrays bug made it into the Sun's database

2004-02-03T13:44:00.000-05:00
One of the bugs we reported to Sun before got posted to Sun's bug database under # 4987749 (although it might take a day or two to propogate). Use this link:

http://developer.java.sun.com/developer/bugParade/bugs/4987749.html

Draft on some causes of spam

2004-01-21T16:52:00.000-05:00
I posted a new draft which is VERY RAW about some causes of the spam problem. Of course, I can be completely wrong and off the wall, but it is something that might end up being useful. You can find it here.

The recent proliference of Anti-Spam Groups

2004-01-15T16:46:00.000-05:00
There seem to be a recent blooming of new anti-spam groups, must be the upcoming arrival of Spring. The ASRG has been around since last spring and MSFT/AOL/Yahoo AntiSpam Alliance has been around since the early summer. However recently two new groups have come up, and its a bit unclear what they do. One is the Anti-Phishing Working Group which targets "phishing" of credit card numbers. Since this is usually done via spam, the connection is obvious. No membership list of their website. The other, is a new MAAWG started by OpenWave and its customers. It would be interesting to see what develops...

Just some misc updates:
1. The ASRG got a new website, with a cute name - asrg.sp.am (sp.am domain curtesy of my ASRG co-chair, John Levine).
2. I switched RSS readers to a cross platform one called AmphetaDesk. Its written as a local webserver and is cross platform, so I can use the same settings on both Windows and Linux.
3. Upgraded Linux to Redhat Fedora Core 1. Not much difference from my old RH9. Maybe one of these days I will try out Debian.

Another Week, Another JVM bug

2004-01-12T15:48:00.000-05:00
A copy of the bug report we submitted to Sun is included here for the curious:

release: 1.4.2
hardware: x86
OSversion: Linux
synopsis: ArrayOutOfBounds Exception on LinkedList with Mutliple Thread WITH -server
description: FULL PRODUCT VERSION :
Tested VMs:
* 1.4.1_03-b02 - does not fail
* 1.4.2_02-b03 - fails
* 1.4.2_02-b28 - fails
* 1.4.2_03-b02 - fails

FULL OS VERSION :
Redhat Linux 8, kernel v 2.4.24
Windows 2000 SP2
Windows XP SP1

EXTRA RELEVANT SYSTEM CONFIGURATION :
Tested hardware:
* Linux - Desktop Intel P-II 333Mhz
* Windows 2000 - Desktop AMD Athlon
* Windows XP - AMD Athlon

A DESCRIPTION OF THE PROBLEM :
When operating on LinkedLists in a multithread environment such as a J2EE server (JBoss in our case),
with the "-server" option turned on, there the toArray(Object []) method of the LinkedList class throws
occasional ArrayOutOfBounds errors. There are no synchronization issues since the variables are local to
each thread. This problem DOES NOT happen on the "-client" HotSpot VM. This error DOES NOT OCCUR
on JVM 1.4.1 under Linux.

In our understanding of the server VM document (http://wwws.sun.
com/software/solaris/java/wp-hotspot/#pgfId=1082013), this can be related to the following optimization:

"Range check elimination -- The Java programming language specification requires array bounds checking
to be performed with each array access. An index bounds check can be eliminated when the compiler can
prove that an index used for an array access is within bounds."

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Have multiple threads operate on local LinkedList variables USING the toArray method

EXPECTED VERSUS ACTUAL BEHAVIOR :

EXPECTED -
No errors

ACTUAL -
The following exception is thrown very often:

java.lang.ArrayIndexOutOfBoundsException: 591
at java.util.LinkedList.toArray(LinkedList.java:657)
at Test$RunThread.run(Test.java:153)

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.ArrayIndexOutOfBoundsException: 591
at java.util.LinkedList.toArray(LinkedList.java:657)
at Test$RunThread.run(Test.java:153)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.LinkedList;

/**
* Test case to show ArrayIndexOutOfBoundsException bug on
* toArray(Object[]) call to a LinkedList. This only happens in
* a multithreaded environment and running on a server JVM.
*
* Tested VMs:
* 1.4.1_03-b02 - does not fail
* 1.4.2_02-b03 - fails
* 1.4.2_02-b28 - fails
* 1.4.2_03-b02 - fails
*
* Tested OSs:
* Redhat Linux 8 with 2.4.24 kernel
* Windows 2000 SP 2
* Windows XP SP1
*
* Tested hardware:
* Intel P-II 333Mhz
* AMD Athlon 2500+
* AMD Athlon 1Ghz
*
* To execute this test run:
* java.exe -server Test
*
* A log file will be created along with standard output. If this
* test does not fail for you right away increase the number of runs
* and threads and try again.
*
*/
public class Test {

//main entry
public static void main(String[] args) throws IOException {
Test test = new Test();
test.test();

}//main()

//main run method
public void test() throws IOException {

//modify the following params to increase the number
//of inner and outer iterations as well as the thread count
int runs = 1000;
int threads = 100;
int outerRuns = 20;

//create counter and synchronizer
Counter cnt = new Counter("test.log");
cnt.status("Starting");

try {

//iterate through outer runs
for(int x = 0; x < 10; x++) {
cnt.status("Outer Run Starting: " + x);

//create and initialize all threads
RunThread[] rt = new RunThread[threads];
for(int i = 0; i < threads; i++) {
rt[i] = new RunThread("Thread " + i, runs, cnt);
rt[i].start();
}//for

//tell all threads they can start working
cnt.run();
cnt.status("Outer Run Waiting: " + x);

//wait for about 30 seconds for all of them to complete
int c = 0;
while(cnt.count > 0) {
c++;
try { Thread.sleep(200); }
catch(Throwable e) { e.printStackTrace(); }

if(c > 1500) {
cnt.stop();
cnt.status("Waited too long, aborting");
}//if
}//while

//flag stop to all threads and get ready to
//reinitialize
cnt.stop();
cnt.status("Outer Run Finished: " + x);
}//for
} catch(Throwable e) {

//catch outer iteration exceptions
cnt.status("Outer Failure:");
cnt.error(e);

}//try..catch

//finish up, close log file
cnt.status("Done");
cnt.stop();
cnt.done();
}//main()

//main test thread
private class RunThread extends Thread {
protected int runs;
protected String name;
protected Counter cnt;

public RunThread(String n, int r, Counter c) {
name = n;
runs = r;
cnt = c;

}//RunThread()

//main inner iteration method
public void run() {

//tell counter we are ready to start
cnt.inc(1);

//wait for counter to say its ok to start
while(!cnt.isRunning()) {
try { sleep(100); }
catch(Throwable e) { e.printStackTrace(); }
}//while

//initialize local vars
int lsize = -1;
LinkedList lst = new LinkedList();
Integer[] last = null;

try {

//main inner iteration loop
for(int i = 0; i < runs && cnt.isRunning(); i++) {

//add to list
lst.add(new Integer(i+1));

//record size prior to toArray call
lsize = lst.size();
last = new Integer[lst.size()];

//place toArrayCall
last = (Integer[]) lst.toArray(last);

}//for

//this is a sucessfull complete, tell counter we are done
cnt.inc(-1);
} catch(Throwable e) {

//this is an error complete, still tell counter we are done
cnt.inc(-1);

//try to log the error
try {

//make sure no one else is writing to the log
//at the same time
synchronized(cnt) {

//record failed thread, sizes, error, and list contents
cnt.status("");
cnt.status(name + ": Failed");
cnt.status("Last Size: " + lsize);
cnt.status("Reported Size: " + lst.size());
cnt.status("Feed Size: " + last.length);
cnt.error(e);
cnt.status("Dump: " + lst.toString());
cnt.status("");

}//syncronized

} catch(Throwable ee) { ee.printStackTrace(); }
}//try..catch
}
}

//main counter and synchronizer, takes care of waiting and logs
protected class Counter {
int count = 0;
boolean running = false;
PrintWriter buff; //log buffer

//initialize log file
public Counter(String file) throws IOException {
buff = new PrintWriter(new BufferedWriter(new FileWriter(file)));

}//Counter()

//run state methods
public synchronized void run() { running = true; }
public synchronized void stop() { running = false; }
public synchronized boolean isRunning() { return running; }

//running count increment/decrement method
public synchronized void inc(int i) { count += i; }

//status output method
public synchronized void status(String str) throws IOException {
System.out.println(str);
buff.println(str);
buff.flush();

}//status();

//error output method
public synchronized void error(Throwable e) {
e.printStackTrace();
e.printStackTrace(buff);
buff.flush();
}//status();

//finishing method, log file close
public synchronized void done() {
buff.flush();
buff.close();

}//done()
}//Counter
}//Test

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
1. Use the "-client" VM instead - in our case with J2EE server, there is performance degradation.
2. Use the ListIterator and create the output array manually. Much slower than the toArray(Object)
method of LinkedList if using custom objects.

ASRG Rechartered

2004-01-04T13:04:00.000-05:00
The ASRG was rechartered today. New charter can be found here.

More info on the getBytes() bug

2003-12-23T22:42:00.001-05:00
After some digging in the JVM source code, our CTO came out with the exact source of the problem and a possible workaround. Of course this is all at your risk, so don't blame us or Sun if you have problems. A copy of the message we submited to Sun appears here:

SOURCE CODE TRACE

We took a look at the source code of the JVM. The problem stems from the fact that float values are used to indicate the maximum value of bytes per characters in java.nio.charset.CharsetEncoder.maxBytesPerChar.

The issue is that floats cannot accuratly hold more than 2^24 integer values which is equals to 16,777,216. After that value is reached, the encoding operation in the character set classes incorrectly rounds down the amount of memory needed for the buffer. The correct solution would be to use doubles instead, or account for the round off problem by increasing the buffer size.

SUGGESTED WORKAROUND

The workaround that we are using, is to use to .getBytes() on a substring that is smaller than 16MB, and combined the results by either using a ByteArrayOutputStream or a ByteBuffer.

NOTE: If you are planning on using more than one-byte characters sets, than you have to make sure that your buffer is set accordingly.

More info on the getBytes() bug

2003-12-23T22:42:00.000-05:00
After some digging in the JVM source code, our CTO came out with the exact source of the problem and a possible workaround. Of course this is all at your risk, so don't blame us or Sun if you have problems. A copy of the message we submited to Sun appears here:
SOURCE CODE TRACE

We took a look at the source code of the JVM. The problem stems from the fact that float values are used to indicate the maximum value of bytes per characters in java.nio.charset.CharsetEncoder.maxBytesPerChar.

The issue is that floats cannot accuratly hold more than 2^24 integer values which is equals to 16,777,216. After that value is reached, the encoding operation in the character set classes incorrectly rounds down the amount of memory needed for the buffer. The correct solution would be to use doubles instead, or account for the round off problem by increasing the buffer size.

SUGGESTED WORKAROUND

The workaround that we are using, is to use to .getBytes() on a substring that is smaller than 16MB, and combined the results by either using a ByteArrayOutputStream or a ByteBuffer.

NOTE: If you are planning on using more than one-byte characters sets, than you have to make sure that your buffer is set accordingly.

After two days of bug hunting ... a JVM bug!

2003-12-23T20:28:00.000-05:00
We have been hunting out a nasty bug in our code involving processing of large XML files via Java String, StringBuffer and byte[]. We finally narrowed it down to the following line:
byte[] byteArray = largeString.getBytes();

which fails with:
java.nio.BufferOverflowException
at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
at java.lang.StringCoding$CharsetSE.encode(StringCoding.java:340)
at java.lang.StringCoding.encode(StringCoding.java:374)
at java.lang.StringCoding.encode(StringCoding.java:380)
at java.lang.String.getBytes(String.java:590)

and guess what, it fails, and its now our code either :( After doing some head scratching and checking around Sun's site, this came out to be a JVM bug # 4949631.

P.S. As I was searching through the Sun's bug site, it was really sad to see the same bug being mentioned over and over, and being closed because the submitters could not provide normal code to reproduce it. Good thing to keep in mind when submitting bug reports.

AOL offers $299 PC ... with StarOffice

2003-12-04T01:34:00.000-05:00
According to this News.com story, AOL is offering a M$FT Windows XP PC for $299 if you sign up for AOL Service for a year. They also have a website "299pcdeal.com". BUT, the most interesting stuff is on their site and documentation. This PC comes with ... AOL Office by Sun - which according to the documentation look like a repacked version of StarOffice.

RSS is great!

2003-11-25T12:01:00.000-05:00
I recently took the plunge and tried out some RSS readers. I like the results - instead of checking websites for news, I can now work on things and have the headlines pop up in my systray or my phone. With a quick look I can see which headlines interest me, and which do not, saving on time and bandwidth.

I have tried out several RSS readers, and my current favorites are Nick Bradbury's FeedDemon for Windows ((in BETA), StandAlone's HandRSS for Palm OS and Sean Gallagher's News Spotter for Linux.

My current favorite feeds are: CNN, CNET News.com, GrokLaw, InfoWorld's top technology news, Kuro5hin.org, National Weather Service alerts, Microsoft Watch, Slashdot, The Register, Yahoo's Top Stories, and Wired News.

Another Plan for Spam

2003-11-22T23:42:00.000-05:00
There have been rumors for a few months about the secretive Microsoft / AOL alliance against spam. A recent story revealed some of those plans:

"One organization working on sender-authentication mechanism is a commercial alliance comprising the biggest consumer e-mail providers: Microsoft, Yahoo, America Online and Earthlink."

"Under the proposal, ISPs and any other organization with their own domain name system (DNS) would use a private key in their mail servers to place an encrypted code in the header of each piece of outgoing mail. When the mail arrived at its destination, the receiving mail server would get the sender's public key from its DNS server to decrypt the header, thus verifying the message's origin.

If the message is spam, or even a legitimate marketing message the receiver doesn't want, then email from that DNS can be blacklisted, or automatically blocked. "Once you have identity, then you can establish reputation and trust," Libbey said. "Those are really important concepts in e-mail."

Yahoo has done some proof-of-concept testing of the idea internally, but the technology is still at the early stages of development and no timetable for general release has been set."

Comment on Mossberg's praise of C/R

2003-11-18T01:40:00.000-05:00
This is written in response to a recent column by Walter Mossberg that was printed in the Wall Street Journal. In this column, Walter calls challenge/response technology "the only truly effective method for fighting spam". The following response from myself was sent to Walter.

It is hardly possible to call C/R "the only truly effective method". The more correct description would be "a rather effective method TODAY", as I am going to explain.

The basic C/R process has three different approaches[4]: (1) autheticating the sender, (2) making sure the same exact message originated from the sender, and (3) making sure the sender is human by presenting some graphic (Turing tests). There are numerous problems with these three approaches (see [4] section 2.2].

The first approach where a sender is being autheticated by sending an email to his address, puts an additional burden on the sender. The problem also is that many C/R systems are not implemented properly[2][5] and can end up challenging other machines or even each other[2]. Furthermore, nothing stops spammers from setting up automated systems which respond to such challenges[8]. Mailing lists are also affected by C/R systems to a degree where some list operators have refused to respond to C/R messages all together because of the burden[8]. Additionally, given enough incentive spammers can forge the return address of someone from the receiver's whitelist, granting them access[2]. There are also numerous privacy issues[8].

The second approach which asks the sender's server whether a specific message originated from that server, is not widely used today because of a lack of protocols to enable such transactions [1] (the ASRG is currently considering one such protocol called CRI [4]). The problems inherent with this approach is the lack of a standard protocol [1], inability to people who are travelling to send email from different servers[1], and the need to the sender's server to keep copies of all sent messages which raises significant privacy issues[1].

The third approach, being the most popular today, includes in the message a link to an image or an image that presents the sender with a puzzle, usually some letters or numbers, that cannot be readily recognized by a computer, but can be easily read by a human being. These are called "reverse Turing tests" [4]. There are numerous problems with these tests. First of all, they cannot be used by disabled users [4], which has been recently pointed out by the W3C in their draft on the issue [6]. Second, it has not been proven that these images cannot be recognized by a computer, and some approaches have been broken by spammers. Third, given enough incentive a spammer can hire people in developed countries to answer the challenges, still regaining the cost advantage[8].

However, the best example of spammers's innovation ability is the recent finding of a spammer who setup a free porn site which serves a Turing test challenge every five or ten images. People visiting the site, responded to the challenges solving spammer's problem[7]. While this particular case was not used to break C/R systems, but rather registration forms, the point is the same.

While challenge/response is an effective method of fighting spam today, with some anti-spam companies deploying the technology reporting over 99% block rates, nevertheless it is not the ultimate or the "only" solution to spam. Additionally, the percentage of deployed C/R systems is so low, that spammers do not have the incentive to break them[8]. This is akin to having a security product that has never been tested in public - until hackers actually start attacking, one cannot claim that it is effective. The real test of C/R will come once a significant number of C/R systems are deployed, and spammers will try to break them. It is then that the C/R technology will either win or fall by the wayside like many others.

Therefore, challenge / response is only one of the many tools used to fight spam, and cannot be called "the only truly effective method" by any strech of imagination. One thing must be kept in mind that C/R addresses the issue of forgery in email[1], something that is a general authetication problem which has many solutions. C/R is not the only solution for that problem out there, there are many others which may or may not operate better[1][2]. Putting things in perspective, the spam problem has many facets, and authetication vs. forgery is only one of them. An effective response to spam requires a combination of many efforts on many fronts including technical, legal, and social. Even within the technical realm, an effective solution requires an on-going, adaptive effort, with stochastic rather than complete results, utilizing multiple, adaptive techniques[3].

References:
1. Leibzon, W.; "Email Path Verification", ASRG,
http://www.elan.net/~william/asrg-emailpathverification-presentation.pdf
2. Levine, John R.; "Technical Responses to Spam", Taughannock Networks,
http://www.taugh.com/spamtech.pdf
3. Crocker, D., Levine, J., and Schryver, V.; "Technical Considerations for Spam Control Mechanisms", ASRG,
http://www.ietf.org/internet-drafts/draft-crocker-spam-techconsider-02.txt
4. Dean, E., and Shafranovich, Y.; " Challenge / Response Interworking (CRI) Framework for Challenge / Response Email Systems", ASRG,
http://www.ietf.org/internet-drafts/draft-irtf-asrg-cri-00.txt
5. Templenton, B.; "Proper principles for Challenge/Response anti-spam systems"
http://www.templetons.com/brad/spam/challengeresponse.html
6. May, M.; "Inaccessibility of Visually-Oriented Anti-Robot Tests: Problems and Alternatives"; W3C,
http://www.w3.org/TR/turingtest/
7.Spice, B,; "CMU student taps brain's game skills", Pittsburgh Post-Gazette,
http://www.post-gazette.com/pg/03278/228349.stm
[ALSO see: https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07899.html]
8. ASRG Mailing List,
https://www1.ietf.org/mail-archive/working-groups/asrg/current/maillist.html
http://news.gmane.org/gmane.ietf.asrg

Added PGP key

2003-11-14T15:55:00.000-05:00
I finally got around to installing GnuPG into my Mozilla client with EnigMail for Mozilla. Going over my ancient keys, I discovered one from back in 1995 when I used PGP 2.x. Well, I decided to get a new key, so here it is.

Changes at the ASRG

2003-11-14T02:06:00.000-05:00
A leadership change took place today at the ASRG. My co-chair in the group, Dr. Paul Q. Judge of CipherTrust, stepped down today and was replaced by Dr. John R. Levine of I.E.C.C. (in case you are wondering it stands for "The Invincible Electric Calculator Company" according to the homepage :) ).

On a related note, my company's ISP had a DDOS attack. I wonder if this has any relevance to my work in the ASRG, especially considering recent attacks on SpamHaus.

P.S. I got linked from someone else's webpage, thanks katie-and-rob.org.

SPEWS and CAPTCHAS

2003-11-06T14:58:00.000-05:00
Well, it seems that SPEWS is back, no explanation on what happened of course.

Also, today I ran across an article about how a spammer was able to defeat the visual Turing tests used by some C/R systems:
But at least one potential spammer managed to crack the CAPTCHA test. Someone designed a software robot that would fill out a registration form and, when confronted with a CAPTCHA test, would post it on a free porn site. Visitors to the porn site would be asked to complete the test before they could view more pornography, and the software robot would use their answer to complete the e-mail registration.

This was originally posted on Matt McCay's weblog. There is also a weblog entry by Liudvikas Bukys asking for specific examples such as this one.

SPEWS.ORG domain name has been killed

2003-11-05T19:23:00.000-05:00
According to WHOIS records at PIR the domain name of SPEWS.ORG has been deactivated:

Domain ID: D74783489-LROR
Domain Name: SPEWS.ORG
Created On: 07-Jul-2001 19:50:12 UTC
Last Updated On: 03-Nov-2003 15:44:43 UTC
Expiration Date: 07-Jul-2008 19:50:12 UTC
Sponsoring Registrar: CSL Computer Service Langenbach GmbH (R25-LROR)
Status: INACTIVE

The name server is pointing to INVALID-ADDRESS.JOKER.COM. Looks like the spammers are at it again. For those folks that want to get to SPEWS, use SPEWS.US. There is also a Usenet thread on this.

When Big Companies Send Spam - No one cares

2003-11-03T20:22:00.000-05:00
Among the tons of spam that I usually receive, two particular pieces have caught my attention. One was an advertisement for some photo-mosaic thingy BUT it was sent to an email address at my domain that has only been used for one single company - Godiva Chocolate. Now their privacy policy OBVIOUSLY states that they WILL NEVER sell your information. Well guess what - they LIED!!!!! And to rub it in, they are even certified by the BBB. I have emailed them but so far - no response. If my second email does not get a response, it is time to go to the BBB and the FTC (yes, privacy policies can be enforced by the FTC).

Going to exhibit #2, is an email that was received today actually on behalf of Gartner, Inc. and sent by Exact Target, LLC of Indiana. OF COURSE they have a no-spam policy and a privacy policy TOO. Well guess what - the email address they emailed to, was the one used for my cell phone which I NEVER share with anyone since it sends messages DIRECTLY to my phone. Now suprise, suprise - so much for "opt-in" and "anti-spam policies".

To make things more interesting, they are based in Connecticut and the company that actually sent the email in Indiana. BOTH states have anti-spam laws (see SpamLaws.com). The Indiana one required "ADV" and an unsubscribe link BOTH of which were absent. So I FAXED them a letter with the original spam message (all 16 pages of it. he he), and we'll see what happens.

The lesson of this is that even the big guys endulge in spam no matter how they may scream that they don't. Perhaps the consent framework or a Federal Law is the only thing that can stop them...

For interested parties, I saved copies of both spam emails, contact me if you want to see them

New Draft of the Consent Framework and Java Musings

2003-11-02T23:16:00.000-05:00
Almost forgot to mention - a few days ago I posted a new draft of the consent framework. This draft (v0.03) has less insurance-policy language and more plain English, plus some pictures.

Another thought while I am at it - if you ever use Java Servlets, and wonder why everything stops working after switching to "multi-form" format - its not a bug but a feature. Pick up a copy of Apache Commons / File Upload package to fix the problem.

Some housekeeping

2003-11-02T22:43:00.000-05:00
Its been a while since I updated my homepage so tonight I decided to do a little bit of housekeeping: just some links added and a copy (PDF / PPT) of my presentation from the Open Group meeting has been added.

I recently got the new Handspring Treo 600 with a camera and am currently experimenting with a picture moblog. So far the only thing that has made it there is the cat but I am hoping to find some more novel pictures to post.

This month I also switched hosting from my company to this weird new provider called "1 and 1" - they are offering FREE three years of hosting so I jumped at the deal. Considering that some of the stuff on my site might get SlashDoted due to the ASRG affiliation, I rather not take chances with my company's system but with someone else's - especially if its free!

Unfortunatly, in the process of switching a rather weird thing happened - all email to my domain bounced for about two days. That has been the longest amount of time either my personal or business email has been down, EVER. A very weird disconnected feeling occured to me when that happened.

Some comments on another Verisign Story

2003-10-17T01:39:00.000-04:00
A recent News.com story has the following quote:

Still, a lot of people in the Internet community were quite surprised by Site Finder--and then you had complaints surfacing that it was not complying to approved standards.

Let's break the argument down: The claim that Site Finder was nonstandard and that we should have informed the community that we were doing something nonstandard--excuse me: Site Finder is completely compliant to standards that have been out and published by the IETF (Internet Engineering Task Force) for years. That's just a misnomer. The IAB (Internet Architecture Board) in its review of Site Finder said the very same thing--that VeriSign was adhering to standards.

The second claim, that we brought it out without testing--Site Finder had been operational since March or April, and we had been testing it with individual companies and with the DNS traffic at large. Ninety-nine percent of the traffic is pure HTTP (Hypertext Transport Protocol), and so it handles it the way it should. Just so you know, our customer service lines went from 800 or 900 calls on the first day to almost zero right now. For every customer who had a Site Finder issue, the remediation took less than 12 hours.

First of all, 90% of traffic is HTTP yet many email and spam systems where broken due to a non-compliant SMTP server. And second, it took more than 12 hours to replace the server with a compliant one. Note the phrase "For every customer" which seems not to include the regular Internet folks and us at the IRTF.

Spam, ASRG, consent and "herding cats"

2003-10-07T20:40:00.000-04:00
The ASRG is moving slowly towards its goals. Apparently that has been way tooo slow to some people. Recently, I ran across an post somewhere that was asking for a opinion of the ASRG. The response was not positive. It seems that many hardcore anti-spammers consider us either too soft-spined, consisting solely of goofy salesmen and kooks, or being pro-spam by pushing the notion of "consent" into email. Nothing arouses the passions of anti-spam fighters more than an wolf in sheep's clothing - an anti-spam group pushing spam. Unfortunately, the consent framework has not made things better since it is in its early draft stage and its not too clear (in the words of an ASRG member "this thing looks like an insurance policy").

We take on the notion of consent as a concept that would allow for unification of various anti-spam tools and proposals into once concrete architechture. This would not mean that senders get to see everyone's consent - rather receivers can express consent or what kind of emails they want to receive, and the tools will enforce that decision. This would give the freedom to every user to pick and choose their anti-spam preferences in a standard way, while allowing various anti-spam tools to interpreted. However, in some ways this is akin to treating the symptoms of a decease instead of the cause. On one hand, if the email system allows for consent and denial of consent this would imply that less spam goes through. On the other hand, the fundamental problems causing spam do not go away. Long-term if the consent framework kills most spam, people would be satisfied since small amounts of spam only cause small problems. But changing the wiring of the Internet long-term might be an option.

The last comment I wanted to add is one about "herding cats". This has been a phrase many times used to refer to getting engineers to work together, and in IETF/IRTF lingo has come to mean getting volunteers to do work. This has been a constant problem in all groups, not just the ASRG and sometimes is very frustrating. But those are the facts of life, and we are forced to work with what we have. So, I keep on "herding cats" and hopefully get the ASRG move along a bit more...

That's all folks.

REMINDER: ICANN's meeting on Verisign is TODAY

2003-10-07T02:40:00.000-04:00
ICANN is having a meeting TODAY on Verisign. For more information see:
1. Announcement.
2.Agenda.
3. SOME attendees (note the NTIA and DHS people).

Alas, I will not make but I am sure that ICANN Watch will be providing a host of coverage along with blogs and the media. Oh well...

Verisign caves in

2003-10-03T17:07:00.001-04:00
Verisign decided to cave in to ICANN and suspend their SiteFinder service according to a press release. Some VERY INTERESTING quotes:

"Without so much as a hearing, ICANN today formally asked us to shut down the Site Finder service."
Hmm, hint, hint here - how about the fact that you turned on the service without a hearing?

"The next several weeks will be a test as to whether innovation will occur within the Internet infrastructure."
So now Verisign is the "innovator" on the Internet and I guess all the IETF/IRTF/IAB folks (myself included) and everyone else is against innovation.

This can be summed up very well:

Power corrupts, and absolute power corrupts absolutely (Lord Acton)

Verisign caves in

2003-10-03T17:07:00.000-04:00
Verisign decided to cave in to ICANN and suspend their SiteFinder service according to a press release. Some VERY INTERESTING quotes:
"Without so much as a hearing, ICANN today formally asked us to shut down the Site Finder service."
Hmm, hint, hint here - how about the fact that you turned on the service without a hearing?
"The next several weeks will be a test as to whether innovation will occur within the Internet infrastructure."
So now Verisign is the "innovator" on the Internet and I guess all the IETF/IRTF/IAB folks (myself included) and everyone else is against innovation. The wonders of the business world.

ICANN gives Verisign 48 hours to shut down SiteFinder

2003-10-03T11:11:00.000-04:00
Just saw an article on ICANN Watch this morning, informing us that ICANN is asking VeriSign to shut down SiteFinder in 48 hours or "ICANN will be forced to take the steps necessary to enforce VeriSign's contractual obligations". The original announcement can be found on ICANN's website together with a letter to Verisign. More adjustments to spam tools...

An October Update and some musings on Verisign

2003-10-03T02:03:00.000-04:00
As I said on my homepage - "but do not expect frequent updates". So I finally got around to updating this blog.

Well, the past week or so has been pretty busy. The ASRG had seen some pretty heavy posting volume although today it has slowed down a little. We finally got all of the various RMX people together so they can start working on a common proposal and lots of other good efforts are under way as well.

Also, ICANN has announced a meeting to discuss Verisign's recent behavior - it will be held October 7th in Washington, DC. I looked over the invitees list and there are 23 people listed as of right now, whose affiliations are very interesting. They seem to me to line up as follows:

Defendants - Verisign and the company providing the whole DNS/search thing - Paxfire
Competition - mainly other gTLD Registraries - EDUCAUSE (.edu and they are non-profit), Neustar (.us, .kids.us, .cn and .biz), and Afilias (.info and they also provide the backend for .org, which is operated by PIR / ISOC), etc.
"Uncle Sam" - NTIA - they OWN (in the fullest sense of that word) the DNS system which ICANN operates under contract further subcontracting .COM and .NET to VeriSign
Lawyers - someone from the Cardozo School of Law, plus I am sure many others
Internet standards folks - IETF and affiliated private individuals, etc.
Big Business - AT&T, Namespace Strategy Group, etc.
Non-Profits - SDF, etc.
Consultants - Summit Strategies International?, others, etc.
Others - all the ones I can't decipher

Pretty interesting possibilities, eh? So the defendants will argue that they didn't do anything wrong, the competition and big business will either defend them so they can do the same thing, or attack them in hopes of getting the .COM and .NET contracts while the non-profits and the IETF folks will preach standards adherence. Lawyers and consultants will hover over like vultures in hopes of getting some carrion from whomever loses (just think "class action lawsuit" - don't even think, there are a few filed already!). Meanwhile ICANN and "Uncle Sam" (who legally is the owner of the DNS system and is the final decision maker), will try to decide who is right (although if the US Government owns DNS, and taxpayers own the government, that means I own the DNS system, and I get to decide to who wins...). Anyway, this sounds like a bad joke - "a lawyer, a businessman and a g-man...." ANYWAY, I wish that someone from the anti-spam community would be there but maybe the IETF folks will keep anti-spam solutions in mind when discussing the issue. So we'll see what happens with this...

But in any case, the IAB has already stated:
"Proposed guideline: If you want to use wildcards in your zone and understand the risks, go ahead, but only do so with the informed consent of the entities that are delegated within your zone." and that "We recommend that any and all TLDs which use wildcards in a manner inconsistent with this guideline remove such wildcards at the earliest opportunity."
Now the IAB "provides oversight of, and occasional commentary on, aspects of the architecture for the protocols and procedures used by the Internet." And I think they are pretty experienced with the Internet, since they were around since back in 1979.

So in conclusion I think we should trust what they recommend that "any TLDs which use wildcards..." (hint, hint) "remove such wildcards at the earliest opportunity"

Some maintenance and housekeeping

2003-09-24T00:07:00.000-04:00
Did some housekeeping on the weblog: changed the template, archiving options and enabled title display. Also added some extra links and sections to the homepage.

Updated link to Verisign DNS petition

2003-09-22T12:29:00.000-04:00
Here is an updated link sent in by George Kirikos:

http://www.whois.sc/verisign-dns/

"All your email belongs to us" - Verisign

2003-09-19T02:12:00.000-04:00
The "Snubby rejection daemon" of Verisign fame (see http://news.com.com/2100-1032-5077530.html) has been now upgraded to version 1.5! It accepts EHLO and most SMTP commands, and looks RFC compliant.

Additionally, as per below log, it accepts mail addressed to as per section 4.5.1 of RFC 2821. It does not accept mail addressed to , so misaddressed mail bounces UNLESS the sender's MTA/MUA uses the format. The mail log appears below:

-------------snip------------
open 64.94.110.11 25

220 Snubby Mail Rejector Daemon v1.5 ready
HELO snubby.org
250 snubby
RSET
250 Ok
HELO snubby.org
250 snubby
MAIL FROM:<>
250 Ok
RCPT TO:
250 Ok
RCPT TO:
550 : Client host rejected: The domain you are
trying to
send mail to does not exist.
DATA
354 End data with .
Subject: ASRG test
From: research@solidmatrix.com

This is a test for the ASRG - see www.irtf.org/asrg

.
250 Ok: queued as E449472F8
RSET
250 Ok
EHLO shdhdh.com
250-snubby
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-XVERP
250 8BITMIME
RSET
250 Ok
QUIT
221 Bye

Connection to host lost.
-------------snip------------

This will happen under the following circumstances (what are the chances I don't know, most email clients do not use the mailbox format):
1. The receiving domain does not exist.
2. The message is addressed to postmaster @ nonexistingdomain.
3. The sender's email client uses the mailbox format for the RCPT TO command: RCPT TO:<postmaster>instead of RCPT TO:<postmaster@domain.com>.

Here is a sample log:

-------------snip------------
open 64.94.110.11 25

220 Snubby Mail Rejector Daemon v1.5 ready
HELO snubby.org
250 snubby
RSET
250 Ok
HELO snubby.org
250 snubby
MAIL FROM:<>
250 Ok
RCPT TO:<postmaster>
250 Ok
RCPT TO:<postmaster@nowoneoeoeoeoisherehehe.com>
550 <unknown[68.27.148.70]>: Client host rejected: The domain you are
trying to
send mail to does not exist.
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: ASRG test
From: research@solidmatrix.com

This is a test for the ASRG - see www.irtf.org/asrg

.
250 Ok: queued as E449472F8
-------------snip------------

CNET article

2003-09-16T19:16:00.000-04:00
A recent CNET article (http://news.com.com/2100-1032_3-5077530.html) slightly misquoted me but that has been fixed.

Petition against Verisign "typos" service

2003-09-16T17:50:00.000-04:00
Here is a link to a petition setup by George Kirikos:

http://www.petitiononline.com/icanndns/

"All your typos belong to us" - Verisign

2003-09-16T15:07:00.001-04:00
I am sure people have heard by now about Verisign's recent changes to the .COM/.NET servers - now all your typos belong to them - all non-existing domain names resolved to sitefinder.verisign.com. For some more information checkout these links:

Original Verisign announcement of the change:
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07318.html

Messages regarding the "fake" SMTP server:
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07323.html
http://www.ietf.org/mail-archive/ietf/Current/msg22247.html

Reports of anti-spam tools failing:
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07326.html
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07328.html
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07318.html

Reports of SMTP failures:
http://www.ietf.org/mail-archive/ietf/Current/msg22254.html
http://gnso.icann.org/mailing-lists/archives/ga/msg00329.html

ISP reactions:
https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg07329.html
http://gnso.icann.org/mailing-lists/archives/ga/msg00326.html

Additional DNS problems:
http://gnso.icann.org/mailing-lists/archives/ga/msg00310.html
http://gnso.icann.org/mailing-lists/archives/ga/msg00317.html
http://www.merit.edu/mail.archives/nanog/msg13673.html

Trademark issues:
http://gnso.icann.org/mailing-lists/archives/ga/msg00321.html

Plus:
NANOG List Archive
IETF List Archive
ASRG List Archive
ICANN GNSO List Archive

The ASRG

2003-08-24T01:46:00.000-04:00
Well its official - I got appointed as a co-chair of the Anti-Spam Research Group (ASRG) of the IRTF.

I wonder what the spammers think about this....

Homepage housekeeping

2003-08-22T16:55:00.000-04:00
Spent some time cleaning up the HTML, fixing up some text and putting in a nice graphic from the Internet Traffic Report.

SCO declares GPL invalid, huh?

2003-08-14T15:42:00.000-04:00
I saw this in the Wall Street Journal this morning (here is a link, subscription required), before this got posted on SlashDot.

From the WSJ article:
"Now, SCO is preparing to wheel out the software-industry equivalent of a nuclear bomb: It will argue that the GPL itself is invalid, says SCO's lead attorney, Mark Heise of Boies Schiller & Flexner LLP. Mr. Heise says the GPL, by allowing unlimited copying and modification, conflicts with federal copyright law, which allows software buyers to make only a single backup copy. The GPL "is pre-empted by copyright law," he says."

Trip down memory lane

2003-08-10T02:55:00.000-04:00
Tonight I tried again to remove my old homepage based at CompuServe. That required installing some ancient software like CompuServe v3 and Microsoft's Web publishing wizard. So far nothing worked with my current Windows XP installation. Will try again.

Outsourcing and open sourcing

2003-08-08T16:21:00.000-04:00
All these stories about large companies (Sprint, JP Morgan, and others) outsourcing to India brings some thoughts.

A lof the jobs moving overseas are programming jobs. If a company uses open-source products, and most of the IT effort involved is not programming but rather integration or service, than it would follow that those jobs should stay in the US while the actual programming is done by the open source community (it is hard to imagine a company that does service and support on your operating system to have technicians fly in from India). Could switching to open source save the high-tech IT industry in the US?

2003-07-13T11:59:00.000-04:00
After a few days of BlogSpot downtime, the service was finally brought back up and I get to refresh this blog.