Mail.dll .NET email component includes high accuracy anti-spam filter.

It uses enhanced naive Bayesian classifier, specifically modified to handle email messages. Bayesian spam filters are a very powerful technique for dealing with spam.

In our tests we achieved 99,6% accuracy with very low false positive spam detection rates (9 false positives in 54’972 emails tested – that’s 0.016%).

Training

First in the learning phase, you need to teach the classifier to recognize spam and non-spam (ham) messages. You need to prepare 100-200 spam and ham messages.

I suggest using following folder structure:

“Learn” folder is used for training the filter. Both spam and ham folders should contain around 100-200 messages each (the more the better). The number of messages in spam and ham folders must be equal. You can find a spam archive on the bottom of the article.

Messages must be in eml format with correct line endings (rn or 13 10 hex).

Now we use SpamFilterTeacher class to teach BayesianMailFilter:

// C#
using Lesnikowski.Mail.Tools.Spam;

BayesianMailFilter filter = new BayesianMailFilter();
SpamFilterTeacher teacher = new SpamFilterTeacher(filter);
teacher.TeachSpam(@"c:bayeslearnspam");
teacher.TeachHam(@"c:bayeslearnham");

Testing

“Test” folder is used for testing our filter:

// C#

SpamTestResults r = teacher.Test(
    @"c:bayestestspam",
    @"c:bayestestham");

Console.WriteLine(r);
r.FalsePositives.ForEach(Console.WriteLine);
r.NotMarkedAsSpam.ForEach(Console.WriteLine);

The results should be similar to this:

Accuracy=0.9949, False positives=9, Not marked as spam=271, Tests count=54972
c:bayestestham�16874.eml
...


When the filter is trained and the results are satisfactory, you can save it to disk:

// C#

filter.Save("c:\20111022.mbayes");

Using

You can load the filter from disk and check individual messages:

// C#

BayesianMailFilter filter = new BayesianMailFilter();
filter.Load("c:\20111022.mbayes");

// you can use Mail.dll to download mesage from POP3 or IMAP server:
string eml = ... 

IMail email = new MailBuilder().CreateFromEml(eml);

SpamResult result = filter.Examine(email);
Console.WriteLine(result.Probability);
Console.WriteLine(result.IsSpam);

If the filter incorrectly recognizes the message you can train it again:

// C#

filter.LearnSpam(email);
// - or -
filter.LearnHam(email);

filter.Save("c:\20111022.mbayes");

Spam archives

For most recent spam you can check this great archive: http://www.untroubled.org/spam/.
Unfortunately messages don’t have correct extension (*.eml) and line endings are incorrect.

You can download spam archive including 7874 spam messages from Oct 2011 here:
http://www.lesnikowski.com/mail/spam/spam201110.zip

OAuth is an open protocol to allow secure API authorization in a simple and standard method from desktop and web applications.

In this post I’ll show how to access Gmail account using 3-legged OAuth authentication method. The key advantage of this method is that it allows an application to access user email without knowing user’s password.

You can read more on OAuth authentication with Google accounts here:
http://code.google.com/apis/accounts/docs/OAuth_ref.html

Gmail IMAP and SMTP using OAuth:
http://code.google.com/apis/gmail/oauth/protocol.html

If your application/website is not registered, you should use following key and secret:
consumer key: “anonymous”
consumer secret: “anonymous”

Remember to add reference to Maill.dll and appropriate namespaces.

// C#

using Lesnikowski.Client.IMAP;
using Lesnikowski.Client.Authentication;
using Lesnikowski.Client.Authentication.Google;

const string userEmailAccount = "pat@gmail.com";
const string consumerKey = "anonymous";
const string consumerSecret = "anonymous";

GmailOAuth oauth = new GmailOAuth(
    consumerKey, consumerSecret);

string url = oauth.GetAuthorizationUrl("http://localhost:64119/");

Process.Start(url);
// You can use Response.Redirect(url) in ASP.NET

string oauthVerifier = HttpUtility.UrlDecode(Console.ReadLine());
// You can use Request["oauth_verifier"].ToString() in ASP.NET

oauth.GetAccessToken(oauthVerifier);

using (Imap client = new Imap())
{
    client.ConnectSSL("imap.gmail.com");
    string oauthImapKey = oauth.GetXOAuthKeyForImap();
    client.LoginOAUTH(oauthImapKey);

    // Now you can access user's emails
    //...

    client.Close();
    oauth.RevokeToken(oauthImapKey);
}

1.
GmailOAuth.GetAuthorizationUrl method returns url you should redirect your user to so he can authorize access.
As you can see Mail.dll is asking for access to user’s email information and Gmail access:

2.
If you don’t specify callback parameter, user will have to manually copy&paste the token to your application:

In case of a web project, you can specify a web address on your website. oauth_verifier will be included as the redirection url parameter.

After the redirection, your website/application needs to read oauth_verifier query parameter:

3.
GmailOAuth.GetAccessToken method authorizes the token.

4.
GmailOAuth.GetXOAuthKeyForImap method uses Google API to get the email address of the user, and generates XOAuth key for IMAP protocol (you can use GetXOAuthKeyForSmtp for SMTP).

5.
GmailOAuth.RevokeToken method revokes XOAuth key, so no further access can be made with it.

…and finally VB.NET version of the code:

' VB.NET

Imports Lesnikowski.Client.IMAP
Imports Lesnikowski.Client.Authentication
Imports Lesnikowski.Client.Authentication.Google

Const userEmailAccount As String = "pat@gmail.com"
Const consumerKey As String = "anonymous"
Const consumerSecret As String = "anonymous"

Dim oauth As New GmailOAuth(consumerKey, consumerSecret)

Dim url As String = oauth.GetAuthorizationUrl("http://localhost:64119/")

Process.Start(url)
' You can use Response.Redirect(url) in ASP.NET

Dim oauthVerifier As String = HttpUtility.UrlDecode(Console.ReadLine())
' You can use Request["oauth_verifier"].ToString() in ASP.NET

oauth.GetAccessToken(oauthVerifier)

Using client As New Imap()
	client.ConnectSSL("imap.gmail.com")
	Dim oauthImapKey As String = oauth.GetXOAuthKeyForImap()
	client.LoginOAUTH(oauthImapKey)

	' Now you can access user's emails
	'...

	client.Close()
	oauth.RevokeToken(oauthImapKey)
End Using

OAuth is an open protocol to allow secure API authorization in a simple and standard method from desktop and web applications.

In this post I’ll show how to access Gmail account using 2-legged OAuth authentication method. The basic idea is that domain administrator can use this method to access user email without knowing user’s password.

You can read more on OAuth authentication with Google accounts here:
http://code.google.com/apis/accounts/docs/OAuth_ref.html

Gmail IMAP and SMTP using OAuth:
http://code.google.com/apis/gmail/oauth/protocol.html

Remember to add reference to Maill.dll and appropriate namespaces.

// C#

using Lesnikowski.Client.IMAP;
using Lesnikowski.Client.Authentication;
using Lesnikowski.Client.Authentication.Google;

const string consumerKey = "example.com";
const string consumerSecret = "secret";
const string email = "pat@example.com";

Gmail2LeggedOAuth oauth = new Gmail2LeggedOAuth(
    consumerKey, consumerSecret);

using (Imap client = new Imap())
{
    client.ConnectSSL(TestConstants.GmailImapServer);

    string oauthImapKey = oauth.GetXOAuthKeyForImap(email);

    client.LoginOAUTH(oauthImapKey);

    //...

    client.Close();
}

' VB.NET

Imports Lesnikowski.Client.IMAP
Imports Lesnikowski.Client.Authentication
Imports Lesnikowski.Client.Authentication.Google

Const  consumerKey As String = "example.com"
Const  consumerSecret As String = "secret"
Const  email As String = "pat@example.com"

Dim oauth As New Gmail2LeggedOAuth(consumerKey, consumerSecret)

Using client As New Imap()
	client.ConnectSSL(TestConstants.GmailImapServer)

	Dim oauthImapKey As String = oauth.GetXOAuthKeyForImap(email)

	client.LoginOAUTH(oauthImapKey)

	'...

	client.Close()
End Using

Here are the google apps configuration screens:

Recently some of the IMail interface address properties changed its types. The reason for this was to handle email groups nicely.

Now To, Cc, Bcc and ReplyTo are of IList<MailAddress> type (That is IList Of MailAddress in VB.NET).

In those collections you can find two kinds of objectes:

• MailBox class which represents single mailbox,
• MailGroup class which represents group of email addresses.

Here’s the simplest code that prints only mailboxes:

// C#
using Lesnikowski.Mail.Headers;

private string PrintMailboxes(IEnumerable<MailAddress> addresses)
{
   List<string> parts = new List<string>();
   foreach (MailBox mailbox in addresses.OfType<MailBox>())
   {
       parts.Add(string.Format("{0} <{1}>",
           mailbox.Name,
           mailbox.Address));
   }
   return string.Join(", ", parts.ToArray());
}

' VB.NET
Imports Lesnikowski.Mail.Headers

Private Function PrintMailboxes(addresses As IEnumerable(Of MailAddress)) As String
  Dim parts As New List(Of String)()
  For Each mailbox As MailBox In addresses.OfType(Of MailBox)()
      parts.Add(String.Format("{0} <{1}>", _
		mailbox.Name, _
		mailbox.Address))
  Next
  Return String.Join(", ", parts.ToArray())
End Function

Here’s a slightly more complicated version that handles groups also:

// C#
using Lesnikowski.Mail.Headers;

private string PrintAllAddresses(IEnumerable<MailAddress> addresses)
{
   List<string> parts = new List<string>();
   foreach (MailAddress address in addresses)
   {
       if (address is MailGroup)
       {
           MailGroup group = (MailGroup)address;
           parts.Add(string.Format("{0} ({1})",
               group.Name,
               PrintAllAddresses(group.Addresses))); // recursion
       }
       if (address is MailBox)
       {
           MailBox mailbox = (MailBox)address;
           parts.Add(string.Format("{0} <{1}>",
               mailbox.Name,
               mailbox.Address));
       }
   }
   return string.Join(", ", parts.ToArray());
}

' VB.NET
Imports Lesnikowski.Mail.Headers

Private Function PrintAllAddresses(addresses As IEnumerable(Of MailAddress)) As String
   Dim parts As New List(Of String)()
   For Each address As MailAddress In addresses

      If TypeOf address Is MailGroup Then
         Dim group As MailGroup = DirectCast(address, MailGroup)
         parts.Add(String.Format("{0} ({1})", _
            group.Name, _
            PrintAllAddresses(group.Addresses))) ' note the recursion
      End If

      If TypeOf address Is MailBox Then
         Dim mailbox As MailBox = DirectCast(address, MailBox)
         parts.Add(String.Format("{0} <{1}>", _
            mailbox.Name, _
            mailbox.Address))
      End If

   Next
   Return String.Join(", ", parts.ToArray())
End Function

You can use SupportedAuthenticationMethodsto retrieve all authentication methods supported by the server:

// C#

using (Imap client = new Imap())
{
    client.ConnectSSL("imap.example.org");
    client.UseBestLogin("user", "password");

    Console.WriteLine("Supported methods:");

    foreach (var method in client.SupportedAuthenticationMethods())
    {
        Console.WriteLine(method.Name);
    }

    Console.WriteLine("Supports CramMD5:");

    bool supportsCramMD5 = client.SupportedAuthenticationMethods()
        .Contains(ImapAuthenticationMethod.CramMD5);

    Console.WriteLine(supportsCramMD5);

    client.Close();
}

' VB.NET

Using client As New Imap()
    client.ConnectSSL("imap.example.org")
    client.UseBestLogin("user", "password")

    Console.WriteLine("Supported methods:")

    For Each method As ImapAuthenticationMethod In client.SupportedAuthenticationMethods()
	    Console.WriteLine(method.Name)
    Next

    Console.WriteLine("Supports CramMD5:")

    Dim supportsCramMD5 As Boolean = client.SupportedAuthenticationMethods() _
        .Contains(ImapAuthenticationMethod.CramMD5)

    Console.WriteLine(supportsCramMD5)

    client.Close()
End Using

For example Gmail produces following output:

Supported method s:
IMAP4rev1
UNSELECT
IDLE
NAMESPACE
QUOTA
ID
XLIST
CHILDREN
X-GM-EXT-1
UIDPLUS
COMPRESS

Supports CramMD5:
False

We take great care for the API to look similar for all protocols (IMAP, POP3, SMTP).

Here’s the sample for POP3:

// C#

using (Pop3 client = new Pop3())
{
    client.ConnectSSL("pop3.example.org");
    client.UseBestLogin("user", "password");

    Console.WriteLine("Supported methods:");

    foreach (var method in client.SupportedAuthenticationMethods())
    {
        Console.WriteLine(method.Name);
    }

    Console.WriteLine("Supports CramMD5:");

    bool supportsCramMD5 = client.SupportedAuthenticationMethods()
        .Contains(Pop3AuthenticationMethod.CramMD5);

    Console.WriteLine(supportsCramMD5);

    client.Close();
}

' VB.NET

Using client As New Pop3()
    client.ConnectSSL("pop3.example.org")
    client.UseBestLogin("user", "password")

    Console.WriteLine("Supported methods:")

    For Each method As Pop3AuthenticationMethod In client.SupportedAuthenticationMethods()
	    Console.WriteLine(method.Name)
    Next

    Console.WriteLine("Supports CramMD5:")

    Dim supportsCramMD5 As Boolean = client.SupportedAuthenticationMethods() _
        .Contains(Pop3AuthenticationMethod.CramMD5)

    Console.WriteLine(supportsCramMD5)

    client.Close()
End Using

Here’s the sample for SMTP:

// C#

using (Smtpclient = new Smtp())
{
    client.Connect("smtp.example.org");
    client.UseBestLogin("smtp", "password");

    Console.WriteLine("Supported methods:");

    foreach (var method in client.SupportedAuthenticationMethods())
    {
        Console.WriteLine(method.Name);
    }

    Console.WriteLine("Supports CramMD5:");

    bool supportsCramMD5 = client.SupportedAuthenticationMethods()
        .Contains(SmtpAuthenticationMethod.CramMD5);

    Console.WriteLine(supportsCramMD5);

    client.Close();
}

' VB.NET

Using client As New Smtp()
    client.Connect("smtp.example.org")
    client.UseBestLogin("user", "password")

    Console.WriteLine("Supported methods:")

    For Each method As SmtpAuthenticationMethod In client.SupportedAuthenticationMethods()
	    Console.WriteLine(method.Name)
    Next

    Console.WriteLine("Supports CramMD5:")

    Dim supportsCramMD5 As Boolean = client.SupportedAuthenticationMethods() _
        .Contains(SmtpAuthenticationMethod.CramMD5)

    Console.WriteLine(supportsCramMD5)

    client.Close()
End Using