Business Development, Strategy Alignment and Consulting News from The Business City London -

By Erik Rozman

E-mail

As we all know human beings like and sometimes even need to communicate. Without communication we would simply not feel alive. The basic form of communication is speech. Speech can be defined as a direct form of communication yet human beings needed to developed many indirect forms of communication.

Indirect forms of communication allow us to communicate, even if the person, with whom we want to communicate is far away. The most well known form of indirect communication is mail (or snail-mail).

With mail you deliver your message to the Post Office and they deliver it for you, they may cooperate with different Post Offices based in different countries and eventually your message will reach its destination (We hope). The form of most messages delivered by mail is a simple written message- a letter.

Even though letters are a popular form of passing messages people wanted to be able to pass objects to each other so the mail companies started delivering packages-packages could include letters and objects. The main problem with conventional mail is delivery time. It takes quite a large amount of time to deliver a package from one side of the globe to the other.

If you need to deliver an important document or a drawing(diagram, picture, ect.) to one of your overseas clients standard mail may be too slow for you. I can see some eyebrows being raised-what does he mean slow, there are companies today that can deliver a package within 48 hours to every point on the globe(almost)!!! Yet there is one factor that backs my assumption here- the Internet.

The Internet

The Internet is a huge web of interconnected Local Area Netwroks(LAN) populated by computers. Each LAN encompasses a local area such as a campus or company.

These campuses and companies can be reached in a matter of seconds since they are interconnected (as I stated earlier). The global Internet (note that I write it with a capital 'I') is based on the TCP/IP protocol for transfering data amongst different computers (hosts) based in different LANs.

This level of accessibility allows us to send information from a LAN in one country (in Europe) to a LAN in Washington (US) in a matter of seconds or minutes (some delay may be caused by the mail servers and slow hardware).

The architects of the Internet saw and understood the potential of this level of accessibility and decided to build on it to provide us with a new and highly improved mailing system- the Electronic Mail system. This system allows us to send text and other file attachments around the globe almost instantaneously.

We will take an overall look at this system and its protocols.

E-mail overview Email systems are based on different components and protocols:

Components
1. User agents
2. Queues
3. Mailboxes
4. MTA (Message Transfer Agents)

Protocols
1. SMTP
2. POP3
3. IMAP4

The mail process is initiated by a user (there are some cases in which the mail process is triggered automatically).The user employs a User Agent(a mail client program) to write the message and add attachments(if needed) and using the Agent he sends the message to the outgoing messages Queue.

The Message Transfer Agent polls the Queue periodically and when it finds messages in the Queue it attempts to send them to the MTA responsible for the recipient's mailbox. When the message reaches the recipients MTA the MTA places it into the users mailbox where it awaits retrieval by the User Agent. As you may note the mail delivery process is divided into different stages and at each stage different components and protocols are used to relay the message to the next component until it reaches its final destination I will provide an explanation for each component and its related protocol.

The User Agent

Any mail client program that can interact with message queues and mailboxes using POP3 (Post Office Protocol) or IMAP4 (Internet Mail Access Protocol) can be called a user agent. The two protocols that user agents use to retrieve messages from mailboxes are POP3 and IMAP4.

Each of the two protocols allows a different degree of control over the mailboxes and queues:

POP3(Post Office Protocol Version 3)

This protocol allows a user to download and manipulate to some extent mail messages, this protocol is quite simplistic and doesn't allow any special actions. POP3 connections are made at port 110 (TCP). Once the connection has been established on the defined port the POP3 server greets the client (by issuing the +OK text_string ) and enters the AUTHORIZATION stage.

During the AUTHORIZATION stage the user has to identify him self by issuing the USER command and a string that identifies his mailbox. After identifying the mailbox the user has to issue the PASS command and his password string. The server may acknowledge any of the commands by using the +OK reply or deny access by issuing the -ERR reply.

UA: Connects at TCP port 110
S: +OK POP3 ready

UA: USER davidm
S: +OK davidm is valid
UA: PASS secret
S: +OK davidm's mailbox has 2 messages (160 bytes)

After the user is authenticated and the mailbox is located the system switches into the TRANSACTION stage. While in the transaction stage the user may manipulate his mailbox by using different commands. The STAT command may be used to check the number of messages waiting and their total size.

The POP3 mechanism enumerates message sequentially thus the first message is 1 the second 2 ect (messages marked as deleted can not be referred to using this enumeration). The LIST command may be used in conjunction with this enumeration to provide the size of a specific message.

The most important feature of POP3 is to allow users to retrieve messages, the command that triggers this action is RETR and a message number (the message number is required).

After retrieving the message the user may want to remove it from the server to free up storage resources, for this task the user agent issues the DELE command and the message's number, at this stage the message is marked as deleted but it is not yet deleted from the server. The RSET command is the undo command -if the user wants to unmark the messages that are marked as deleted he can issue, using the user agent, an RSET command. An simple yet cool sounding command is the NOOP command. The NOOP command simply checks the connections state, if the connection is a valid one the response will be +OK.

After the user has manipulated his massage he has to issue the QUIT command. This command finalizes the user's decision and puts the mechanism into its final stage called the UPDATE stage. Note that the QUIT command may be issued during the AUTHORIZATION stage, the result will be the disconnection of the TCP session.

In the update stage the users mailbox is updated according to the users commands- this mainly concerns the deletion of messages since messages marked deleted are deleted at this stage only. If the connection is broken before the update stage any messages marked as deleted will be retained on the server and unmarked.

These are the basic commands as defined by RFC1939 that defines POP3, the RFC also defines a few optional (may not be implemented) commands from which I see as one of the most important commands the TOP command. The TOP command may be issued only during the TRANSACTION stage and it allows a user(using an agent) to download a specific message(message enumeration) and a specific number of lines from that message.

IMAP4(Internet Message Access Protocol)

Allows a greater degree of control over mailboxes-header download and partial download of messages, IMAP connects on port 143(TCP). The IMAP4 protocol functions in the same manner as the POP3 protocol yet it allows a higher degree of control over mailboxes and the IMAP4 protocol supports three models of E-mail usage:

Offline model-all mail is downloaded and manipulated on the client computer- an example for such a protocol is POP3.
Online Model-all changes a mail manipulation is done on the server the mail is never downloaded to the client, a web access client can be given as an example for this model.
Disconnected model- a composite of both the offline and online model. The user manipulates mail on both the client and the server.

The IMAP4 listens on TCP port 143 and once a client's connects the protocol greets the client. At this stage the client can start sending commands to the server, each client command begins with a unique tab(A0001,A0002,A0003, ect.) allowing the client to send multiple commands at once and receive answers to the appropriate commands(identified by the tag). The servers response to a task states its status:

1.	OK- indicating success.
2.	NO- indicating failure.
3.	BAD-indicating an unknown command or syntax.

Mailboxes and messages are handled differently by IMAP4 in comparison to POP3, under IMAP4. Each mailbox has a unique identifier validity value and each message in the mailbox is identified by both a unique identifier (UID of a 32 bit value) and a message sequence number that identifies the relative position of the message in the mailbox (this value may change while the UID stays the same). Each message in a mailbox may have different flags set to allow the system to understand the message's status, there are two kinds of flags:

1. Permanent flags.
2. Session-only.

As I mentioned earlier a flag sets a messages status:

\seen - Message has been read.
\Answered - Message has been answered.
\Flagged - Message has been marked for special attention.
\Deleted - Message is deleted (later removal).
\Recent - Message arrived recently (first session after arrival).

After the initial connection between the client and the server the interaction enters a stage called the NON-AUTHENTICATED stage. In this stage the client can choose an authentication mechanism and has to authenticate itself. Commands used in this stage are:

AUTHENTICATE-this command allows the user to choose an authentication mechanism.
LOGIN- the client uses this command to send a username and password to the server.

After a successful login the interaction switches into the AUTHENTICATED stage, commands that can be used at this stage are:

SELECT-in order to select a specific mailbox.
EXAMINE-selects a mailbox for examination (read only).
CREATE-used to create a mailbox-will not overwrite existing mailboxes.
DELETE-permanently erases the mailbox.
RENAME-renames a mailbox.
LIST-returns all names available to a user.
STATUS-This command checks the status of a specific mailbox.

After a mailbox is selected it can be operated upon and the interaction enters its SELECTED stage. The commands that define the operation that can be done in this stage:

CHECK-this commands allows you to receive the status of your mailbox regarding site and memory usage.
CLOSE- the command removes any messages marked deleted permanently and moves back to the AUTHENTICATED stage.
EXPUNGE- removes all messages marked as deleted
SEARCH- searches a mailbox for a message that contains a given criteria.

There are a few IMAP4 commands that can be issued in any state:

CAPABILITY-requests the list of functions a server supports.
NOOP- this command allows you to reset an inactivity timer because it mainly alerts the server that you are still on-line and does nothing else.
LOGOUT - sends a request to disconnect the session

For the full implementation details on IMAP4 I advise you to read RFC2060.

The Queues

Allow storage of mail message before being passed on to different message transfer agents. The reason for the existence of a queue is to allow a message to be delivered even if the recipient MTA is down for some reason. The message is placed in the queue and the queue is checked at pre-configured intervals by the MTA.

When it finds a message in the queue it attempts to deliver it to the recipient MTA. If it is unsuccessful it will reattempt the delivery at set intervals until it reaches the configured threshold for a non deliverable message status. At that stage the sender will be notified.

Message Transfer Agent

Use the SMTP protocol (using NVT ASCII) to pass messages between themselves thus each MTA can perform two distinct and different roles - client and server. The SMTP protocol defines the following standards:

SMTP connections are based upon port 25 of the TCP protocol.
When the connection (TCP:25) is established the server MTA sends a 220 reply code and it's fully qualified domain name.
The client answers with a HELO and its fully qualified domain name.
At this stage the client invokes the MAIL command and identifies the sender. The server acknowledges the sender by the 250 reply code.
The client MTA identifies the recipients of the message (RCPT) and the server MTA acknowledges (250) them or denies(550) their existence.
At this stage the client informs the server that it is ready to transfer the mail message contents by issuing a DATA command, to which the server answers with a 354 message. To signal the end of the data transfer the client sends a carriage return, line feed, period, carriage return, line feed.
If there are more messages to deliver to this MTA they will be sequentially delivered-if no messages are to be delivered the client will issue a QUIT command and the server MTA should respond with a 221 command saying it agrees to disconnect.
If the server MTA has messages it needs to deliver to the client MTA it can reverse the connection and become the client MTA by issuing the TURN order. This pattern is more efficient than tearing down and rebuilding the connection in the opposite direction.

As you can see SMTP is built on transferring simple ASCII messages. Having understood the principles let us see an example of a real connection:

S:220 mail.xyz.com
C:HELO mail.crs.com
S:250 mail.xyz.com
C:MAIL FROM <david@crs.com>
S:250 ok
C:RCPT <leon@xyz.com>
S:250 Recipient ok
C:RCPT <wrong@xyz.com>
S:550 No such user
C:DATA S:354 Start mail input
C:Transfer Message Body
C:.
S:250 Message received
C:QUIT
S:221 OK

As we can see there are 5 main commands used in SMTP: HELO, MAIL, RCPT, DATA and QUIT. There are a few more commands we should be aware of:

TURN - allows the server MTA to turn into a client MTA.
VRFY - verifies the existence of a recipient.
EXPN - expands a mailing list without sending messages to the members.

Mailboxes When we send mail we usually specify the recipients address on the envelope, when we send E-mail we have to specify the recipients Electronic Mail address. An E-mail address specifies the location of its assigned mailbox and is actually made up of two distinct pieces of information:

1. The user's alias in the mail system (may be synonymous with the user's ID).
2. The servers location- the servers location is actually portrayed by a domain suffix.

The format of an E-mail is as follows: david@xyz.com.

When an MTA needs to deliver a message to the user whose alias is david and is located at xyz.com the MTA turns to the Domain Name System for name resolution (Fully Qualified Domain Name to IP address).

The DNS contains zone files in which it records different Resource Records (host name to IP, Canonical Names ect.). The problem in the specific case of name resolution is that the E-mail address gives no information about the machine that hosts the recipient MTA.

The problem was solved by creating a special Resource Record (RR) called a Mail Exchanger(MX), MX records specify fully qualified domain names of machines that host MTAs for the specific domain. This allows the sending MTA to simply query for an MX record of a specific domain. MX records allow you to include a priority for each record thus allowing some level of fault tolerance. After the source MTA identifies the recipient MTA it opens a TCP session on port 25 and starts sending SMTP commands.

E-mail addresses can have different relationships with recipients-one to one, many to one and one to many:

One to One- an Alias mapped to a specific recipient.
Many to One- a Number of aliases that point to the same recipient.
One to Many- in this case there is one alias that actually points to many recipients-this relationship is also called a Mailing List. If a message needs to be delivered to a group of people instead of mailing the same message to each and every member of the group a Mailing List can be created. The Mailing List will have one alias (a special mailbox) that receives a message and sends it out to all aliases its list.

As you can see we have covered most aspects of E-mail, one last aspect remains:
Attachments.

RFC 822 defining the standards for SMTP clearly states that all information passed between different MTAs should be ASCII based, what happens if we want to send files that are binary and not ASCII??

The solution was provided by RFC 1521 which introduces a new standard called MIME- Multipurpose Internet Mail Extensions. The theory is quite simple- take the non-ASCII information you want to transfer and turn it into ASCII and then transfer it. When the recipient receives the information it has to be converted back to binary form from its ASCII form.

In order to allow the recipient to convert the information back again we need to provide assistance The recipient can not know the original form of conversion that was emplyed upon the message so we just need to tell it by using the mail messages headers. RFC 1521 defines five types of encoding types- once the encoding type is known to the recipient he can decode the information passed on to him.

PROMOTION

Only 32.00€

Get complete protection from the most dangerous threats on the internet - worms, viruses, trojans, spyware, and adware.

AVG Anti-Virus

incl.

Anti-Spyware

"1 computer 1 year"

Get it for 32€ incl. VAT!

Save up to 15€!

PROMOTION