mimedefang-protocol.7.in

.\" $Id$
.\""
.TH MIMEDEFANG-PROTOCOL 7 "8 February 2005"
.UC 4
.SH NAME
mimedefang-protocol \- Conventions used by \fBmimedefang\fR(8) to
communicate with filter programs.

.SH DESCRIPTION
\fBmimedefang\fR(8) and \fBmimedefang-multiplexor\fR(8) provide a
simplified mechanism for hooking scripts and programs into
Sendmail's milter API.

The milter API is multi-threaded and written in C; \fBmimedefang\fR
lets you write single-threaded filters written in the language of your
choice.  Some of the flexibility and speed of milter is sacrificed,
but the ease of writing filters more than compensates for this slight
loss.

This manual describes how \fBmimedefang\fR communicates with the
filter program, and gives you enough information to write your own
filter program as a replacement for \fBmimedefang.pl\fR if you
wish.

.SH PROTOCOL OVERVIEW

The protocol is a simple file-based protocol.  For each invocation of
a filter, \fBmimedefang\fR creates a unique working directory and
populates it with files.  It calls the filter, which is expected to
populate the working directory with more files, which communicate the
scan results back to \fBmimedefang\fR.  This simple mechanism allows
you to easily write filters in scripting languages without worrying
about C-level details.

.SH FILTER INVOCATION

The filter program may be invoked in one of five ways:

.TP
\fIfilter_prog\fR \fIdirectory\fR
If the program is invoked with a single argument which is an absolute
path name (called the \fBworking directory\fR, the program is expected
to perform filtering in that directory and then exit.

.TP
\fIfilter_prog\fR \fB\-server\fR
If the program is invoked with the single argument \fB\-server\fR, it
is expected to run as a server.  See SERVER MODE for details.

.TP
\fIfilter_prog\fR \fB\-serveru\fR
If the program is invoked with the single argument \fB\-serveru\fR, it
is expected to run as a server.  In addition, anything it prints to
file descriptor 3 is used to update the "worker status" field in the
multiplexor.  This lets the filter inform administrators exactly what
it is doing.  (See the \fB\-Z\fR option to \fBmimedefang-multiplexor\fR.)

.TP
\fIfilter_prog\fR \fB\-embserver\fR
Similar to \fB\-server\fR, but used by the embedded Perl code.
The program should run any initialization routines and then exit.
The multiplexor will subsequently call the Perl routine
\fBdo_main_loop\fR when it is time for the worker to begin running
in server mode.

.TP
\fIfilter_prog\fR \fB\-embserveru\fR
Similar to \fB\-embserver\fR with the additional magic of updating
the worker status from data written to file descriptor 3.

.SH INITIAL FILE LAYOUT

When the filter begins a scan, it should change directories to the
working directory.  In that directory, it will find the following
files.

.TP
.B INPUTMSG
A file containing the complete input e-mail message, including headers.

.TP
.B HEADERS
A file containing just the headers, one per line.  Headers which are
continued over several lines in the original message are collapsed
into a single line in this file.

.TP
.B COMMANDS
A file containing a list of commands.  Each command is a single
letter and may be followed by arguments.  Each command is on its
own line.

.SH THE COMMANDS FILE

All commands have their arguments encoded as follows: All characters outside
the range 33 to 126 ASCII, as well as the characters "%", "\\",
"'" and double-quote, are replaced by a percent sign followed by two
hex digits specifying the character's numerical value.  The filter
must un-escape the arguments when it reads the COMMANDS file.

.\""Fix emacs highlighting

The commands from the C to Perl filters are:

.TP
.B S\fIsender\fR
The sender of the message.

.TP
.B s\fIesmtp_arg\fR
An ESMTP argument associated with the sender (such as SIZE=54432).  There
is one \fBs\fR line for each ESMTP argument.

.TP
.B U\fIsubject\fR
The message subject.

.TP
.B X\fImessage_id\fR
The Message-ID.

.TP
.B R\fIrecipient\fR \fImailer\fR \fIhost\fR \fIaddr\fR
A recipient.  There is one \fBR\fR line for each recipient.  The
\fImailer\fR, \fIhost\fR and \fIaddr\fR parts of the line are the
values of the Sendmail {rcpt_mailer}, {rcpt_host} and {rcpt_addr}
macros if they are available, or "?" if not.

.TP
.B r\fIesmtp_arg\fR
An ESMTP argument associated with the \fImost recent\fR recipient
(such as NOTIFY=never).  There is one \fBr\fR line for each SMTP argument.

.TP
.B !
If this command is present, there are suspicious characters in the message
headers.

.TP
.B ?
If this command is present, there are suspicious characters in the message
body.

.TP
.B I\fIhost_addr\fR
The SMTP relay host's IP address in dotted-quad notation.

.TP
.B i\fIidentifier\fR
An identifier generated by MIMEDefang.  On a given host, this identifier
is very likely to be unique over a timespan of about 24 years.

.TP
.B J\fIhost_addr\fR
The "real" SMTP relay host's IP address in dotted-quad notation.
Multi-stage MIMEDefang relays can use a special IP validation
header so that even the innermost MIMEDefang relay can see the
"original" relay's IP address.

.TP
.B H\fIhost_name\fR
The SMTP relay host name.

.TP
.B E\fIargument\fR
The argument to the SMTP "EHLO" or "HELO" command.

.TP
.B Q\fIqid\fR
The message's Sendmail queue-ID.

.TP
.B =\fImacro\fR \fIval\fR
Set the value of the specified Sendmail macro to \fIval\fR.  Both
\fImacro\fR and \fIval\fR are percent-encoded, but the single space
character between them is not.

.SH FILTER OPERATION

When the filter performs a scan, it can make use of all the information
in the files mentioned previously.  If the filter needs temporary
working files, it should create a subdirectory under the working directory
for its own use.  In this case, you do not have to clean up your working
files, because \fBmimedefang\fR deletes the working directory when the
filter returns.

.SH FINAL FILE LAYOUT

The filter communicates the results of the scan back to \fBmimedefang\fR
by creating additional files in the working directory.  The most
important file is called RESULTS, and it contains a list of one-letter,
one-line commands back to the filter.  As usual, command arguments
are percent-escaped.  The commands from the filter back to \fBmimedefang\fR
are:

.TP
.B B\fIcode\fR \fIdsn\fR \fIreply_text\fR
Bounce (reject) the message with the specified SMTP reply code, DSN code
and reply text.

.TP
.B D
Silently discard the message and pretend it was delivered.

.TP
.B T\fIcode\fR \fIdsn\fR \fIreply_text\fR
Return an SMTP temporary failure code with the specified SMTP code, DSN
and reply text.

.TP
.B C
Replace the message body.  If this command is present, the file
NEWBODY must contain the new message body.

.TP
.B M\fIheader_val\fR
Replace the MIME Content-Type header with a new value.  Used to change
MIME boundaries or convert non-MIME to MIME messages.

.TP
.B H\fIheader\fR \fIval\fR
Add a new header \fIheader\fR with value \fIval\fR.  The \fIheader\fR
should not contain a colon.  Each of \fIheader\fR and \fIval\fR is
percent-escaped, but the single space between them is not.

.TP
.B N\fIheader\fR \fIindex\fR \fIval\fR
Adds a new header \fIheader\fR with value \fIval\fR in position
\fIindex\fR.  An index of zero specifies that the new header should
be prepended before all existing headers.

.TP
.B I\fIheader\fR \fIindex\fR \fIval\fR
Replace the \fIindex'th\fR occurrence of \fIheader\fR with value
\fIval\fR.  The \fIindex\fR is 1-based.  The \fIheader\fR should not
contain a colon.  Each of \fIheader\fR, \fIindex\fR and \fIval\fR is
percent-escaped, but the single space between them is not.

.TP
.B J\fIheader\fR \fIindex\fR
Delete the \fIindex\fR'th occurrence of \fIheader\fR.

.TP
.B R\fIrecip\fR
Add a new recipient \fIrecip\fR to the message.

.TP
.B S\fIrecip\fR
Delete \fIrecip\fR from the list of message recipients.

.TP
.B f\fIsender\fR
Change the envelope sender to \fIsender\fR.  This is only supported by
Sendmail 8.14.0 and higher.

.TP
.B F
Indicate that we have finished issuing commands.  Anything after
an F line is ignored.

.SH SERVER MODE

In server mode, \fBmimedefang-multiplexor\fR runs the filter program
continuously in a server loop.  The filter program reads commands from
standard input, and writes results to standard output.  The filter program
\fImust\fR exit shortly after it sees EOF on its standard input.  If it
does not exit within 10 seconds, it will be terminated with SIGTERM.  If
that still does not work, then after a further 10 seconds, it is killed
with SIGKILL.

.TP
.B SERVER COMMANDS

All server commands are single line commands.  Each command is followed
by a space-separated list of arguments; each argument is
percent-encoded.  The commands defined are:

.TP
.B ping
Elicits a reply of "PONG" from the server.

.TP
.B scan \fIqueue_id\fR \fIdir\fR
Run a scan for the mail identiefied by the Sendmail queue-ID \fIqueue_id\fR
in the directory \fIdir\fR.  The command is terminated with a newline.
The server must write a newline-terminated "ok" if the scan completed
successfully, or "error: msg" if something went wrong.

.TP
.B relayok \fIip_addr\fR \fIhostname\fR \fIclient_port\fR \fIdaemon_ip\fR \fIdaemon_port\fR
Test whether or not to accept a connection from the specified host.
The server must write "ok 1" if we will accept the connection, or "ok
0 error_message code dsn" if not.  It can indicate a temporary failure by
writing "ok -1 error_message code dsn".  Note that even if the connection is
accepted, a later scan can still reject the message based on other
criteria.  "ip_addr" is the IP address of the relay and "hostname" is the
hostname (if it could be determined; otherwise, the IP address in
square brackets).

.TP
.B helook \fIip_addr\fR \fIhostname\fR \fIhelo_string\fR \fIclient_port\fR \fIdaemon_ip\fR \fIdaemon_port\fR
Test whether or not to accept the HELO/EHLO command.  The
server must write "ok 1" if we will accept the mail attempt, or "ok 0
error_message code dsn" if not.  "ok -1 error_message code dsn"
indicates a temporary failure.
\fIhelo_string\fR is the argument to the HELO/EHLO command.  The remaining
arguments have the same meaning as in \fBrelayok\fR.

.TP
.B senderok \fIsender_addr\fR \fIip_addr\fR \fIhostname\fR \fIhelo_string\fR \fIdir\fR \fIqueue_id\fR [\fIesmtp_args\fR...]

Test whether or not to accept mail from the specified sender.  The
server must write "ok 1" if we will accept the mail attempt, or "ok 0
error_message code dsn" if not.  "ok -1 error_message code dsn"
indicates a temporary failure.  Note that even if the sender is
accepted, a later scan can still reject the message based on other
criteria.  "sender_addr" is the sender's e-mail address. The "ip_addr"
and "hostname" arguments are as in \fBrelayok\fR. "helo_string" is
the argument to the SMTP HELO/EHLO command. "dir" is the MIMEDefang
spool directory, and "queue_id" is the Sendmail queue identifier.

The optional "esmtp_args" are space-separated, percent-encoded ESMTP
arguments supplied with the MAIL FROM: command.

.TP
.B recipok \fIrecip_addr\fR \fIsender_addr\fR \fIip_addr\fR \fIhostname\fR \fIfirst_recip\fR \fIhelo_string\fR \fIdir\fR \fIqueue_id\fR [\fIesmtp_args\fR...]
Test whether or not to accept mail for the specified recipient.  The
server must write "ok 1" if we will accept it, or "ok 0 error_message
code dsn" if not.  ok -1 error_message code dsn" indicates a temporary-failure.

"recip_addr" is the argument to the RCPT TO: command, and
"first_recip" is the argument to the first RCPT TO: command for this
message.  Other arguments are as in \fBsenderok\fR.

.TP
.B map \fImap_name\fR \fIkey\fR
If you are using a map socket (the \fB\-N\fR option to \fBmimedefang-multiplexor\fR), then the server should look up the key \fIkey\fR in the map
\fImap_name\fR.  The server should print a single line to standard output.  The
first word on the line should be one of OK, NOTFOUND, TEMP, TIMEOUT or PERM,
indicating a successful lookup, absence of the key, a temporary failure,
a timeout or a permanent failure, respectively.  This should be followed
by a space and a percent-encoded string representing the value of the key
(if it was found) or an optional error message (if something went wrong.)

.TP
.B tick \fIband\fR
The filter should run \fBfilter_tick\fR with the specified
\fIband\fR argument.  It should print a single line to standard output:

tock \fIband\fR

.TP
.B Additional Commands
The filter can define a function \fBfilter_unknown_cmd\fR that
can extend the list of server commands.  If you do this, make sure
all of your commands start with an upper-case letter to avoid
conflicts if more built-in commands are defined in the future.

.TP
.B SERVER REPLIES

The reply codes are:

.TP
.B ok [\fIreturn_code\fR] [\fIparameters\fR]
The operation completed successfully.  Some operations have an
associated return code, and possibly other parameters as well.
See the source code for the gory details.

.TP
.B error: \fIMessage\fR
The operation failed.  \fIMessage\fR may give additional details.

.PP
In server mode, you should \fInot\fR write anything to standard output
except reply codes, or the multiplexor will become confused.  You
should \fInot\fR terminate the program in server mode; simply echo an
\fBerror:\fR reply and return to the server loop.

When you send a reply code back to the multiplexor, be sure to terminate
it with a newline, and to flush standard output.  If your program uses
the Standard I/O library, standard output may not be flushed immediately,
and \fBmimedefang-multiplexor\fR will wait forever for the filter's reply,
and eventually kill the filter on the assumption it has hung up.

In server mode, if the filter program receives a SIGINT signal, it must
terminate.  This is used by \fBmimedefang-multiplexor\fR to terminate
workers after they have processed a given number of e-mail messages.

.SH AUTHOR
\fBmimedefang\fR was written by Dianne Skoll <dfs@roaringpenguin.com>.
The \fBmimedefang\fR home page is \fIhttps://www.mimedefang.org/\fR.

.SH SEE ALSO
mimedefang.pl(8), mimedefang(8), mimedefang-multiplexor(8), mimedefang-filter(5)