Development Question: Interfacing Perl with Sendmail 
Author Message
 Development Question: Interfacing Perl with Sendmail

Hi:

I am currently developing a mass mailling tool in Perl which runs on a Linux
system with sendmail.  I am concerned about monitoring and potentially
controlling the network and CPU utilization of sendmail as I don't want to
overload the server's resources and would like to have some sort of feedback
mechanism to enable the Perl code to throttle back if necessary.  My
questions are therefore:

1. Does sendmail have some internal mechanisms for monitoring network and
CPU utilization?  If so, where can I get more info about this?

2. When sending emails from PERL, as I see it I have two options: invoke the
"sendmail" binary directly, or use an SMTP perl client to send the mail to
the localhost.  I am wondering if these two methods differ in the way they
impact sendmail's resource monitoring facilities, how it manages its queues
etc.  Is there a preferred method?

3. Are there any ideas on how best to send a large number of emails, without
overloading network or CPU resources?

Thanks in advance,
Keith



Sat, 15 Jan 2005 16:26:39 GMT  
 Development Question: Interfacing Perl with Sendmail

Quote:

> I am currently developing a mass mailling tool in Perl which runs on a Linux
> system with sendmail.  I am concerned about monitoring and potentially
> controlling the network and CPU utilization of sendmail as I don't want to
> overload the server's resources and would like to have some sort of feedback
> mechanism to enable the Perl code to throttle back if necessary.  My
> questions are therefore:

> 1. Does sendmail have some internal mechanisms for monitoring network and
> CPU utilization?  If so, where can I get more info about this?

> 2. When sending emails from PERL, as I see it I have two options: invoke the
> "sendmail" binary directly, or use an SMTP perl client to send the mail to
> the localhost.  I am wondering if these two methods differ in the way they
> impact sendmail's resource monitoring facilities, how it manages its queues
> etc.  Is there a preferred method?

> 3. Are there any ideas on how best to send a large number of emails, without
> overloading network or CPU resources?

Take a look at bulk_mailer program and sympa list manager.

In short make you perl script submit messages via multiple parrallel SMTP
sessions, send specific number of messages per single session e.g. max 50
sessions at any momemt, 30 messages per session. In pre 8.12 sendmail use ONEX
SMTP command (it is sendmail specific).

--

366A 5DD7 7707 379C 9251   32AE C948 0BD2 7D99 688A   expires: 2003-01-25
I may disagree with the following *random epigram* :
"And remember: Evil will always prevail, because Good is dumb."
                -- Spaceballs



Sat, 15 Jan 2005 18:01:00 GMT  
 Development Question: Interfacing Perl with Sendmail

Quote:


> > I am currently developing a mass mailling tool in Perl which runs on
> > a Linux system with sendmail.  I am concerned about monitoring and
> > potentially controlling the network and CPU utilization of sendmail
> > as I don't want to overload the server's resources and would like to
> > have some sort of feedback mechanism to enable the Perl code to
> > throttle back if necessary.
[snip]
> In short make you perl script submit messages via multiple parrallel
> SMTP sessions, send specific number of messages per single session
> e.g. max 50 sessions at any momemt, 30 messages per session. In pre
> 8.12 sendmail use ONEX SMTP command (it is sendmail specific).

Actually, max *one* session, and as many messages as you have *all* sent
via that one session, will likely be better.

--

pack 'u', pack 'H*', 'ab5cf4021bafd28972030972b00a218eb9720000';



Sun, 16 Jan 2005 08:06:50 GMT  
 Development Question: Interfacing Perl with Sendmail
Benjamin/Andrzej:

Can you briefly describe what ONEX does and how this if more efficient?  I
am specifically concerned about network utilization and being able to
control the loading of the network interface.  Does sendmail have mechanisms
either implicit or explicit to control this sort of thing?  Can you recomend
any documents?

Thx
Keith


Quote:


> > > I am currently developing a mass mailling tool in Perl which runs on
> > > a Linux system with sendmail.  I am concerned about monitoring and
> > > potentially controlling the network and CPU utilization of sendmail
> > > as I don't want to overload the server's resources and would like to
> > > have some sort of feedback mechanism to enable the Perl code to
> > > throttle back if necessary.
> [snip]
> > In short make you perl script submit messages via multiple parrallel
> > SMTP sessions, send specific number of messages per single session
> > e.g. max 50 sessions at any momemt, 30 messages per session. In pre
> > 8.12 sendmail use ONEX SMTP command (it is sendmail specific).

> Actually, max *one* session, and as many messages as you have *all* sent
> via that one session, will likely be better.

> --

> pack 'u', pack 'H*', 'ab5cf4021bafd28972030972b00a218eb9720000';



Sun, 16 Jan 2005 17:55:35 GMT  
 Development Question: Interfacing Perl with Sendmail

Quote:

> Benjamin/Andrzej:

> Can you briefly describe what ONEX does and how this if more efficient?

ONEX is sendmail's extension of SMTP protocol.
Without it pre 8.12 sendmail will fork separate process for every message
received via the same smtp session.

If you send 200+ messages over single SMTP connection pre 8.12 sendmail will
try to fork 200 new processes.
Sendmail does not try instant delivery when system load is to high, but it
takes some time to detect that load is too high [feedback loop delay].

Quote:
> I am specifically concerned about network utilization and being able to
> control the loading of the network interface.  Does sendmail have mechanisms
> either implicit or explicit to control this sort of thing?  Can you recomend
> any documents?

It is a long story.
I have already recommended relevant documentation in previous post:
"Take a look at bulk_mailer program and sympa list manager."

IMHO on long run making perl send mass mailing messages directly via "plain
SMTP" may be a better option than using MTA which is optimized/designed for
another requirements. It will allow to handle much more efficiently
"personalized" mailing with tiny personalization e.g. the same message but with
recipients name in the body.

--
Andrzej [pl2en: Andrew] Adam Filip http://www.polbox.com/a/anfi/
I may disagree with the following *random epigram* :
An idea is an eye given by God for the seeing of God.  Some of these eyes
we cannot bear to look out of, we blind them as quickly as possible.
                -- Russell Hoban, "Pilgermann"



Sun, 16 Jan 2005 20:41:00 GMT  
 Development Question: Interfacing Perl with Sendmail

Quote:



> > > I am currently developing a mass mailling tool in Perl which runs on
> > > a Linux system with sendmail.  I am concerned about monitoring and
> > > potentially controlling the network and CPU utilization of sendmail
> > > as I don't want to overload the server's resources and would like to
> > > have some sort of feedback mechanism to enable the Perl code to
> > > throttle back if necessary.
> [snip]
> > In short make you perl script submit messages via multiple parrallel
> > SMTP sessions, send specific number of messages per single session
> > e.g. max 50 sessions at any momemt, 30 messages per session. In pre
> > 8.12 sendmail use ONEX SMTP command (it is sendmail specific).

> Actually, max *one* session, and as many messages as you have *all* sent
> via that one session, will likely be better.

You are right but only in typical mail sending scenario (up to 20-30
messages) for pre 8.12 sendmail [ -> you are wrong ].
When you want your program/script send send 200+ messages there is a
different story. You can very easily create DOS attack tool [see "mass
mailing tool" phrase].

In pre 8.12 sendmail (with no ONEX) sendmail will start special process to
deliver every received message just after it is received [in 8.12 sendmail
ONEX is implied]. ONEX makes sendmail accept next message over SMTP session
after initial (first) delivery attempt. Using multiple ONEX sessions makes
your program/script control number of sendmail processes delivering
submitted messages - you take load control from sendmail designers into
your own hand.
Sendmail is optimized for "normal" email traffic pattern, mass mailing is
not in that category.

--
Andrzej [pl2en: Andrew] A. Filip http://www.polbox.com/a/anfi/
I may disagree with the following *random epigram* :
I owe the public nothing.
                -- J.P. Morgan



Sun, 16 Jan 2005 20:44:00 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. Development Question: Interfacing Perl with Sendmail

2. question about speeding perl development

3. Sendmail question, and LWP question

4. Newbie question about SendMail (I think that is what my question is about :)

5. perl user interface interface to tar?

6. Question about development environment on NT from unix perler

7. Perl/C/TCL interface question

8. Off Topic Perl Email Question: Sending mail to AOL and Outlook Express with SENDMAIL

9. sendmail/perl questions

10. Question about Sendmail, Perl, and CGI

11. Sendmail/Perl question

12. Perl to DB2 interface question

 

 
Powered by phpBB® Forum Software