David Hrbáč

avgd stops listening on port 54322

David Hrbáč — Mon, 30 Aug 2010 08:08:44 +0000

Last days I’m experiencing avgd not responding. Avgd stops to listen on port 54322, amavisd-new timeouts on talking to AV. Mails are being held and queue is slowly increasing its number. It’s strange, that I’m experiencing it not only on one production system…

Aug 29 04:59:46 rakosnicek amavis[25940]: (25940-01-5) (!)AVG Anti-Virus av-scanner FAILED: run_av error: Too many retries to talk to 127.0.0.1:54322 (timed
out) at (eval 111) line 373.\n

It seems working after changing a little bit AVG configuration with following values:

avgcfgctl -w Default.setup.features.antispam=false
avgcfgctl -w Default.tcpd.avg.limiter_start=150
avgcfgctl -w Default.tcpd.avg.limiter_stop=200
avgcfgctl -w Default.tcpd.avg.timeout=5000
avgcfgctl -w Default.tcpd.smtp.enabled=false
avgctl --restart=tcpd

Firefox 3.6.2 and Thunderbird 3.0.3 for CentOS 5.4

David Hrbáč — Wed, 24 Mar 2010 09:08:58 +0000

It’s a quite long time I have announced Firefox 3.6 and Thunderbird 3.0 packages for CentOS 5. Announcement has been submitted via CentOS mail-list. See http://lists.centos.org/pipermail/centos-devel/2010-February/005416.html and http://lists.centos.org/pipermail/centos/2010-February/090418.html.

Packages have been tested by users and have been always within the testing repository only. Today I’m pushing new firefox 3.6.2 into the stable repository together with xulrunner, mozilla-filesystem, and thunderbird. Packages are to be found here:

http://fs12.vsb.cz/hrb33/el5/hrb/stable/i386/repoview/

http://fs12.vsb.cz/hrb33/el5/hrb/stable/x86_64/repoview/

Nagios and dpt-i2o-raidutils, raidutil check

David Hrbáč — Fri, 05 Feb 2010 08:24:49 +0000

We do check our production boxes with hardware raid cards on raid status. We are using our script to call raidutil, which is running fine, but not under nagios user. Permissions for the files are as follows:

-rwxr-xr-x 1 root root 254708 Aug 20  2008 raideng
-rwxr-xr-x 1 root root 255880 Aug 20  2008 raidutil

Solution is to add proper rights to utils with:

chmod u+s raid*

It should look like this:

-rwxr-xr-x 1 root root 254708 Aug 20  2008 raideng
-rwxr-xr-x 1 root root 255880 Aug 20  2008 raidutil

Sed to correct path lines

David Hrbáč — Thu, 04 Feb 2010 12:22:56 +0000

Sed one-liner to change correct paths. Before:

php_admin_value open_basedir /srv/www/www.domena.tld/www

After:

php_admin_value open_basedir /srv/www/www.domena.tld

Sed script:

sed -i "s/\(.*\)php_admin_value open_basedir \(.*\)\/www$/\1php_admin_value open_basedir \2/g" /etc/httpd/conf/httpd.conf

Upgrade DAViCal 0.9.7.2 to 0.9.8

David Hrbáč — Wed, 03 Feb 2010 09:54:31 +0000

DAViCal upgrade is quite easy:

[root@fs12 html] wget http://debian.mcmillan.net.nz/packages/davical/davical_0.9.8.orig.tar.gz
[root@fs12 html] tar xvzf davical_0.9.8.orig.tar.gz
[root@fs12 html] cd davical-0.9.8.orig/dba
[root@fs12 dba] su postgres
bash-3.00$ ./update-davical-database 
The database is version 8.1 currently at revision 1.2.5.
Applying patch 1.2.6.sql ... succeeded.
Applying patch 1.2.7.sql ... succeeded.
Successfully applied 2 patches.
Supported locales updated.
Updated view: dav_principal.sql applied.
CalDAV functions updated.
RRULE functions updated.
Database permissions updated.
bash-3.00$ exit
[root@fs12 html] ln -s davical-0.9.8.orig davical

That’s all.

Zoner Antivirus with Amavisd-new

David Hrbáč — Mon, 01 Feb 2010 21:05:04 +0000

Today I have decided to test Amavisd-new with free Zoner Antivirus for Linux. First of all you have to download the package, (test it), install, and change the ownership. Then edit the configuration.

wget http://update.zonerantivirus.com/download/zav-1.2.2-redhat-i586.rpm --nodeps
rpm -Uhv zav-1.2.2-redhat-i586.rpm --test
rpm -Uhv zav-1.2.2-redhat-i586.rpm
chown amavis: /opt/zav -R
chown amavis: /var/run/zav -R
vi /etc/zav/zavd.conf

So, we have changed the ownership. Now we have to change the daemon user to amavis too.

# user under which to run the daemon
ZAVD_USER               = "amavis"
ZAVD_GROUP              = "amavis"
# Your license key for accessing ZAV update
UPDATE_KEY              = "11111-22222-SAMPLE-33333-44444"

We can start and update ZAV with:

/etc/init.d/zavd start
/etc/init.d/zavd update

Finally we have to change Amavisd-new configuration by adding the following:

 ### http://www.grisoft.com/
 ['Zoner Anti-Virus',
#  '/opt/zav/bin/zavcli', '-z /var/run/zav/zavd.sock {}',
  '/opt/zav/bin/zavcli', ' {}',
  [0,1,2,3,4], [5,6,7,11],
  qr/^.*:\ (SUSPICIOUS|PROBINFECTED|INFECTED)\ \((.*)\)$/
]

Send file as attachment from commnad line

David Hrbáč — Mon, 18 Jan 2010 13:04:28 +0000

There’s an easy way of sending files as attachment from command line:

uuencode file name | mail -s "Subject" "to@tld.cz"  -- -ffrom@tld.cz

Yes, there is double hyphen and -f before the from email address.

pure-ftpd with geoip

David Hrbáč — Fri, 16 Oct 2009 15:31:48 +0000

During the last few weeks we are experiencing compromised FTP accounts within our production. These accounts seem to be stolen from infected Windows client machines, infected by malware or trojans. As far as I know the issues, everyone of them is using Total Commander… TC seems to save the credentials in plain text form, unencrypted. Fine.

I have been going through pure-ftpd log to gather ips that had been used to upload files. I’ve found that most of them were outside our country, China, Romania, Russia, etc. So I decided to limit users set and allow only ips from Czech Republic. I wanted to go with iptables geoip module, which is working very fine. But I have found geoip patch for pure-ftpd. I have corrected it a little bit and finally created RPM packages for Centos4/5. You can find the packages within my repository. Feel free to use…

Packages

http://fs12.vsb.cz/hrb33/el5/hrb/stable/i386/repoview/pure-ftpd.html
http://fs12.vsb.cz/hrb33/el5/hrb/stable/x86_64/repoview/pure-ftpd.html

http://fs12.vsb.cz/hrb33/el4/hrb/stable/i386/repoview/pure-ftpd.html
http://fs12.vsb.cz/hrb33/el4/hrb/stable/x86_64/repoview/pure-ftpd.html

The patch

diff -ur pure-ftpd-1.0.22/src/ftpd.c pure-ftpd-1.0.22-geoip/src/ftpd.c
--- pure-ftpd-1.0.22/src/ftpd.c 2009-09-17 09:38:04.000000000 +0200
+++ pure-ftpd-1.0.22-geoip/src/ftpd.c   2009-09-17 09:34:03.000000000 +0200
@@ -34,6 +34,8 @@
 # include "osx-extensions.h"
 #endif
 
+#include "GeoIP.h"
+
 #ifdef WITH_GSSAPI
 # include "auth_gssapi.h"
 #endif
@@ -4818,7 +4820,7 @@
 die(421, LOG_ERR, MSG_GETPEERNAME ": %s" , strerror(errno));
 }
 fourinsix(&amp;peer);
-    if (checkvalidaddr(&amp;peer) == 0) {
+    if (checkvalidaddr(&amp;peer) == 0 || STORAGE_FAMILY(ctrlconn) != AF_INET) {
 die(425, LOG_ERR, MSG_INVALID_IP);
 }
 #ifndef DONT_LOG_IP
@@ -4854,6 +4856,37 @@
 *host = '?';
 host[1] = 0;
 #endif
+    do {
+        char line[LINE_MAX];
+        GeoIP *gi;
+        const char *country;
+        char *sep;
+        int found = 0;
+        FILE *fp;
+
+        gi = GeoIP_new(GEOIP_STANDARD);
+        country = GeoIP_country_code_by_name(gi, host);
+        if (country == NULL || *country == 0 ||
+            !(fp = fopen(CONFDIR "/pureftpd-restricted-countries.txt", "r"))) {
+            break;
+        }
+        while (fgets(line, sizeof line, fp) != NULL) {
+            if (*line == '#') {
+                continue;
+            }
+            if ((sep = strchr(line, '\n')) != NULL) {
+                *sep = 0;
+            }
+            if (strcasecmp(line, country) == 0) {
+                found++;
+                break;
+            }
+        }
+        fclose(fp);
+        if (found == 0) {
+            die(425, LOG_ERR, MSG_INVALID_IP);
+        }
+    } while(0);
 iptropize(&amp;peer);
 logfile(LOG_INFO, MSG_NEW_CONNECTION, host);
 
diff -ur pure-ftpd-1.0.22/src/Makefile.am pure-ftpd-1.0.22-geoip/src/Makefile.am
--- pure-ftpd-1.0.22/src/Makefile.am    2006-04-25 10:15:54.000000000 +0200
+++ pure-ftpd-1.0.22-geoip/src/Makefile.am      2009-09-17 09:37:14.000000000 +0200
@@ -16,7 +16,7 @@
 
 pure_ftpd_LDADD = \
        ../puredb/src/libpuredb_read.a \
-       @LDAP_SSL_LIBS@ @GETLOADAVG_LIBS@ @BONJOUR_LDADD@
+       @LDAP_SSL_LIBS@ @GETLOADAVG_LIBS@ @BONJOUR_LDADD@ -lGeoIP
 
 pure_ftpd_CFLAGS = -DINCLUDE_IO_WRAPPERS=1
 
diff -ur pure-ftpd-1.0.22/src/Makefile.in pure-ftpd-1.0.22-geoip/src/Makefile.in
--- pure-ftpd-1.0.22/src/Makefile.in    2006-04-25 10:45:12.000000000 +0200
+++ pure-ftpd-1.0.22-geoip/src/Makefile.in      2009-09-17 09:35:43.000000000 +0200
@@ -221,7 +221,7 @@
 target_alias = @target_alias@
 pure_ftpd_LDADD = \
        ../puredb/src/libpuredb_read.a \
-       @LDAP_SSL_LIBS@ @GETLOADAVG_LIBS@ @BONJOUR_LDADD@
+       @LDAP_SSL_LIBS@ @GETLOADAVG_LIBS@ @BONJOUR_LDADD@ -lGeoIP
 
 pure_ftpd_CFLAGS = -DINCLUDE_IO_WRAPPERS=1
 pure_ftpd_SOURCES = \

AVG 8.5 with Amavisd-new

David Hrbáč — Tue, 13 Oct 2009 13:35:31 +0000

Amavisd-new is a quite powerful tool capable to co-operate with a large amount of AV scanners. Today I have decided to test it with free AVG Antivirus for Linux. First of all you have to download the package, (test it), install, and change the ownership. Then edit the configuration.

wget http://download.avgfree.com/filedir/inst/avg85flx-r287-a2632.i386.rpm
rpm -Uhv avg85flx-r287-a2632.i386.rpm --test
rpm -Uhv avg85flx-r287-a2632.i386.rpm
chown amavis: /opt/avg -R
vi /opt/avg/avg8/etc/init.d/avgdinit.conf

So, we have changed the ownership. Now we have to change the daemon user to amavis too.

# user under which to run the daemon
SUSER=amavis

We can start AVG with:

/etc/init.d/avgd start

Finally we have to change Amavisd-new configuration by commenting out AVG section and changing the port:

 ### http://www.grisoft.com/
 ['AVG Anti-Virus',
   \&amp;ask_daemon, ["SCAN {}\n", '127.0.0.1:54322'],
   qr/^200/, qr/^403/, qr/^403 .*?: ([^\r\n]+)/ ],

Benchmarking php with apc, memcached

David Hrbáč — Wed, 23 Sep 2009 07:56:43 +0000

As we are very extensively using techniques to speed up web applications we build or host, I’d like to share some benchmarks. One of our clients is running site on our machines which is using SOAP/WSDL to get data. The application is extremely slow, result are very poor.

Apache is able to server this webapp within cca 23 pages per second, which is bad. Here are the ab results:

[root@webmel4 ~]# ab -n 400 -c 5 http://beta.somedomain.cz/
This is ApacheBench, Version 2.0.41-dev &lt;$Revision: 1.141 $&gt; apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
 
Benchmarking beta.somedomain.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Finished 400 requests
 
Server Software:        Apache
Server Hostname:        beta.somedomain.cz
Server Port:            80
 
Document Path:          /
Document Length:        45829 bytes
 
Concurrency Level:      5
Time taken for tests:   17.637183 seconds
Complete requests:      400
Failed requests:        0
Write errors:           0
Total transferred:      18473376 bytes
HTML transferred:       18348354 bytes
Requests per second:    22.68 [#/sec] (mean)
Time per request:       220.465 [ms] (mean)
Time per request:       44.093 [ms] (mean, across all concurrent requests)
Transfer rate:          1022.84 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        6    6   0.2      6       9
Processing:   176  212  26.1    207     380
Waiting:      142  172  21.4    169     277
Total:        182  218  26.1    213     386
 
Percentage of the requests served within a certain time (ms)
  50%    213
  66%    221
  75%    225
  80%    230
  90%    252
  95%    272
  98%    290
  99%    325
 100%    386 (longest request)

We decided to implement memcached which means some small code rewrite, which is quite simple. After that apache serves the pages within the 47 pages per second. Nice but still poor.

[root@webmel4 ~]# ab -n 400 -c 5 http://beta.somedomain.cz/
This is ApacheBench, Version 2.0.41-dev &lt;$Revision: 1.141 $&gt; apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
 
Benchmarking beta.somedomain.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Finished 400 requests
 
Server Software:        Apache
Server Hostname:        beta.somedomain.cz
Server Port:            80
 
Document Path:          /
Document Length:        11128 bytes
 
Concurrency Level:      5
Time taken for tests:   8.403883 seconds
Complete requests:      400
Failed requests:        0
Write errors:           0
Total transferred:      4579944 bytes
HTML transferred:       4455233 bytes
Requests per second:    47.60 [#/sec] (mean)
Time per request:       105.049 [ms] (mean)
Time per request:       21.010 [ms] (mean, across all concurrent requests)
Transfer rate:          532.13 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        4    5   1.2      6      16
Processing:    45   98  40.8     88     249
Waiting:       37   81  34.0     73     218
Total:         51  104  40.7     94     253
 
Percentage of the requests served within a certain time (ms)
  50%     94
  66%    113
  75%    131
  80%    139
  90%    165
  95%    177
  98%    205
  99%    231
 100%    253 (longest request)

Finally we decided to add apc php cache. Speed up is quite better cca 130 pages per second.

[root@webmel4 ~]# ab -n 400 -c 5 http://beta.somedomain.cz/
This is ApacheBench, Version 2.0.41-dev <$Revision: 1.141 $> apache-2.0
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
 
Benchmarking beta.somedomain.cz (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Finished 400 requests
 
Server Software:        Apache
Server Hostname:        beta.somedomain.cz
Server Port:            80
 
Document Path:          /
Document Length:        11128 bytes
 
Concurrency Level:      5
Time taken for tests:   3.70608 seconds
Complete requests:      400
Failed requests:        0
Write errors:           0
Total transferred:      4584288 bytes
HTML transferred:       4459266 bytes
Requests per second:    130.27 [#/sec] (mean)
Time per request:       38.383 [ms] (mean)
Time per request:       7.677 [ms] (mean, across all concurrent requests)
Transfer rate:          1457.69 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        6    6   0.2      6       9
Processing:    21   31   8.2     30      83
Waiting:       14   23   7.9     22      76
Total:         27   37   8.2     36      89
 
Percentage of the requests served within a certain time (ms)
  50%     36
  66%     39
  75%     41
  80%     43
  90%     47
  95%     51
  98%     60
  99%     78
 100%     89 (longest request)

So, as you can see, there are ways to speedup the web apps not mentioning that the app itself can be better. We have seen the code and it seems to me that the programmer did not optimize anything within the code.

technology	requests per sec	speed
php	22.68	100%
php+memcachedphp	47.6	210%
+memcached+apc	130.27	574%

Benchmarks were run against CentOS 4.8 machine running:

httpd-2.0.52-41.ent.4.centos4
memcached-1.4.1-2.el4.hrb
php-5.1.6-3.el4s1.10
php-pecl-apc-3.0.19-1.el4.hrb
php-pecl-memcache-3.0.4-3.el4.hrb

You can find non distribution packages within my repos.