DeutschEnglish

Submenu

 - - - By CrazyStat - - -

8. November 2019

rspamd and redis: Random Connection refused errors

Filed under: Server Administration — Tags: , , , , , — Christopher Kramer @ 19:35

For some time, I am using rspamd as Spamfilter. By the way, I can really recommend it over Spamassasin / Amavisd-new solutions. I configured rspamd to use a redis database for the bayes filter, neural network etc.

Everything worked well until after some time, probably due to an update, it started spitting out Connection refused errors like these:

2019-11-07 06:25:10 #32043(normal) rspamd_redis_stat_keys: cannot get keys to gather stat: Connection refused
2019-11-07 06:25:15 #32042(normal) <4n3xhp>; lua; neural.lua:855: cannot get ANN data from key: rn_default_default_t66yaxz4_0; Connection refused
2019-11-07 06:25:15 #32044(normal) <4n3xhp>; lua; neural.lua:1101: cannot get ANNs list from redis: Connection refused
2019-11-07 06:25:15 #32040(controller) <4n3xhp>; lua; neural.lua:1135: cannot exec invalidate script in redis: Connection refused

Not every connection to redis failed, but quite a lot. This is a Debian Stretch system with a local redis 3.2.6 database and rspamd 2.1 running on the same host. So network problems couldn’t be it. The redis logs did not even mention the failed connections, the MONITOR command of redis also did not show anything for the refused connections. Restarting redis didn’t change anything, but restarting rspamd solved the problem for about an hour, when the problem started again. I tried adjusting the linux open files limit, the number of maximum redis connections and timeout settings without any luck.

Finally, I found the setting that solved the problem. My redis configuration in /etc/rspamd/local.d/redis.conf had looked like this:

password = "somePassword";
write_servers = "localhost";
read_servers = "localhost";

Changing it to this solved the problem:

password = "somePassword";
write_servers = "127.0.0.1:6379";
read_servers = "127.0.0.1:6379";

The problem seems to be that localhost both resolves to an IPv4 address and an IPv6 address. Redis, however, is configured to only listen on the IPv4 address in /etc/redis/redis.conf:

bind 127.0.0.1

Rspamd seems to sometimes use the IPv4 address, which works, and sometimes the IPv6 address, which doesn’t. This is why the problem seems to occur randomly.

Hope this helps somebody to find the issue faster. If this saved your day, please drop a comment.

Recommendation

Try my Open Source PHP visitor analytics script CrazyStat.