DeutschEnglish

Submenu

 - - - By CrazyStat - - -

8. November 2019

rspamd and redis: Random Connection refused errors

Filed under: Server Administration — Tags: , , , , , — Christopher Kramer @ 19:35

For some time, I am using rspamd as Spamfilter. By the way, I can really recommend it over Spamassasin / Amavisd-new solutions. I configured rspamd to use a redis database for the bayes filter, neural network etc.

Everything worked well until after some time, probably due to an update, it started spitting out Connection refused errors like these:

2019-11-07 06:25:10 #32043(normal) rspamd_redis_stat_keys: cannot get keys to gather stat: Connection refused
2019-11-07 06:25:15 #32042(normal) <4n3xhp>; lua; neural.lua:855: cannot get ANN data from key: rn_default_default_t66yaxz4_0; Connection refused
2019-11-07 06:25:15 #32044(normal) <4n3xhp>; lua; neural.lua:1101: cannot get ANNs list from redis: Connection refused
2019-11-07 06:25:15 #32040(controller) <4n3xhp>; lua; neural.lua:1135: cannot exec invalidate script in redis: Connection refused

Not every connection to redis failed, but quite a lot. This is a Debian Stretch system with a local redis 3.2.6 database and rspamd 2.1 running on the same host. So network problems couldn’t be it. The redis logs did not even mention the failed connections, the MONITOR command of redis also did not show anything for the refused connections. Restarting redis didn’t change anything, but restarting rspamd solved the problem for about an hour, when the problem started again. I tried adjusting the linux open files limit, the number of maximum redis connections and timeout settings without any luck.

Finally, I found the setting that solved the problem. My redis configuration in /etc/rspamd/local.d/redis.conf had looked like this:

password = "somePassword";
write_servers = "localhost";
read_servers = "localhost";

Changing it to this solved the problem:

password = "somePassword";
write_servers = "127.0.0.1:6379";
read_servers = "127.0.0.1:6379";

The problem seems to be that localhost both resolves to an IPv4 address and an IPv6 address. Redis, however, is configured to only listen on the IPv4 address in /etc/redis/redis.conf:

bind 127.0.0.1

Rspamd seems to sometimes use the IPv4 address, which works, and sometimes the IPv6 address, which doesn’t. This is why the problem seems to occur randomly.

Hope this helps somebody to find the issue faster. If this saved your day, please drop a comment.

Recommendation

Try my Open Source PHP visitor analytics script CrazyStat.

6 Comments »

  1. Thanks a lot for this entry. It really saved my day.

    Comment by alien lilfe form — 28. January 2020 @ 18:41

  2. Even with your help, i did not work
    but i solved it by just renaming the redis.conf file so it would not read it at all

    and now it works, on config default perfect.

    Comment by Erik Reemst — 3. March 2020 @ 19:33

  3. Had the same problem, duckduckgo brought me here, the solution worked.

    I think the problems is that I had both “127.0.0.1 localhost” and “::1 localhost” in /etc/hosts and the server was not configured to actually use ipv6.

    Removed the ipv6 entry from hosts and disabled ipv6 in kernel entirely, using “localhost” worked from then on.

    Rspamd 2.5.

    Comment by Terlisimo — 24. July 2020 @ 01:28

  4. It saved my day with the neural plugin, I got:

    neural.lua:1352: cannot exec invalidate script in redis: Connection refused

    Changing `neural.conf` from `servers = “localhost”;` to `servers = “127.0.0.1”;` fixed it.

    Comment by nh2 — 12. October 2020 @ 05:29

  5. Wow … I don’t know for how much longer I’d have tried to hunt down the root of the problem if I hadn’t stumbled upon your solution. It worked like a charm for me. Thanks a lot!

    Comment by fritteli — 5. April 2021 @ 23:14

  6. Thanks for saving me a night full of troubleshooting 🙂

    Comment by Thomas — 18. May 2022 @ 17:04

RSS feed for comments on this post. TrackBack URL

Leave a comment