DeutschEnglish

Submenu

 - - - By CrazyStat - - -

21. May 2015

wp-cli: syntax error, unexpected ‘?’, error connecting to the database

Filed under: Server Administration,Windows,Wordpress — Christopher Kramer @ 10:55

Running wp-cli on a website that was set up by somebody else resulted in a strange error:

PHP Parse error:  syntax error, unexpected '?' in phar:///usr/local/wp-cli.phar/php/wp-cli.php(23) : eval()'d code on line 1
Error:
<h1>Fehler beim Aufbau der Datenbank-Verbindung</h1>
...

The database settings in wp-config.php were all correct and WordPress was also working, so this did not seem to be the problem. In the end, I found out the reason was a byte order mark at the beginning of wp-config.php that had been inserted probably by Windows Editor. To solve the issue, the easiest way is to open the file in a good editor, for Windows I recommend PSPad, copy everything into the clipboard (CTRL +A, CTRL + C), create a new empty file (CTRL + N), paste everything (CTRL + V), save it (CTRL + S) as wp-config.php and upload the file.

 

Recommendation

Try my Open Source PHP visitor analytics script CrazyStat.

20. March 2015

Remove duplicate mails from a maildir

Filed under: Linux,Server Administration — Tags: , , , , — Christopher Kramer @ 19:30

In case somehow, you got duplicate mails in a maildir, like me, maybe this is helpful to you:

find /var/mail/user -type d -name cur -print0 | xargs -0 /usr/bin/fdupes -n | less

Of course you need fdupes installed. On debian:

apt-get install fdupes

The above command is the dry run, showing you the duplicates.

Then to delete the duplicates, leaving only the first found file:

find /var/mail/user -type d -name cur -print0 | xargs -0 /usr/bin/fdupes -ndN | less

Use the same thing again with “new” instead of “cur” if you have duplicate unread mails. This only looks at duplicates within the same folder, so you can also safely run it on /var/mail for all users.

Note that this assumes the duplicates have exactly the same content. If you want to clean duplicates which are a bit different, consider this tool.

14. March 2015

Bind nameserver: view cache entries

Filed under: Linux,Server Administration — Tags: , , , , — Christopher Kramer @ 15:35

If you want to view the cache entries of your bind nameserver, this is how it works on bind9 on Debian:

rndc dumpdb
less /var/cache/bind/named_dump.db

Tested on Debian Wheezy. Hope this helps somebody.

6. March 2015

Owncloud: Upgrading to owncloud 8 fails (Integrity constraint violation in oc_filecache)

Filed under: Linux,PHP,Server Administration — Tags: , — Christopher Kramer @ 21:33

First, the upgrade to owncloud 8 in the web GUI failed. Then, I performed the upgrade in the console like this:

sudo -u www-data php /var/www/owncloud/occ upgrade

This gave an error like this (I don’t remember exactly):

An exception occurred while executing 'INSERT INTO "oc_filecache" ... Integrity constraint violation: key s_storage_path_hash is not unique
...
Upgrade failed

This owncloud is using a MySQL DB.

So what in the end solved the problem: First put Owncloud in maintainance mode:

sudo -u www-data php /var/www/owncloud/occ maintenance:mode --on

Then make sure no occ processes of old upgrade attempts are running. If there are, kill them. Then clear the oc_filecache using this MySQL command:

TRUNCATE oc_filecache;

Don’t worry, it will populate itself again during the upgrade. Then restart the upgrade:

sudo -u www-data php /var/www/owncloud/occ upgrade

This might take a lot of time! Better run this on a screen (see screen tutorial if you don’t know how) so it does not stop when your SSH connection breaks.

Spamassasin: MISSING_SUBJECT in every mail even though mail has a subject

Filed under: Linux,Server Administration — Tags: , , , — Christopher Kramer @ 09:48

I noticed the MISSING_SUBJECT rule matched for every mail recently even though the mails contained a subject header. Finally I found this in the spamd.log:

warn: Possible unintended interpolation of @yahoo in string at /etc/spamassassin/local.cf, rule MY_SPAMMY_YAHOO, line 1.
warn: rules: failed to compile Mail::SpamAssassin::Plugin::Check::_head_tests_0_4, skipping:
warn:  (Global symbol "@yahoo" requires explicit package name at /etc/spamassassin/local.cf, rule MY_SPAMMY_YAHOO, line 1.)

The reason was a rule like this:

header          MY_SPAMMY_YAHOO        To =~ /myspammyaddress@yahoo.de/
score           MY_SPAMMY_YAHOO        3
describe        MY_SPAMMY_YAHOO        Mail to myspammyaddress@yahoo.de is mostly spam

Escaping the @ in the rule fixed the Problem:

header          MY_SPAMMY_YAHOO        To =~ /myspammyaddress\@yahoo.de/

Hope this helps someone to spot the error faster.

The problem is that spamassasin does not autolearn ham if the ham messages match the MISSING_SUBJECT rule. So an error like this basically not only breaks lots of tests but also autolearning of ham. So lesson learned: better always look at the spamd.log after creating a new rule, even if you think you know what you are doing 😉

9. December 2014

Icinga: Group all services in a servicegroup instead of using a wildcard

Filed under: Linux,Server Administration — Tags: , , , , , , — Christopher Kramer @ 16:15

At some places you can use the * wildcard as a service description (which requires use_regexp_matching=0), but sometimes it does not seem to work:

Error: Could not expand services specified

Therefore, I simply wanted to group all services in one servicegroup. That’s quite easy if you use a generic service template as a basis for all services. First, create a servicegroup “allservices”:

define servicegroup {
        servicegroup_name               allservices
        alias                           All Services
}

Then edit your generic service template (see servicegroups line):

# generic service template definition
define service{
        name                            generic-service ; The 'name' of this service template
        active_checks_enabled           1       ; Active service checks are enabled
        passive_checks_enabled          1       ; Passive service checks are enabled/accepted
        parallelize_check               1       ; Active service checks should be parallelized (disabling this can lead to major performance probl$
        obsess_over_service             1       ; We should obsess over this service (if necessary)
        check_freshness                 0       ; Default is to NOT check service 'freshness'
        notifications_enabled           1       ; Service notifications are enabled
        event_handler_enabled           1       ; Service event handler is enabled
        flap_detection_enabled          1       ; Flap detection is enabled
        failure_prediction_enabled      1       ; Failure prediction is enabled
        process_perf_data               1       ; Process performance data
        retain_status_information       1       ; Retain status information across program restarts
        retain_nonstatus_information    1       ; Retain non-status information across program restarts
        notification_interval           0               ; Only send notifications on status change by default.
        is_volatile                     0
        check_period                    24x7
        normal_check_interval           5
        retry_check_interval            1
        max_check_attempts              4
        notification_period             24x7
        notification_options            w,u,c,r
        contact_groups                  admins
        servicegroups                   allservices ; ADD THIS TO ADD ALL SERVICES INTO THE allservices GROUP
        register                        0       ; DONT REGISTER THIS DEFINITION - ITS NOT A REAL SERVICE, JUST A TEMPLATE!
        }

This assumes all your services use this template like this (see use line):

define service {
        hostgroup_name                  ssh-servers
        service_description             SSH
        check_command                   check_ssh
        use                             generic-service  ; USE THE TEMPLATE ABOVE
        notification_interval           0 ; set > 0 if you want to be renotified
}

 

Now you can easily use this servicegroup for example in serviceescalations (see servicegroup_name line):

 define serviceescalation{
        hostgroup_name          intranet-servers
        servicegroup_name       allservices
        first_notification      1
        last_notification       0
        notification_interval   1440
        contact_groups          intranet-admins
        }

Hope this helps somebody. I guess it works the same way in Nagios.

13. August 2014

Icinga: Monitor refused mails in postfix mailqueue

Filed under: Linux,Server Administration — Tags: , , , , , , , , , , — Christopher Kramer @ 12:09

In case your server gets listed on blacklists, mails will get refused by destination servers and stick in the deferred mail queue for some time until the sender finally gets a mailer daemon.

As it takes some time until the sender gets the mailer daemon and informs the server admin, it would be better if you could directly get notified by Icinga/Nagios when a mail is in the deferred queue because the destination server refused it.

Therefore I wrote a small shell script which I want to share with you here. I am assuming Debian Wheezy with Icinga and a postfix mailserver.

Create the shell script with the actual plugin in

/usr/lib/nagios/plugins/check_mailq_blacklist :
#!/bin/sh
# detects if mails in mail queue were refused by destination server (because of blacklist?)
# From http://blog.christosoft.de/2014/08/icinga-monitor-refused-mails-postfix-mailqueue/
# Version: 2017-03-07

if mailq | grep -qP "(refused to talk to me(?!(.*out of connection slots)))|(unsolicited mail originating from your IP)|(temporarily deferred due to user complaints)"
then
  mails=`mailq | grep -oP "(refused to talk to me(?!(.*out of connection slots)))|(unsolicited mail originating from your IP)|(temporarily deferred due to user complai$
  echo "$mails mail(s) were refused, check mailq!"
  if [ "$mails" -le 10 ] && [ "$mails" -gt 1 ]; then
    # 2-10 mails -> warning
    echo "\nWarning. | refused=$mails;2;11;0"
    return 1;
  fi
  if [ "$mails" -gt 10 ]; then
    # more than 10 mails -> critical
    echo "\nCriticial! | refused=$mails;2;11;0"
    return 2;
  fi
  return 1;
else
  echo "Ok, there seems to be no refused mail in the mailq | refused=0;2;11;0"
  exit 0;
fi

This will check for the texts “refused to talk to me” (not followed by “out of connection slots”) and “unsolicited mail originating from your IP” in the mailq output. These are the most common errors you get when the destination server has your server’s IP blacklisted.  In case at least one mail was refused, this causes a warning state in icinga. If more than 10 mails were refused, it causes a critical state.

Now you need to make this script executable:

chmod +x /usr/lib/nagios/plugins/check_mailq_blacklist

Now create the config file for the plugin in

/etc/nagios-plugins/config/mailq_blacklist.cfg :
# 'check_mailq_blacklist' command definition
define command{
        command_name    check_mailq_blacklist
        command_line    /usr/lib/nagios/plugins/check_mailq_blacklist
}

So now we have the command and need to define a service that uses it. Let’s say we use this locally for localhost. In

/etc/icinga/objects/localhost_icinga.cfg

add:

define service{
        use                             generic-service
        host_name                       localhost
        service_description             Mail Queue Refused Mail
        check_command                   check_mailq_blacklist
        }

This is it, just restart icinga and you are done:

service icinga restart

I hope this is of use to somebody.

Of course it is also useful to monitor in Icinga, if you are on some of the most used blacklists. A script to do this can be found here.

27. May 2014

Debian Linux: Update packages automatically

Filed under: Linux,Server Administration — Tags: , , , , , — Christopher Kramer @ 20:50

Here is how you configure automatic (security) updates on Debian:

aptitude install unattended-upgrades

Here you can configure it:

nano /etc/apt/apt.conf.d/50unattended-upgrades

This could look like this:

Unattended-Upgrade::Origins-Pattern {
        "origin=Debian,archive=stable,label=Debian-Security";
};
Unattended-Upgrade::Package-Blacklist {
// add packages here that need manual steps like this:
//        "vim";
}
Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::MinimalSteps "true";
Unattended-Upgrade::InstallOnShutdown "false";
Unattended-Upgrade::Mail "root";
Unattended-Upgrade::MailOnlyOnError "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "false";
Acquire::http::Dl-Limit "200";

Now create the following file:

nano /etc/apt/apt.conf.d/02periodic

With this content:

// Enable the update/upgrade script (0=disable)
APT::Periodic::Enable "1";

// Do "apt-get update" automatically every n-days (0=disable)
APT::Periodic::Update-Package-Lists "1";

// Do "apt-get upgrade --download-only" every n-days (0=disable)
APT::Periodic::Download-Upgradeable-Packages "1";

// Run the "unattended-upgrade" security upgrade script
// every n-days (0=disabled)
// Requires the package "unattended-upgrades" and will write
// a log in /var/log/unattended-upgrades
APT::Periodic::Unattended-Upgrade "1";

// Do "apt-get autoclean" every n-days (0=disable)
APT::Periodic::AutocleanInterval "7";

Of course the server needs to be able to send mails so it can send mails in case of problems.

You can test it like this:

# mail -s test mail@example.com
My testmail
.
EOT

Hope this helps somebody.

Update: Some updates caused dpkg questions about changed config files and therefore failed.

For example php5-fpm did ask this:

Setting up php5-fpm (5.4.4-14+deb7u10) ...

Configuration file `/etc/php5/fpm/pool.d/www.conf'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** www.conf (Y/I/N/O/D/Z) [default=N] ? dpkg: error processing php5-fpm (--configure):
 EOF on stdin at conffile prompt

This caused PHP5-FPM to stop and all PHP sites to show an Internal Server Errror…

So you don’t run into this problem, create /etc/apt/apt.conf.d/local with this content:

Dpkg::Options {
   "--force-confdef";
   "--force-confold";
}

This tells DPKG to keep the old config file. It will create .dpkg-dist files with the package distributer’s version. More information on this can be found here.

9. May 2014

Updating to PHP 5.4 causes missing Text

Filed under: PHP,Server Administration — Tags: , , , , , , — Christopher Kramer @ 14:29

After updating from PHP5.3 to PHP 5.4, on some sites text was missing. No error could be found in the error log so I had to dig into the code to find out what was going on.

The root cause is that with PHP5.4, the default character set expected by htmlentites(), htmlspecialcharacters() and html_entity_decode() changed from ISO-8859-1 to UTF-8. So if a script passes ISO-8859-1 characters like German “Umlaute” (öäüÖÄÜß) to one of these functions without specifying the charset with the corresponding parameter, these functions will return an empty string. And unfortunately, with PHP 5.4, they also removed the error message that PHP 5.3 recorded in the logfile in this case. This makes finding the problem a lot more difficult.

So what can you do about it? You could

  1. Use PHP 5.3 😉
    Here is a blog post on downgrading to PHP 5.3. on Debian Wheezy
  2. change the used charset to UTF-8
    This might require changing the character set in files, databases or config files, depending on what is used on the site.
    I explained in a blog post how to change the charset in Typo3 to UTF-8 back in 2012.
  3. Provide ISO-8859-1 as a parameter to all calls of htmlspecialcharacters() etc.

So for the third option, what you have to do is find places like this:

htmlspecialchars($string);

And replace them with something like:

htmlspecialchars($string, ENT_COMPAT | ENT_XHML, 'ISO-8859-1');

The problem is that it’s hard to do this automatically. What is easy to do, is replace all htmlspecialchars()-calls with calls to htmlspecialchars_PHP5-3() etc. and place these functions there:

function htmlspecialchars_PHP5-3($string, $ent=ENT_COMPAT, $charset='ISO-8859-1') {
    return htmlspecialchars($string, $ent, $charset);
}

function htmlentities_PHP-5-3($string, $ent=ENT_COMPAT, $charset='ISO-8859-1') {
    return htmlentities($string, $ent, $charset);
}

function html_entity_decode_PHP-5-3($string, $ent=ENT_COMPAT, $charset='ISO-8859-1') {
    return html_entity_decode($string, $ent, $charset);
}

So just do a search & replace over all files and make sure that all scripts have a file included that contains these functions.

18. January 2014

Linux: get members of a group / get groups of a user

Filed under: Linux,Server Administration — Tags: , , , , — Christopher Kramer @ 00:23

This might sound easy with /etc/group and /etc/passwd, but what if you use libnss-mysql for example and need to try if it works?

So just forget grepping config files. Here are the real commands:

Members of a group:

getent group GROUPNAME

Replace GROUPNAME with the group you want to check.

Groups of a user:

groups USERNAME

Replace USERNAME with the name of the user to check.

 

Checked on Debian, not sure if it works on any Linux/Unix.

« Newer PostsOlder Posts »