DeutschEnglish

Submenu

- - - By CrazyStat - - -

26. March 2017

Zimbra: fix corrupt index open_conversation in an mboxgroup MySQL-DB

Filed under: DBMS,Linux,Server Administration — Tags: , , , , , , — Christopher Kramer @ 20:50

Just wanted to upgrade a Zimbra server from 8.7.3 to 8.7.5. The upgrade always asks you whether to check database integrity. Even it was only a minor upgrade, I chose yes to be on the safe side. And it turned out the MySQL DB was indeed corrupt.

I had seen corrupt zimbra dbs a lot and the “MySQL crash recovery” guide in the zimbra wiki always helped out. But not this time.

I tried the crash recovery as explained in the wiki. When doing dumps, four mboxgroup-databases always failed because the index open_conversation of the table open_conversation was corrupt. As the guide explains, I increased innodb_force_recovery step by step from 1 to the maximum 6, but the error did not go away.

So here is what helped:

  1. Try to create the dumps as explained in the crash recovery guide. You will get errors like this: 
    Dumped mboxgroup8
mysqldump: Error 1712: Index open_conversation is corrupted when dumping table `open_conversation` at row: 0
Dumped mboxgroup9

    This means that mboxgroup9 (not 8!) is corrupt. Write down all the mboxgroup numbers where an error appeared.

  2. Remove  innodb_force_recovery from the my.cnf if you inserted it
  3. Login as zimbra 
    su zimbra
  4. Restart the mysql server 
    mysql.server restart
  5. Load the MySQL account data into shell variables 
    source ~/bin/zmshutil ; zmsetvars
  6. Log into MySQL using the root account 
    mysql -u root --password=$mysql_root_password
  7. Open the first database that is corrupt: 
    USE mboxgroup9;
  8. Repair the corrupt open_converstation table: 
    OPTIMIZE TABLE open_conversation;

    Note: If this fails, check if you really removed innodb_force_recovery from the my.cnf!

  9. Go back to step 7 and open the next database that is corrupt until all have been repaired.
    Now exit the MySQL prompt:

    exit;
  10. You can now continue with the crash recovery, it should now create all dumps correctly. But if the open_conversation tables where the only corruption problem, you could also just stop here as this should have fixed the corruption. In my case, I jus started the upgrade again and let it verify message store database integrity again, and this time it completed with “No errors found”. 🙂
  11. Clean up the MySQL dumps 
     rm -R /tmp/mysql.db.list /tmp/mysql.sql/

Please let me know if this helped you or if you have some additions.

Recommendation

Try my Open Source PHP visitor analytics script CrazyStat.

5. June 2015

Zimbra: Redirect http to https

Filed under: Linux,Security,Server Administration — Tags: , , , , — Christopher Kramer @ 11:15

Zimbra without Proxy (pre 8.5)

That’s the easy way how you can enforce https encrpytion by redirecting http to https:

su – zimbra
zmtlsctl redirect
zmcontrol stop
zmcontrol start

Works at least on Zimbra 8.0 and I think should also work on 7

Zimbra with Proxy (required from 8.5+)

With Zimbra 8.5+, a Proxy is required. This makes the configuration a little different. To configure the proxy to redirect http to https, run:

su zimbra
~/libexec/zmproxyconfig -e -w -o -a 8080:80:8443:443 -x both  -H `zmhostname`
# if your proxy is local:
zmprov ms `zmhostname` zimbraReverseProxyMailMode redirect
# if your proxy is proxy.server.name
zmprov ms proxy.server.name zimbraReverseProxyMailMode redirect
zmcontrol restart

With the latest Zimbra versions, the restart is not even necessary, it automatically detects the change within 2 minutes.

Hope this helps somebody!

2. June 2015

Zimbra: Using zmpostfixpolicyd to stop backscatter

Filed under: Linux,Server Administration — Tags: , , , — Christopher Kramer @ 19:20

Your Zimbra is backscattering? This is what may help you:

Update 2018-08-21:

Before doing anything else, make sure you don’t have a CatchAll defined for your domain. To do so:

su - zimbra
zmprov gd example.com | grep zimbraMailCatchAllAddress 
# should be empty. NOT @example.com
zmprov gd example.com | grep zimbraMailCatchAllForwardingAddress
# should be empty. NOT @example.com
zmprov gd example.com | grep zimbraMailTransport 
# should be lmtp:zimbra.example.com:7025 . NOT e.g. smtp:relayhost.com:587

If any of this is not as should be, then change it:

zmprov md domain.com zimbraMailCatchAllAddress ""
zmprov md domain.com zimbraMailCatchAllForwardingAddress ""
zmprov md domain.com zimbraMailTransport lmtp:zimbra.example.com:7025

Note that in the last line, you need to give the FQDN of your Zimbra server, instead of zimbra.example.com.

You may also read the Zimbra wiki about catchalls.

Original article:

A Zimbra 8.0 Server (on Ubuntu) was backscattering. Unfortunately Zimbra servers cannot use the postfix setting

smtpd_reject_unlisted_sender=yes

when an alias domain is setup. Everybody says it is easy, just install the policyd as a recipient_restriction and that’s it. All you need to do is:

su - zimbra
zmlocalconfig -e postfix_enable_smtpd_policyd=yes
zmprov mcf +zimbraMtaRestriction "check_policy_service unix:private/policy"
zmmtactl stop
zmmtactl start

Unfortunately, this did not work for me with Zimbra 8.0. The server was still backscattering.

First, I enabled verbose output for the policyd. To do so, open /opt/zimbra/postfix/conf/master.cf.in (the .in is important) in a texteditor and search for this part:

%%uncomment LOCAL:postfix_enable_smtpd_policyd%%policy    unix  -       n       n       -       -       spawn
%%uncomment LOCAL:postfix_enable_smtpd_policyd%%        user=zimbra argv=/usr/bin/perl /opt/zimbra/libexec/zmpostfixpolicyd

Add a -v ad the end of the command:

%%uncomment LOCAL:postfix_enable_smtpd_policyd%%policy    unix  -       n       n       -       -       spawn
%%uncomment LOCAL:postfix_enable_smtpd_policyd%%        user=zimbra argv=/usr/bin/perl /opt/zimbra/libexec/zmpostfixpolicyd -v

Then (as zimbra user), recreate the master.cf and restart postfix with these commands:

zmmtactl stop
zmmtactl start

Now when a mail is received (from an external sender), you will see output like this in your mail log (on Debian/Ubuntu: /var/log/mail.log):

/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: ccert_fingerprint=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: sasl_method=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: sasl_sender=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: size=3428
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: helo_name=[...]
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: reverse_client_name=[...]
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: queue_id=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: encryption_cipher=AECDH-AES256-SHA
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: encryption_protocol=TLSv1.2
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: etrn_domain=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: ccert_subject=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: request=smtpd_access_policy
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: protocol_state=RCPT
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: stress=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: recipient=does-not-exist@example.com
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: sasl_username=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: ccert_pubkey_fingerprint=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: instance=7289.556db23f.b6b7d.0
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: protocol_name=ESMTP
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: encryption_keysize=256
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: recipient_count=0
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: ccert_issuer=
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: sender=[...]
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: client_name=[...]
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Attribute: client_address=[...]
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Recipient Domain: example.com
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Recipient userid: does-not-exist
/opt/zimbra/libexec/zmpostfixpolicyd[29326]: Action: dunno

If you see these logs, then zmpostfixpolicyd is set up correctly in postfix. If you don’t, check the postfix configuration.

But the policy daemon always returned “dunno”, even if the recipient did not exist. Therefore, I looked into the perl code of the policy daemon (/opt/zimbra/libexec/zmpostfixpolicyd):

The original code includes this part:

    if (lc(($ldap->get_values("zimbraDomainType"))[0]) eq "alias") {
      my $robject = ($ldap->get_values("zimbraMailCatchAllForwardingAddress"))[0];
      syslog $syslog_priority, "Real Domain: %s", $robject if $verbose;
      @attrs=('1.1');
      $mesg = $ldap->search_s(
                "",
                LDAP_SCOPE_SUBTREE,
                "(&(|(zimbraMailDeliveryAddress=$user"."$robject)(zimbraMailDeliveryAddress=$daddr)(zimbraMailAlias=$user".
                "$robject)(zimbraMailAlias=$daddr)(zimbraMailCatchAllAddress=$user"."$robject)(zimbraMailCatchAllAddress=$robject)".
                "(zimbraMailCatchAllAddress=$daddr))(zimbraMailStatus=enabled))",
                \@attrs,
                0,
                $result
              );
      $ent = $ldap->first_entry();
      $ldap->unbind;
      if ($ent != 0) {
        return "dunno";
      } else {
        return "reject 5.1.1 Mailbox unavailable";
      }
    } else {
      $ldap->unbind;
      return "dunno";
    }

So basically, this code only rejects mail to recipients for alias-domains, but accepts all mail to domains that are not alias-domains. I adjusted the code as follows:

    if (lc(($ldap->get_values("zimbraDomainType"))[0]) eq "alias") {
      my $robject = ($ldap->get_values("zimbraMailCatchAllForwardingAddress"))[0];
      syslog $syslog_priority, "Real Domain: %s", $robject if $verbose;
      @attrs=('1.1');
      $mesg = $ldap->search_s(
                "",
                LDAP_SCOPE_SUBTREE,
                "(&(|(zimbraMailDeliveryAddress=$user"."$robject)(zimbraMailDeliveryAddress=$daddr)(zimbraMailAlias=$user".
                "$robject)(zimbraMailAlias=$daddr)(zimbraMailCatchAllAddress=$user"."$robject)(zimbraMailCatchAllAddress=$robject)".
                "(zimbraMailCatchAllAddress=$daddr))(zimbraMailStatus=enabled))",
                \@attrs,
                0,
                $result
              );
      $ent = $ldap->first_entry();
      $ldap->unbind;
      if ($ent != 0) {
        syslog $syslog_priority, "Existing Alias-Recipient";
        return "dunno";
      } else {
        syslog $syslog_priority, "Non-Existing Alias-Recipient";
        return "reject 5.1.1 Mailbox unavailable";
      }
    } else {


      @attrs=('1.1');
      $mesg = $ldap->search_s(
                "",
                LDAP_SCOPE_SUBTREE,
                "(&(|(zimbraMailDeliveryAddress=$user"."$domain)(zimbraMailDeliveryAddress=$daddr)(zimbraMailAlias=$user".
                "$domain)(zimbraMailAlias=$daddr)(zimbraMailCatchAllAddress=$user"."$domain)(zimbraMailCatchAllAddress=$domain)".
                "(zimbraMailCatchAllAddress=$daddr))(zimbraMailStatus=enabled))",
                \@attrs,
                0,
                $result
              );
      $ent = $ldap->first_entry();
      $ldap->unbind;
      if ($ent != 0) {
        syslog $syslog_priority, "Existing Recipient";
        return "dunno";
      } else {
        syslog $syslog_priority, "Non-Existing Recipient";
        return "reject 5.1.1 Mailbox unavailable";
      }
    }

 

Then I restarted postfix as a zimbra user with:

zmmtactl stop
zmmtactl start

And finally, the policy daemon rejected mails to unexisting recipients 🙂

/opt/zimbra/libexec/zmpostfixpolicyd[5799]: Recipient Domain: example.com
/opt/zimbra/libexec/zmpostfixpolicyd[5799]: Recipient userid: does-not-exist
/opt/zimbra/libexec/zmpostfixpolicyd[5799]: Non-Existing Recipient
/opt/zimbra/libexec/zmpostfixpolicyd[5799]: Action: reject 5.1.1 Mailbox unavailable
postfix/smtpd[5791]: NOQUEUE: reject: RCPT from [...]: 554 5.1.1 <does-not-exist@example.com>: Recipient address rejected: Mailbox unavailable; [...]

Please use the described solution with care. It is my workaround, it is working on the system in question. It might not work with another setup. If you use it, test thoroughly if only the correct mails get rejected.

A Zimbra update will overwrite the zmpostfixpolicyd code and thus revert the change, so you might need to readjust the code after an update.

This drove me nuts and I hope this information is of help for somebody. Please comment if you have a better solution or find the described solution useful.

11. September 2012

Zimbra: Setting up a free (real) “commercial” SSL certificate

Filed under: Linux,Security,Server Administration — Tags: , , , , , — Christopher Kramer @ 20:40

I recently wrote about how to configure a new self-signed certificate for Zimbra.

Today I want to explain you how you can do even better and setup a real SSL certificate by Startcom which will make those annoying browser warning messages go away 😉 And the best: It is all for free, thanks to Startcom! It is also important to have a real SSL-certificate for use with most smartphones.

Please note that you have to respect Startcom’s certificate policy, which might require a non-free class 2 certificate for your commercial use. See the policy for details. Thanks to Thomas for this remark. With “commercial” certificate, I here mean a “real” CA-signed certificate, which Zimbra calls “commercial”, no matter whether used in a commercial context or not.

First, create a Certificate Request (CSR) in Zimbra. To do so, ssh into your server, login as root and issue a command like this:

/opt/zimbra/bin/zmcertmgr createcsr comm -new -keysize 2048 -digest sha256
-subject "/C=US/ST=CA/L=Sunnyvale/O=Yahoo/OU=Zimbra Collaboration Suite/CN=host.example.com"
-subjectAltNames host.example.com

This is one line. What is important here, is that you use the domain name for which the certificate should be issued at the end (instead of host.example.com). Startcom will ignore everything else anyway, so you can leave country, state, organization and so on as is or change it, doesn’t matter. What is also important is that you define a keysize of 2048 as Startcom won’t accept 1024 bit keys anymore. This parameter is not documented in the wiki yet, and seems to be available for newer versions of Zimbra only. With a little trick, it should also be also possible for older versions of zimbra such as 5 (but better update to the new version anyway…).

Next thing you do is register at startcom if you do not have an account yet. You’ll get an S/MIME certificate by Startcom for free which you need to login to their control panel. Your browser will generate the secret certificate and store it in its internal storage. I’d recommend you to backup this certificate – you will not be able to login into your startcom account if you loose it.

Then at startcom, you use the validation wizard to validate your domain. This will send a mail to postmaster/hostmaster/webmaster@host.example.com (you can choose which one) with host.example.com being your domain. So you need access to one of these mailboxes to prove that you own the domain.

Then use the certificate wizard at Startcom to create a new certificate. Skip the certificate creation step! Instead, past the CSR created by Zimbra ( /opt/zimbra/ssl/zimbra/commercial/commercial.csr ) into the webform. (Better always create private certificates yourself, never use certificate generators by somebody else, not even the CA.)

Once the certificate is created by Startcom (usually takes some minutes), install it as described in the zimbra wiki:

  1. Store the new (public) certificate you get from Startcom somewhere (e.g. /root/commercial.crt )
  2. Download the root CA certificate 
    wget -O /root/ca.pem https://www.startssl.com/certs/ca.pem
  3. Download the intermediary certificate from startcom 
    # If your certificate is class 1:
wget -O /root/ca_intermediary.crt https://www.startssl.com/certs/class1/sha2/pem/sub.class1.server.sha2.ca.pem
# If your certificate is class 2:
wget -O /root/ca_intermediary.crt https://www.startssl.com/certs/class2/sha2/pem/sub.class2.server.sha2.ca.pem
  4. Combine them: 
    cat /root/ca_intermediary.crt /root/ca.crt > /root/ca_chain.crt
  5. Verify your certificate: 
    /opt/zimbra/bin/zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key /root/commercial.crt /root/ca_chain.crt
  6. Deploy your certificate: 
    /opt/zimbra/bin/zmcertmgr deploycrt comm /root/commercial.crt /root/ca_chain.crt
  7. Check: 
    /opt/zimbra/bin/zmcertmgr viewdeployedcrt
  8. Restart Zimbra 
    su – zimbra
zmcontrol stop
zmcontrol start

I hope this post was useful so some of you. If you have problems with one of the steps, just ask in the comments.

Read here how to redirect http to https to enforce the use of https.

9. September 2012

Zimbra: Creating a new self-signed SSL certificate

Filed under: Linux,Server Administration — Tags: , , , , , , , , , — Christopher Kramer @ 10:04

I recently had to recreate the SSL certificate of a Zimbra server and surprisingly it was not as easy as the documentation looked like, so I’d like to document how it is done and make comments on some difficulties that might come up.

So this is how it is done (on a Ubuntu Server running Zimbra Network edition 6.0.16 GA):

  1. SSH into the server, login as root
  2. Switch to the zimbra-user using 
    su - zimbra
  3. Then run the following commands: 
     sudo /opt/zimbra/bin/zmcertmgr createca -new
 sudo /opt/zimbra/bin/zmcertmgr deployca
 sudo /opt/zimbra/bin/zmcertmgr deploycrt self
  4. Restart Zimbra. To do so, as user zimbra, issue these commands (no sudo here): 
    /opt/zimbra/bin/zmcontrol stop
/opt/zimbra/bin/zmcontrol start

So the difficulties I had and some remarks:

  • sudo kept asking me for a password when I typed in 
    sudo zmcertmgr createca -new

    Seems I am not the only one with this problem. The zmcertmgr command is white-listed in /etc/sudoers so you should normally not be asked for a password. Run the following command to edit /etc/sudoers (do not edit it in any other way!)

    visudo

    So make sure in this file the following line is included:

    %zimbra ALL=NOPASSWD:/opt/zimbra/bin/zmcertmgr

    The % at the beginning seems to belong there. Note that the zimbra wiki has typo (zmvertmgr) in this line.
    But although I had this line in there, sudo kept asking me for the password. So what finally worked was invoking zmcertmgr with the complete path (as done above).
    Update: It seems I had a typo in here myself. Make sure it is “zmcertmgr”  and not “zmzertmgr” 😉
    Thanks to the comment by erolha!

  • In the Zimbra Release notes, the last command for updating the certificate is 
    sudo zmcertmgr deploycrt self -new

    I got this error:

    Can't deploy cert for -new.  Unknown service.

    Without -new (and the complete path), it went through well.

  • No zimbra documentation I found mentions that a restart of zimbra is required, but without a restart, the old certificate was still used when opening the webmailer or the admin interface via https.

 

I hope I could help some of you that run into one of these problems.