# Upgrade iRedMail from 0.9.6 to 0.9.7

Attention

Check out the lightweight on-premises email archiving software developed by iRedMail team: Spider Email Archiver.

Remote Upgrade Assistance

Check out our remote upgrade support if you need assistance.

## ChangeLog

- Nov 12, 2017: Add Fail2ban jail `nginx-http-auth`.
- Jul 3, 2017: Mention how to upgrade uwsgi (OpenBSD only), iRedAdmin and iRedAPD.
- Jul 2, 2017: Mention Roundcube 1.3.0 requires PHP 5.4.
- Jul 1, 2017: Initial publish.

## General (All backends should apply these steps)

### Update `/etc/iredmail-release` with new iRedMail version number

iRedMail stores the release version in `/etc/iredmail-release` after installation, it's recommended to update this file after you upgraded iRedMail, so that you can know which version of iRedMail you're running. For example:

    0.9.7

### Upgrade iRedAPD (Postfix policy server) to the latest stable release (2.1)

Please follow below tutorial to upgrade iRedAPD to the latest stable release: Upgrade iRedAPD to the latest stable release

### Upgrade iRedAdmin (open source edition) to the latest stable release (0.8)

Please follow this tutorial to upgrade iRedAdmin open source edition to the latest stable release: Upgrade iRedAdmin to the latest stable release

### Upgrade Roundcube webmail to the latest stable release (1.3.0)

Roundcube 1.3

- Roundcube 1.3 requires at least PHP 5.4. If your server is still running PHP 5.3 and cannot upgrade to 5.4, please upgrade Roundcube to the latest 1.2 branch (1.2.5) instead.
- Roundcube 1.3 no longer supports IE < 10 and old versions of Firefox, Chrome and Safari.
- Roundcube 1.3 uses jQuery 3.2 and will not work with current jQuery mobile plugin. If you use any third-party plugin, please check its website to make sure it's compatible with Roundcube 1.3 before upgrading.

With the release of Roundcube 1.3.0, the previous stable release branches 1.2.x and 1.1.x will switch in to LTS low maintenance mode which means they will only receive important security updates but no longer any regular improvement updates.

> There're several security fixes in Roundcube 1.2.4 and 1.2.5, all users are encouraged to upgrade it as soon as possible. For more details about this release, please check Roundcube release notes:

> - 1.2.4
> - 1.2.5

Please follow Roundcube official tutorial to upgrade Roundcube webmail to the latest stable release immediately:

- How to upgrade Roundcube.

### Fixed: improper order of Postfix HELO restriction rules.

iRedMail-0.9.6 and earlier releases didn't configure Postfix to apply custom HELO restriction rule before FQDN helo hostname check and DNS verification, this way you cannot whitelist some bad HELO hostnames. Please follow steps below to fix it.

- Open file `/etc/postfix/main.cf` (Linux/OpenBSD) or `/usr/local/etc/postfix/main.cf` (FreeBSD), find parameter `smtpd_helo_restrictions` like below:

    smtpd_helo_restrictions =
        permit_mynetworks
        permit_sasl_authenticated
        reject_non_fqdn_helo_hostname
        reject_unknown_helo_hostname
        check_helo_access pcre:/etc/postfix/helo_access.pcre

- Move the `check_helo_access` line after `permit_sasl_authenticated`:

    smtpd_helo_restrictions =
        permit_mynetworks
        permit_sasl_authenticated
        check_helo_access pcre:/etc/postfix/helo_access.pcre
        reject_non_fqdn_helo_hostname
        reject_unknown_helo_hostname

- Reloading or restarting Postfix service is required.

### Fixed: incorrect owner and permission for rotated Dovecot log files

Attention

This is applicable to Linux, not FreeBSD and OpenBSD.

iRedMail-0.9.6 and earlier releases have an incorrect logrotate setting for Dovecot log file, it causes all Dovecot log files are empty due to no required permission to open log files. Please follow steps below to fix it.

Please open file `/etc/logrotate.d/dovecot`, find line below:

        create 0600 vmail vmail

Remove above line and save the change.

### Fixed: incorrect session.save_path in php-fpm pool config file on RHEL/CentOS

Attention

This is applicable to RHEL/CentOS system, and Nginx web server.

iRedMail-0.9.6 doesn't set path for `session.save_path` parameter in php-fpm pool config file `/etc/php-fpm.d/www.conf`, please fix it with steps below:

- Open file `/etc/php-fpm.d/www.conf`, find line:

    php_value[session.save_path] = "/var/lib/php/session"

- The directory name should be `sessions` (ends with `s`), not `session`. So please change it to:

    php_value[session.save_path] = "/var/lib/php/sessions"

- Restarting php-fpm service is required:

    service php-fpm restart

### Fixed: incorrect freshclam setting `UpdateLogFile`

Attention

This is applicable to RHEL/CentOS system.

With iRedMail-0.9.6, freshclam program cannot update ClamAV signatures due to improper log file permission, please open its config file `/etc/freshclam.conf` or `/etc/clamav/freshclam.conf`, comment out setting `UpdateLogFile` to use syslog for logging.

    #UpdateLogFile ...          # <- Comment out this parameter
    LogSyslog true              # <- Make sure you have this line. If not present, please add it manually.

### Fail2ban: fixes an improper filter and add new filter rule

iRedMail-0.9.7 fixes an improper filter for Dovecot log file which may cause incorrect ban, and adds a new filter for Roundcube log file to help ban bad client while Roundcube is running behind a proxy server.

- On Linux:

    cd /etc/fail2ban/filter.d/
    rm -f dovecot.iredmail.conf roundcube.iredmail.conf
    wget https://github.com/iredmail/iRedMail/raw/1.0/samples/fail2ban/filter.d/dovecot.iredmail.conf
    wget https://github.com/iredmail/iRedMail/raw/1.0/samples/fail2ban/filter.d/roundcube.iredmail.conf

Restarting Fail2ban service is required.

### Fail2ban: Add new jail for Nginx

Attention

This is applicable if you run Nginx as web server.

Let's add a new jail to stop bad clients which tried to perform http basic auth but failed.

Create file `/etc/fail2ban/jail.d/nginx-http-auth.local` with content below:

Attention

If directory `/etc/fail2ban/jail.d/` doesn't exist, you can append content below in file `/etc/fail2ban/jail.local` instead.

    [nginx-http-auth]
    enabled     = true
    filter      = nginx-http-auth
    action      = iptables-multiport[name=nginx, port="80,443", protocol=tcp]
    logpath     = /var/log/nginx/error.log

Restarting Fail2ban service is required.

### NEW: New backup script for SOGo

Attention

This is not applicable to SOGo-2.x because it doesn't support backing up all users' data with command `sogo-tool backup /path/to/backup/dir ALL`.

iRedMail has script `/var/vmail/backup/backup_mysql.sh` (or `backup_pgsql.sh`) to backup SOGo database by dumping whole database to a plain SQL file as backup. It's not ideal because:

- it's hard to restore single user's data with a plain SQL file.
- if SOGo changes some SQL structure, it's hard to restore all data.

This new script does backup with `sogo-tool backup` command to avoid issues mentioned above, you can restore a single user's data or all users data with `sogo-tool restore`.

Please follow steps below to setup this daily cron job.

- Download backup script. We store it under `/var/vmail/backup`, if you prefer a different directory, feel free to change the directory name used in commands below:

    cd /var/vmail/backup/
    wget https://github.com/iredmail/iRedMail/raw/1.0/tools/backup_sogo.sh
    chmod 0400 backup_sogo.sh

- This script will create new directory under `/var/vmail/backup` like below to store backup files:

    /var/vmail/backup
                |- sogo/
                    |- 2017/                # <- year
                        |- 03/              # <- month
                            |- 22.tar.bz2   # <- day (file name is: <day>.tar.bz2)

If you want to use another directory to store backup files, please open file `backup_sogo.sh`, update variable `BACKUP_ROOTDIR` with the new directory.

- Run command `crontab -e -u root` to setup root user's cron job. Add content below as new job:

    # SOGo: backup all users' data at 3:05AM everyday.
    5   3   *   *   *   bash /var/vmail/backup/backup_sogo.sh

### OpenBSD: Upgrade uwsgi to the latest 2.0.15

uwsgi is the interface between Nginx and iRedAdmin, so if you're running iRedAdmin, it's recommended to upgrade uwsgi to the latest version, 2.0.15.

Steps: Download the latest uwsgi, compile it, then restart uwsgi service.

    cd /root/
    ftp https://projects.unbit.it/downloads/uwsgi-2.0.15.tar.gz
    tar zxf uwsgi-2.0.15.tar.gz
    cd uwsgi-2.0.15
    python setup.py install

uwsgi should be succesfully installed, then restart uwsgi service:

    rcctl restart uwsgi

## OpenLDAP backend special

### Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

- [CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

- Open the daily MySQL backup script, it's `/var/vmail/backup/backup_mysql.sh` by default. if you use different storage directory during iRedMail installation, you can find the base directory with command `postconf virtual_mailbox_base`.

- Find variable name `CMD_MYSQLDUMP` like below:

    export CMD_MYSQLDUMP="mysqldump ..."

- Make sure it has argument `--skip-comments` like below:

    export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

- Save your change. That's it.

## MySQL/MariaDB backend special

### SQL structure change in `vmail.alias` SQL table

We've made some changes to `vmail.alias` SQL table for easier account management. This change introduces 2 new SQL tables (`forwardings`, `alias_moderators`), and (optionally) dropped few columns in `vmail.alias` table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Warning

Please backup SQL database `vmail` before you run any SQL commands below.

#### Create required new SQL tables

Please connect to MySQL server as MySQL root user, and execute SQL commands below to create required new tables:

    USE vmail;

    CREATE TABLE IF NOT EXISTS alias_moderators (
        id BIGINT(20) UNSIGNED AUTO_INCREMENT,
        address VARCHAR(255) NOT NULL DEFAULT '',
        moderator VARCHAR(255) NOT NULL DEFAULT '',
        domain VARCHAR(255) NOT NULL DEFAULT '',
        dest_domain VARCHAR(255) NOT NULL DEFAULT '',
        PRIMARY KEY (id),
        UNIQUE INDEX (address, moderator),
        INDEX (domain),
        INDEX (dest_domain)
    ) ENGINE=InnoDB;

    -- Forwardings. it contains
    --  - members of mail alias account
    --  - per-account alias addresses
    --  - per-user mail forwarding addresses
    CREATE TABLE IF NOT EXISTS forwardings (
        id BIGINT(20) UNSIGNED AUTO_INCREMENT,
        address VARCHAR(255) NOT NULL DEFAULT '',
        forwarding VARCHAR(255) NOT NULL DEFAULT '',
        domain VARCHAR(255) NOT NULL DEFAULT '',
        dest_domain VARCHAR(255) NOT NULL DEFAULT '',
        -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
        is_list TINYINT(1) NOT NULL DEFAULT 0,
        -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
        is_forwarding TINYINT(1) NOT NULL DEFAULT 0,
        -- defines whether it's a per-account alias address. 0=no, 1=yes.
        is_alias TINYINT(1) NOT NULL DEFAULT 0,
        active TINYINT(1) NOT NULL DEFAULT 1,
        PRIMARY KEY (id),
        UNIQUE INDEX (address, forwarding),
        INDEX (domain),
        INDEX (dest_domain),
        INDEX (is_list),
        INDEX (is_alias)
    ) ENGINE=InnoDB;

#### Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

Attention

If you're running Python 3, please download this file instead: `https://github.com/iredmail/iRedMail/raw/master/tools/migrate_sql_alias_table.py`, and run it with command `python3 migrate_sql_alias_table.py`.

    cd /root/
    wget https://github.com/iredmail/iRedMail/raw/1.0/tools/migrate_sql_alias_table.py
    python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user `vmailadmin`:

- /opt/www/iredadmin/settings.py (Debian/Ubuntu)
- /var/www/iredadmin/settings.py (CentOS/OpenBSD)
- /usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
- /usr/local/www/iredadmin/settings.py (FreeBSD)

#### Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is `/usr/local/etc/postfix/mysql`.

    cd /etc/postfix/mysql/
    perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
    perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
    perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
    perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
    perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

#### Drop unused SQL columns and records in `vmail.alias` table

Warning

- Make sure you have a backup of SQL database `vmail`.
- Please also upgrade iRedAPD and iRedAdmin-Pro, they need the new SQL structure too.

After migration, `vmail.alias` table contains few sql columns we will never use, also old records (accounts) will cause ghost accounts if we don't remove them.

Please connect to MySQL server as MySQL root user, then execute SQL commands below:

    USE vmail;

    -- Remove non-mail-alias account
    DELETE FROM alias WHERE islist <> 1;

    -- Remove per-domain catch-all account
    DELETE FROM alias WHERE address=domain;

    -- Drop unused columns
    ALTER TABLE alias DROP COLUMN goto;
    ALTER TABLE alias DROP COLUMN moderators;
    ALTER TABLE alias DROP COLUMN islist;
    ALTER TABLE alias DROP COLUMN is_alias;
    ALTER TABLE alias DROP COLUMN alias_to;

### Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

- [CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

- Open the daily MySQL backup script, it's `/var/vmail/backup/backup_mysql.sh` by default. if you use different storage directory during iRedMail installation, you can find the base directory with command `postconf virtual_mailbox_base`.

- Find variable name `CMD_MYSQLDUMP` like below:

    export CMD_MYSQLDUMP="mysqldump ..."

- Make sure it has argument `--skip-comments` like below:

    export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

- Save your change. That's it.

## PostgreSQL backend specific

### SQL structure change in `vmail.alias` SQL table

We've made some changes to `vmail.alias` SQL table for easier account management, this change introduces 2 new SQL tables (`forwardings`, `alias_moderators`), and (optionally) dropped few columns in `vmail.alias` table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Warning

Please backup SQL database `vmail` before you run any SQL commands below.

#### Create required new SQL tables

Please run shell commands below to connect to PostgreSQL server as `vmailadmin` user first:

    su - postgres
    psql -U vmailadmin -d vmail

Then execute SQL commands below to create required new tables:

    CREATE TABLE alias_moderators (
        id SERIAL PRIMARY KEY,
        address VARCHAR(255) NOT NULL DEFAULT '',
        moderator VARCHAR(255) NOT NULL DEFAULT '',
        domain VARCHAR(255) NOT NULL DEFAULT '',
        dest_domain VARCHAR(255) NOT NULL DEFAULT ''
    );

    CREATE INDEX idx_alias_moderators_address ON alias_moderators (address);
    CREATE INDEX idx_alias_moderators_moderator ON alias_moderators (moderator);
    CREATE UNIQUE INDEX idx_alias_moderators_address_moderator ON alias_moderators (address, moderator);
    CREATE INDEX idx_alias_moderators_domain ON alias_moderators (domain);
    CREATE INDEX idx_alias_moderators_dest_domain ON alias_moderators (dest_domain);

    CREATE TABLE forwardings (
        id SERIAL PRIMARY KEY,
        address VARCHAR(255) NOT NULL DEFAULT '',
        forwarding VARCHAR(255) NOT NULL DEFAULT '',
        domain VARCHAR(255) NOT NULL DEFAULT '',
        dest_domain VARCHAR(255) NOT NULL DEFAULT '',
        -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
        is_list INT2 NOT NULL DEFAULT 0,
        -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
        is_forwarding INT2 NOT NULL DEFAULT 0,
        -- defines whether it's a per-account alias address. 0=no, 1=yes.
        is_alias INT2 NOT NULL DEFAULT 0,
        active INT2 NOT NULL DEFAULT 1
    );
    CREATE INDEX idx_forwardings_address ON forwardings (address);
    CREATE INDEX idx_forwardings_forwarding ON forwardings (forwarding);
    CREATE UNIQUE INDEX idx_forwardings_address_forwarding ON forwardings (address, forwarding);
    CREATE INDEX idx_forwardings_domain ON forwardings (domain);
    CREATE INDEX idx_forwardings_dest_domain ON forwardings (dest_domain);
    CREATE INDEX idx_forwardings_is_list ON forwardings (is_list);
    CREATE INDEX idx_forwardings_is_forwarding ON forwardings (is_forwarding);
    CREATE INDEX idx_forwardings_is_alias ON forwardings (is_alias);

    -- Grant required privilege to vmail user
    GRANT SELECT ON TABLE forwardings to vmail;
    GRANT SELECT ON TABLE alias_moderators to vmail;

#### Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

    cd /root/
    wget https://github.com/iredmail/iRedMail/raw/1.0/tools/migrate_sql_alias_table.py
    python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user `vmailadmin`:

- /opt/www/iredadmin/settings.py (Debian/Ubuntu)
- /var/www/iredadmin/settings.py (CentOS/OpenBSD)
- /usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
- /usr/local/www/iredadmin/settings.py (FreeBSD)

#### Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is `/usr/local/etc/postfix/pgsql`.

    cd /etc/postfix/pgsql/
    perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
    perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
    perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
    perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
    perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

#### Drop unused SQL columns in `vmail.alias` table

Warning

- Make sure you have a backup of SQL database `vmail`.
- Please also upgrade iRedAPD and iRedAdmin-Pro, they need the new SQL structure too.

After migration, few columns in `vmail.alias` table are not used anymore, it's ok to drop them. But it's strongly recommended to keep them for few more days until you can confirm all features are working as expected.

    su - postgres
    psql -d vmail

    -- Remove non-mail-alias account
    DELETE FROM alias WHERE islist <> 1;

    -- per-domain catch-all account
    DELETE FROM alias WHERE address=domain;

    -- Drop unused columns
    ALTER TABLE alias DROP COLUMN goto;
    ALTER TABLE alias DROP COLUMN moderators;
    ALTER TABLE alias DROP COLUMN islist;
    ALTER TABLE alias DROP COLUMN is_alias;
    ALTER TABLE alias DROP COLUMN alias_to;