Home
Products
Download
Buy Now
Support
Company

Help Index

 

Email Access via Z-Push (aka ActiveSync) [BETA]

ActiveSync is the Microsoft protocol for the groupware synchronisation of mobile devices. By installing the open source Z-Push php library as described below you can provide ActiveSync access to your surgemail email server. This notably provides:

  • More efficient mail synchronisation and sending mechanism than normal IMAP + SMTP (in terms of mobile device resources at the expense of server resources)
  • Instant "push notification" of new mail - of particular note on iOS devices (which do not support the IMAP IDLE alternative)

This guide provides a discussion on the comparitive technical merits. Also instructions are provided for installation if you choose to install this library.

ActiveSync Quick Start for administrators

  1. Install Z-Push and apply needed configuration tweaks
  2. Test and Login with your client of choice:
    • iOS : Specify username (eg user@domain.com), password. No autoconfiguration, once connection fails enter servername.

Technology Overview

ActiveSync

ActiveSync is the Microsoft protocol that allows mobile devices to efficiently synchronise with Microsoft Exchange. Some non exchange servers also implement some or all of the ActiveSync protocol. In the case of the use of Z-Push with surgemail only email can be synchronised at this stage. (On Exchange and some third party servers ActiveSync can also be used to synchronise contacts, calendar events, todo items and notes).

The ActiveSync protocol is a binary XML protocol across http. The protocol is specifically designed with efficient use from mobile devices in mind. As such the protocol is optimised for low bandwith, high latency connections. Also the protocol is designed for minimum number of whole request round trips. This means that the protocol can use many of the same techniques used to speed up access to websites. This is as opposed to imap or smtp which is a two way handshaking tcp protocol which can be both quite slow and costly in terms of battery consumption across high latency connections.

Push Implementation

Getting ActiveSync instant push notification on mobile devices - particulary iOS devices - may well be the primary reason you are wanting ActiveSync. However this will introduce significant additional server loading, so it should be carefully considered and monitored whether this is is actually a desirable feature on your server.

The client (ie phone) uses long polling http with 30 minute timeouts on the http requests. This means if no mail arrives there is no traffic on the tcp channel and the phone radios remain in low power receive only mode - so long battery life for you!

However, serverside these http requests are served as long polling web requests and processed in php. When the request comes in it consumes the resouces of a web request - see further discussion below - and keeps this open for up to 30 minutes, or until a new message arrives, or the client device is switched off. In addition to this an imap channel is opened to surgemail which is kept open but does not transition to imap idle. Instead every 5 seconds an IMAP status command is sent to surgemail for each monitored folder. Every 5 minutes a more comprehensive check is completed to make doubly sure that no new messages have been received.

Z-Push can be served as part of the surgemail webserver, or as part of the apache webserver. I believe slightly less server resources will be used if you install z-push under apache than under surgemail. These are the "continually in use" server resources per ActiveSync device that is configured to use push notification:

  • Surgemail: Surgemail web thread + php process + surgemail imap thread + repeated mail checks
  • Apache: Apache thread from thread pool + php thread from thread pool + surgemail imap thread + repeated mail checks

The above is the "standard" Z-Push configuration. We will continue to investigate the nicest ways to make this less resource intensive. At the very least you could install Z-Push on a server other than your surgemail server in which case the resource loading on the surgemail server is significantly less.

Mail Check and Send

Mail is received and accessed by connecting to surgemail using IMAP. This is done from php using php_imap.

Mail is sent from php through surgemail by making use of the "sendmail stub" binary which surgemail installs.

Installation Notes (under SurgeMail)

Under surgemail, php-cgi is used. This is the simplest but least efficient way of running php. You can also host z-push under apache in which case, make sure you are using a more efficient method for calling php (such as mod_php or fastcgi).

1. Setup php prerequisites

Access to surgemail email via ActiveSync is based on the open source Z-Push php library. As such you will need to have a suitable php configuration installed that includes php_imap and php-process. The easiest way to do that is to make sure you have the CalDav calendaring up and running.

Also it best to make sure these are set in the php.ini to go with php-cgi that you are running: 

   php_flag magic_quotes_gpc off
   php_flag register_globals off
   php_flag magic_quotes_runtime off
   php_flag short_open_tag on

2. Download Z-Push library

Download the Z-Push library from Z-Push website (version v2.2.0 has been tested to work).

Unzip the contents into surgemail/phplib/z-push, and suitably set permissions (unix only) 

   cd /usr/local/surgemail/phplib
   wget http://download.z-push.org/final/2.2/z-push-2.2.0-1934.tar.gz
   tar -xzf z-push-2.2.0-1934.tar.gz
   mv z-push-2.2.0-1934 z-push
   chown -R mail.mail z-push

Z-Push is configured using the file surgemail/phplib/z-push/config.php. At very least you will need to configure it to use "BackendIMAP" instead of "BackendZarafa". Z-Push also needs a state directory and log directory to be preconfigured. The defaults set in config.php are fine on unix (/var/lib/z-push/ and /var/log/z-push/) but can be changed if so desired - and need to be changed on windows.

Configure z-push to use imap and use ip address based connection: 

   define('BACKEND_PROVIDER', "BackendIMAP");
   define('IMAP_SERVER', '127.0.0.1');

And setup the z-push directories: 

   mkdir /var/lib/z-push/
   chown -R mail.mail /var/lib/z-push/
   mkdir /var/log/z-push/
   chown -R mail.mail /var/log/z-push/

On windows you will need to change the directory specification to directory of your choice (and update the creation process based on that): 

   define('STATE_DIR', 'd:/web/z-push/');
   define('LOGFILEDIR', 'd:/log/z-push/');

There are a variety of other z-push configuration options changable in config.php that may be useful including more verbose debug logging.

3. Setup surgemail interfacing

Download intefacing script z-push.php from netwinsite.com and save in surgemail/scripts directory: 

   cd /usr/local/surgemail/scripts
   wget https://netwinsite.com/ftp/surgemail/util/z-push_netwin.tgz
   tar -xzf z-push_netwin.tgz
   ls -l | grep z-push.php
   -rw-r--r-- 1 mail mail    158 Jul 28 23:09 z-push.php

Next add this to surgemail.ini 

   g_url_alias from="/Microsoft-Server-ActiveSync" to="/scripts/z-push.php"

and issue a "tellmail reload": 

   root@svr:/# tellmail reload

4. Verify the environment

You can't use ActiveSync with a web browser, but you can access it using a web browser as a test page. Try connecting to your own server on the url: https://your.server.com/Microsoft-Server-ActiveSync or http://your.server.com/Microsoft-Server-ActiveSync .

  • If all is working you will get a simple test page allowing you to login with your email credentials and confirming that all is well and you should connect with your mobile device.
  • If there is a problem with the configuration (eg can't access log files etc) this should be displayed as a relatively helpful exception.

See troubleshooting for further help.

Installation Notes (under Apache)

Under Apache the instructions are much the same as surgemail, other than making sure Apache is correctly configured for the virtual host in question. Also if you host php under apache it is recommended you make use of methods such as mod_php or fastcgi for greater efficiency (as opposed to the cgi calling under surgemail).

1. Setup apache / php prerequisites

You need to have apache running with php 5.1+ installed and php_imap and php-process.

2. Download Z-Push library

Download the Z-Push library from Z-Push website (version v2.2.0 has been tested to work).

The INSTALL file in the z-push distribution contains very good instructions, and some of the configuration can be done in one of several ways, but in a nutshell: 

   wget http://download.z-push.org/final/2.2/z-push-2.2.0-1934.tar.gz
   tar -xzf z-push-2.2.0-1934.tar.gz
   mv z-push-2.2.0-1934 /usr/share/z-push

Z-Push is configured using the file /usr/share/z-push/config.php. At very least you will need to configure it to use "BackendIMAP" instead of "BackendZarafa". Z-Push also needs a log and state directories to be preconfigured. The defaults set in config.php are fine (/var/lib/z-push/ and /var/log/z-push/) but can be changed if so desired.

Configure z-push to use imap and use ip address based connection: 

   define('BACKEND_PROVIDER', "BackendIMAP");
   define('IMAP_SERVER', '127.0.0.1');

And setup the z-push directories (owned by webserver process or globally writable): 

   mkdir /var/lib/z-push/
   chown -R www-data.www-data /var/lib/z-push/
   mkdir /var/log/z-push/
   chown -R www-data.www-data /var/log/z-push/
or 
   mkdir /var/lib/z-push/
   chmod 777 /var/lib/z-push/
   mkdir /var/log/z-push/
   chmod 777 /var/log/z-push/

3. Setup apache interfacing

Add apache alias (in one of the apache configuration files) 

   vi /etc/apache2/conf.d/z-push.conf
   -- z-push.conf begin --
   Alias /Microsoft-Server-ActiveSync /usr/share/z-push/index.php
   -- z-push.conf end --

Also a customer reported that with mod_fcgi an apache rewite rule may be required to pass authorisation through: 

   SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1

Make sure PHP has suitable flags set: 

   vi /usr/share/z-push/.htaccess
   -- .htaccess begin --
   php_flag magic_quotes_gpc off
   php_flag register_globals off
   php_flag magic_quotes_runtime off
   php_flag short_open_tag on
   php_admin_flag safe_mode off
   -- .htaccess end --

Restart apache 

   service apache2 restart

4. Verify the environment

You can't access ActiveSync with a web browser, but you can access it using a browser as a test page. Try connecting to your own server on the url: https://your.server.com/Microsoft-Server-ActiveSync or http://your.server.com/Microsoft-Server-ActiveSync .

  • If all is working you will get a simple test page allowing you to login with your email credentials and confirming that all is well and you should connect with your mobile device.
  • If there is a problem with the configuration (eg can't access log files etc) this should be displayed as a relatively helpful exception.

See troubleshooting for further help.

Extras

Auto configuration

ActiveSync supports autoconfiguration, so that users only have to enter their email address and password. This autoconfiguration works based on a web request to the primary domain hostname webserver. eg if you have email address user@domain.com that checks their mail on mail.domain.com, the following would need to be served on the "domain.com" webserver.

The autodiscover url is: http://domain.com/autodiscover/autodiscover.xml and the response must be formatted as documented in MSDN.

At this stage there is no explicit support for this, so the user needs to explicitly enter the server name.

Troubleshooting

This has been tested to successfully work on iOS and Android mobile devices.

If you get into trouble start by checking the log files in the z-push configured log directory. ie. z-push.log and z-push-error.log

Next check your php log file and webserver logs.

Also good additional installation notes can be found in the INSTALL file in the z-push distribution.

Lastly feel free to email surgemail-support@netwinsite.com for further help or suggested improvements.