MDaemon to Microsoft Exchange migration

Posted in Blogs,Sys Admin Corner,Ubuntu by admin on September 14th, 2011

 

Every modern company wishing (and willing to pay the premium) to move forwards in regards to their electronics communication with the rest of the world, has definitely thought of the Microsoft Exchange Server.

That was my case at my current job position as a network/system administrator. Among a reasonable amount of available mailing systems, Microsoft’s Exchange Server was a very strong candidate, fulfilling our requirements.

There were many parameters involved in the final decision (eg. once off price, recurring fees, scalability, stability, functionality, management, value adding services etc), and Exchange Server’s biggest disadvantage might have been it’s high licensing price. But since my company was willing to pay the additional cost, I was more than happy to carry on with this project.

As the post’s subject indicates, our previous mailing system was  Alt-N’s MDaemon (v11.0.3 Pro at that time), and that’s where my odyssey begins! Migrating 100+ mailboxes, as transparently as possible to Microsoft’s Exchange Server 2010 SP1! Any kind of email loss was definitely out of the question. The same goes for any kind of office hours service disruption, and hopes migrating all the accounts throughout a weekend, were soon proven daydreaming material.

I’ve been through various migration tutorials, but some of them were risky, others were incomplete and some others unclear. Despite not having any straight forward procedure to follow, all the reading I went through, gave me a better general idea on how to make my own migration plan from the one mailing system to the other.

Right now I have just completed this project has successfully, so I decided to make this write up, trying to provide as straight forward instructions as possible for a complete MDaemon to Exchange Server migration. I hope you will find my information useful.

To start with, Figure 1 shows a basic diagram of the original mail flow within the company’s network. Any other L2 & L3 devices in between, not affecting the mail service, have been left out.

Figure 1

Roughly, our domains’ MX records, point to a specific public IP address of our firewall, which is NAT’ed to the mail server. This means that all incoming emails are downloaded first by the firewall, where they are virus scanned and checked again anti-spam databases and other static rules.

All valid emails, are then forwarded to the MDaemon server’s internal IP address, which does its own checks, and finally delivers them to the corresponding user’s mailbox and a third party email archiver.

Figure 2

Due to the large amount of user accounts, as well as their high volume of data in each mailbox, direct migration was out of the questions. So I followed a server co-existence approach (Figure 2). In this scenario the two independent mailing systems work simultaneously, both serving users under the same domain name.

 

 

To achieve such a scenario, you need to perform the following configurations on the two servers:

Screenshot 1
  • Setup MDaemon to relay all emails sent to non-existing users, to another server, the Exchange Server in our case. Navigate to Setup -> select “Default Domain & Servers” option and highlight the “Unknown Mail” tab  (Screenshot 1). You have to enable advanced options, and enter your Exchange server’s hostname or internal IP address and receiving port. You might also have to drop some NDR (Non Delivery Report) settings, to avoid possibly false undeliverable notifications. NDRs are controlled by the top boxes under the same tab (“Incoming mail sent to local users who don’t exist should be…”). In my case I unticked all 3 of them.
Screenshot 2
  • Setup MDaemon to accept emails relayed from the Exchange Server, to other local or Internet recipients (Screenshot 2). To do this, you have navigate to Setup -> select “Security Settings” option and highlight the “Relay Control” tab. There, you have to loosen up the relay security settings, and allow TRUSTED hosts/IPs relay through MDaemon. Then make sure that you add your Exchange Server’s Hostname and internal IP address under the “Trusted Hosts” tab.
Screenshot 3
  • Setup a receiving connector on Exchange Server, to trust and accept emails relayed from the MDaemon server, for domain’s mailboxes unknown to MDaemon. This is done from Server Configuration -> Hub Transport -> Receive Connectors. Enter MDaemon’s IP address as the remote server, under the Network tab (Screenshot 3). To simplify things, I chose to allow Basic Authentication (under the connector’s Authentication tab), for all groups.
Screenshot 4
  • Setup a send connector on Exchange Server (Organization Configuration -> Hub Transport -> Send Connectors), of SMTP type and * as the Address Space. Under Network tab, select to route emails via a smarthost, and enter your MDaemon’s address as one (Screenshot 4). You can also set your preferred Authentication Method, by pressing the “Change” button below. This connector will be used to relay all outgoing emails via the MDaemon server, for mailboxes not existing on the Exchange server. * Remember: after every Hub Transport configuration, the corresponding Windows “Microsoft Exchange * ” service, requires a restart to apply the changes.

 

At this point, if you are using a software or hardware email archiving solution, you must be careful. Even though all incoming/outgoing emails pass through MDaemon server which is configured to archive them; the internal communication between users residing on the Exchange Server, is just local. Those emails never reach MDaemon, therefore not archived.

But at the same time, having both servers archiving your emails, can results into duplicate copies. There is no easy solution for this issue, so you have to compromise some email archiver space. After all, typically such software/appliances have advanced compression algorithms, saving up space from duplicate copies. So, as soon as some users have been migrated to the Exchange Server, it’s wise to setup the email archiving (journal) functionality (Screenshot 5).

Screenshot 5

Some other helpful steps I performed are listed below. They are not compulsory for the migration process:

  • To make the Exchange Server directly accessible from the outside world throughout the migration process, so users can access their new mailboxes, I assigned it its own public IP address. Also made sure that the firewall will allow access to important ports (eg 25, 110, 443, 465, 567, 993 etc), to enable use of OWA/IMAP/SMTP etc services from outside the company.
  • Getting an SSL certificate from a CA, is highly recommended (if you are paying for a fully licensed Exchange Server, this is not the part where you cut back costs). Useful instructions on how to get the certificate installed can be found here.

Up to this point, you should have a functional co-existence setup. You can create some Exchange mailboxes and run some internal tests and external tests to verify its correct operation.

The only thing missing, is your data! You can say we are “half way there”, but when you are done with this project and look back on how much time you spent struggling with each and every mailbox to migrate as transparently as possible, this parts feels like FOREVER! But anyway, stay focused and pay attention to the following steps. We can finally start the actual data migration!

  • Export MDaemon’s accounts list. Click on Accounts -> Exporting -> Export accounts to a Comma Delimited file. This might take a while, but eventually you will have a list of all mailboxes, including their username and password in a CSV format (Screenshot 6). This might not sound very secure, the server admin knowing every user’s password, but comes handy in situations like this.
Screenshot 6

After spending some time, researching for an email migration solution, to make my life easier, my research quickly came to an end with only a small amount of tools promising to help me carry out the task. Most of the tools were carrying a high price tag, per migrated mailbox. Other tools were too intrusive, involving the user’s Outlook Exporting his/her mailbox(es) into PST files and then Importing them to the Exchange Server mailbox via Outlook again.

Finally, the most appropriate tool for the task was just under my nose. IMAPsync is included in Ubuntu’s repositories and of course it’s free and open source under WTFPL license. This tools could be ran from the server or any PC, and would login into 2 IMAP accounts and synchronize their contents. It’s a higher level migration than I was hoping, over the IMAP protocol, therefore slower; but I was happy as long as it worked! It’s a great tool, don’t forget to donate! :)

Oh the irony… migrating to a Microsoft product, using Linux! :)

  • Installing IMAPsync under Ubuntu, is as simple as typing:

sudo apt-get install imapsync

in a terminal window. Then you can read its manual, typing  ” man imapsync ”   in the terminal window again. Alternatively, you can save your time, and use the same command I used and provide in this post, at a later stage.

After this, we are ready to migrate our first mailbox. Here is a the process I followed for each mailbox migration:

  1. Create a mailbox for a new/existing Active directory user on Exchange Server (eg. user@example.com). In the mailbox’s properties, add any email aliases and forwardings you might require.
  2. Make sure that you know user’s both MDaemon (exported earlier in CSV) and Active Directory credentials (you will have to ask the user for this, or reset the password yourself)
  3. Rename the MDaemon email account to something else (eg. from user@example.com to user-0@example.com), so all new emails for user@example.com will be forwarded to the Exchange Server account.
  4. Find the MDaemon’s maillists the renamed account (user-0@example.com) is member of (in account’s settings, Options Tab).
  5. Open individually every mail list, and replace the renamed account (user-0@example.com) with the account’s original name, eg. user@example.com (has to be typed/added manually).
  6. Remove any aliases/forwardings assigned to the MDaemon mailbox, and add them to the Exchange Server mailbox. Skipping this step can lead into tricky situations, where emails are not sent or received.
  7. Migrate the MDaemon mailbox to Exchange account using the IMAPsync tool. I used the following command command:
    imapsync –host1 MdaemonIP –user1 MDaemonUser –password1 MDaemonPass –host2 ExchangeIP –user2 ExUser –password2 ExPass –authmech2 PLAIN –syncinternaldates
    MDaemonIP: MDaemon server’s internal IP address
    MDaemonUser: migrating mailbox (eg. user-0@example.com)
    MDaemonPass: the account’s password exported earlier
    ExchangeIP: Exchange Server’s internal IP address
    ExUser: Active Directory username
    ExPass: Active Directory username
  8. Meanwhile, you can setup the user’s new Exchange mailbox to his Outlook, while it is being populated by old emails, as well as newly arriving ones, so he will still be able to work.
  9. Ideally the user should not move any emails around folders during this process.
  10. In case of a huge mailbox, if the user needs to access some old emails, you can correct the MDaemon’s IMAP account’s credentials in Outlook (eg. set account’s username to: user-0@example.com).
  11. User should NOT move/delete any emails from MDaemon mailbox, while migration procedure is taking place.
  12. When IMAPsync process has completed, it is safe to disable the MDaemon account, and remove it from the user’s Outlook.

Plenty of loooong endless days/weeks/months later repeating the above procedure, you should be done! I know it’s not the fastest method, but it’s the safest one, since you gradually move forwards to Exchange Server, troubleshooting any issues coming up and avoiding repeating possible mistakes.Using IMAPsync you can perform a mass migration, but personally, I wouldn’t risk it!

When you are done migrating user accounts, remember to recreate the Distribution Lists on Exchange Server, and disable them on MDaemon.

When this last step is finally completed, it is time to kill the MDaemon! Make sure that you perform this task during non-working hours! As Murphy’s law states: “If anything can go wrong, it will!” :)

What’s left to be done is simple. But if anything goes wrong, it means downtime on the mail services. Something personally I couldn’t risk.

  1. Remove the MDaemon-specific send/receive connectors, and setup your corresponding connectors to communicate with the Internet directly. Restart the Transport Service
  2. Change the NAT (or whatever method you use) of the MDaemon’s public IP address, to also point to the Exchange Server.
  3. Specify to the firewall, to relay emails to the Exchange Server’s internal IP address from now on, and remove/replace any other MDaemon instances in its configuration
  4. Cross your fingers and send plenty of external test emails, from various webmails to your company’s mailboxes
  5. Unplug MDaemon server’s network connection for the next couple of days, just to make sure that everything works smoothly, and have MDaemon standby, in case something goes wrong, to revert back to (mailflow-wise).

That is all more or less! This is how I spent my “free time” for the past several weeks/months!

My name is Frank, and I have been MDaemon-free 10 days now! :)

I hope someone out there will find my experience useful, and benefit off it!

 

Frank

6 Responses to “MDaemon to Microsoft Exchange migration”

  1. Andy Says:

    While migrating from MDaemon to Exchange 2003, I found that I had to use a regular expression to correctly map sent dates to received dates for messages that I myself had sent. The syntax is:


    ./imapsync --regexmess 's{A(.*?(?!^$))^Date:(.*?)$}{$1Date: $2nReceived: From; $2}gxms' --host1 host1.example.com --host2 host2.example.com --user1 user1 --password1 pass1 --user2 user2 --password2 pass2

    Thanks for a useful writeup on the migration process.

    Andy

  2. Andy Says:

    In my experience, the section “Setup MDaemon to relay all emails sent to non-existing users, to another server” didn’t work. MDaemon just refused the emails outright if it didn’t recognise the user.

    Instead, I had to go into each account as I moved it from MDaemon to Exchange and tell it to forward emails directly to the Exchange server:
    Edit Accuont… > Forwarding:
    – Tick “This account is currently forwarding mail”
    – enter the receiving address on the Exchange server
    > Advanced Forwarding Options:
    – Forward the message to this domain: [enter the IP of the Exchange server]

    Note that using this method you shouldn’t remove aliases from the account, nor rename the account name, otherwise MDaemon will not recognise it.

  3. Frank Says:

    Thank you very much for your feedback Andy.
    Differences could be caused by different a MDaemon version, or maybe having setup my MDaemon installation in the past with some settings slipped from my post.

    Glad I could be of some help. I hope you are done with your migration odyssey :)

  4. Jim Says:

    Hi Frank,

    I have a slightly different setup in that I have taken over a setup that is mostly Exchange but some still Mdaemon. Having looked at their setup they seem to be using a forwarder as an intermediatory between Mdaemon and Exchange for the Mdaemon accounts.
    A couple of things I have noticed that are different on this setup. First, the server is under trusted hosts but only as a wildcard for the whole IP scope the exchange servers sit on, the forwarders are there as well as their full IP addresses.
    Secondly, Under relay control there are no ticks in the bottom 6 tick boxes, where you said to loosen up security.
    Also, under Unknown Mail tab, nothing set in there.

    So, what I want to know is. Should I try setting it up the way you have and see if that works or is it more trouble than it is worth trying to play with those settings?

    Unfortunately I need to be able to understand this setup before I can move forward with any migration, so as not to loose any current data or cause any e-mail outages.

  5. Frank Says:

    Hi Jim. I will do my best to rewind my brain back 1+ year to help you :)
    What kind of forwarder are you talking about?
    My relay control settings might have been a bit of an overkill. But since I was working on a live environment, and got the desired functionality, I didn’t play around with it any further.
    What are you trying to achieve? Get migration of all users to Exchange and getting MDaemon out of the picture?

  6. Jim Says:

    Hi Frank.

    The forwarder is an Imap Forwarder as far as I can tell.

    What I am initially trying to achieve is just a full understanding of how the mail is currently routed with the end aim to migrate all users over to Exchange. But for now they are having a few occasional problem where the exchange server stops working, so with a better understanding I should hopefully be able to pinpoint whether it is Mdaemon that is stopping it working, the firewall or Exchange itself. When I say it stops working it just stops getting anything routed through it. A reboot is the only thing to do that has got it back up and running.

Leave a Reply

You must be logged in to post a comment.