Blog Archives

Goodlink 5 deployment issue

I have been working on deploying multiple Goodlink 5 messaging servers recently, and came across a problem with the user migration feature in Goodlink 5.  Normally if you have more than one Good messaging server, you have the option to “change good messaging server” when you right-click on a user account.  So in my deployment when I didn’t see this option enabled, I had to call Goodlink support.  What I found out made sense, but is definately irritating. 

I called Goodlink support before deploying Goodlink version 5 to ask about deployment options and get some advice on my project.  I asked about the user migration feature and using multiple GoodAdmin mailboxes, and was told that it should work fine.  Now here is where the issue comes in.  In my deployment scenario, I wanted to deploy multiple individual Goodlink 5 servers in various locations/offices around the world.  The limitation here is network bandwidth and latency, due to the fact that the offices getting these Goodlink 5 servers are spread across the globe on various speed networks, and some of the offices in Asia for example, have very high network latency and bandwidth is limited.  To help with this situation, I asked if it would be advisable to use multiple GoodAdmin mailboxes, one for each new Goodlink 5 server, that way, each location that got a Goodlink 5 server and had a local Exchange server, would get its own GoodAdmin mailbox homed on the local Exchange server.  This would increase performance, reduce the time it takes to re-connect all the mailboxes in the event of a reboot, and provided a better solution for my deployment project. 

When you use a single consolidated GoodAdmin mailbox, and you have more than one Goodlink server which is connected to the network on a low bandwidth link or one that has high latency, it is a good idea to have separate GoodAdmin mailboxes, to keep network traffic local to the office in question.  Its faster, more reliable, and if the central Exchange server that houses the consolidated GoodAdmin mailbox were to go down, all your Good Messaging servers would be rendered useless until the Exchange server is back online and the GoodAdmin mailbox is available again. 

So my scenario of using more than a single consolidated GoodAdmin mailbox was a good idea, still works well and does help with performance and all the other reasons I mentioned above.  The drawback is that in this configuration, from within the Good Messaging console, you only see the server you are connected to under the “Good messaging servers” folder.  This also means that the “change good messaging server” option is grayed out and unavailable.  The fact is that the GoodAdmin mailbox is what keeps track of how many servers you have, but they all have to be configured to use the same GoodAdmin mailbox in order for the server to “know” about the other Good Messaging servers you have in your network.  This basically means that the user migration option is not going to work for us. 

This is not a problem, its just an inconvenience.  And it would have been nice to have known this initially when I talked with Good support eariler.  Lately no matter what company I deal with, I usually end up getting conflicting information about their products and features based on who you talk to.  The first time I call and get info on something, I hear one thing, then when I am working on the deployment and have a question or problem and call back, I hear something completely different.  This is irritating and can cause some major problems for large projects. 

Advertisements

Pasting links from OWA

I deal with Good support now and then for various technical issues with their mobile e-mail solution (Goodlink).  Today it appears they sent out a survey request to all of their customers, but I had to chuckle a little when I read the message and found they had pasted in the survey link through OWA.  If you read below, you will find the actual URL is missing the first h in the (http), and the actual linked URL is an Exchange OWA redirect link.  This happens when you copy a link from OWA and paste into another message.  By default in OWA, links point to the Exchange redirector and then take you to the link specified at the end of the redirect URL. 

I run into this issue myself from time to time and forget about this until I get a reply back to something I sent out with a complaint that the URL I sent does not work.  To get around I usually paste the link into notepad so I Can get a plain text copy of the link, and then I remove the Exchange redirector part.  I then paste in the plain text link which most e-mail programs and even OWA convert to a hyperlink automatically for you. 

I’m not trying to gripe about this flub, but find it humorous that such a mistake can be made by anyone and its nice to know I’m not the only person out there who forgets about this from time to time and sends out links that don’t work.  Now naturally a technically apt person could easily extract the correct link and get where they need to go, but the average person is going to reply and complain that you sent them a bad link.  Not just that, but now we all have the OWA logon link to their Exchange mailboxes.  (Don’t worry Good (motorola), I’ve hidden the actual OWA Link for your privacy) 🙂  (NOTE: I did receive a correction email from them shortly after receiving the message below).

Dear Valued Motorola Good Technology Group Customer:

Thank you for your recent inquiry into our support team. We appreciate the opportunity to serve you, and help you and your company meet your mobile messaging needs. To gauge the level of support you, and other customers are receiving, and to better understand where our strengths and weakness are, we are asking for your help.

By clicking on the following link and taking a brief six question survey, you will help us gauge the level of support we are providing our customers. This will allow us to know where our strengths are, and where we need improvement:

ttp://www.zoomerang.com/recipient/survey-intro.zgi?p=maskedsurveyidforprivacy <https://nosy.notmotorola.com/exchweb/bin/redir.asp?URL=http://www.zoomerang.com/recipient/survey-intro.zgi?p=maskedsurveyidforprivacy>

Please spend the two minutes it will take to respond to the survey questions so we will know how to better serve you in the future.

Sincerely,

The Good Technology Group Technical Support Team

Details on DST patches, Goodlink issues and resulting patch problems.

This post is going to be very technical in nature and hopefully someone will find it useful. This all started because of the patches for Exchange 2003 due to Daylight Saving time updates. Below you will find detailed information on the DST patches, resulting problems I had with Goodlink, permissions issues I encountered, and resulting side effects of a supported fix from Microsoft. I’ll break it down into sections, relating to each type of issue I encountered.

Daylight Saving Patches

After installing DST patches on Exchange 2003, I found myself unable to mount the stores on all of my Exchange 2003 servers. The information store service would start, but none of the stores would mount from ESM. After some researching I found this article from Microsoft that was supposed to fix the problems I encountered. After the 930241 patch was installed, my stores would mount and everything seemed to be working fine. After applying these updates, Exchange seemed to be working fine at the time, all my message tests were good and I didn’t see any obvious signs of a problem.

Goodlink and the Send-As permission

After applying the DST patches and the update from MS KB 930241, I found that my Goodlink devices were not working. At first it was affecting everyone, no one was sending or receiving goodlink messages from their handhelds. I did some checking and found this article on the good.com website. Now on a previous case with Good support, we went through and modified the correct permissions for the Goodadmin account in advance, so that we wouldn’t have this issue. It turned out that some of our Exchange objects were not inherriting permissions correctly, so when we applied the DST patches along with 930241, our goodlink system lost the send as permission required to operate properly. I got on the phone with Good support and we went through and manually reset the permissions again, ensuring that all Exchange objects were inherriting the proper permissions. I had to restart the Exchange services on all 4 servers to make the changes effective, and once done, Goodlink resumed working normally. This all happened between a Firday night and a Saturday morning. So not only do you need to ensure you grant the send-as right in AD to the GoodAdmin account, but I’d strongly recommend making sure you have all the inheritance setup properly for all Exchange objects as well. I think our permission issues were remnants of our setup of a mixed environment with Exchange 5.5 and 2003.

Read the rest of this entry

My year of Hell in 4 days

There is a Star Trek Voyager episode entitled “Year of Hell”. Its about a span of 365 days in the life of the crew where pretty much anything that could go wrong, goes go wrong and I think they barely survive (its been a while since I’ve seen it). Well, over the last 4 days, I’ve had my year of Hell starting from Friday night to yesterday around 6:30pm. I’ll summarize:

1. Friday night, I installed DST patches

a. DST patch caused a problem where my info stores would not
mount in Exchange 2003
b. The DST patches also broke the goodlink system for mobile e-mail
c. Turned out that MS had changed the way the permissions work in
Exchange and AD and we had to re-set the permissions on our
GoodAdmin account
c. Spent up until 11:30pm working on the DST patches and fixing the
goodlink account while on the phone with Good support

2. Saturday – all day

a. Checked the server from previous night to ensure everything is
still working
b. Found everything to still not be working.
c. Found some objects in Exchange 2003 were not
inherriting permissions correctly. Had to continue
resetting permissions for those items and testing.
d. Continued working with Good support and testing and finally got the
Goodlink mobile e-mail working
e. Had to follow a workaround by allowing inherritance on the
AdminSDHolder AD object and allowing time for replication
f. Spent the rest of the day testing the mobile e-mail system paranoid
that it would fail again.
g. Abby got sick Friday night, so I got maybe 2 hours of sleep Friday
night and Sat Morning

3. Sunday – all morning

a. Continued testing the mobile e-mail system, all tests were working
ok, external and internal.
b. I stayed home with Abby all day to take care of her, she was really
sick and had a very high fever.
c. Got my first almost full night of sleep, since Liz was kind enough to
stay in the living room with Abby

4. Monday – All DAY!!!

a. Started getting complaints that people couldn’t open some e-mail
messages in Outlook
b. Got some reports that people were missing e-mails, that the volume
of messages received over the weekend and morning were very low
c. To make it more frustrating, all the external tests and internal tests
I was doing were working fine
d. Checked the mail queues in Exchange 2003 and found hundreds of
queued messages, they were stuck in the MTA queue
e. Nothing I tried would release the messages
f. I called Microsoft and paid my $245 fee for support, and spent about
8 hours on the phone with 2 separate Microsoft engineers.
g. Once I got through to the escalation engineer, we got the issue fixed
within 3 hours or so.
h. Turned out that MS had made two code changes to alter the way
Exchange does recipient and sender address formatting or lookups.
i. The quick fix was to uninstall patch 930241. So I did, and that
restored functionality completely, for both symptoms.
j. I also worked with MS to run some store traces to try to
identify the issue.
k. After some minor permissions adjustments for two servers who were
still not inherriting permissions correctly, everything started to
work normally.
l. I did NOT have lunch this day, nor anything to drink. I was on the
phone all day!
m. At 6:35pm I finally left the office and went home and crashed!

So as you can see, thanks to Congress for screwing around with the Daylight saving configuration and thanks to Microsoft for making some code changes that had unexpected side effects, I ended up having a very bad few days indeed. I really didn’t get a weekend, no break, barely any sleep. Its going to take me a while to recover from that. I need a vacation! Sarah has off all next week for spring break, I’m thinking of asking for next Wednesday off to break up the work week and give myself a break.

Stressful few days

The last few days have been hectic to say the least. With all the DST preparations that were needed and other daily tasks it was just crazy. Then on Friday evening after installing the DST updates for Exchange 2003, it broke our Goodlink system for mobile e-mail. The cause was a known issue, and I even worked wtih Goodlink previously and made the necessary changes to our permissions to get around it, but for some reason, we still had problems. I had to contact Good support and work with a tech until almost midnight on Friday, and even past noon today (Saturday). I basically had to re-do all the goodlink account permissions and modify some settings in AD and Exchange. I’ll post all the MS KB articles later. It was a long process and I didn’t get the server back into normal working condition until around noon today. It was working ok when I stopped last night just before midnight, but for whatever reason, the problem re-appeared after a few hours sometime overnight.

To make matters worse, Abby started acting sick yesterday, and woke up last night around 1am with a cough high fever. I heard her crying so I went to get her. Liz and I crashed on the couch because of the time and we like sleeping on the couch sometimes. So when I heard Abby crying I went to get her, only to find her in my bed crying. She had gone in my room to find us and we weren’t there. So she got up in our bed and was crying. I brought her out to the couch with me and Liz and I swapper her around and barely got any sleep. So today, I spent a lot of time working on the goodlink issues, but Liz had to go out to a MOPS picnic event (which we were all going to go to) to spend some time with some orphans. I didn’t get to go, I ended up staying home to work and Abby wasn’t well so I also watched Abby and Michael while Liz took Sarah in the Miata.

We didn’t do much of anything today, and Sarah seems to be the only one with any Energy today. Tomorrow is the last day of the strawberry festival. I love their strawberry shortcake!!! And I would love to come home with a flat of strawberries, but I don’t know for sure yet if I’ll get to go. The plan is to grab some cash from the bank tomorrow and sometime after lunch, I’ll run over to Plant city and get some strawberries, and hopefully find a place that has some strawberry shortcake!

Now I’m off to rest with poor sick little Abby and maybe watch a movie or something while collapsing on the couch!

Surprise Weekend

This is going to be a rather long story. Its a tale of surprise and fun.

It all started on Thursday evening while I was at work. I was working on a server migration project involving Goodlink, when things took a turn for the worse. I was on the phone with Goodlink support almost all day. I was having a terrible time getting things to work properly, and ended up having to stay super late on Thursday, I didn’t leave the office until 11pm. I get in to work around 7:45am or so every weekday, so I was at work from 7:45am until 11m, which makes for a very long day! I finally got things stabalized with the help of Goodlink support and started on my way home. Once I got home, around 11:30pm, I walked in the door, hungry (since I hadn’t eaten anything) and tired (obviously), and so I sat down next to Liz who was sitting on the couch. I leaned over to give her a hello kiss, when I saw something out of the corner of my eye that startled me. It was my borhter Jeremy who’s hand was reaching up towards me in an attempt to surprise me! I jumped up off the couch in surprise and realized it was him. They were totally playing me and had even more surprised in store for me.

We talked for a bit and got a snack and a drink and finally went to bed around 12:30am. The next morning we got up around 8 or so, and I was still in bed when there was a ring at the doorbell. I had a suspicion as to who it might be, but Liz asked me to get the door, so I did and found my mom standing out on the porch. Turned out they had both flown in that Thursday morning and spent the day with Liz and the kids. I had to work late so it totally screwed up their surprise plans and they had to completely change around what they were going to do. So we got to spend the whole weekend (I took Friday off too) with my mom and Jeremy. Click more or the post title to read more about the weekend…
Read the rest of this entry

Recent tech delights

1. Server 2003 clustering with Exchange virtual cluster. All on HP equipment, including MSA1500 SAN, redundant fibre channel switches and DL580 servers. Was able to run several tests and everything worked perfectly.

2. Spam filter changes. Currently I'm using an unlimited license of NoSpamToday for servers from Byteplant.com. Their spam filtering product has been great for my company in general, but spamassassin takes a great deal of tweaking and configuration, so I've been digging through perl type code going through and adding my own filters. I recently added a keyword filter which I generated by hand based on reports from NST on mail that made it through. I ran it all weekend and spam rates are down. I can't wait to see the log tomorrow and see what effect it had on spam rates in general on a normal working day.

3. Exchange 2007. I downloaded a VHD of Microsoft Exchange 2007 to play around with. I like the new interface, but its going to take some getting used to, since they moved everything around. There are also some new things I'm going to have to read up on before actually working with it in a live test. Overall I think its good, and can't wait to test it more.

4. Server 2003 R2. I've been able to setup R2 for Server 2003 and use some of the new file server tools. I love the new reporting, and file server management tools that are now available.

5. This isn't a new thing, but I am really enjoying ActiveSync and DirectPush on my Cingular 8125. Its awesome to have full connectivity to both my company e-mail system through Goodlink and my personal e-mail system at home. I've only had small setup issues in the beginngin with this, but its been working great for me for a few months now. If I ever lose this ability, I'm going to feel like I'm missing my right arm or somethign…

Thats it for now.]]>

Cingular 8125 – and Exchange 2003

WOW, I just got a Cingular 8125 handheld smartphone. I must say that I love it! Its got everything I need to stay “well connected”. I run Exchange 2003 at home for my personal e-mail, so I can sync my new smartphone with my home exchange server with ActiveSync and also get my work email through Goodlink. The goodlink interface on this smartphone is great, its really fast and convenient. Pocket outlook is ok, not as good as goodlink, but very nice none the less. I had some trouble getting active sync setup. I had trouble becuase I had forms based authentication enabled with SSL, which my self signed cert was not trusted by the smartphone by default. I just discovered I could export the cert in DER format and simply open it on my smartphone to import it. This would have cleared up my issue, but I had already purchased a godaddy cert during my troubleshooting process. This is better anyway since now people I host e-mail for in my family won’t get that annoying security prompt now.

Company Migration Project

1. Little to no end user interruption
2. No loss of email connectivity
3. All work must be done by internal IT staff
4. No third party tools can be used

Existing Network:

1 NT4 PDC in NYK and 1 NT4 BDC in Switzerland. Two Exchange 5.5 servers in NYK, one in Switzerland and one in Hong Kong. There are also various software packages that sit on top of the Exchange infrastructure that must remain intact, such as RightFax (email faxing), Goodlink (mobile email) etc. Exchange is installed on member servers, none of the Exchange servers are installed on a PDC/BDC.

Desired result:

2003 Active Directory Domain with Exchange 2003 Enterprise Edition. Exchange servers will be in two-server clusters. 4 hub offices will house the Exchange clusters.

Migration Path:

In order to accomplish the above design goals and satisfy all the requirements, taking into account the existing network infrastructure, the following design should prove to be successful.

First, we will install a new server into our existing NT4 domain and make it a BDC for the domain. We will then, promote the new BDC to a PDC and allow time for replication. Once replication is complete, we will take the old PDC offline (by unplugging network connection or shutting down). We will then upgrade the new NT4 PDC to Server 2003 and run DCPROMO to install Active Directory. This process should preserve all existing user accounts, machine accounts, groups, permissions, etc. So far we satisfy all requirements.

Next, we upgrade or replace all existing NT4 BDCs. Once all NT4 servers are removed, we can then upgrade our new Active directory domain to Server 2003 Native Mode. As no more NT4 servers will be participating as domain controllers.

NOTE: During this time, all existing Exchange 5.5 servers will be maintained as NT4 member servers of the 2003 domain. Exchange 5.5 will continue to handle mail for us until the upgrade to Exchange 2003.

Exchange 2003 Migration:

For this part, we will install our first Exchange 2003 server (on a 2003 Enterprise Edition server installed as a member server) into our existing Exchange 5.5 site. To the exchange site, we are just adding a new server. The Exchange deployment tools will walk us through installing the Active Directory Connector and all necessary connection agreements. The SRS will also be installed. One Exchange 2003 server per hub office will be installed initially. Once we verify that the ADC is working properly and the “Move mailbox” wizard is available, all Exchange 5.5 user mailboxes will be moved to an Exchange 2003 server. Once all Exchange 5.5 mailboxes, public folders, distribution lists, custom forms, etc, are replicated over to the new Exchange 2003 servers, the existing Exchange 5.5 servers will be shut down to verify connectivity and that no “behind the scenes” issues exist. Once our Exchange organization has been verified to be functioning correctly with no further references to the Exchange 5.5 servers, we can then begin to de-commission them. This will be done by removing all references to Exchange 5.5 as replication partners on all public folders, and other exchange resources. Then the servers can be deleted from the Exchange 5.5 server administrator.

During this migration process, the only end user interruption noticed will be during the move mailbox process, as users will be logged out of their exchange mailboxes during a mailbox move. Mail flow is not affected since we installed Exchange 2003 into our existing Exchange 5.5 site, so the routing group is the same. The only remaining task to complete is something I left out above. Before Exchange 5.5 can be shut down or de-commissioned, the SMTP connector will need to be moved from one of the Exchange 5.5 servers to one of the Exchange 2003 servers. Once this is complete, and mail flow has been verified, then the Exchange 5.5 servers can be removed.

The end result is a quick, efficient migration/upgrade to Server 2003 and Exchange 2003. A final note here is on Clustering. The reason we did not use our clusters to install the first Exchange 2003 server is that there are certain components of Exchange that will not function on a cluster. Such as the SRS and ADC. This is why we will be using a standalone server in all hub offices for the initial move to Exchange 2003. Once we have Exchange 2003 up and running globally, we can then introduce our Exchange 2003 Clusters and then move mailboxes once more to the clusters. Once finished, the SMTP (bridgehead) can be moved to the appropriate cluster and the initial Exchange 2003 standalone servers can be removed.

The initial plan was to do a parallel migration, basically creating a whole new system in parallel to our existing system. This plan has many problems and would not have worked for us. End users would have received all new machine profiles, outlook profiles would have been lost, etc. This would have created too much work for our internal IT staff and caused too much interruption to end user connectivity. Not to mention mail flow and interoperability with Exchange 5.5 and Exchange 2003 is much more complicated when installed in separate Exchange Organizations. Third party tools would almost certainly be needed to maintain the level of co-existence we would have needed.
]]>