On Wednesday, Verizon received an errant disconnect order from the ISP, Intermedia. Luckily, the facilities were not reallocated. Verizon got the line back up around 10am on Saturday, about 48 hours after the disconnect. Downtime extended from Thursday 10am to Saturday 10am.

On Saturday morning, a new primary MX record was added to the backup DNS for vees.net and epistolary.org for a machine on a remote network to accept and deliver all mail arriving for either of those domains to a temporary file.
Shortly after that, the T1 connectivity returned and mail was redirected back to the main servers.

This is in line with the disaster recovery plan which switches the mail over after 48 hours and "Temporarily Down" web pages (with an appropriate robots.txt file to prevent poisioning of the search engines) set up at a remote location answering for all epistolary.org and vees.net hosts.

No mail was lost, and all mail was delivered successfully within the standard 5 day time frame. Mail from UMBC was delayed until 72 hours later due to an MTA size issue. This issue was also found and repaired.

In the future, there will be a secondary server with a mirrored config available on a seperate network to keep everything going.

No Comments | #332

Unless noted, all content on epistolary.org is © Copyright 1999-2008 to Rob Carlson with all rights reserved. All information is verified when possible, cited as appropriate and applied in the real world at your own risk. Send all feedback to rob@vees.net.