WJTech Logo

Welcome

WJTech Network Status Report

Be sure to always refresh/reload your browser when visiting this page.

The purpose of this site is to inform clients of the West Jersey Technologies Family of Networks of any problems that may be occurring anywhere on the network. This site is hosted outside of the WJTech network, so it should be available even if our other services are down. Bookmark this page so that you may check for updates if you experience slow service or are unable to connect to your website, or are unable to send or receive email using the WJTech mail servers. The email address below will work even if all WJTech mail servers are down.

Below you will find Server Status and Announcements.

WJTech email and phone

Server Status

Server Name
Status
Location
Scheduled Maintenance
avenger
normal
Burlington, NJ
None scheduled
centurion
normal
Clifton, NJ
None scheduled
colossus
normal
Burlington, NJ
None scheduled
defiant
normal
Clifton, NJ
None scheduled
excelsior
normal
Burlington, NJ
None scheduled
guardian
normal
Burlington, NJ
None scheduled
intrepid
normal
Burlington, NJ
None scheduled
orion
normal
Secaucus, NJ
None scheduled
vindicator
normal
Clifton, NJ
None scheduled

 

Announcements

Date/Time
Issue
Status
Jan. 22, 2010
9:25 AM
Please note that we are scheduling a network maintenance on Saturday, January 23rd from 2AM to 4AM to replace one of our core routers at the Fortress ITX Data Center in Clifton, NJ. If the window goes smoothly, there will be no network impact, but there is always a possibility for issues to arise so it would be wise to expect downtime for the duration of the window, just to be safe.
Completed
Jan. 7, 2010
7:25 AM

Update: Jan 15, 2010: From the Fortress ITX Data Center in Clifton, NJ:

Yesterday morning [Jan 7] at approximately 9AM EST, we experienced a failure in one of our core routers which caused some network outages for approximately 30 minutes. Initially, we found a faulty supervisor card in the router. The card was immediately replaced, and the router was restored back onto the network.

Shortly after returning traffic to the router, the newly replaced supervisor card failed. During this second failure, traffic that was passing over that router was essentially black holed and not delivering properly via BGP sessions over the other routers as it should have been. In other words, failover routers were not seeing this traffic as a true failure and thus did not shift traffic properly as it should have.

We have since removed that router from our network and will be replacing it entirely as the issue is more of an upstream problem of the supervisor card on the router itself. We apologize for any inconvenience that this outage may have caused to you, and we will update you on the replacement of the router when it will be deployed. We thank you for your patience and understanding in this matter and appreciate your continued support and patronage.

Update:11:30 AM: Network stable. We are waiting for a report from the data center about what happened.

Update:10:20 AM: Connections are becoming more stable. We will continue to monitor. More to follow.

Update:9:40 AM: Intermittent connection outages continue. Servers remain stable. More to follow.

Update:7:52 AM: Outages started again. More to follow.

Update:7:40 AM: All connections reestablished. Appears to have been a problem with an incoming router at the data center. All servers were up during the outages.

Intermittent outages of connections to servers at the Fortress ITX Data Center in Clifton, NJ. This started at about 6:30 AM. More to follow

Resolved
Oct. 22, 2009
10:00 PM

Update: Oct. 22, 10:45 AM: Server has been re-booted. Additional re-boots may be necessary later today. Please check this page before beginning extensive editing of your site. We apologize for any inconvenience this may cause, and thank you for your patience.

There will be an unscheduled reboot of server defiant at 10:40 AM.

Completed
Oct. 12, 2009
12:05 PM

A fake email is being sent out purporting to be from the system administrator of wjtech.net. It states that there will be a server upgrade on October 16. It tell you to download a patch and run it on your local computer. DO NOT DO THIS. The patch probably is a virus that will compromise you computer.

If you have any questions, do not hesitate to get in touch with us at the email address or phone number listed at the top of this page.

Completed
Sept. 14, 2009
3:20 PM

Update: 4:50 PM: We were experiencing some unusual activity on server centurion. After reboot, server is up and running smoothly. We will continue to monitor.

There will be an unscheduled reboot of server centurion at 3:30 PM.

Completed
Sept. 4, 2009
6:00 PM

Update: Oct. 12, 12:05 PM: Only a few site still to be moved.

Update: Sept. 14, 4:50 PM: Still moving sites.

Beginning today, Friday September 4, 2009, and continuing over the weekend, we will be moving all sites on server centurion to a new server. The new server (named defiant) will have a different control panel for managing security, email, and websites.

One of the biggest changes will be how email is handled. You will be receiving instructions on how to access you email shortly. This email will be sent from the old server, so you will be able to receive it just as you always have. A link to these instructions will also be posted here. Note: Your email address WILL NOT change.  

Your actual site will be visible on the internet during the transition, but once switched to the new server, some functions may not be available for a short while.

If you must make critical updates to your site right now, let me know and I will hold off until they are complete.

If you have any questions, do not hesitate to ask.

In Progress
June 11, 2009
6:45 PM

Update: 7:20 PM: Server centurion is back on line. We are running various test on the server. If additional reboots are necessary, they will take place between 5 AM and 8 AM tomorrow (June 12) morning.

Unscheduled reboot of server to take place at 7:15 PM

Completed
Nov 3 , 2008
4:00 AM

Update: 8:50 AM: Problem tracked to faulty DNS (nameserver) information. Info corrected. You may still experience connection problems over the next several hours until the corrected info makes it way around the internet.

Update: 8:00 AM: Still experiencing intermittent connections problems, although all servers seem to be running normally. Still under investigation.

Outside connections with the Fortress ITX Data Center in Clifton, NJ are down going up and down.

Resolved
Oct. 9, 2008
7:20 AM
There will be a network maintenance window from 12:00 Midnight to 4:00 AM Sunday, October 12th.  The window is to upgrade one of our distribution switches. We are allowing four hours for the maintenance, but the actual downtime should be much less if the upgrade goes smoothly.
Completed
Sept 25, 2008
9:15 AM
There will be a network maintenance window from 12:00 to 1:00 AM Sunday, September 28th.  The window is to upgrade the supervisor card in one of our distribution switches. We are allowing one hour for the maintenance, but the actual downtime should be much less if the upgrade goes smoothly.
Completed
June 24, 2008
8:15 AM

Update: June 24, 1:00 PM: Full connections restored.

Outside connections with the Fortress ITX Data Center in Clifton, NJ are degraded. Once again, the problem seems to be related to Comcast lines between North and South Jersey. Monitoring service from 14 different connections world-wide showing no problems.

Resolved
June 10, 2008
10:45 PM

Update: June 11, 6:30 AM: Full connections restored. All Fortress servers remained fully functional all night, but problems with Comcast lines between south and north NJ caused connection problems. The routine maintenance started yesterday has been postponed until we are sure all connections are once again rock solid.

Some (but not all) outside connections with the Fortress ITX Data Center in Clifton, NJ have been lost. Problem seems to be related to Comcast. Monitoring service from 14 different connections world-wide showing no problems.

Resolved
June 10, 2008
5:55 AM

Update: June 10, 7:00 AM: Completed for today. Will resume tomorrow from 5 to 7 AM

Routine maintenance on server centurion will require several reboots of the server. Baring complications, downtime will be no more than 2 minutes for each reboot, but could be as long as 15 minutes. All reboots should be completed by 7:00 AM

Completed
May 12, 2008
10:45 AM

Update: May 19, 7:50 AM: Remainder of upgrade is on hold while we work out some problems on a test server.

Update: May 15, 7:22 AM: Server back up. All services available.

Update: May 15, 7:18 AM: Server down. Will be back up shortly.

Update: May 15, 6:35 AM: There will be a brief outage sometime between 6:50 and 7:30 this morning.

Update: May 14, 7:20 AM: We are still researching upgrade issues, so system changes will not be made until Thursday morning.

Update: May 13, 8:00 AM: Because of problems encounter during this mornings upgrades, we will be working on the server again tomorrow morning between 5 and 8 AM. There should be no additional interruption in service for the rest of today.

Update: May 13, 7:05 AM: Server back up. All services available.

Update: May 13, 6:50 AM: Server down. Should be back up shortly.

Update: May 13, 6:30 AM: Major updating has started.

Server centurion will undergo major upgrades/updates today and Tuesday morning. You may experience brief interruptions in service today. The main changes will take place tomorrow morning between 5 and 8 AM. Outages of 15 minutes or more may occur. Particularly susceptible to outage will be the Horde webmail service.

Please check back here for all status updates.

We apologize for any inconvenience this may cause, and thank you for your patience.

Pending
May 1, 2008
12:10 AM

Update May 11, 9:00 AM: Permanent solution in place.

Update May 5, 10:00 AM: Temporary work-around in place. You may edit sites.

Update May 3, 11:20 AM: Unable to edit sites. Still working to solve this problem.

Experiencing problems with some Joomla sites.

Resolved
Mar. 23, 2008
5:45 PM

Update May 12, 10:30 AM: The cause of this lock-up has not been found, however, the problem has not reoccurred. We will continue to monitor the server.

Apache server for centurion locked up at about 5 PM. Service was restored at 5:40 PM. Mail service was not affected by the web server outage. The same thing happened exactly two weeks ago (Sunday at about 5 PM). We are looking at our schedulers for something that is triggering the problem.

Server Being Monitored
Mar. 12, 2008
8:40 AM

Yesterday afternoon's disruption in network services was caused by a BGP (Border Gateway Protocol) blip on one of our core routers. Upon reset of the BGP session, normal service was restored.

We apologize for any inconvenience this may cause, and thank you for your patience.

Resolved
Mar. 9, 2008
6:10 PM
Apache server for centurion locked up at about 5 PM. Service was restored at 6:05 PM. Reason for lock-up is under investigation. Mail service was not affected by the web server outage.
See Update Above
Mar. 5, 2008
3:20 PM

Update 6:00 PM: problem was between south-central NJ and North Jersey/New York. No problems at data center.

Update 4:08 PM: Problem seems to be related to Comcast. Dial-up connection from Burlington NJ working. Monitoring service from 14 different connections world-wide showing no problems.

Some (but not all) outside connections with the Fortress ITX Data Center in Clifton, NJ have been lost.

Resolved
Mar. 5, 2008
7:15 AM

We will be performing network maintenance during the following times this weekend:

Maintenance 1 = 10 PM to 3 AM March 7th into March 8th 2008

Maintenance 2 = 10 PM to 3 AM March 8th into March 9th 2008

During these times, there will be periods of connectivity loss, and while we will do everything possible to minimize the downtime, you should expect up to 45 minutes of downtime. Hopefully this downtime will be much less, but we are blocking the window periods during these maintenance periods to the hours above to be safe.

Geek info: We will be upgrading equipment and routes while implementing major adjustments on our Distribution switches, Core/Edge routers and Layer2 connectivity in our New York City POP.

We apologize for any inconvenience this may cause, and thank you for your patience.

Pending
Feb. 25, 2008
5:30 AM

Update at 6:45: Maintenance complete. There will be additional maintenance periods this week, but they will all take place between 5 and 7 AM.

Server undergoing maintenance. May be down for brief periods between 5: 45 and 7 AM

Completed
Feb. 24, 2008
8:30 AM

Update at 10:20 AM: Because of some unexpected problems with this mornings maintenance, including being down for about 15 minutes, we are postponing the remainder of the upgrades until early tomorrow morning. The procedures will begin at about 5 AM and should be completed before 7 AM.

We apologize for any inconvenience today's brief outage may have caused.

Update at 9:42 AM: Server is back on line. More info to follow.

Update at 9:30 AM: Did not recover from a reboot. Server down. Should be up in a few minutes.

Routine maintenance on server centurion will require several reboots of the server. Baring complications, downtime will be no more than 2 minutes for each reboot.
Domain Administrators: Before beginning extensive changes to your website, check here for estimated boot times.

Next reboot: between 9 and 9:30 AM

Completed
Feb. 4, 2008
6:30 PM
Migration of accounts from defiant to centurion is complete. Server defiant will be taken off line for a complete operating system and software upgrade.
Completed
Jan. 30, 2008
7:20 AM

Migration of accounts from defiant to centurion continues.

Note: It appears that most of the problems with duplicate (and multiple) copies of an email being delivered occur in MS Outlook. Please be sure your version of Outlook is up to date.

Completed
Jan. 24, 2008
7:15 PM

Migration of accounts from defiant to centurion continues.

Note: because of the higher security standards used on server centurion, you must log into your email account using both user name and domain: e.g., username@yourdomain.com instead of just username. This restriction will go into effect on Saturday, January 26. Be sure to update you mail program (Outlook Express, Thunderbird, etc.)

Note: The higher security standards may require you to change your password to one that is more secure then the one you are now using. If your password no longer works, contact you domain administrator, and he/she will reset it for you.

Completed
Jan. 22, 2008
10:15 AM
Moving control of networked servers to centurion. You may experience brief email delays both sending and receiving. If this occurs over a period of more than 10 minutes, please contact support using the above address and a non-wjtech email account (yahoo, hotmail, etc.)
Completed
Jan. 20, 2008
10:24 AM

Update: Jan. 20, 10:31 AM - Problem solved.

Problems connecting to server centurion.

Resolved
Jan. 15, 2008
10:40 AM

Update: Jan. 20, 5:50 AM - Accounts being moved to new server.

Update: Jan. 16, 9:50 PM - We have reinstalled all of the Spam Guardian/Spamassassin software which we think was partially responsible for multiple copies of the same email being received by some clients. We will continue to monitor the situation.

Update: Jan. 16, 12 noon - Problem persists. Work continues.

Update: Jan. 16, 6:45 AM - We are working on a possible solution involving the spam protection program. Email service may be interrupted briefly over the next several hours. If email fails, please try again in a few minutes. Spam protection will also be turned off for a few minutes.

We have received reports of duplicate and even multiple emails, as well as emails bounced back to the sender with "failed" messages. We are going to try to repair the email server while it remains live, but in all likelihood, we will have to take it down for up to 2 hours while we do a complete rebuild.  If we do take it down, it will be between 5 and 7 AM EST Wednesday or Thursday morning. More to follow.

Completed
Nov. 20, 2007
6:20 AM

Update: 10:45 AM - Only server defiant will be moved today. The move should take place between 2 PM and 5 PM, but may be as late as 8 PM.

On Tuesday, Nov. 20, between the hours of 10 AM EST and 5 PM, servers centurion and/or defiant at the Fortress ITX Data Center in Clifton, NJ will be moved to new racks. The new racks are equipped to better allow WJTech to manage the servers. Downtime should be no more than 10 minutes. As soon as we have a more exact time for the move of your individual server, we will email you.

We apologize for the short notice and any inconvenience this may cause.

Completed
Nov. 15, 2007
12:55 PM

Servers vindicator, defiant, and centurion all off line for switch maintenance and reboot.

Update: Although down longer then expected, all servers are back on line. Total time ~30 minutes.

Completed
Nov. 13, 2007
1:15 PM
Over the course of the next 3 days, beginning Wednesday November 14th through Friday November 16th, we will be performing routine maintenance on all of our network switches. During this time we will need to reboot each switch individually, causing a period of downtime of approximately 2-3 minutes for any server attached to that particular switch. However, in the event of any issues arising, services may be unreachable for a period of up to 10-15 minutes. We will be rebooting the switches during the hours of 10AM EST until 12 Midnight. Unfortunately, due to the large number of switches we must upgrade and reboot, no exact timeframe can be given for these reboots. Network administrators will be standing by and monitoring each individual switch as it is rebooted to rectify any problems immediately.
Completed
Nov. 6, 2007
9:50 AM

On Tuesday, Nov. 6, between the hours of 12 Noon EST and 2 PM, the server vindicator at the Fortress ITX Data Center in Clifton, NJ will be moved to new a rack. The new rack is equipped to better allow WJTech to manage the server. Downtime should be no more than 10 minutes.

We apologize for any inconvenience this may cause, and thank you for your patience.

Completed
Nov. 4, 2007
6:20 AM

On Tuesday, Nov. 6, between the hours of 10 AM EST and 5 PM, servers at the Fortress ITX Data Center in Clifton, NJ will be moved to new racks. The new racks are equipped to better allow WJTech to manage the servers. Downtime should be no more than 10 minutes. As soon as we have a more exact time for the move of your individual server, we will email you.

We apologize for any inconvenience this may cause, and thank you for your patience.

See Update Above
Oct. 26, 2007
9:50 AM
Everything running smoothly now. Server defiant never went down and operated the whole time under relatively light load. Problem may have been with the Comcast network running from the south toward North New Jersey.
Resolved
Oct. 26, 2007
8:55 AM
Experiencing intermittent networking problems on connection to server defiant.
See Update Above

 

Remember, The Future is today

 

Copyright ©2004-2009 West Jersey Technologies