|

Welcome
WJTech Network Status Report
Be sure to always refresh/reload
your browser when visiting this page.
The purpose of this site is to inform clients of the
West Jersey Technologies Family of Networks of any problems that may
be occurring anywhere on the network. This site is hosted outside of
the WJTech network, so it should be available even if our other services
are down. Bookmark this page so that you may check for updates if you
experience slow service or are unable to connect to your website, or
are unable to send or receive email using the WJTech mail servers.
The email address below will work even if all WJTech mail servers are
down.
Below you will find Server Status and Announcements.

Server Status
Server Name |
Status |
Location |
Scheduled Maintenance |
avenger |
normal |
Burlington, NJ |
None scheduled |
centurion |
normal |
Clifton, NJ |
None scheduled |
colossus |
normal
|
Burlington, NJ |
None scheduled |
defiant |
normal
|
Clifton, NJ |
None scheduled |
excelsior |
normal |
Burlington, NJ |
None scheduled |
guardian |
normal |
Burlington, NJ |
None scheduled |
intrepid |
normal |
Burlington, NJ |
None scheduled |
orion |
normal |
Secaucus, NJ |
None scheduled |
vindicator |
normal |
Clifton, NJ |
None scheduled |
Announcements
Date/Time |
Issue |
Status |
Jan. 22, 2010
9:25 AM |
Please note that we are scheduling a network maintenance on Saturday, January 23rd from 2AM to 4AM to replace one of our core routers at the Fortress ITX Data Center in Clifton, NJ. If the window goes smoothly, there will be no network impact, but there is always a possibility for issues to arise so it would be wise to expect downtime for the duration of the window, just to be safe. |
Completed |
Jan. 7, 2010
7:25 AM |
Update: Jan 15, 2010: From the Fortress ITX Data Center in Clifton, NJ:
Yesterday morning [Jan 7] at approximately 9AM EST, we experienced a failure in one of our core routers which caused some network outages for approximately 30 minutes. Initially, we found a faulty supervisor card in the router. The card was immediately replaced, and the router was restored back onto the network.
Shortly after returning traffic to the router, the newly replaced supervisor card failed. During this second failure, traffic that was passing over that router was essentially black holed and not delivering properly via BGP sessions over the other routers as it should have been. In other words, failover routers were not seeing this traffic as a true failure and thus did not shift traffic properly as it should have.
We have since removed that router from our network and will be replacing it entirely as the issue is more of an upstream problem of the supervisor card on the router itself. We apologize for any inconvenience that this outage may have caused to you, and we will update you on the replacement of the router when it will be deployed. We thank you for your patience and understanding in this matter and appreciate your continued support and patronage.
Update:11:30 AM: Network stable. We are waiting for a report from the data center about what happened.
Update:10:20 AM: Connections are becoming more stable. We will continue to monitor. More to follow.
Update:9:40 AM: Intermittent connection outages continue. Servers remain stable. More to follow.
Update:7:52 AM: Outages started again. More to follow.
Update:7:40 AM: All connections reestablished. Appears to have been a problem with an incoming router at the data center. All servers were up during the outages.
Intermittent outages of connections to servers at the Fortress ITX Data Center in Clifton, NJ. This started at about 6:30 AM. More to follow |
Resolved |
Oct. 22, 2009
10:00 PM |
Update: Oct. 22, 10:45 AM: Server has been re-booted. Additional re-boots may be necessary later today. Please check this page before beginning extensive editing of your site. We apologize for any inconvenience this may cause, and thank
you for your patience.
There will be an unscheduled reboot of server defiant at 10:40 AM. |
Completed |
Oct. 12, 2009
12:05 PM |
A fake email is being sent out purporting to be from the system administrator of wjtech.net. It states that there will be a server upgrade on October 16. It tell you to download a patch and run it on your local computer. DO NOT DO THIS. The patch probably is a virus that will compromise you computer.
If you have any questions, do not hesitate to get in touch with us at the email address or phone number listed at the top of this page. |
Completed |
Sept. 14, 2009
3:20 PM |
Update: 4:50 PM: We were experiencing some unusual activity on server centurion. After reboot, server is up and running smoothly. We will continue to monitor.
There will be an unscheduled reboot of server centurion at 3:30 PM. |
Completed |
Sept. 4, 2009
6:00 PM |
Update: Oct. 12, 12:05 PM: Only a few site still to be moved.
Update: Sept. 14, 4:50 PM: Still moving sites.
Beginning today, Friday September 4, 2009, and continuing over the weekend, we will be moving all sites on server centurion to a new server. The new server (named defiant) will have a different control panel for managing security, email, and websites.
One of the biggest changes will be how email is handled. You will be receiving instructions on how to access you email shortly. This email will be sent from the old server, so you will be able to receive it just as you always have. A link to these instructions will also be posted here. Note: Your email address WILL NOT change.
Your actual site will be visible on the internet during the transition, but once switched to the new server, some functions may not be available for a short while.
If you must make critical updates to your site right now, let me know and I will hold off until they are complete.
If you have any questions, do not hesitate to ask. |
In Progress |
June 11, 2009
6:45 PM |
Update: 7:20 PM: Server centurion is back on line. We are running various test on the server. If additional reboots are necessary, they will take place between 5 AM and 8 AM tomorrow (June 12) morning.
Unscheduled reboot of server to take place at 7:15 PM |
Completed |
Nov 3 , 2008
4:00 AM |
Update: 8:50 AM: Problem tracked
to faulty DNS (nameserver) information. Info corrected. You may
still experience connection problems over the next several hours
until the corrected info makes it way around the internet.
Update: 8:00 AM: Still experiencing
intermittent connections problems, although all servers seem
to be running normally. Still under investigation.
Outside connections with the Fortress ITX Data Center
in Clifton, NJ are down going up and down. |
Resolved |
Oct. 9, 2008
7:20 AM |
There will be a network maintenance window from
12:00 Midnight to 4:00 AM Sunday, October 12th. The window
is to upgrade one of our distribution switches.
We are allowing four hours for the maintenance, but the actual
downtime should be much less if the upgrade goes smoothly. |
Completed |
Sept 25, 2008
9:15 AM |
There will be a network maintenance window from
12:00 to 1:00 AM Sunday, September 28th. The window is to
upgrade the supervisor card in one of our distribution switches.
We are allowing one hour for the maintenance, but the actual downtime
should be much less if the upgrade goes smoothly. |
Completed |
June 24, 2008
8:15 AM |
Update: June 24, 1:00 PM: Full
connections restored.
Outside connections with the
Fortress ITX Data Center in Clifton, NJ are degraded. Once
again, the problem seems to be related to Comcast lines between
North and South Jersey. Monitoring service from 14 different connections
world-wide showing no problems. |
Resolved |
June 10, 2008
10:45 PM |
Update: June 11, 6:30 AM: Full
connections restored. All Fortress servers remained fully functional
all night, but problems with Comcast lines between south and
north NJ caused connection problems. The routine maintenance
started yesterday has been postponed until we are sure all connections
are once again rock solid.
Some (but not all) outside connections with the
Fortress ITX Data Center in Clifton, NJ have been lost. Problem
seems to be related to Comcast. Monitoring service from 14 different
connections world-wide showing no problems. |
Resolved |
June 10, 2008
5:55 AM |
Update: June 10, 7:00 AM: Completed
for today. Will resume tomorrow from 5 to 7 AM
Routine maintenance on server centurion will require
several reboots of the server. Baring complications, downtime will
be no more than 2 minutes for each reboot, but could be as long
as 15 minutes. All reboots should be completed by 7:00 AM |
Completed |
May 12, 2008
10:45 AM |
Update: May 19, 7:50 AM: Remainder
of upgrade is on hold while we work out some problems on a test
server.
Update: May 15, 7:22 AM: Server
back up. All services available.
Update: May 15, 7:18 AM: Server
down. Will be back up shortly.
Update: May 15, 6:35 AM: There
will be a brief outage sometime between 6:50 and 7:30 this morning.
Update: May 14, 7:20 AM: We
are still researching upgrade issues, so system changes will
not be made until Thursday morning.
Update: May 13, 8:00 AM: Because
of problems encounter during this mornings upgrades, we will
be working on the server again tomorrow morning between 5 and
8 AM. There should be no additional interruption in service for
the rest of today.
Update: May 13, 7:05 AM: Server
back up. All services available.
Update: May 13, 6:50 AM: Server down. Should
be back up shortly.
Update: May 13, 6:30 AM: Major updating has started.
Server centurion will undergo major upgrades/updates
today and Tuesday morning. You may experience brief interruptions
in service today. The main changes will take place tomorrow morning
between 5 and 8 AM. Outages of 15 minutes or more may occur. Particularly
susceptible to outage will be the Horde webmail service.
Please
check back here for all status updates.
We apologize for any inconvenience this may cause, and thank
you for your patience. |
Pending |
May 1, 2008
12:10 AM |
Update May 11, 9:00 AM: Permanent
solution in place.
Update May 5, 10:00 AM: Temporary work-around
in place. You may edit sites.
Update May 3, 11:20 AM: Unable
to edit sites. Still working to solve this problem.
Experiencing problems with some Joomla sites. |
Resolved |
Mar. 23, 2008
5:45 PM |
Update May 12, 10:30 AM: The cause of this
lock-up has not been found, however, the problem has not reoccurred.
We will continue to monitor the server.
Apache server for centurion
locked up at about 5 PM. Service was restored at 5:40 PM. Mail
service was not affected by the web server outage. The same thing
happened exactly two weeks ago (Sunday at about 5 PM). We are looking
at our schedulers for something that is triggering the problem.
|
Server Being Monitored |
Mar. 12, 2008
8:40 AM |
Yesterday afternoon's disruption in network services was caused
by a BGP (Border Gateway Protocol) blip on one of our core routers.
Upon reset of the BGP session, normal service was restored. We apologize for any inconvenience this may cause, and thank
you for your patience. |
Resolved |
Mar. 9, 2008
6:10 PM |
Apache server for
centurion locked up at about 5 PM. Service was restored at 6:05
PM. Reason for lock-up is under investigation. Mail service was
not affected by the web server outage. |
See Update Above |
Mar. 5, 2008
3:20 PM |
Update 6:00 PM: problem was between south-central
NJ and North Jersey/New York. No problems at data center.
Update 4:08 PM: Problem seems to be related
to Comcast. Dial-up connection from Burlington NJ working. Monitoring
service from 14 different connections world-wide showing no problems.
Some (but not all) outside connections with the Fortress ITX
Data Center in Clifton, NJ have been lost.
|
Resolved |
Mar. 5, 2008
7:15 AM |
We will be performing network maintenance during the following
times this weekend:
Maintenance 1 = 10 PM to 3 AM March 7th into March 8th 2008
Maintenance 2 = 10 PM to 3 AM March 8th into March 9th 2008
During these times, there will be periods of connectivity loss,
and while we will do everything possible to minimize the downtime,
you should expect up to 45 minutes of downtime. Hopefully this
downtime will be much less, but we are blocking the window periods
during these maintenance periods to the hours above to be safe.
Geek info: We will be upgrading equipment and routes while implementing
major adjustments on our Distribution switches, Core/Edge routers
and Layer2 connectivity in our New York City POP.
We apologize for any inconvenience this may cause, and thank
you for your patience.
|
Pending |
Feb. 25, 2008
5:30 AM |
Update at 6:45: Maintenance complete. There
will be additional maintenance periods this week, but they will
all take place between 5 and 7 AM.
Server undergoing maintenance. May be down for brief periods
between 5: 45 and 7 AM
|
Completed |
Feb. 24, 2008
8:30 AM |
Update at 10:20 AM: Because of some unexpected
problems with this mornings maintenance, including being down
for about 15 minutes, we are postponing the remainder of the
upgrades until early tomorrow morning. The procedures will begin
at about 5 AM and should be completed before 7 AM.
We apologize for any inconvenience today's brief outage may
have caused.
Update at 9:42 AM: Server is back on line.
More info to follow.
Update at 9:30 AM: Did not recover from a reboot.
Server down. Should be up in a few minutes.
Routine maintenance on server centurion will require several
reboots of the server. Baring complications, downtime will be
no more than 2 minutes for each reboot.
Domain Administrators: Before beginning extensive changes to
your website, check here for estimated boot times.
Next reboot: between 9 and 9:30 AM
|
Completed |
Feb. 4, 2008
6:30 PM |
Migration of accounts
from defiant to centurion is complete. Server defiant will be
taken off line for a complete operating system and software upgrade. |
Completed |
Jan. 30, 2008
7:20 AM |
Migration of accounts from defiant to centurion continues.
Note: It appears that most of the problems with duplicate (and
multiple) copies of an email being delivered occur in MS Outlook.
Please be sure your version of Outlook is up to date.
|
Completed |
Jan. 24, 2008
7:15 PM |
Migration of accounts from defiant to centurion continues.
Note: because of the higher security standards used on server
centurion, you must log into your email account using both user
name and domain: e.g., username@yourdomain.com instead of just
username. This restriction will go into effect on Saturday, January
26. Be sure to update you mail program (Outlook Express, Thunderbird,
etc.)
Note: The higher security standards may require you to change
your password to one that is more secure then the one you are
now using. If your password no longer works, contact you domain
administrator, and he/she will reset it for you.
|
Completed |
Jan. 22, 2008
10:15 AM |
Moving control
of networked servers to centurion. You may experience brief email
delays both sending and receiving. If this occurs over a period
of more than 10 minutes, please contact support using the above
address and a non-wjtech email account (yahoo, hotmail, etc.) |
Completed |
|
Update: Jan. 20, 10:31 AM - Problem solved.
Problems connecting to server centurion.
|
Resolved |
|
Update: Jan. 20, 5:50 AM - Accounts being moved
to new server.
Update: Jan. 16, 9:50 PM - We have reinstalled
all of the Spam Guardian/Spamassassin software which we think
was partially responsible for multiple copies of the same email
being received by some clients. We will continue to monitor the
situation.
Update: Jan. 16, 12 noon - Problem persists.
Work continues.
Update: Jan. 16, 6:45 AM - We are working
on a possible solution involving the spam protection program.
Email service may be interrupted briefly over the next several
hours. If email fails, please try again in a few minutes. Spam
protection will also be turned off for a few minutes.
We have received reports of duplicate and even multiple emails,
as well as emails bounced back to the sender with "failed" messages.
We are going to try to repair the email server while it remains
live, but in all likelihood, we will have to take it down for
up to 2 hours while we do a complete rebuild. If we do
take it down, it will be between 5 and 7 AM EST Wednesday or
Thursday morning. More to follow.
|
Completed |
|
Update: 10:45 AM - Only server defiant will
be moved today. The move should take place between 2 PM and 5
PM, but may be as late as 8 PM.
On Tuesday, Nov. 20, between the hours of 10 AM EST and 5 PM,
servers centurion and/or defiant at the Fortress ITX Data Center
in Clifton, NJ will be moved to new racks. The new racks are
equipped to better allow WJTech to manage the servers. Downtime
should be no more than 10 minutes. As soon as we have a more
exact time for the move of your individual server, we will email
you.
We apologize for the short notice and any inconvenience this
may cause.
|
Completed |
Nov. 15, 2007
12:55 PM |
Servers vindicator, defiant, and centurion all off line for
switch maintenance and reboot.
Update: Although down longer then expected,
all servers are back on line. Total time ~30 minutes.
|
Completed |
|
Over the course
of the next 3 days, beginning Wednesday November 14th through
Friday November 16th, we will be performing routine maintenance
on all of our network switches. During this time we will need
to reboot each switch individually, causing a period of downtime
of approximately 2-3 minutes for any server attached to that
particular switch. However, in the event of any issues arising,
services may be unreachable for a period of up to 10-15 minutes.
We will be rebooting the switches during the hours of 10AM EST
until 12 Midnight. Unfortunately, due to the large number of
switches we must upgrade and reboot, no exact timeframe can be
given for these reboots. Network administrators will be standing
by and monitoring each individual switch as it is rebooted to
rectify any problems immediately. |
Completed |
|
On Tuesday, Nov. 6, between the hours of 12 Noon EST and 2 PM,
the server vindicator at the Fortress ITX Data Center in Clifton,
NJ will be moved to new a rack. The new rack is equipped to better
allow WJTech to manage the server. Downtime should be no more
than 10 minutes.
We apologize for any inconvenience this may cause, and thank
you for your patience.
|
Completed |
Nov. 4, 2007
6:20 AM |
On Tuesday, Nov. 6, between the hours of 10 AM EST and 5 PM,
servers at the Fortress ITX Data Center in Clifton, NJ will be
moved to new racks. The new racks are equipped to better allow
WJTech to manage the servers. Downtime should be no more than
10 minutes. As soon as we have a more exact time for the move
of your individual server, we will email you.
We apologize for any inconvenience this may cause, and thank
you for your patience.
|
See Update Above |
Oct. 26, 2007
9:50 AM |
Everything running
smoothly now. Server defiant never went down and operated the
whole time under relatively light load. Problem may have been
with the Comcast network running from the south toward North
New Jersey. |
Resolved |
Oct. 26, 2007
8:55 AM |
Experiencing intermittent
networking problems on connection to server defiant. |
See Update Above |
Remember, The Future
is today
Copyright ©2004-2009
West Jersey Technologies
|
|