Study of a Local Area Network
The City of
To: City of Fort Pierce
From: Megabyte Concepts
Subject: WAN/LAN Analysis
Date: April
14, 2005
Introduction
The City of
type one shielded cabling with IBM IDC connectors attached to loop wire concentrators and related MAU’s creating a LAN at each of the six sites. Each site was connected through an IBM 5394 controller, an Addtron CSU/DSU, and a Mitel digital modem over a data T-1 frame relay circuit or WAN connection. At this time the City had a sequential or ring logical topology and a bus physical topology.
Logical Network Overview
The City of Fort Pierce’s current computer infrastructure consists of 6 sites or local area networks (LAN’s) containing anywhere from 4 to 100 plus nodes per site that are all connected via a private statewide frame relay network. The physical sites are as follows: the main and largest site is at City Hall with over 100 nodes, the next largest is the Public Works compound that has about 24 nodes, Solid Waste which is separate and within the compound has about 10 nodes.
The Fort Pierce Police Station has an entirely separate network and administration consisting of 400 plus nodes; however, this portion of the network connectivity discussed in this paper for security related purposes contains 5 nodes not related to the other 400 plus nodes, The Indian Hills Golf Course contains about 10 nodes, and The City of Fort Pierce Marina contains 5 nodes. Each of the six sites consists of a broadcast logical topology and a star physical topology via Ethernet LAN technology. At each site, originally there were Intel 24port 10/100 intelligent Hubs that were later upgraded to 3COM 3300 24 port 10/100 switches. City Hall consists of three floors that house the departments of Finance/MIS, City Clerk, City Attorney, City Manager, Planning and Zoning, Community Response, & Code Enforcement, Purchasing, Human Resources and Engineering. This site contains 8 3COM 3300 24 port 10/100 switches combined in 2 stacks with a 1 gigabit backbone via two management consoles for each stack of four switches. The two stacks of 3COM switches are connected together via 4 dedicated 100 megabit full duplex ports obtaining an 800 megabit throughput to almost match the 1 gigabit backbone within each stack of four switches. The entire building is wired via category 5E non-plenum cabling with no wireless technology throughout the building, the Category 5E wiring channels into category 5E patch panels and then into the 3COM 3300 10/100 switches. This physical type of Ethernet LAN technology was chosen during Y2K due the ease of installation, relative low cost, relative speed benefits attained over copper wiring and other related computer services that could be rolled out over this physical computing technology.
From the 3COM switches, each site
or LAN connects to an internally managed Cisco Router, a Sonicwall Firewall,
and then into the State of
WAN Connectivity and Hardware
As mentioned, the City utilizes and
manages Cisco routers internally for network address translation (NAT),
transport layer firewall techniques, and for other purposes related to internal
security, control, and network connectivity.
The City of
Incidentally, the State of Florida, Sprint, and Bell South
are all involved in this statewide WAN to provide private network WAN services,
24 hours/7 days a week network management, monitoring, and technical support
down to the router provided to the customer for all municipalities throughout
the state of Florida. The technical
support staff that manages the network for
LAN Server Hardware and Software – 5 Remote Sites
Each of the five remote sites has an IBM X205 or X206 dedicated file server with a Pentium 4, 2 gigahertz up to a 3.2 gigahertz processor with 512 megabytes up to 1 gigabyte of memory per server. Therefore, each remote site which consists of the City Marina, Golf Course, Police Station, Public Works, and Solid Waste has a dedicated file server network. Each server utilizes SCSI hardware for its tape drive and a single hard drive per server.
Raid 5 technology is not used at these remote site IBM servers and therein necessitates the need for local LAN tape backups. Each hard drive has 40 to 80 gigabytes of SCSI hard disk capacity along with SCSI 4mm DAT tape drive technology that matches the SCSI hard drive capacity at each server. The IBM servers at each site all use Windows 2003 Server – Standard Edition for their respective network operating systems. With Windows 2003 Server installed, each IBM server becomes a domain controller for each site, each server also utilizes Microsoft’s Software Update Services server or SUS server to download Microsoft Windows operating system patches at a single location per site and then the SUS server distributes and installs the windows patches to all workstations across the respective local area network. Each IBM server at each site also utilizes E-policy Orchestrator from Network Associates to role out and continually update the Mcafee desktop firewall and Mcafee antivirus programs, engines, and antivirus database files to continually secure the local area network infrastructure. Each file server connects to the local area network via an integrated IBM 10/100/1000 network adapter (NIC) or 3COM PCI 10/100 NIC which directly connects to the 3COM 3300 24 port 10/100 switch at each site via a category 5E twisted pair cable and RJ-45 interfaces. Again, from the 3COM switches the connectivity continues to the internal Cisco 26XX services router, to a Sonicwall Soho 3 or Pro 330 firewall, to the external state provided Cisco 17XX or 26XX series router or gateway and out over the state’s frame relay network to the main site’s gateway or Cisco 26XX series router, down to the Sonicwall Pro 330 Firewall into another 26XX series router and into the two stacks of 3COM 3300 10/100 switches at City Hall or to another remote site, or to the internet, etc.
At each of the 5 remote sites or
LAN’s, the current physical limitations are related to the maximum category 5E
cable length of 328 feet before another hub, switch, or repeater is
required. Some remote sites contain only
one 3COM 3300 24 port 10/100 switch, therefore; some sites can currently only
support 24 nodes, unless more switches are added, etc. At the
Indian Hills Golf Course, the category 5E limits have been surpassed in this
LAN by a single Category 5E cable that is 398 feet long ran underground in
conduit between two buildings, the cable fails the category 5E tests, but the
cable runs at 100 megabits in full duplex mode between two switches. So, in this case the transmission limitations
are being stretched to meet some temporary constructions needs at this
site. Also, the City of Fort Pierce has installed
its own single mode fiber optic connection within and between some of its
adjacent sites to further reduce costs such as eliminating a full T-1 leased
line at $1,000.00 per month while simultaneously improving security and other
addressing other needs. One mile of
fiber has been run in conduit underground between the Police Station and the
Golf Course; thereby, eliminating one leased digital circuit and cutting the
cost in half for the remaining leased digital circuit between the two sites
that have now become one larger site or local area network. The State’s router, as well as the City’s
router and firewall have also been eliminated at one site within this
transition. This reduces costs and eases
administration of each site in the local area network. The City of
This installation process eliminated the need of additional switches that were necessary between the buildings compared with single mode fiber optic cables to connect all three building to one server to form a local area network beyond the limitations of conventional copper wiring. Security and stability were also enhanced and the Solid Waste department was also connected to Public Works to form a larger local area network by traversing across the main road within the compound with another single mode fiber optic cable of just over 600 meters or 1800 plus feet in length. This eliminated another Full T-1 leased digital circuit and eliminated the need of another state router, City router, and associated firewall. Costs of $1,000.00 per month were negated as well as the additional costs of administration of these WAN infrastructure components. Ease of network administration was also made easier by allowing two sites to become one and improved the ability to administer and secure the now larger local area network.
LAN AS400/Server Hardware and Software – Primary Site
At the City of Fort Pierce’s main site at City Hall, a full T-1 digital data circuit at 1.5 megabits is being utilized to connect to all 5 remote sites and vice versa, in tandem, email, telnet, and internet or http services are accessible from this site. A proxy server firewall used at this site limited internet access through this site when the IBM AS400 was shutdown. However, the proxy firewall was removed and now all sites have internet access through their respective gateways or routers. City Hall contains a model 520 AS400 with 600 gigabytes of RAID 5 storage that encompasses about 28 SCSI hard drives in the RAID 5 array internal to the AS400. All nodes whether remote or local connect to this mid range computer system for email, telnet, and data storage services.
This IBM AS400 is rated at over
1000 CPW’s and over 375 batches cycles and this mid range computer also
contains over 8 gigabytes of main memory and utilizes a 64-bit bus architecture
throughout the computer. The City’s IBM
AS400 also contains 6 internal IBM Netfinity servers, two of the servers are
Pentium III’s @ 700 megahertz with 512 and 768 megabytes of memory, the other
four servers are Pentium IV’s @ 2 gigahertz with 1 gigabyte of memory
each. All 6 servers share internally
with the AS400 the 600 gigabyte RAID 5 array,
and other related devices and peripheral hardware such as an IBM
keyboard, IBM flat screen LCD monitor, mouse, CD-ROM/DVD-ROM, 2 external SCSI
tape drives, 4 internal redundant power supplies and one large external backup
uninterruptible power supply. The 6
internal servers each have an internal 10/100/1000 IBM 64bit PCI (NIC) adapter
to connect to the 3COM 3300 24 port 10/100 switches at Full-Duplex 100 megabit
speed. Also, each of the 6 internal servers
connects to the IBM AS400 with a virtual IBM (NIC) adapter at 1 gigabit speed. All 6 servers use Microsoft Windows Server
2003 – Standard Edition. The Microsoft
network operating system was chosen by the City of
LAN Server Software and Software Services – Primary Site
This main site at City Hall also contains Lotus Notes Domino groupware, running natively on the AS400 versus on a typical Microsoft server platform. The City’s purposes for this are to utilize the enhanced security features of the AS400 code written in RPG of which its operating system is the OS400. The City IBM AS400 OS400 operating system level is at V5R3 (Version 5, Revision 3) running at a security level of 40. The highest security level which ranges from 10 to 50 is 50. Entities such as the Pentagon operate at a security level of 50 which is very drastic compared to level 40 security or below. Levels 10 – 30 are relatively weak, 40 is medium strength security, and 50 is rated as a very high level of operating security. But any organization can implement level 50 security if necessary. The AS400 operating system is relatively impervious to viruses, worms, Trojan horses, etc. And therein is why the City uses the AS400 RAID 5 array to store user data via home directories, process email, other groupware functions, and as a mid range computer system to perform many other computing based services for the entire City’s computing needs. As mentioned earlier, the City’s IBM AS400 hosts the Lotus Notes groupware program for Domino, DNS, DHCP, and other server based services can also be implemented and shared from this platform. However those network services are generally performed at the server level. One IBM Netfinity server functions as a Domain controller with DNS, DHCP, and SUS services.
Another Netfinity server is dedicated as a peer domain controller or backup domain controller providing backup DNS, DHCP server services and combines the Network Associates Mcafee E-Policy Orchestrator services, that provide rogue node detection, desktop firewall services, and of course, antivirus, anti-worm, and many other services including reporting on the networks status and health of nodes attached to the local area network. Group Policies for security, other user right issues, and software upgrades are propagated throughout the entire local area network at this site and as each server/domain controller at the 5 remote sites. Each of the other four IBM Netfinity servers running Microsoft Windows 2003 Server – Standard Edition are dedicated to one application each such as Track It version 6.5 which is used to enhance and enforce group policy issues and collect information about the nodes throughout the City’s network. Another Netfinity server is dedicated to a Windows based Credit Card Purchasing and Procurement program, another Netfinity server is dedicate to Cognos which is database query or data mining and reporting program along with a SQL 2000 database server. And the final Netfinity server is dedicated to WinDiag which is another server based software package that remotely collects node information, assists in enforcing group policy, and assist in rolling out software updates and installations via *.msi files or other similar types of remote installation files. This program also allows for remote administration of any computer on the entire network. The AS400 also has software installed from Bytware Software, Inc., one package from this software company is called Messenger Plus which monitors all AS400 activity and pages and notifies the City’s MIS Department by various messaging mediums about all events on the AS400 and associated 6 internal IBM Netfinity Servers.
The City also has purchased Standguard antivirus for the AS400 from the same company. This software is used to protect the root drive where all the users PC data is stored and therefore, protected.
LAN Server Hardware and Software– Primary Site
Attached to the same local area network are various standalone administration servers for other various server based software packages. The City has a standalone Lotus Notes administration version 6.5.3 server/PC that uses an IBM X206 server with 1 gigabyte of memory and an 80 gigabyte SCSI hard drive and an IBM 80 gigabyte SCSI tape drive.
This administration server/PC uses
Microsoft Windows XP Professional as it operating system and software named
Group Software made from Group Software Technologies in
This administration server/PC uses
Microsoft Windows XP Professional as its operating system and DMS software from
SunGuard H.T.E., Inc. to manage all AS400 data in document form in and out of
the IBM AS400. Also, SunGuard H.T.E.,
Inc. is the City of
LAN Workstation Hardware and Software – All Sites
The workstations at City Hall and the 5 aforementioned remote sites all utilize IBM PC’s. Of the 150 plus workstations throughout the City that attach to the AS400 for services, about 50% are IBM PC 300 GL’s with Pentium III processors running at 500 megahertz to 900 megahertz with 128 to 512 megabytes of memory and 20 to 40 gigabyte IDE hard drives. The other 50% are newer IBM Pentium IV NetVista class PC’s, they range from 2.0 gigahertz to 3.2 gigahertz with 512 megabytes of memory and 80 gigabyte SCSI hard drives. All IBM PC’s also have CD-ROM‘s, CD-R’s, and DVD-ROM drives. Some of these IBM PC’s also have Travan 7 or TR-7 IDE technology tape drives to match each PC’s respective hard drive capacity for local backups. Most of the IBM workstations are used to access the AS400 via a telnet session over 5250 IBM Client Access Express version V5R3 software with an I Services Navigator that allows GUI access with respect to the signed on user rights and privileges of the user for objects such as printers and related spool files in the printers outq. These workstations also use the current Lotus Notes Client version 6.5.3 to access the groupware functions such as calendaring and for email access. These workstations also have access to the internet and are monitored for activity via Track It version 6.5 and WinDiag software. The Sonicwall Firewalls at all sites are setup to block workstation access to various topical websites such as weapons, gambling, pornography, as well as black and white lists, and various rules for blocking internet protocol addresses, ports, and winsockets. These workstations also have programs installed such as a multi user license edition Microsoft Office 2003 Professional, Adobe Acrobat 7.0 Professional, and various licensed software packages for scanning, editing scanned data, and printing.
For remote sites such as the City Marina, these IBM PC workstations have a Progress 9 database engine installed to access the Marina Program on their server to monitor and adjust the boat slips as customers come and leave the marina. The Golf Course workstations have Rectrac version a4 that is used to account for all golf related transactions such as goods purchased in the pro shop, greens fees, cart fees, and buckets of driving range balls for practice. The Golf Course also has a PC dedicated to the Rain Bird program and hardware to control the entire irrigation system of the 18 hole Golf Course itself. The Public Works department has a work order program in an SQL database in which they track all of the work orders for various City related infrastructure projects. The Engineering department has GIS programs and Auto Cadd installed on their workstations for engineering projects as well as traffic light control and monitoring programs and related equipment. The Police Station has numerous programs installed on their workstations for all types of purposes. Most workstations have a network drive or home directory to store data over the network and as a central point of backup and data archiving on the local area network server or on the AS400. Some of these network drives are shared for all or certain departments and/or workstations and related users to access and exchange or update the data as necessary with applicable rights access based upon the signed on user on the workstation.
Computer Warranty and Costs
These IBM Pentium III and all other newer workstation’s all have 3 year extended 24/7 onsite warranty and support as well as all IBM servers which have the same type of warranty and support.
The IBM AS400 also has 24/7 support
via the lease contract agreement with IBM and a notation is that all AS400’s
are monitored throughout the world in various locations. The IBM group that monitors the City’s AS400
is located in
Each standalone IBM X205 or X206 series server costs the City about $2,500.00, each IBM NetVista workstation costs the City around $1,000.00 to $1,500.00 depending upon the PC configuration. Some software packages such as Rectrac cost over $30,000 with annual support at around $5,000.00 for Rectrac and up to over $30,000 for support for the IBM AS400 per year.
All computer hardware, software, and technical support is purchased either by Florida State contract or through a single source vendor such as Microsoft, IBM , or a business partner of those respective corporations. An IBM 3 year onsite 24/7 support agreement for each workstation costs $199.00.
LAN and WAN Printers
There are numerous types of printers directly connected to these workstations, servers and as standalone printer servers over the network. Most workstation attached printer are from Hewlett Packard and are LaserJet’s or color DeskJet’s such as an HP1300C or an HP 1700 C respectively. These types of printers connect to each workstation either by a parallel port or a USB port and cable to the PC. They also have 3 year extended 24/7 warranties and they cost the City $150 up to over $500.00 respectively. The 3 year 24/7 extended printer warranty usually cost $49.00 to $99.00. These workstation printers may be shared over the local area network or even the WAN. They can print any data from a Windows based program or from the AS400 via a 5250 Client Access Express Printer emulation session via telnet. There are about 10 Toshiba, Fujitsu and various other manufactures of network based print server all in one copiers, printers, and scanners throughout City Hall and the remote sites to augment a local workstations printer.
These printers are connected to the various local area networks as a print server based printer via a category 5E cable connected from the 3COM 3300 24 port 10/100 switch to a proprietary print server or directly to a NIC within the printer/copier/scanner. They are made shareable over the LAN’s, WAN, and they can even print data from the AS400. They are fast, reliable, and cost over $50,000 including the associated print server.
They also have technical support contracts that cost over $1,000.00 per year. The IBM AS400 has an impact printer, an IBM 6400 that prints various forms, log files, and numerous reports for all the financial applications. Some of the forms it produces are W-2’s, 1099’s, various license renewals, etc. and other financial reports are printed on green bar paper. This printer is connected to City Hall’s local area network via a category 5E cable directly to the built in NIC adapter within the printer. This printer can also be printed to from any Windows based workstations as this printer is shareable as well across the entire local area network.
AS400 Printers
The City of
These printers are accessible to print to from the AS400. The City also has an IBM Infoprint 40 LaserJet network printer that uses magnetic toner to print payroll and pension checks, direct deposits, and accounts payable checks.
This printer is connected directly from its built in 10/100 NIC adapter via a category 5E cable to a 3COM 3300 24 port 10/100 switch. This printer of course is not shareable across the network and is only accessible from the AS400 which creates the checks to print via form server software within the AS400. The IBM Infoprint 40 cost the City $40,000.00. The IBM 6400 impact printer cost the City over $30,000.00 and the 6 IBM 3230 impact printers cost the city over $5,500.00 each including the associated printer servers. All of these printers listed about have annual renewal technical support agreements through IBM and there costs vary according to the respective type of printer covered under warranty and support agreement. The AS400 has three dumb terminals with 2 attached printers that are attached to the AS400 via twin ax, type 1 shielded cable with an IDC connector through a loop wire concentrator.
LAN and WAN IP Schema
All 6 sites have public and private
addressing for their respective local area network. These 6 IP schemas for the public side of the
City’s IP schema within the State of
City E-mail and Web Site
At the City’s main site at City
Hall, the AS400 as well as the native running Lotus Notes Domino server have a
static address. As mentioned earlier in
this research, NAT services are being performed to translate the private local
area network addresses into public addresses for routing and other TCP/IP
connectivity purposes such as public static forwarding IP addresses. Enclosed within this research is a diagram of
how all 6 sites or local area networks connect within the state provided frame
relay wide area network. The City of
MIS Division
Various departments such as
Purchasing, Public Works, Engineering, Planning &Zoning, and Community
Response have separate and redundant networks, web sites, internet access
providers (ISP’s) and additional email accounts that are not covered in this
research. These networks create extra cost and maintenance beyond the scope of
this study. The MIS division is not an
autonomous department with its own staff and budget. The Finance department overseas the operation
of the MIS division and the Finance department controls the finances of the
division within Finance department’s budget.
The MIS staff consisted of the MIS Director and MIS Operator until
2002. Two positions were then created,
MIS PC/Network Systems Analyst and MIS PC Technician which brings the
department to a total of 4. The ratio of
users to tech support is very high at over 40 to 1. Due to this fragmented environment, most
departments within the City of
There are procedures on generating payroll, pension, and accounts payable checks as well as most other important documents and forms as mentioned earlier to be produced by City staff.
Backup and Recovery
The City of
The City of
The City of
Each month, two tapes are used for each of the six computers to be backed up , i.e. A and B or C and D in a bimonthly rotation in which two backups are done such as A and B in one month and C and D in the next month in case one backup fails during a restore due to a number of reasons such as a corrupt tape media, etc. One tape is of each computer backed up is stored offsite, either tape A or tape B, etc. Each tape whether off or on site is stored in a safe location in a fire proof vault and then in a fireproof safe. Other automated daily backups are conducted on a work day basis during the evening of each working day on all servers throughout City Hall, some remote sites, and the AS400. These tapes also follow the A, B, C, and D grandfathering tape backup method, but on a weekly versus monthly basis. Other backups are backups are conducted such as when payroll or pension or various fiscal and calendar year process have been completed. Backups are also performed on any server or AS400 computer upgrade before and after the upgrade is conducted to enable restore capabilities in case of an upgrade failure or other related upgrade problem. In the Finance department, every PC is fully backed up twice as well and stored with the AS400 and server backups. The PC’s in the Finance department which includes the MIS division also follow the same A, B, C, and D grandfather tape backup rotation.
MIS Budget
A budget estimate, if the MIS division
was a City chartered department would probably
encompass over a $1,000,000.00 per year for staff salary and benefits, for all the
City computer related software, hardware, and most importantly, additional and staff necessary to properly
manage the City of
The City’s MIS approximated current budget under Finance is under $300,000 a year. Each department contributes out of their respective budget for the necessary hardware and software purchases necessary to maintain and upgrade the existing City’s computer infrastructure. Recommendations for improving the City’s current MIS situation would be to petition the City Manager who controls all staff decisions and the City Commission to allow the MIS division to become a chartered MIS department with an approved budget to conduct the hiring of MIS staff and therein, begin to centralize all computer hardware, software, training, and various other computer services and needs to one standard to be lead under like kind leadership, technical capability, and vision.
This would improve the non cohesive
work environment the City appears to currently endure and improve productivity
for the City to function and ultimately to better respond to the citizens of
the City of
Future Projects
Future projects for the MIS Division of the City of Fort Pierce are to establish a client to LAN VPN service for all MIS staff to remotely access the network for offsite administration, LAN to LAN VPN services for vendors such as SunGuard H.T.E., Inc., IBM, Live Vault, and other vendors to remotely connect, administer, and backup hardware and/or software on our LAN’s throughout our frame relay WAN via the internet. Fully implement Questys archiving software through the City, implement Cognos database query program throughout the City wide network, and fully implement an intranet with Naviline 3.0.
Other items are to continue enhancing existing security hardware and related software such as: adware, spybot, and malware detection and removal software package for all network nodes that can be centrally administered, updated, and distributed.
To upgrade the SUS server services
to Microsoft’s new method of installing operating systems fixes and patches
across the enterprise wide network when it becomes available. Finally, the MIS division is responsible for
the City’s telephone system; the main PBX system located at City Hall is a Mitel
2000 SX PBX with two smaller, but like kind telephone PBX systems at Public
Works and the Police Station. The Police
Station also has its own voicemail system.
City Hall has a standalone Microsoft Windows 2000 Professional based PC
which runs a voicemail program for most of the City’s voicemail needs. At some point in the future, these services
will be anticipated to move over to Voice over IP technology (VOIP) through the
local area network and out over the frame relay network or WAN for LAN network based
telephone and voicemail services. These
are but a few of the projects within the next 3 years that the MIS staff of the
City of
Suggestions on Improvement and Conclusion
During the next three years, growth
problems will primarily continue to accrue due to the current status of the MIS
division and its limited ability to role out new services for all computer
users, as well as security and other computer networking enhancements necessary
to carry the City of
The suggestions on improving the City’s entire computer system will not occur without increasing current costs because financial and other support in all its forms was never fully given since the City was computerized in 1979. If the resources were poured into this important City wide support level division to grow, then the enormous resources required today would not be an all at once or nothing issue. The enormous accrued costs due to outsourcing most kinds of City needed computer related services and support would be mitigated by growing the MIS division to support the City computer infrastructure and in turn, the enormous financial impact necessary would be compensated for by eliminating the high costs of outsourcing by investing those saved funds into the MIS Division. For the long term better computer related cohesion would produce faster and better worker productivity and output which would more than make up for the capital infusion necessary to place this City’s MIS division on the proper track to become a full fledged MIS Department for the entire City’s computing needs.
Please note: all computer related information and
financial information was obtained from City of
