I N D I A   P R I V A T E   L I M I T E D

INTEGRATED INFOSOLUTIONS



LOGICAL TOPOLOGIES FOR DATA CENTER



DATACENTER NETWORKING PROTOCOLS


Data centers contain many network transmission protocols for communication between electronic equipment. Ethernet and Fibre Channel are the dominant networks, with Ethernet providing a local area network (LAN) between users and computing infrastructure while Fibre Channel provides connections between servers and storage to create a storage area network (SAN). To design a structured cabling system for a data center, the designer should understand the different protocols that are used in each area of the data center.


>    LAN Protocols



Ethernet
Ethernet is the most widely installed LAN data transmission technology and is standardized as IEEE 802.3. Ethernet is typically used in data center backbones to transmit data packets from the core router to the access switch to the server network interface card (NIC). Ethernet is now deployed in data center switch networks with optical connectivity in the backbone and copper connectivity that addresses short-length equipment interconnects. Data center Ethernet deployments operate at speeds of 1G and 10G utilizing predominately OM3 and OM4 multimode optical fiber. Multimode fiber installations usually operate at 850 nm with VCSEL transceivers.



>    SAN Protocols



a) Fibre Channel
Fibre Channel is a high-performance, low latency, duplex fiber serial link application with data rates of 1 Gb/s, 2 Gb/s, 4 Gb/s, 8 Gb/s, 10 Gb/s and 16 Gb/s. It provides a very reliable form of communication that guarantees delivery of information. Fibre Channel is used in the data center to transmit data from the server host bus adapter (HBA) to the SAN director to the SAN storage. Similar to Ethernet, OM3 and OM4 fibers are the dominant fibers and media type used in the SAN network.

b) Fibre Channel over Ethernet
Data centers utilize multiple networks that present operational and maintenance issues as each network requires dedicated electronics and cabling infrastructure. As previously discussed, Ethernet (LAN) and Fibre Channel (SAN) are the typical networks in a data center. FCoE is simply a transmission method in which the Fibre Channel frame is encapsulated into an Ethernet frame at the server. The server encapsulates Fibre Channel frames into Ethernet frames before sending them over the LAN and de-encapsulates them when FCoE frames are received. Server I/O consolidation combines the NIC and HBA cards into a single converged network adapter (CNA) which reduces server cabling and power/cooling needs. At present, the Ethernet frame is removed at the Ethernet edge switch to access the Fibre Channel frame which is then transported to the SAN directors. Fibre Channel is a deterministic protocol that guarantees delivery of information.




While standards help guide the data center physical infrastructure, the data center logical infrastructure does not have a standards body helping with design. Logical architectures vary based on customer preference and are also guided by the electronics manufacturers. Though a standard does not exist, there are some common architecture best practices that can be followed. Most logical architectures can be broken into four layers :



>    Core



The core layer provides the high-speed connectivity between the data center and the campus network. This is typically the area where multiple ISPs provide connections to the internet.



>    Aggregation



The aggregation layer provides a point where all server area devices can share common applications such as firewalls, cache engines, load balancers and other value-added services. The aggregation layer must be able to support multiple 10G and 1G connections to support a high-speed switching fabric.



>    Access



The access layer provides the connectivity between the aggregation layer shared services and the server farm. Since additional segmentation may be required in the access area, three different segments are needed :
a) Front-end segment – This area contains web servers, DNS servers, FTP and other business application servers.
​b) Application segment – Provides the connection between the front-end servers and the back-end servers.
​c) Back-end segment – Provides connectivity to the database servers. Also provides access to the storage area network (SAN).



>    Storage



The storage layer contains the Fibre Channel switches and other storage devices such as magnetic disc media or tape.



TIER RATINGS FOR DATA CENTERS


Additional considerations when planning a data center infrastructure include redundancy and reliability. TIA-942 describes redundancy using four tiers to distinguish between varying levels of availability of the data center infrastructure. The tiers used by this standard correspond to industry tier ratings for data centers, as defined by the Uptime Institute. The tiers are defined as Tier I, II, III and IV, where a higher tier rating corresponds to increased availability. Tier ratings are specified for various portions of the data center infrastructure, including telecommunications systems architectural and structural systems, electrical systems and mechanical systems. Each system can have a different tier rating, however; the overall data center tier rating is equal to the lowest of the ratings across the infrastructure.


>     Tier I Data Center : Basic
>     ​Tier II Data Center : Redundant Components
>     ​TierIIIDataCenter: Concurrently Maintainable
>     ​Tier IV Data Center : Fault Tolerant



Data center cabling infrastructures must provide security and enable 24 x 365 x 7 uptime. Tier 4 data centers have uptime requirements of 99.995 percent, less than one-half hour per year.



>    Flexibility



With the constant in data centers being change, the cabling infrastructure must be modular to accommodate changing requirements and easy to manage and adjust for minimal downtime during moves, adds and changes.



>    Scalability



Cabling infrastructures must support data center growth, both in addition of system electronics and increasing data rates to accommodate the need for more bandwidth. The  infrastructure must be able to support existing serial duplex transmission and provide a clear migration path to future parallel optic transmission. In general, the infrastructure should be designed to meet the challenges of the data center over a 15- to 20-year service life.



APPLICABLE STANDARD : TIA-942


TIA-942, Telecommunications Infrastructure Standards for Data Centers, was released in April 2005. The purpose of this standard is to provide information on the factors that should be considered when planning and preparing the installation of a data center or computer room. TIA-942 combines within a single document all of the information specific to data center applications. This standard defines the telecommunications spaces, infrastructure components and requirements for each within the data center. Additionally, the standard includes guidance as to recommended topologies, cabling distances, building infrastructure requirements, labeling and administration, and redundancy.



DATA CENTER SPACES AND INFRASTRUCTURE





















The main elements of a data center, defined by TIA-942, are :


-      Entrance room (ER)

-      ​Main distribution area (MDA)

-      ​Horizontal distribution area (HDA)

-      ​Equipment distribution area (EDA)

-      ​​Telecommunications room (TR)


The components of the cabling infrastructure, as defined by TIA-942, are as follows :


-      ​Horizontal cabling

-      Backbone cabling

-      ​Cross-connect in the ER or MDA

-      ​Main cross-connect in the MDA

-      ​Horizontal cross-connect in the TR, HDA, MDA

-      ​Zone outlet or consolidation point in the ZDA

-      ​Outlet in the EDA


​In a data center, including HDAs, the maximum distance allowed for horizontal cabling is 90 m, independent of media type. With patch cords, the maximum channel distance allowed is 100 m, assuming 5 m of patch cord at each end of the channel for connection to end equipment. When a ZDA is used, horizontal cabling distances for copper may need to be reduced. Depending on the type and size of the data center, the HDA may be collapsed back to the MDA. This is a typical design for enterprise data centers. In this scenario, the cabling from the MDA to the EDA, with or without a ZDA, is considered horizontal cabling. In a collapsed design, horizontal cabling is limited to 300 m for optical fiber and 90 m for copper. TIA-942 defines the maximum distance for backbone cabling as being application and media dependent.




DESIGN CONSIDERATIONS


There are two types of environments in the data center: local area networks (LANs) and storage area networks (SANs). A LAN is a network linking multiple devices in a single geographical location. Typical LAN speeds are 1 Gb or 10 Gb Ethernet. A SAN is an area in the network linking servers to storage equipment, which introduces the flexibility of networking to servers and storage. Speeds are typically 2G, 4G, 8G or 10G Fibre Channel.



















​When designing a data center, several factors should be taken into consideration, including standards compliance. TIA-942, Telecommunications Infrastructure Standard for Data Center, details several of the factors that should be considered when designing a data center. When implementing a structured cabling solution, the standard recommends a star topology architecture to achieve maximum network flexibility. TIA-942 outlines additional factors crucial to data center design, including recognized media, cable types, recommended distances, pathway and space considerations and redundancy. In addition to standards compliance, the need for infrastructure flexibility to accommodate future moves, adds and changes due to growth, new applications, data rates and technology. Advancements in system equipment must be considered.



DATACENTER NEEDS


As data centers face the continued need to expand and grow, the fundamental concerns are constant. Data center infrastructures must provide reliability, flexibility and scalability in order to meet the ever-changing data center network.


>    Reliability



>    ​Network Efficiency



Data centers have seen significant growth in size and numbers in the past few years and should continue to see significant growth in the future as networks continue to evolve and move toward 100 Gigabit Ethernet. Due to the considerable growth in data centers, there is a need to have simple, efficient cabling solutions that maximize space and facilitate reduced installation time and costs. Preterminated solutions are often the preferred solution as they provide higher fiber density, reduced installation time and the ability to easily facilitate moves, adds and changes (MACs). For example Corning Cable Systems’ pre-terminated optical fiber cabling solutions streamline the process of deploying an optical network infrastructure in the data center. A modular design guarantees compatibility and flexibility for all optical connectivity and easily scales as demands dictate and requirements change. The preterminated solutions also manage fiber polarity, virtually eliminating it as a concern in network design, installation or reconfiguration.



End users are looking for a higher performance, low profile solution for a more effective overall operation of the network. Manageability is essential; without it, the cabling infrastructure takes over the data center in a short amount of time. To increase control over the data center infrastructure, structured cabling should be implemented. The key benefit of structured cabling is that the user regains control of the infrastructure rather than living with an unmanageable buildup of patch cords and an abundance of unidentifiable cables.



>    ​Flexibility and Scalability



Flexibility and scalability of the cabling infrastructure allow quick and easy changes with little to no impact on the day-to-day operation of the data center, as well as reduced risk that tomorrow’s technology will render an obsolete infrastructure. Scalability of the data center is essential for migration to higher data rates and for adding capacity without major disruption of operations. The initial data center must be designed so it can be scaled quickly and efficiently as the requirements change. To meet the requirements and demands of the data center, the topology in the data center, as well as the actual components used to implement the topology, must be explored. Both topology and components, if chosen correctly, create an effective network, save time and money, and create efficiency, manageability, flexibility and scalability in the data center.



FUNDAMENTAL ISSUES


>    Manageability



>    Storage



Storage Area Network
This area of the data center provides the back-end connection to data. This area contains many types of storage devices. The protocols used to communicate in this area are Fibre Channel Ethernet and small computer system interface (SCSI).



Point of Presence (PoP) Zone
This area of the data center is sometimes referred to as the “meet me” room. It is typically the area where the service provider enables access to their networks. This area contains many routers and core switches.

Server Area Zone
​This area of the data center provides the front-end connection to the database servers. This area contains many switches and servers. The protocols used to communicate in this area are 1 Gigabit and 10 Gigabit Ethernet.

DATA CENTER DESIGN & CONSTRUCTION


A data center, as defined in TIA/EIA-942, Telecommunications Infrastructure Standard for Data Centers, is a building or portion of a building whose primary function is to house a computer room and its support areas. The main functions of a data center are to centralize and consolidate information technology (IT) resources, house network operations, facilitate e-business and to provide uninterrupted service to mission-critical data processing operations.


​Data centers can be classified as either enterprise(private) data centers or co-location(co-loc)/ hosting(public) data centers. Enterprise data centers are privately owned and operated by private corporate, institutional or government entities. Enterprise data centers support internal data transactions and processing, as well as Web Services and are supported and managed by internal IT support. Co-loc data centers are owned and operated by telcos or unregulated competitive service providers and offer outsourced IT services. Services that data centers typically provide include Internet access, application or web hosting, content distribution, file storage and backup, database management, fail-safe power, HVAC controls, security and high-performance cabling infrastructure.

























FUNCTIONAL AREAS


>     Switching