Transcription

HomeSearchCollectionsJournalsAboutContact usMy IOPscienceThe Gigabit Link Interface Board (GLIB), a flexible system for the evaluation and use of GBTbased optical linksThis article has been downloaded from IOPscience. Please scroll down to see the full text article.2010 JINST 5 1007)View the table of contents for this issue, or go to the journal homepage for moreDownload details:IP Address: 137.138.62.77The article was downloaded on 24/11/2010 at 10:16Please note that terms and conditions apply.

P UBLISHEDBYIOP P UBLISHINGFORSISSAR ECEIVED: October 8, 2010ACCEPTED: October 29, 2010P UBLISHED: November 19, 2010TOPICAL WORKSHOP ON E LECTRONICS20–24 S EPTEMBER 2010,A ACHEN , G ERMANYFORPARTICLE P HYSICS 2010,P. Vichoudis,1 S. Baron, V. Bobillier, S. Haas, M. Hansen, M. Joos and F. VaseyCERN,1211 Geneva 23, SwitzerlandE-mail: [email protected] BSTRACT: The Gigabit Link Interface Board (GLIB) is an evaluation platform and an easy entry point for users of high speed optical links in high energy physics experiments. Its intended useranges from optical link evaluation in the laboratory to control, triggering and data acquisition fromremote modules in beam or irradiation tests. The GLIB is an FPGA-based Advanced MezzanineCard (AMC) conceived to serve a small and simple system residing either inside a Micro Telecommunications Computing Architecture (µ TCA) crate, or on a bench with a link to a PC. This paperpresents the architecture of the GLIB, its features as well as examples of its use in different setups.K EYWORDS : Data acquisition circuits; Modular electronics; Digital electronic circuits1 Correspondingauthor.c 2010 IOP Publishing Ltd and SISSAdoi:10.1088/1748-0221/5/11/C110072010 JINST 5 C11007The Gigabit Link Interface Board (GLIB), a flexiblesystem for the evaluation and use of GBT-basedoptical links

ContentsIntroduction12Implementation23Typical use cases44Summary and status61IntroductionThe Gigabit Link Interface Board (GLIB) is an evaluation platform and an easy entry point forusers of high speed optical links in high energy physics experiments. Its intended use ranges fromoptical link evaluation in the laboratory to control, triggering and data acquisition from remotemodules in beam or irradiation tests. Each GLIB card can process data to/from four Small FormFactor Pluggable Plus (SFP ) transceiver modules, each operating at bi-directional data rates of upto 6.5 Gbps. This performance matches comfortably the specifications of the GigaBit Transceiver(GBT) / Versatile Link project [1, 2] with its targeted data rate of 4.8 Gbps. In its simplest form,one GLIB board thus interfaces with up to four GBT channels.Figure 1 highlights the baseline configuration of a GBT — Versatile Link — GLIB system. Onthe left side, front-end (FE) ASICs are electrically connected to the GBT ASIC through e-links [3]while the GBT high-speed serial data streams are converted to/from the optical domain through theVersatile Link FE transmitter/receiver. On the other end, the GLIB converts data to/from the opticaldomain, implements the GBT protocol and codes/decodes the user payload at the link back-end.The GLIB I/O capability can be further enhanced with two FPGA Mezzanine Cards (FMCs) [4].This gives users the flexibility to adapt the GLIB interface to their system, for instance by addingconnectivity to the Timing, Trigger and Control (TTC) network at the backend, or connectingto e-links at the front-end. Figure 2 illustrates a case where two GLIB boards are interconnectedFigure 1. The GLIB board in a GBT-Versatile Link system.–1–2010 JINST 5 C110071

back-to-back, allowing implementing and experimenting with GBT-based systems well before fullfledged GBT ASICs become available.2ImplementationThe GLIB is an Advanced Mezzanine Card (AMC) [5] in Double-Width Mid-Size format (180 mm 148 mm 18 mm) conceived to serve a small and simple system residing either inside a MicroTelecommunications Computing Architecture (µ TCA) [6] crate or on a bench with a link to a PC.Figure 3 shows a block diagram of the GLIB. The card is based on the XC6VLX130T FPGA ofthe Virtex-6 family by Xilinx Inc. The FPGA includes twenty Multi-Gigabit Transceivers (MGTs)operating at up to 6.5Gbps that are organized in groups of four (quads).Concerning the AMC high speed serial connectivity, the GLIB provides two Gigabit Ethernet(GbE) and two 2nd generation four-lane PCI Express (PCIe Gen2 4x) interfaces. It is importantto mention that the GLIB gives users the possibility of implementing various other high speed serial data protocols (for custom applications) instead of PCIe. This is possible mainly thanks toGLIB’s sophisticated clock distribution circuitry that is based on cross-point switches and programmable clock multipliers. This circuitry offers a large selection of input clock sources (AMCclocks, the FMC clocks, front panel clock connector or on-board oscillators).The two high-pin-count FMC sockets each provide up to 80 user-specific differential I/O pairsdirectly connected to the FPGA as well as two differential clock inputs and two differential clockoutputs. The primary FMC also provides four optional 6.5Gbps transceiver lines, thus allowingextending the optical I/O capability to eight links by using e.g. a quad-SFP FMC1 that is commercially available. Additionally, many other FMC functionalities could be implemented such as: FMC for TTC reception that could be implemented either based on custom ASICs or commercial Clock and Data Recovery ICs. E-link FMCs for direct communication with front-end modules not equipped with GBT. Other optical interface extension FMCs equipped with other types of optical modules wherevarious protocols could be implemented (e.g. 10 Gigabit Ethernet).1 Themaximum component height allowed on the component side of a mid-size AMC is 12.9 mm. The maximumheight of an FMC with 8.5 mm and 10 mm stacking height is 11.4 mm and 12.9 mm, respectively. A 10 mm stackingheight FMC can accommodate SFP cages, since their height is typically 9.7 mm.–2–2010 JINST 5 C11007Figure 2. Back-to-back interconnected GLIB boards with customization mezzanines (TTC and E-Link).

For the board initialization when in an xTCA environment as well as for other management tasks(e.g. temperature/voltage monitoring) a Module Management Controller (MMC) mezzanine card [7]is used. The MMC is based on a microcontroller by ATMEL.For configuration purposes, a very flexible Joint Test Action Group (JTAG) circuitry basedon a Complex Programmable Logic Device (CPLD) is also available. The CPLD acts as a JTAGswitch selecting the JTAG master source between the dedicated JTAG connectors, MMC or AMCJTAG lines for the configuration of the FPGA and the associated EEPROM. The JTAG switch isalso useful for boundary scan testing purposes.The GLIB carries a GbE PHY as well as an Ethernet plug in order to interface to a PC througha standard Ethernet cable, while in bench-top operation. The Ethernet plug is located in the interiorof the board, thus it is not accessible when the GLIB is installed in a crate. Interfacing to a PCwhile the GLIB is used on a bench could be also achieved with a PCIe adapter card attached to theAMC edge connector.For temporary data storage, two 72Mb SRAM devices are available on-board.2 The GLIB hasthe possibility of using higher capacity (up to 1Gb) SRAM devices once they become available.It is important to note that the two SRAMs have independent address/data buses.For the powering of the AMC, high efficiency switching regulator modules with improvedthermal performance are used, requiring minimal cooling. Concerning the cooling of the FPGA,2 On-boardSRAM devices have been used instead of Single In-line Memory Modules (SIMM) due to the lack ofspace in the printed circuit board.–3–2010 JINST 5 C11007Figure 3. Block diagram of the GLIB AMC.

the use of a low-profile heat sink is sufficient for in-crate operation. For bench-top operation, theuse of a heat sink equipped with a cooling fan is foreseen.Figure 4 shows the component placement (top-side view). The AMC edge connector is locatedon the left side of the card while the front panel is on the right. The orientation of the FMC socketsis such that the primary FMC is accessible by the front panel and the secondary from the rear.Additionally, the front panel provides access to the four sockets for SFP modules as well as toan external clock connector. The switching regulator modules, as well as other active components(e.g. the clock distribution circuitry) are not shown in figure 4 since they are placed on the bottomside of the board.3Typical use casesThe flexibility of the GLIB with its variety of optical and electrical interfaces enables it to be usedin various configurations. Three possible setups are discussed below. Figure 5 shows a bench-topbeam test setup. In this setup, the GLIB communicates with the front-end via GBT links. TheTTC information is received by the GLIB through an FPGA Mezzanine Card (TTC FMC, shownas a small yellow rectangular block). The configuration of the GLIB as well as the transfer of–4–2010 JINST 5 C11007Figure 4. Preliminary placement (component side).

Figure 6. Bench-top front-end module test setup.the GBT payload from/to the PC is done through a GbE connection. The SRAM can be used forintermediate storage, if necessary.Figure 6 shows a bench-top setup for testing front-end modules not equipped with GBT. Forthe direct communication with the front-end chips, the GLIB is equipped with an e-link FMC emulating a GBT-e-port functionality (shown as a large yellow square block). The GLIB is alsoequipped with a TTC FMC. The configuration of the GLIB as well as the transfer of the GBT payload from/to the PC is done through a GbE connection. The SRAM can be used for intermediatestorage, if needed.For the powering of both bench-top setups highlighted in figures 5 and 6, an external powersupply is used. Figure 7 shows a system test setup in a µ TCA crate. The crate is managed bya MicroTCA Carrier Hub (MCH) controlled through a GbE link. The GLIB is equipped with aTTC FMC. The GLIB communicates with the front-end via GBT links. The configuration of theGLIB as well as the transfer of the GBT payload from/to an AMC CPU is done through the MCH–5–2010 JINST 5 C11007Figure 5. Bench-top beam test setup.

PCIe switch. The AMC CPU can store the data to an AMC storage medium through a SerialAdvanced Technology Attachment (SATA) link.4Summary and statusThis paper has introduced the GLIB, an evaluation platform currently under development for usersof high speed optical links in high energy physics experiments. Its architecture and main featuresas well as some of the various possible configurations have been presented. The PCB layout ofthe board is on-going and is expected to be completed by the end of 2010. A first prototype of theboard is expected in early 2011. We envisage that the GLIB board will be easily adapted to variousother applications thanks to its variety of interfaces. More details and up-to-date information aboutthe GLIB project can be found in the GLIB project web page [8].AcknowledgmentsThe authors would like to thank many colleagues who are contributing to this work, in particularGregory Iles of Imperial College and Jean-Pierre Cachemiche of Centre de Physique des Particulesde Marseille.References[1] P. Moreira et al., The GBT, a proposed architecture for multi-Gb/s data transmission in high energyphysics, in Proceedings of the Topical Workshop on Electronics for Particle Physics TWEPP-07,CERN-2007-07 (2007) pp. 332–336.[2] L. Amaral et al., The versatile link, a common project for super-LHC, 4 JINST 2009 P12003–6–2010 JINST 5 C11007Figure 7. µ TCA crate system test setup.

[3] S. Bonacini et al., e-link: A radiation-hard low-power electrical link for chip-to-chip communication,in Proceedings of the Topical Workshop on Electronics for Particle Physics TWEPP-09,CERN-2009-06 (2009) pp. 422–425.[4] ANSI/VITA,57.1-2008 (R2010), http://www.vita.com/fmc.html.[5] PICMG, AMC.0 R2.0, November 15 ons2.cfm?thetype One&thebusid 1.[6] PICMG, MTCA.0 R1.0, July 6 ons2.cfm?thetype One&thebusid 5[8] The GLIB team, GLIB project public page, 2010 JINST 5 C11007[7] J.P. Cachemiche, MMC mezzanine board – Specification v2.0, private communication.

Figure 1 highlights the baseline configuration of a GBT — Versatile Li nk — GLIB system. On the left side, front-end (FE) ASICs are electrically connected to the GBT ASIC through e-links [3] while the GBT high-speed serial data streams are converted to/from the optical domain through the