A nanophotonic interconnection system, although equivalent to an electrical one on the principle (carriage and switching of information - while showing drastically improved performances), has its own peculiarities and design constraints. To create a new generation of photonic-enabled interconnection networks, the research effort must employ a co-design strategy that concurrently investigates computer and photonic-enabled network architecture designs. This re-assessment of the distributed computing architecture (i.e. from single Chip-Multiprocessor based systems to large scale, supercomputer typed systems) impacts in turn the application programming, which should be included in the co-design. Using our custom, in house designed simulations suite PhoenixSim 2.0, we, along with our collaborators model numerous multiprocessor architectures and networks.
A central component in our research effort is the development of an integrated modeling and design environment spanning multiple levels of abstraction from the physical level through the circuit and micro-architectural level up to the system level. Our targeted methodology has been presented in this paper .
Our environment is called Phoenix Sim 2.0 and consists of three modules. The hardware level tool translates the characteristics of nanophotonic elements located along each link into insertion losses, power penalties and crosstalk metrics. This tool is based on models proposed in the literature but also on in-house developed ones. In particular, we recently we reported progresses on the modeling of integrated Silicon Photonics switch.
Starting from an initial architecture (bus, network), this tool is able to automatically assemble different designs, typically varying the scale, the nominal-per-wavelength datarate, and the number of wavelengths. Specific algorithms are used to identify the worst-case signal impairments under dynamic traffic. Each architecture is then analyzed and optimized to maximize throughput while maintaining signal integrity in any traffic condition.
Architectures developed and optimized with the physical layer can then be tested under dynamic traffic conditions using LightweightSim discrete-event simulator. A LightweightSim simulation can involve different type of traffic injectors. Using statistical traffic generators, the impact of the networking protocol and the associated contention resolution mechanisms present in the network can be statistically measured, which gives a rough estimate of the global system performance. In this case, LWSim translate the action taken at the micro-scale by the protocols and the devices into macro-scale statistical latency and bandwidth measurements. Check in particular this paper where we analyze different reservation protocols for an all-optical switch, and this study on the consequences in terms of energy of various reservation schemes.
LWSim also supports application dependent traffic. Specific injectors and receivers are created in the network simulation structure. In parallel, threads are created and associated with portions of code. Messages generated in that code are passed to the injectors, emitted in the network, received by the receivers and passed back to the threads. In this way, cross-influences between the executed code and the network can be analyzed.
The PhoenixSim 2.0 suite takes advantage of Javanco , a scientific computing toolbox. Javanco includes in particular versatile graph handling and visualization functionalities. Network layout displayed in above have been generated and displayed with it. Furthermore, LWSim simulations can be visualized live and network elements characteristics can also be modified through the graphical interface while the simulation is running. These features can be exploited to graphically check that a simulation model is correctly implemented, and that changes in the configuration change the performance as predicted. Javanco also integrates a graphical interface permitting an eased creation of simulation batches, which is extremely useful for design exploration. This semi-automated batch creation system is coupled with a result collection systems with integrated chart drawing functionalities.
This effort aims to address the detrimental communications constraints imposed by bandwidth density limitations and vastly growing power consumption of current electronically interconnected systems. We aim to develop optical interconnection network architectures for rack to warehouse scale distributed computing systems. We leverage our modeling and simulation environment (PhoenixSim) to obtain energy-performance data movement analysis and optimize the systems for diverse workloads types.
Check out our recent Hot Interconnect paper which compares various Chip-to-Chip silicon photonics based interconnection architectures in terms of realized bandwidth and energy-efficiency. According to our predictions, the critical metric of 1pJ/bit can be approached, in particular if inter-chip traffic is sustained at Tb/s rates.
This effort includes the analysis of the workload distribution over multiple node on the traffic intensities. We also analyze how interconnection networks with distinct capabilities affect distributed tasks. In this context, we proposed at the IA^3 workshop (part of SuperComputing'13 conference) a synthetic task model to ease such analyses.
Interaction with other simulation platforms
Acknowledging the major role played by data-movement in future systems performance, computing system simulators, as the Structural Simulation Toolkit (SST) effort of Sandia National Laboratories/DOE, are modeling interconnection networks with increasing fidelity. We are collaborating toward a seamless integration of optical-based data-movement capabilities in such simulators (along the lines of this position paper). This work is realized in the context of two Sandia/DOE funded research projects, Data-Movement Dominates and Co-Design and Optimization of Silicon Photonic Enabled Communication Architecture for High Performance Combustion Simulations.
Leveraging SST's full application simulation capabilities, we have been able to analyze the negative impact of the connection-oriented nature of optical communications on classical scientific application performance, and envisage network management policies able to mitigate this negative impact. Another ongoing research effort consists in identifying the optical interconnection network design leading to a minimized energy dissipation when executing a scientific workload. If scarcely dimensioned optical networks are less consuming, they also cause application slow-down which extends the whole system on-time. In contrast, ultra-wide networks show an important energy dissipation but lead to expedited application execution.
Toward Exascale computing thanks to photonic interconnection systems
An important effort of the modeling and simulation team consists in investigating how emerging technologies like silicon based nanophotonics may enable efficient Exascale computing, i.e. allow a super-computer to attain 10^18 operations per second. The extreme requirement of Exascale computing make optical transmission technologies potentially the sole enabling technology. This offer a unique chance for photonics to definitively penetrate in the computer architecture.
This research effort led to an estimation of the switch radix required to realize an Exascale computer. We also recently compiled an extensive review of major challenges associated with introducing Silicon Photonics in Exascale systems.