Stochasticity is ubiquitous in the world around us. However, our predominant computing paradigm is deterministic. Random number generation (RNG) can be a computationally inefficient operation in this system especially for larger workloads. Our work leverages the underlying physics of emerging devices to develop probabilistic neural circuits for RNGs from a given distribution. However, codesign for novel circuits and systems that leverage inherent device stochasticity is a hard problem. This is mostly due to the large design space and complexity of doing so. It requires concurrent input from multiple areas in the design stack from algorithms, architectures, circuits, to devices. In this paper, we present examples of optimal circuits developed leveraging AI-enhanced codesign techniques using constraints from emerging devices and algorithms. Our AI-enhanced codesign approach accelerated design and enabled interactions between experts from different areas of the microelectronics design stack including theory, algorithms, circuits, and devices. We have demonstrated optimal probabilistic neural circuits using magnetic tunnel junction and tunnel diode devices that generate an RNG from a given distribution.
“Efficiently produce a large number of random numbers and make them widely available throughout the circuits and architecture such that different types of RNG can be performed in different parts of the computation, as the application requires”.
The key challenge from the circuit and architecture level is to identify potential mechanisms for mapping the inherent stochasticity of our devices (Thrust 1) with the required probability distributions for our algorithms (Thrust 3).
What probabilistic logic elements that leverage the intrinsic physical properties of devices (such as magnetic-induced correlations) can be developed to provide more sophisticated probabilistic behaviors?
What probabilistic neural architecture can AI-guided co-design identify that incorporates the properties of stochastic devices, probabilistic logic, and neural algorithm requirements?
To enable the design of circuit topology and circuit parameters, we use AI-enhanced evolutionary optimization and reinforcement learning. AI/ML techniques can be used to optimize multiple objectives; in this case, we may be interested in how accurate the circuit performs the task using a given device model, while simultaneously minimizing latency and/or energy usage. We leverage abstracted device models for faster iteration over the design space. AI/ML methods can also be used to evolve both the topology of the circuit and the parameters simultaneously. Evolutionary algorithms can potentially be used to creatively discover new ways to design circuits or leverage the underlying device characteristics and have been found to have surprising creativity in the design of novel solutions to a variety of problems. On the other hand, reinforcement learning techniques can leverage existing designs and domain knowledge for circuit design. Further Bayesian and evolutionary multi-objective optimization can further help optimize our fitness function/reward functions for the evolutionary and reinforcement learning algorithms respectively.
We anticipate the AI/ML-guided design will also inform design of our stochastic devices and enable the design of creative circuits and architectures for our application. We will develop key RNG scientific kernels that exploit probabilistic devices and algorithms. These simulators and scientific kernels will be critical building blocks to feed into the architecture design search.
Copyright © 2023 Funded under the DOE National Laboratory Announcement Microelectronics Co-Design Research. SAND2022-3825 C and SAND2022-16196 O
We acknowledge support from the DOE Office of Science's Advanced Scientific Computing Research (ASCR) program and Basic Energy Sciences (BES) programs, Microelectronics Co-Design project COINFLIPS.