With the proliferation of ultra-high-speed mobile networks and device numbers connected to the Internet, people's expectations for computing performance have been increased by leaps and bounds. The processors under the traditional von Neumann architecture are constrained by energy consumption and bandwidth, which leads to the inability to carry the massive data processing demands. On the other hand, the emergence of artificial intelligence, especially artificial neural networks, poses great challenges to the slowing down of Moore's Law. Endowed with the inherent characteristics of high speed, low energy consumption and parallel processing, photonics is expected to achieve higher-performance information processing and even solve the complex algorithm problems that cannot be solved by microelectronic processors cooperating with electronics^1,2. It is worth noting that the training process of neural networks heavily relies on massive matrix operations during the feed-forward and backpropagation stages^3-5 which determines the computation time and energy consumption of many workloads. The optical matrix-vector multiplier (MVM) has the potential to greatly reduce the power consumption of information transmission, increase the operating speed of the device, as well as expand the information transmission capacity^6,7. Therefore, MVM is expected to be viable candidate for high-performance deep neural network acceleration hardware. In addition, many discrete numerical operations are based on the matrix so that MVM can perform various related operations, such as vector inner product operations, discrete Fourier operations⁸ and convolution⁹.

Initially, researchers implemented optical multipliers in free space^10-12. However, the traditional optical MVM adopts detachable components, which is not conducive to the scaling and integration of the overall structure. The development of photonic integration technology has made it possible to implement multipliers on-chip. The on-chip multipliers can be divided into three types according to the principle and the realization structure: wavelength division multiplexing (WDM) type, multiplane light conversion (MPLC) type, and Mach-Zehnder interferometer (MZI) type⁵. In a WDM-MVM, wavelengths are used to differentiate signals input in parallel. In addition to the initial 4 × 4 microring multiplier demonstration¹³, multipliers that can implement negative modulation matrix loading^14-17 and non-volatile loading of modulation elements^18-21 have also been demonstrated. The MPLC-MVM can load the matrix, which is decomposed into a series of programmable unitary diagonal matrices and unitary diffraction matrices²². Tang et al. experimentally demonstrated a ten-port single optical processor²³. In the MZI-MVM, it is also necessary to utilize the triangular²⁴r rectangular decomposition algorithm²⁵o convert an ordinary matrix into the multiplication of several unitary matrices and diagonal matrices²⁶. MZI-MVMs are often adopted to accelerate the optical neural networks (ONN) linear calculation part^27-30.

WDM-MVM usually requires multiple lasers to support multiple wavelengths. Modulation element loading requires various algorithms in MPLC-MVM and MZI-MVM. In order to reduce the cost and system complexity, in the current work, an on-chip optical signal processor was proposed to perform matrix-vector multiplication based on mode division multiplexing (MDM), which consists of a mode multiplexer, a multimode beam splitter, a mode demultiplexer, MZI modulators, and a power combiner. To the best of our knowledge, it is the first time that the MDM has been introduced to realize MVMs. As a proof-of-concept, a 4 × 4 MDM-MVM was demonstrated on silicon-on-insulator (SOI) platform with 4-level modulation.

Matrix-vector multiplication can be described by the following formulas:

where, I is an n-dimensional input vector; M is an m×n modulation matrix; O is an m-dimensional result vector; I_j, M_ij, and O_i are elements of I, M, O, respectively. The proposed architecture for performing the multiplication of a 4 × 4 matrix M and a 4 × 1 vector I is shown in Fig. 1a. Four incoherent fundamental mode (TE₀) light signals were input at the ports I₁-I₄. The elements of vector I were sequentially loaded on these four signals, which can be achieved by modulating the signal intensity. Each input vector element corresponds to a different mode (TE₀-TE₃), combined to the bus waveguide by the asymmetric directional coupler (ADC) based mode multiplexer^31,32. Later, a 1 × 4 multimode power beam splitter was utilized to divide the light into four even parts, containing all the four modes. Each part of the multimode signal was sequentially demultiplexed into four TE₀ lights by the four-channel mode de-multiplexer. Then the intensities of all 16 channels were modulated by the 4 × 4 modulator array loaded with matrix M. All the 4 × 4 multiplication processes were performed simultaneously. Finally, the four TE₀ signals of each channel were output to the single-mode waveguide through the 4 × 1 beam combiner, realizing the addition operation. Four optical powers detected by the off-chip PD array represent the elements of the result vector O.

In the MDM-MVM, the mode multiplexer and demultiplexer are realized by ADCs based on mode evolution, as shown in Fig. 2a. The TE₀ light is input at waveguide A. In the counter-tapered coupling region, a particular high-order mode is coupled into the bus waveguide B, while other lower-order modes are transmitted in waveguide B without interference. In order to achieve high mode conversion efficiency, the counter-tapered coupling region should be carefully designed following the phase-matching condition. A 220 nm-thick strip waveguide was chosen to construct the device. Under such a geometric structure, calculation of the effective indices for different modes of the strip waveguides with different widths at 1.55 µm wavelength was conducted, as shown in Fig. 2b. Assuming that the multiplexer is to implement conversion coupling from the TE₀ mode to the TE_i mode. The width at the input of waveguide A (Wa₁) needs to support the TE₀ mode, and the width at the output (Wa₂) is required to be small enough so as to reduce the coupling of waveguide B to waveguide A. The width at the input of waveguide B (Wb₁) must support the transmission of the original modes (TE₀-TE_i-1). Wb₂ needs to be greater than the minimum width corresponding to TE_i, while at the same time smaller than the minimum width for exciting TE_i+1 to avoid conversion of the TE₀ mode into a higher-order mode. Parameters such as lengths and gaps were optimized with the adoption of the finite-difference time-domain (FDTD) method. The optimized parameters of the three multiplexers are shown in Table 1. Fig. 2c shows the calculated field distribution and the mode conversion rate.

The structure of multimode waveguide crossing with an obliquely embedded subwavelength grating (SWG)³³ (Fig. 3a) was employed as the multimode power splitter. Due to the thin-film interference effect, the SWG reflector can split the multimode incident beam into two equal beams simultaneously. Since the effective refractive index of the SWG can be adjusted by changing the duty cycle (f_swg), the splitting ratio can be adjusted arbitrarily with a large bandwidth. In consideration of the fabrication feasibility, the SWG pitch Λ_swg and SWG width w_swg were chosen to be 500 nm and 200 nm, respectively. According to the simulation, when the multimode waveguide width was set as 15 µm, and the f_swg as 0.4, a splitting ratio of about 50 : 50 was achieved. The uniformity of the splitting ratio of each mode and the crosstalk between modes are shown in Fig. 3b. In order to reduce the difficulty and complexity of the fabrication process, the 1 × 4 beam splitter is realized with the adoption of three 50 : 50 beam splitters cascaded in a parallel structure (Fig. 3c).

The modulator is realized with the MZI based on thermal tuning, forming a modulation matrix to verify the feasibility of the MVM. The electrode in the heating area is titanium nitride (TiN), the length of the heating arm is about 300 µm, the resistance is about 1.8 kΩ, and the width of the heating waveguide is 500 nm. The beam combiner is achieved by a 4 × 1 MMI beam combiner. Assuming that the input power of each Input Port is equal, the power output from input port 1, port 2, port 3, and port 4 to output port accounts for 23.69%, 24.48%, 24.48%, and 23.69%, respectively.

Fig. 1b, c show the fabricated and wire bonded SOI photonic chip and the microscope image of the MDM-MVM device, respectively. The input is the TE₀ light, marked as I_j-TE_j-1 (j = 1, 2, 3, 4) according to the modes multiplexed into the bus waveguide, and the modulation MZI array can be marked as M_ij in the same way. The corresponding output port is O_i. It should be noted that, in order to improve the compactness of the device and reduce the footprint, 90° corner bends are adopted to change the transmission direction of the optical path³⁴.

The experimental setup for the device characterization is shown in Fig. 4. To avoid disturbing coherence of different input ports in MMI, an amplified spontaneous emission source (ASE) was adopted as the light source. The ASE was connected to a commercial-grade fiber optic splitter, which splits the original light into four parts as input vector. It is evident that modulators can be connected after the beam splitter to load different input vectors. To generate the modulated signals, a multi-channel direct-current power supply was utilized, which generates sixteen parallel electrical drive signals. Each set of four signals represents the ith row of the modulation matrix M and is applied in parallel to the MZI modulator corresponding to O_i. The light on the chip is emitted from the coupling grating, which can be measured by an optical spectrum analyzer (OSA) to evaluate the static spectral response of the MVM. The chip can also be connected to PDs so as to convert the inner product optical power into an electrical signal.

In the modulation matrix section, analysis on the computational accuracy of the MVM is a pivotal aspect. The factors affecting accuracy originate from various aspects, some of which can be eliminated. For instance, the imbalance in the splitting ratio between different ports of the splitter can be addressed by setting different power detection thresholds for each output port O_i of the multiplier. Additionally, the losses of the splitter and mode multiplexer for different modes, along with the differences in the dynamic range of the MZIs, can lead to variations in the optical power of the same level due to the different mode channels through which the light passes. This can be compensated by normalizing the four MZIs corresponding to the same output port. This means that each MZI's modulation range is constrained within the overall minimum and maximum optical power (Pmin and Pmax) for the four paths, as depicted in Fig. 5a. Within this modulation range, four random numbers were selected to form a modulation vector, which is later subjected to multiplication and addition (MAC) with a fixed input vector. The measured results were compared against the expected analytically calculated multiplication result. The results of 10,000 calculations were scaled to the range [0,1] and plotted together with the corresponding histogram in Fig. 5b, showing a standard deviation of 0.029. Based on the equation $bitdepth={\log }_2\left(1/\left(3\sigma *2\right)\right)$^20,35, a modulation resolution of 2 bits can be derived.

With the modulation accuracy determined, the “level-voltage” lookup table can be obtained for each MZI. Based on the interval of $(P\text{max}-P\text{min})/(2^2-1)$ as the step size, each MZI is configured with corresponding intensity levels and modulation voltages for each level. By traversing all the modulation vectors, the overlap range of output signal ranges for different result elements can be examined for validating the MVM's capability so as to support 4-levels modulation. This approach guarantees the accuracy and consistency of the performance of MVM across all the possible modulation combinations. The measurement results demonstrate that the fabricated MVM can effectively support modulation with the precision of 2 bits, which further confirms the above calculation. This implies that there are totally $\left(2^2-1\right)\times 4+1=13$ possible elements in the result vector. Fig. 6 illustrates the waveforms of the four drive signals, the responses of the dynamic characteristics of each modulator, and the final result of the vector-vector multiplication.

In the characterization system mentioned above, factors that affect the modulation bit resolution also arise from other connected devices, such as the fluctuation of optical power from the light source, the accuracy of voltage loading from the multi-channel voltage source, as well as the stability of the PD. In order to estimate the best bit resolution of the MVM, the general model of finite precision analysis proposed by Li et al³⁵an be utilized, which is based on the assumption that the external devices are ideal and focusing only on the SOI chip. In this case, the extinction ratio (ER) of the modulator is a crucial factor. By inputting the I of 1010 to the MVM and loading 0101 or 1111 signals on each row of the modulation matrix (M_i), the ER of the MVM can be obtained, as is shown in Fig. 7, which varies with wavelength. The utilization of ASE results in an effective ER of the multiplier, which is the average value within the wavelength range of the light source. When employing a non-coherent single-wavelength light source, the ER of the multiplier can be fixed at a higher value specific to that wavelength, which can reach up to 15 dB. Based on the formula $1/\text{ER}<0.5/(2^{\text{bit}\text{depth}}-1)$derived from the model, the MVM can support a precision of 4 bits. To further enhance the accuracy of MDM-MVM, certain components can be optimized within the current structure. For instance, during testing, it was observed that the power fluctuations resulting from thermal crosstalk between modulators account for approximately 2.15% of the MZI's dynamic range, making it a main contributing factor to the overall deviation. In future work, thermo-electric cooler (TEC) can be implemented to reduce the thermal crosstalk. The modulator can also be replaced, such as by using a GeSi waveguide electro-absorption modulator^36,37. The GeSi modulator can mitigate the error caused by the heat conduction of a certain heater to the adjacent heater. It can also effectively increase the modulation rate, thereby increasing the calculation rate.

To achieve larger-scale multiplication in this architecture, it is necessary to evaluate the optical losses introduced by each system component. Based on individual component characterization, the average loss of the 4-mode multiplexer/demultiplexer is -0.44 dB, the 1 × 4 splitter is -9.80 dB, and the 4 × 1 coupler is -6.26 dB. The MVM principle dictates that the loss of the 1 × 4 splitter should be -6 dB, and the losses of other components should be as close to 0 as possible. It is worth noting that the discrepancy between the actual loss and the ideal loss for the coupler is relatively large, which hinders the scalability of the multiplier. It is the structure of the MMI that leads to the fact that only 1/2portnumber of light can enter the output port for each input port. To address the problem above, the approach of utilizing MDM and replacing the MMI with a mode multiplexer can be adopted. This multiplexer combines the four TE₀ signals into a multimode bus waveguide, enabling highly efficient and low-loss beam combining. This approach could improve the signal-to-noise ratio and reduce the power requirements of the light source. Furthermore, edge couplers^38,39are essential for extracting signals from the chip. Alternatively, on-chip integration of a PD array^40,41 allows for direct electrical signal output. In these two signal output schemes, the difference between the coupling efficiency or conversion efficiency of different modes needs particular attention to ensure the accuracy of the results.

To expand the calculation dimension of MDM-MVM, one approach is to continue increasing the number of the multiplexed modes. It has been reported that 11-mode MDM has been achieved⁴². This suggests that, with minimal changes to the existing architecture, the number of multiplexable modes can be increased by adding ADCs for each mode, increasing the number of parallel layers in the splitter's SWG, increasing the number of MZIs, and utilizing an 8 × 1 MMI for an 8-channel multiplier. However, as this method exhibits a clear upper limit to the increase in dimensionality, another approach that can be taken into consideration is to integrate WDM into the architecture. Except the wavelength-sensitive beam combiners, the other components mentioned above are broadband devices that can be directly used in combination with WDM devices. This allows each input element to be characterized by the intensity of light at a specific wavelength and particular mode. The loading of modulation matrix elements can be achieved by either adding wavelength demultiplexer in front of the existing modulator array or replacing the array with a wavelength-selective modulator array^16,43. In the MDM & WDM-MVM architecture, one of the primary limiting factors has become the losses. Assuming that the multiplier scale is 2ⁿ, based on the power of the light source used and the current measured losses of the components, it can be estimated that the output optical power of the system can be expressed as:

In order to ensure that elements 0 to 1 are distinguishable, the ER of the MZI also needs to be considered inPoutput. Based on the minimum detectable power of the PD, it can be deduced that the maximum value of n is 6, which means the current achievable upper limit for the dimensionality is 64.

Compared with other MVM optical signal processors, the MDM-MVM in the current work is endowed with the advantage of having less stringent requirements on the light source. It does not necessitate multiple lasers or microcombs to support multiple wavelengths. Additionally, MDM-MVM possesses a direct execution capability, which eliminates the requirement of intricate algorithms, owing to the one-to-one mapping relationship between modes and matrix elements. The independence between different channels also makes it easier to analyze and subsequently improve crosstalk and losses in the system. The specific comparisons are presented in Table 2.

In the current work, a novel matrix-vector multiplier based on mode division multiplexing was introduced and demonstrated. The core components of the processor include a mode multiplexer, a multimode beam splitter, a mode demultiplexer, a modulator array, and combiners. Efficient 4-level modulation was achieved with the adoption of an off-chip light source and PDs, with each level of the output signal encoding 2 bits of valuable information. The utilization of MDM obviated the necessity for multiple laser light sources and reduced both the system complexity and overall cost. Lasers and detectors can be further integrated on the same chip, resulting in a fully integrated on-chip optical matrix-vector multiplier.

The proposed MDM-MVM was fabricated on an SOI platform with a 220-nm-thick top silicon layer and a 2-µm-thick buried silicon dioxide layer. 248-nm DUV photolithography defines the patterns and inductively coupled plasma reactive ion etching (ICP-RIE) was employed to form the silicon waveguides. Subsequently, a 1-µm thick silica thin film was deposited as the upper cladding. After the deposition, 100-nm thick strip TiN micro heaters were fabricated on top of the heating arm waveguide. Finally, aluminum was deposited and etched to form the metal wires and pads.

Funding This work was supported by National Major Research and Devel- opment Program (2021YFB2801703), National Natural Science Foundation of China (62135011 & 62105286), “Pioneer”and “Leading Goose”R & D Program of Zhejiang (2022C01103), and the Fundamental Research Funds for the Central Universities.

Declaration of Competing Interests The authors declare no competing interests.