2023 BrainChip Discussion, page-1050

Robust Classification of Contraband Substances using Longwave Robust Classification of Contraband Substances using Longwave Hyperspectral Imaging and Full Precision and Neuromorphic Hyperspectral
Abstract
Several agencies such as the US Department of Homeland Security (DHS) seek to improve the detection of illegal threats and materials passing through Ports of Entry (POE). A combined hardware/software solution that is portable, non-ionizing, handheld, low cost, and fast would represent a significant contribution towards that goal as existing systems do not fulfil many or all of these requirements. To design such a system, Quantum Ventura partnered with Bodkin Design and Engineering to combine long-wave infrared (LWIR) hyperspectral imaging (HSI) with convolutional neural networks (CNNs), implemented on full precision GPUs and neuromorphic computing modules.
Neuromorphic processors implement CNNs with dramatically reduced size, weight, power and cost (SWaP-C) compared to GPU versions. Here we describe converting the 3D CNN into a format that can be run on neuromorphic platforms. We had early access to BrainChip’s software developer kit (SDK) and simulator thus we focused our efforts
on this. We now have access to Intel Neuromorphic Research Consortium and are using it for other projects [11]. BrainChip can support many features of CNNs but not all. For example, it can only accept grayscale or RGB images, not hyperspectral images (HSIs) for convolutional input layers. (For regular input layers, it may be possible to input HSIs but only 4-bit precision can be used at this time.) Because of this, we had to remap the 61 bands of the HSI image into separate “grayscale” input channels and then fuse across input channels in groups. Furthermore, the skip connections in the original 3D CNN are implemented by copying activation values from one neural processor unit (NPU) to another, and then copying them to the original NPU with identical weights of the value 1. This was the recommendation from BrainChip. The AkidaTM chip has 80 NPUs so using a handful of extra NPUs to implement the skip connections would not prevent neuromorphic implementation [12]. In Figure 5, we show the translated CNN
compatible with the BrainChip hardware.