Integrated liquid chromatography-mass spectrometry and deep learning embedded hyperspectral imager for swift and nondestructive quality control of Ginkgo ester crude material

Ginkgo ester (GBE50) is a well-known botanical drug derived from crude material Ginkgo Folium . Nonetheless, a swift and nondestructive method for full quality inspection on Ginkgo Folium is still lacking. Herein, an interpretable deep learning embedded hyperspectral imaging methodology was established to control the quality of Ginkgo Folium . Hyperspectral images were collected from 102 batches of Ginkgo Folium ranging from 389.8 nm to 1048.2 nm. The contents of three flavonol glycosides and four terpene lactones in Ginkgo Folium were unambiguously determined by well-validated high-performance liquid chromatography (HPLC) and ultra-high performance liquid chromatography-triple quadrupole-mass spectrometry (UHPLC-TQ-MS) methods, whilst the moisture content measured through a moisture meter. In order to realize the real-time full quality inspection, competitive adapative reweighted sampling-partial least squares regression (CARS-PLSR) and seven deep learning algorithms (ResNet50, AlexNet, GoogLeNet, DenseNet121, ResNeXt50, Inception-V4 and Inception-ResNet-V2) were attempted to establish a robust quantitative calibration model embedded hyperspectral imager. As a result, DenseNet121 algorithm showed the most excellent predictive performance. The R p 2 values of quercetin, kaempferol, isorhamnetin, bilobalide, ginkgolide A, ginkgolide B, ginkgolide C, and moisture were determined to be 0.9969, 0.9913, 0.9965, 0.9988, 0.9951, 0.9952, 0.9988 and 0.9981, respectively, and the RMSEP values were 0.0084, 0.0094, 0.0103, 0.0053, 0.0056, 0.0099, 0.0084 and 0.0050, respectively. Grad-CAM was used to unlock the blackbox of DenseNet121model and interpret the model. It’s believed that the established DenseNet121 embedded hyperspectral imager can rapidly and nondestructive determine the eight quality attributes and provide a tool for full quality inspection on Ginkgo Folium .