Efficiency-Enhanced Densenet Architectures: An Exploration of Multi-Kernel, Multi-Branch Structures for Achieving Optimal Trade-Off Between Parameters and Accuracy - Shaikh Aga Mohammad, Shaikh Abdus Samad; J., Gitanjali (International Journal of Intelligent Systems and Applications in Engineering, 2024)
In this work, we present intricately woven investigative one-layer structure designs for the Dense-Block of the DenseNet, intending to improve performance in visual recognition tasks. Developing a robust representation for accurate visual recognition is a critical challenge that requires more than just increasing the depth and width of neural networks. Therefore, we have devoted significant effort to developing new One-Layer Structures (OLSs) for the Dense-Block. Our proposed OLSs are comprising multiple branches of stacks of 1×1, 3×3, and 5×5 convolutional layers. We recommend replacing the standard OLS of Dense-Block with one of these proposed OLSs. Our proposed OLSs are lightweight, simple, and optimally arranged, making them an ideal choice for optimizing network performance. We organized them into three families: 1.0 and 1.1, 2.0 to 2.3, and 3.0 to 3.3. To evaluate the effectiveness of our proposed models, we conduct experiments on the three benchmark datasets: Imagenette, CIFAR-10, and CIFAR-100. The investigation of DenseNet models enhanced with OLSs up to version 3.3 provides a nuanced understanding of the intricate relationship between model complexity, computational efficiency, and accuracy. Through meticulous analysis on multiple datasets, a consistent pattern of parameter and FLOP reduction is observed, indicating progressive refinement in model architecture. OLS 2.X versions achieve accuracy values ranging from 94.84% to 95.31% on CIFAR-10, 80.33% to 80.69% on CIFAR-100, and 93.325% to 93.478% on Imagenette demonstrating that the integration of OLSs contributes positively to model performance.