In critical fields such as automotive, medicine, and defense, ensuring the reliability of microelectronics has been paramount given the extensive nature of their globalized supply chain. Automated visual inspection (AVI) of printed circuit boards (PCBs) offers a solution through computer vision and deep learning to automate defect detection, component verification, and quality assurance. In this paper, our research follows this precedent by introducing a novel dataset and annotations to train artificial intelligence (AI) models for extracting PCB connectivity components. Utilizing high-resolution images, and state-of-the-art instance segmentation models, this study aims to examine the difficulties in this implementation and lay the groundwork for more robust automated visual inspection.

Due to the continuous outsourcing of printed circuit board (PCB) fabrication, PCB counterfeits and Trojans have increased by a significant margin, and this has necessitated rapid and advanced hardware assurance techniques. PCB Image segmentation is the primary step in PCB assurance. Over the years, few PCB component segmentation methods have been proposed and none of those have provided a definite benchmark of performance. Besides those methods haven’t discussed how the performance is correlated with underlying data or annotation quality. In this work, we present a benchmark on PCB image segmentation along with a high-quality dataset. In addition, we explore how annotation quality affects component segmentation and present possible future research directions to work with coarse annotations to alleviate the human effort behind full data annotation tasks. We have analyzed the performance of the preferred Deep Neural Network (DNN) architecture with the data annotation quality and presented the direction to leverage the outcome with limited quality annotations. Finally, we present the qualitative as well as the quantitative results to demonstrate the performance of our techniques and provide observations and future research directions on the overall task.

Antenna-in-packaging (AiP) enables the next generation of high-performance wireless 5G mmWave communication and beyond by incorporating antenna arrays in small form factors using System in Package (SiP) technology. The trend toward heterogeneous integration and advanced packaging will likely introduce more complexity to the semiconductor supply chain. In addition, there is also the risk of becoming more susceptible to security vulnerabilities associated with advanced packaging. This paper provides an overview of the supply chain vulnerabilities in advanced packaging and heterogeneous integration, followed by the existing security, reliability issues, and assurance of AiP. Apart from discussing existing physical modalities of AiP assurance and vulnerabilities, we propose Radio Frequency Fingerprint (RFF) as a new physical modality for AiP assurance. We also discuss possible future research direction and application of RFF in AiP assurance.

The increasing demand for semiconductor chips and the outsourcing of chip fabrication have heightened vulnerability to hardware security threats. While optical probing has been used extensively for semi-invasive/non-invasive attacks, its resolution limits and obsolescence in advanced technologies have necessitated exploring other techniques. Electron-beam probing (EBP) has emerged as a powerful method, offering 20x better spatial resolution than optical probing, and applies to sub- 7nm flip-chips and advanced 3D architecture systems. However, the increased resolution of EBP also poses a threat to sensitive information on these advanced chips, calling for developing countermeasures to secure assets. By understanding the capability of EBP, the potential of using EBP to extract sensitive data such as encryption keys, soft IP, neural network parameters, and proprietary algorithms will be discussed. This paper delves into the principles behind EBP, its capabilities, challenges for this technique, and potential applications in failure analysis and potential attacks. It highlights the need for developing effective countermeasures to protect sensitive information on advanced node technologies.

This presentation addresses the issue of counterfeiting in the semiconductor industry. It begins with a review of the global supply chain and the various forms of counterfeiting taking place. It then identifies assets that require tamper protection and the types of attacks to which they are prone. It also presents several approaches for physical inspection and assurance at the IC and system level.

Probing and imaging techniques that are conventionally used for failure analysis pose a major threat to the confidentiality and the integrity of data stored in non-volatile memory (NVM) cells integrated into a silicon chip. These techniques fall under the umbrella of physical attacks, which unlock tremendous capabilities for an attacker trying to access secret information stored in a target NVM. How vulnerable an NVM cell is to these attacks depends on device physics and the operational principles of the memory cell. The wide range of emerging NVM technologies opens new opportunities for attackers. Without significant attention to these emerging threats, confidential data stored in NVMs can get compromised without much effort, given access to advanced failure analysis tools. We aim to show how attackers can use their knowledge of how a memory device works to find out a suitable probing or imaging modality to extract the stored secret.

This paper evaluates several approaches for automating the identification and classification of logos on printed circuit boards (PCBs) and ICs. It assesses machine learning and computer vision techniques as well as neural network algorithms. It explains how the authors created a representative dataset for machine learning by collecting variants of logos from PCBs and by applying data augmentation techniques. Besides addressing the challenges of image classification, the paper presents the results of experiments using Random Forest classifiers, Bag of Visual Words (BoVW) based on SIFT and ORB Fully Connected Neural Networks (FCN), and Convolutional Neural Network (CNN) architectures. It also discusses edge cases where the algorithms are prone to fail and where potential opportunities exist for future work in PCB logo identification, component authentication, and counterfeit detection. The code for the algorithms along with the dataset incorporating 18 classes of logos and more than 14,000 images is available at this link: https://www.trusthub.org/#/data .

This paper discusses the basic physics of scanning acoustic microscopy, the counterfeit features it can detect, and how it compares with other screening methods. Unlike traditional optical inspection and IR and X-ray techniques, SAM can identify recycled and remarked chips by exposing ghost markings, fill material differences, delaminations from excessive handling, and popcorn fractures caused by trapped moisture. The paper presents several examples along with detailed images of these telltale signs of semiconductor counterfeiting. It also discusses the potential of developing an automated solution for detecting counterfeits on a large scale.

PCB assurance currently relies on manual physical inspection, which is time consuming, expensive and prone to error. In this study, we propose a novel automated segmentation algorithm to detect and isolate PCB components called EC-Seg. Component segmentation and localization is a vital preprocessing step in the automation of component identification and authentication as well as the detection of logos and text markings. As test results indicate, EC-Seg is an efficient solution to automate quality assurance toolchains and also aid bill-of-material (BoM) extraction in PCBs. It also has the potential to be used as a region proposal algorithm for object detection networks and to facilitate sensor fusion involving artifact removal in PCB X-ray tomography.

Semiconductor manufacturing, including the multistep fabrication of ICs and tedious assembly of PCBs, has been outsourced to untrusted regions due to globalization. This invites many problems particularly for PCBs, which are vulnerable to nondestructive methods of attack such as X-ray data collection and surface trace probing. In the case of ICs, high-z materials have proven to be an effective countermeasure to block or scatter X-rays, but PCBs, because of their larger dimensions, are more difficult to fully secure. In this paper, a framework for passively obfuscating the critical connections between components on PCBs is demonstrated. A proof of concept is presented whereby an EDA tool combining the small features of micro electromechanical systems with X-ray simulation and 3D manufacturing processes is used to iteratively optimize a PCB design to thwart reverse engineering and probing attacks.

Object localization is an essential step in image-based hardware assurance applications to navigate the view to the target location. Existing localization methods are well-developed for applications in many other research fields; however, limited study has been conducted to explore an accurate yet efficient solution in hardware assurance domain. To this end, this paper discusses the challenges of leveraging existing object localization methods from three aspects using the example scenario of IC Trojan detection and proposes a novel knowledge-based object localization method. The proposed method is inspired by the 2D string search algorithm; it also couples a mask window to preserve target topology, which enables multi-target localization. Evaluations are conducted on 61 test cases from five images of three node-technologies. The results validate the accuracy, time-efficiency, and the generalizability of the proposed method of locating multi-target from SEM images for hardware assurance applications.

A Bill of Materials (BoM) is the list of all components present on a Printed Circuit Board (PCB). BoMs are useful for multiple forms of failure analysis and hardware assurance. In this paper, we build upon previous work and present an updated framework to automatically extract a BoM from optical images of PCBs in order to keep up to date with technological advancements. This is accomplished by revising the framework to emphasize the role of machine learning and by incorporating domain knowledge of PCB design and hardware Trojans. For accurate machine learning methods, it is critical that the input PCB images are normalized. Hence, we explore the effect of imaging conditions (e.g. camera type, lighting intensity, and lighting color) on component classification, before and after color correction. This is accomplished by collecting PCB images under a variety of imaging conditions and conducting a linear discriminant analysis before and after color checker profile correction, a method commonly used in photography. This paper shows color correction can effectively reduce the intraclass variance of different PCB components, which results in a higher component classification accuracy. This is extremely desirable for machine learning methods, as increased prior knowledge can decrease the number of ground truth images necessary for training. Finally, we detail the future work for data normalization for more accurate automatic BoM extraction. Index Terms – automatic visual inspection; PCB reverse engineering; PCB competitor analysis; hardware assurance; bill of materials

In the hardware assurance community, Reverse Engineering (RE) is considered a key tool and asset in ensuring the security and reliability of Integrated Circuits (IC). However, with the introduction of advanced node technologies, the application of RE to ICs is turning into a daunting task. This is amplified by the challenges introduced by the imaging modalities such as the Scanning Electron Microscope (SEM) used in acquiring images of ICs. One such challenge is the lack of understanding of the influence of noise in the imaging modality along with its detrimental effect on the quality of images and the overall time frame required for imaging the IC. In this paper, we characterize some aspects of the noise in the image along with its primary source. Furthermore, we use this understanding to propose a novel texture-based segmentation algorithm for SEM images called LASRE. The proposed approach is unsupervised, model-free, robust to the presence of noise and can be applied to all layers of the IC with consistent results. Finally, the results from a comparison study is reported, and the issues associated with the approach are discussed in detail. The approach consistently achieved over 86% accuracy in segmenting various layers in the IC.

Printed Circuit Boards (PCBs) play a critical role in everyday electronic systems, therefore the quality and assurance of the functionality for these systems is a topic of great interest to the government and industry. PCB manufacturing has been largely outsourced to cut manufacturing costs in comparison with the designing and testing of PCBs which still retains a large presence domestically. This offshoring of manufacturing has created a surge in the supply chain vulnerability for potential adversaries to garner access and attack a device via a malicious modification. Current hardware assurance and verification methods are based on electrical and optical tests. These tests are limited in the detection of malicious hardware modifications, otherwise known as Hardware Trojans. For PCB manufacturing there has been an increase in the use of automated X-ray inspection. These inspections can validate a PCB’s functionality during production. Such inspections mitigate process errors in real time but are unable to perform highresolution characterization on multi-layer fully assembled PCBs. In this paper, several X-ray reconstruction methods, ranging from proprietary to open-source, are compared. The high-fidelity, commercial NRecon software for SkyScan 2211 Multi-scale X-ray micro-Tomography system is compared to various methods from the ASTRA Toolbox. The latter is an open-source, transparent approach to reconstruction via analytical and iterative methods. The toolbox is based on C++ and MEX file functions with MATLAB and Python wrappers for analysis of PCB samples. In addition, the differences in required imaging parameters and the resultant artifacts generated by planar PCBs are compared to the imaging of cylindrical biological samples. Finally, recommendations are made for improving the ASTRA Toolbox reconstruction results and guidance is given on the appropriate scenarios for each algorithm in the context of hardware assurance for PCBs.

Reverse engineering (RE) is the only foolproof method of establishing trust and assurance in hardware. This is especially important in today's climate, where new threats are arising daily. A Printed Circuit Board (PCB) serves at the heart of virtually all electronic systems and, for that reason, a precious target amongst attackers. Therefore, it is increasingly necessary to validate and verify these hardware boards both accurately and efficiently. When discussing PCBs, the current state-of-the-art is non-destructive RE through X-ray Computed Tomography (CT); however, it remains a predominantly manual process. Our work in this paper aims at paving the way for future developments in the automation of PCB RE by presenting automatic detection of vias, a key component to every PCB design. We provide a via detection framework that utilizes the Hough circle transform for the initial detection, and is followed by an iterative false removal process developed specifically for detecting vias. We discuss the challenges of detecting vias, our proposed solution, and lastly, evaluate our methodology not only from an accuracy perspective but the insights gained through iteratively removing false-positive circles as well. We also compare our proposed methodology to an off-the-shelf implementation with minimal adjustments of Mask R-CNN; a fast object detection algorithm that, although is not optimized for our application, is a reasonable deep learning model to measure our work against. The Mask R-CNN we utilize is a network pretrained on MS COCO followed by fine tuning/training on prepared PCB via images. Finally, we evaluate our results on two datasets, one PCB designed in house and another commercial PCB, and achieve peak results of 0.886, 0.936, 0.973, for intersection over union (IoU), Dice Coefficient, and Structural Similarity Index. These results vastly outperform our tuned implementation of Mask R-CNN.