Int_ML

Int_ML for semiconductor industry with SHAP method

---

Data: Pre-processed data from Intel Minifab AnyLogic Name of data: final_data_values.csv Description of the data:

• 10086 data sets
• 42 features, 1 decision column (pass or fail)

After correlation analysis: 21 features left

---

Steps Conducted in the NoteBook:

0. Intro - Libraries

1. Import Data

2. Preprocess Data 2b. Measure Feature Correlation 2c. Scale/ Normalize/ Standardize the X

3. Train the Dataset with Regression Algorithms 3a. Logistic Regression 3b. K-Nearest Neighbors (Classifier) 3c. Random Forest (Classifier) 3d. Gradient Boosting Decision Tree (Classifier) 3e. Neural Network (MLP Classifier) 3f. Support Vector Machine-Classifier (SVC)

4. Model Selection: Cross Validation

5. Using Shap for Interpretability 5a. Random Forest SHAP 5b. Gradient Boosting Decision Tree SHAP 5c. Logistic Regression SHAP 5d. SVC SHAP 5e. NN/ MLP SHAP 5f. KNN SHAP

6. Interpret SHAP values 6a. Fit to the whole set -- Before Standardizing the X 6b. Fit to the whole set -- After Standardizing the X 6c. Fit to the whole set -- After Modifying the X

---

ML Methods:

3a. Logistic Regression

3b. K-Nearest Neighbors (Classifier)

3c. Random Forest (Classifier)

3d. Gradient Boosting Decision Tree (Classifier)

3e. Neural Network (MLP Classifier)

3f. Support Vector Machine-Classifier (SVC)

---

More Description

Steps 2 We split the dataset randomly, where 20% of the dataset becomes a test set, and 80% of the dataset becomes a train set. It is possible to change this test size. We also present and measure the correlation between variables.

Steps 2b Results: Using Pearson Correlation, there are high correlations among a few features. After eliminating, there are 21 features left. Some of the modifications: Total captures queue, Machine Idle Time, Machine Step Time per week, Machine A utilization all working. See "df14" for more detailed info for feature removal.

Steps 3 Using sklearn, we can import the regression (only logisitcs) and classifier models to train the dataset. We are also using GridSearchCV to find the optimum parameters. We also use 10-fold cross-validation within the grid search, to prevent overfitting.

Steps 4 Cross-validation is a model selection technique that tests how well a model generalizes to an independent data set. The image below shows a k-fold cross validation where packets comprising of 1/k of data alternate to be the test data in every iteration. A score averaging all the test performances can be used to compare various models' performances.

Results: From the previous result, we see that the best models with lowest MAPE are K Nearest Neighbors(KNN) and SVR, nevertheless it is also within ranges with Neural Network. Because they are quite similar in performance. There is nothing wrong with using any of them and choosing the best depending on different datasets and times. We will look at their feature importance ranks, to see which feature contributes the most to the output ($y$).

Steps 5 To interpret the trained models, we are using SHAP (https://github.com/slundberg/shap). Shapley values come from game theory, and explain the allocation of each features to the output (this can be read further here: https://www.nature.com/articles/s42256-019-0138-9.epdf). We can use it on the tree-based algorithms, such as random forest regression and gradient boosting tree regression.

Also, note that you don't need to standardized/ normalized $X$ for partitioning algorithms, including all the tree-based algorithms, such as random forest (RF), or gradient boosting decision tree (GB) regressions. Tree-based algorithms are more robust to numerical instabilities. Since that's the case, we are going to retrain the RF and GB on non-standardized/ non-normalized values.