Merge pull request #12 from khalil-research/NCE

Docs

Author: LucasBoTang

notebooks/03 Training and Testing.ipynb

@@ -413,7 +413,7 @@
    "id": "eebb2ef2",
    "metadata": {},
    "source": [
-    "The core capability of PyEPO is to build optimization models with GurobiPy, Pyomo, or any other solvers and algorithms, then embed the optimization model into an artificial neural network for the end-to-end training. For this purpose, PyEPO implements **SPO+ loss** and **differentiable Black-Box optimizer**, **differentiable perturbed optimizer**, and **Fenchel-Young loss with Perturbation** as PyTorch autograd modules.\n",
+    "The core capability of PyEPO is to build optimization models with GurobiPy, Pyomo, or any other solvers and algorithms, then embed the optimization model into an artificial neural network for the end-to-end training. For this purpose, PyEPO implements **SPO+ loss** and **differentiable Black-Box optimizer**, **differentiable perturbed optimizer**, **Fenchel-Young loss with Perturbation**, **Noise Contrastive Estimation**, and **Learning to Rank** as PyTorch autograd modules.\n",
     "\n",
     "We will train and test the above aproaches."
    ]
@@ -1436,7 +1436,7 @@
    "id": "7eaab10b",
    "metadata": {},
    "source": [
-    "it uses a Noise Contrastive approach to motivate a family of surrogate loss functions, based on viewing non-optimal solutions as negative examples."
+    "It uses a noise contrastive approach to motivate a family of surrogate loss functions, based on viewing non-optimal solutions as negative examples. For the NCE, the cost vector needs to be predicted from contextual data and maximizes the separation of the probability of the optimal solution."
    ]
   },
   {
@@ -1473,8 +1473,8 @@
     "``pyepo.func.NCE`` allows us to use a noise contrastive estimiation loss for training, which requires parameters:\n",
     "- ``optmodel``: an PyEPO optimization model\n",
     "- ``processes``: number of processors for multi-thread, 1 for single-core, 0 for all of cores\n",
-    "- ``solve-ratio``: a ratio between 0 and 1 that denotes for what proportion of cost vectors predicted during training the instantiated optimization problem should be solved. Whenever the optimization problem is solved, the obtained solution is added to the solution pool which is ranked over.\n",
-    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set."
+    "- ``solve_ratio``: the ratio of new solutions computed during training\n",
+    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set"
    ]
   },
   {
@@ -1661,7 +1661,7 @@
    "id": "2327a93b",
    "metadata": {},
    "source": [
-    "The listwise learning to rank loss measures the difference in how the predicted cost vector and the true cost vector rank a pool of feasible solutions, where listwise ranking measures the scores of the whole ranked lists."
+    "A autograd module for listwise learning to rank, where the goal is to learn an objective function that ranks a pool of feasible solutions correctly. For the listwise LTR, the cost vector needs to be predicted from contextual data, and the loss measures the scores of the whole ranked lists."
    ]
   },
   {
@@ -1703,8 +1703,8 @@
     "``pyepo.func.listwiseLTR`` allows us to use a listwise learning to rank loss for training, which requires parameters:\n",
     "- ``optmodel``: an PyEPO optimization model\n",
     "- ``processes``: number of processors for multi-thread, 1 for single-core, 0 for all of cores\n",
-    "- ``solve-ratio``: a ratio between 0 and 1 that denotes for what proportion of cost vectors predicted during training the instantiated optimization problem should be solved. Whenever the optimization problem is solved, the obtained solution is added to the solution pool which is ranked over.\n",
-    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set."
+    "- ``solve_ratio``: the ratio of new solutions computed during training\n",
+    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set"
    ]
   },
   {
@@ -1911,7 +1911,7 @@
    "id": "7477432c",
    "metadata": {},
    "source": [
-    "The pairwise learning to rank loss measures the difference in how the predicted cost vector and the true cost vector rank a pool of feasible solutions, where pairwise ranking aim to learn the relative ordering of pairs of items."
+    "An autograd module for pairwise learning to rank, where the goal is to learn an objective function that ranks a pool of feasible solutions correctly. For the pairwise LTR, the cost vector needs to be predicted from contextual data, and the loss learns the relative ordering of pairs of items."
    ]
   },
   {
@@ -1950,10 +1950,10 @@
    "id": "27d7caf5",
    "metadata": {},
    "source": [
-    "``pyepo.func.listwiseLTR`` allows us to use a listwise learning to rank loss for training, which requires parameters:\n",
+    "``pyepo.func.pairwiseLTR`` allows us to use a listwise learning to rank loss for training, which requires parameters:\n",
     "- ``optmodel``: an PyEPO optimization model\n",
     "- ``processes``: number of processors for multi-thread, 1 for single-core, 0 for all of cores\n",
-    "- ``solve-ratio``: a ratio between 0 and 1 that denotes for what proportion of cost vectors predicted during training the instantiated optimization problem should be solved. Whenever the optimization problem is solved, the obtained solution is added to the solution pool which is ranked over.\n",
+    "- ``solve_ratio``: the ratio of new solutions computed during training\n",
     "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set."
    ]
   },
@@ -2162,7 +2162,7 @@
    "id": "ab4fef25",
    "metadata": {},
    "source": [
-    "The pointwise learning to rank loss measures the difference in how the predicted cost vector and the true cost vector rank a pool of feasible solutions, where pointwise ranking calculates the ranking scores of the items."
+    "An autograd module for pointwise learning to rank, where the goal is to learn an objective function that ranks a pool of feasible solutions correctly. For the pointwise LTR, the cost vector needs to be predicted from contextual data, and calculates the ranking scores of the items."
    ]
   },
   {
@@ -2201,11 +2201,11 @@
    "id": "05dea9c7",
    "metadata": {},
    "source": [
-    "``pyepo.func.listwiseLTR`` allows us to use a listwise learning to rank loss for training, which requires parameters:\n",
+    "``pyepo.func.pointwiseLTR`` allows us to use a listwise learning to rank loss for training, which requires parameters:\n",
     "- ``optmodel``: an PyEPO optimization model\n",
     "- ``processes``: number of processors for multi-thread, 1 for single-core, 0 for all of cores\n",
-    "- ``solve-ratio``: a ratio between 0 and 1 that denotes for what proportion of cost vectors predicted during training the instantiated optimization problem should be solved. Whenever the optimization problem is solved, the obtained solution is added to the solution pool which is ranked over.\n",
-    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set."
+    "- ``solve_ratio``: the ratio of new solutions computed during training\n",
+    "- ``dataset``: a dataset to initialize the solution pool with. Usually this is simply the training set"
    ]
   },
   {

pkg/pyepo/func/blackbox.py

@@ -15,7 +15,7 @@
 
 class blackboxOpt(optModule):
     """
-    A autograd module for differentiable black-box optimizer, which yield
+    An autograd module for differentiable black-box optimizer, which yield
     optimal a solution and derive a gradient.
 
     For differentiable block-box, the objective function is linear and

pkg/pyepo/func/contrastive.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # coding: utf-8
 """
-Noise Contrastive Estimation Loss function
+Noise contrastive estimation loss function
 """
 
 import numpy as np
@@ -15,9 +15,13 @@
 
 class NCE(optModule):
     """
-        An autograd module for the noise contrastive estimation loss.
-        For the noise contrastive loss, the constraints are known and fixed,
-        but the cost vector needs to be predicted from contextual data.
+    An autograd module for noise contrastive estimation as surrogate loss
+    functions, based on viewing non-optimal solutions as negative examples.
+
+    For the NCE, the cost vector needs to be predicted from contextual data and
+    maximizes the separation of the probability of the optimal solution.
+
+    Thus, allows us to design an algorithm based on stochastic gradient descent.
     """
 
     def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
@@ -26,7 +30,7 @@ def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
             optmodel (optModel): an PyEPO optimization model
             processes (int): number of processors, 1 for single-core, 0 for all of cores
             solve_ratio (float): the ratio of new solutions computed during training
-            dataset (None/optDataset): the training data
+            dataset (None/optDataset): the training data, usually this is simply the training set
         """
         super().__init__(optmodel, processes, solve_ratio, dataset)
         # solution pool

pkg/pyepo/func/perturbed.py

@@ -15,7 +15,7 @@
 
 class perturbedOpt(optModule):
     """
-    A autograd module for differentiable perturbed optimizer, in which random
+    An autograd module for differentiable perturbed optimizer, in which random
     perturbed costs are sampled to optimize.
 
     For the perturbed optimizer, the cost vector need to be predicted from
@@ -133,7 +133,7 @@ def backward(ctx, grad_output):
 
 class perturbedFenchelYoung(optModule):
     """
-    A autograd module for Fenchel-Young loss using perturbation techniques. The
+    An autograd module for Fenchel-Young loss using perturbation techniques. The
     use of the loss improves the algorithmic by the specific expression of the
     gradients of the loss.
 

pkg/pyepo/func/rank.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # coding: utf-8
 """
-Learning To Rank Loss functions
+Learning to rank Losses
 """
 
 import numpy as np
@@ -17,9 +17,13 @@
 
 class listwiseLTR(optModule):
     """
-        An autograd module for the listwise learning to rank loss.
-        For the listwise learning to rank loss, the constraints are known and fixed,
-        but the cost vector needs to be predicted from contextual data.
+    An autograd module for listwise learning to rank, where the goal is to learn
+    an objective function that ranks a pool of feasible solutions correctly.
+
+    For the listwise LTR, the cost vector needs to be predicted from contextual
+    data, and the loss measures the scores of the whole ranked lists.
+
+    Thus, allows us to design an algorithm based on stochastic gradient descent.
     """
 
     def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
@@ -28,7 +32,7 @@ def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
             optmodel (optModel): an PyEPO optimization model
             processes (int): number of processors, 1 for single-core, 0 for all of cores
             solve_ratio (float): the ratio of new solutions computed during training
-            dataset (optDataset): the training data
+            dataset (optDataset): the training data, usually this is simply the training set
         """
         super().__init__(optmodel, processes, solve_ratio, dataset)
         # solution pool
@@ -51,11 +55,12 @@ def forward(self, pred_cost, true_cost, reduction="mean"):
             self.solpool = np.concatenate((self.solpool, sol))
             # remove duplicate
             self.solpool = np.unique(self.solpool, axis=0)
+        # convert tensor
         solpool = torch.from_numpy(self.solpool.astype(np.float32)).to(device)
-        # get obj for solpool
-        objpool_c = true_cost @ solpool.T
-        objpool_cp = pred_cost @ solpool.T
-        # get cross entropy loss
+        # obj for solpool
+        objpool_c = true_cost @ solpool.T # true cost
+        objpool_cp = pred_cost @ solpool.T # pred cost
+        # cross entropy loss
         if self.optmodel.modelSense == EPO.MINIMIZE:
             loss = - (F.log_softmax(objpool_cp, dim=1) *
                       F.softmax(objpool_c, dim=1))
@@ -76,9 +81,13 @@ def forward(self, pred_cost, true_cost, reduction="mean"):
 
 class pairwiseLTR(optModule):
     """
-        An autograd module for the pairwise learning to rank loss.
-        For the pairwise learning to rank loss, the constraints are known and fixed,
-        but the cost vector needs to be predicted from contextual data.
+    An autograd module for pairwise learning to rank, where the goal is to learn
+    an objective function that ranks a pool of feasible solutions correctly.
+
+    For the pairwise LTR, the cost vector needs to be predicted from contextual
+    data, and the loss learns the relative ordering of pairs of items.
+
+    Thus, allows us to design an algorithm based on stochastic gradient descent.
     """
 
     def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
@@ -112,24 +121,24 @@ def forward(self, pred_cost, true_cost, reduction="mean"):
             self.solpool = np.unique(self.solpool, axis=0)
         # convert tensor
         solpool = torch.from_numpy(self.solpool.astype(np.float32)).to(device)
-        # get obj for solpool
-        objpool_c = torch.einsum("bd,nd->bn", true_cost, solpool)
-        objpool_cp = torch.einsum("bd,nd->bn", pred_cost, solpool)
+        # obj for solpool
+        objpool_c = torch.einsum("bd,nd->bn", true_cost, solpool) # true cost
+        objpool_cp = torch.einsum("bd,nd->bn", pred_cost, solpool) # pred cost
         # init relu as max(0,x)
         relu = nn.ReLU()
         # init loss
         loss = []
         for i in range(len(pred_cost)):
-            # get best
+            # best sol
             if self.optmodel.modelSense == EPO.MINIMIZE:
                 best_ind = torch.argmin(objpool_c[i])
             if self.optmodel.modelSense == EPO.MAXIMIZE:
                 best_ind = torch.argmax(objpool_c[i])
             objpool_cp_best = objpool_cp[i, best_ind]
-            # get rest
+            # rest sol
             rest_ind = [j for j in range(len(objpool_cp[i])) if j != best_ind]
             objpool_cp_rest = objpool_cp[i, rest_ind]
-            # get loss
+            # best vs rest loss
             if self.optmodel.modelSense == EPO.MINIMIZE:
                 loss.append(relu(objpool_cp_best - objpool_cp_rest).mean())
             if self.optmodel.modelSense == EPO.MAXIMIZE:
@@ -149,9 +158,14 @@ def forward(self, pred_cost, true_cost, reduction="mean"):
 
 class pointwiseLTR(optModule):
     """
-        An autograd module for the pointwise learning to rank loss.
-        For the pointwise learning to rank loss, the constraints are known and fixed,
-        but the cost vector needs to be predicted from contextual data.
+    An autograd module for pointwise learning to rank, where the goal is to
+    learn an objective function that ranks a pool of feasible solutions
+    correctly.
+
+    For the pointwise LTR, the cost vector needs to be predicted from contextual
+    data, and calculates the ranking scores of the items.
+
+    Thus, allows us to design an algorithm based on stochastic gradient descent.
     """
 
     def __init__(self, optmodel, processes=1, solve_ratio=1, dataset=None):
@@ -185,10 +199,10 @@ def forward(self, pred_cost, true_cost, reduction="mean"):
             self.solpool = np.unique(self.solpool, axis=0)
         # convert tensor
         solpool = torch.from_numpy(self.solpool.astype(np.float32)).to(device)
-        # get obj for solpool as score
-        objpool_c = true_cost @ solpool.T
-        objpool_cp = pred_cost @ solpool.T
-        # get squared loss
+        # obj for solpool as score
+        objpool_c = true_cost @ solpool.T # true cost
+        objpool_cp = pred_cost @ solpool.T # pred cost
+        # squared loss
         loss = (objpool_c - objpool_cp).square()
         # reduction
         if reduction == "mean":

pkg/pyepo/func/spoplus.py

@@ -14,7 +14,7 @@
 
 class SPOPlus(optModule):
     """
-    A autograd module for SPO+ Loss, as a surrogate loss function of SPO Loss,
+    An autograd module for SPO+ Loss, as a surrogate loss function of SPO Loss,
     which measures the decision error of optimization problem.
 
     For SPO/SPO+ Loss, the objective function is linear and constraints are