TFSimilarity.models.ContrastiveModel

Model groups layers into an object with training and inference features.

TFSimilarity.models.ContrastiveModel(
    backbone: tf.keras.Model,
    projector: tf.keras.Model,
    predictor: Optional[tf.keras.Model] = None,
    algorithm: str = simsiam,
    **kwargs
) -> None

Args
inputs	The input(s) of the model: a keras.Input object or list of keras.Input objects.
outputs	The output(s) of the model. See Functional API example below.
name	String, the name of the model.

There are two ways to instantiate a Model:

1 - With the "Functional API", where you start from Input, you chain layer calls to specify the model's forward pass, and finally you create your model from inputs and outputs:

import tensorflow as tf

inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)

Note: Only dicts, lists, and tuples of input tensors are supported. Nested inputs are not supported (e.g. lists of list or dicts of dict).

A new Functional API model can also be created by using the intermediate tensors. This enables you to quickly extract sub-components of the model.

Example:

inputs = keras.Input(shape=(None, None, 3))
processed = keras.layers.RandomCrop(width=32, height=32)(inputs)
conv = keras.layers.Conv2D(filters=2, kernel_size=3)(processed)
pooling = keras.layers.GlobalAveragePooling2D()(conv)
feature = keras.layers.Dense(10)(pooling)

full_model = keras.Model(inputs, feature)
backbone = keras.Model(processed, conv)
activations = keras.Model(conv, feature)

Note that the backbone and activations models are not created with keras.Input objects, but with the tensors that are originated from keras.Inputs objects. Under the hood, the layers and weights will be shared across these models, so that user can train the full_model, and use backbone or activations to do feature extraction. The inputs and outputs of the model can be nested structures of tensors as well, and the created models are standard Functional API models that support all the existing APIs.

2 - By subclassing the Model class: in that case, you should define your layers in init() and you should implement the model's forward pass in call().

import tensorflow as tf

class MyModel(tf.keras.Model):

  def __init__(self):
    super().__init__()
    self.dense1 = tf.keras.layers.Dense(4, activation=tf.nn.relu)
    self.dense2 = tf.keras.layers.Dense(5, activation=tf.nn.softmax)

  def call(self, inputs):
    x = self.dense1(inputs)
    return self.dense2(x)

model = MyModel()

If you subclass Model, you can optionally have a training argument (boolean) in call(), which you can use to specify a different behavior in training and inference:

import tensorflow as tf

class MyModel(tf.keras.Model):

  def __init__(self):
    super().__init__()
    self.dense1 = tf.keras.layers.Dense(4, activation=tf.nn.relu)
    self.dense2 = tf.keras.layers.Dense(5, activation=tf.nn.softmax)
    self.dropout = tf.keras.layers.Dropout(0.5)

  def call(self, inputs, training=False):
    x = self.dense1(inputs)
    if training:
      x = self.dropout(x, training=training)
    return self.dense2(x)

model = MyModel()

Once the model is created, you can config the model with losses and metrics with model.compile(), train the model with model.fit(), or use the model to do prediction with model.predict().

Args
x	examples to use for the calibration.
y	labels associated with the calibration examples.
thresholds_targets	Dict of performance targets to (if possible) meet with respect to the calibration_metric.
calibration_metric	- [ClassificationMetric()](classification_metrics/overview.md) used to evaluate the performance of the index.
k	How many neighboors to use during the calibration. Defaults to 1.
matcher	'match_nearest', 'match_majority_vote' or ClassificationMatch object. Defines the classification matching, e.g., match_nearest will count a True Positive if the query_label is equal to the label of the nearest neighbor and the distance is less than or equal to the distance threshold. Defaults to 'match_nearest'.
extra_metrics	List of additional tf.similarity.classification_metrics.ClassificationMetric() to compute and report. Defaults to ['precision', 'recall'].
rounding	Metric rounding. Default to 2 digits.
verbose	Be verbose and display calibration results. Defaults to 1.

Args
distance	Distance used to compute embeddings proximity. Defaults to 'auto'.
kv_store	How to store the indexed records. Defaults to 'memory'.
search	Which Search() framework to use to perform KNN search. Defaults to 'nmslib'.
evaluator	What type of Evaluator() to use to evaluate index performance. Defaults to in-memory one.
embedding_output	Which model output head predicts the embeddings that should be indexed. Defaults to None which is for single output model. For multi-head model, the callee, usually the SimilarityModel() class is responsible for passing the correct one.
stat_buffer_size	Size of the sliding windows buffer used to compute index performance. Defaults to 1000.

Args
x	Examples to be matched against the index.
y	Label associated with the examples supplied.
k	How many neighbors to use to perform the evaluation. Defaults to 1.
extra_metrics	List of additional tf.similarity.classification_metrics.ClassificationMetric() to compute and report. Defaults to ['precision', 'recall'].
matcher	'match_nearest', 'match_majority_vote' or ClassificationMatch object. Defines the classification matching, e.g., match_nearest will count a True Positive if the query_label is equal to the label of the nearest neighbor and the distance is less than or equal to the distance threshold. verbose (int, optional): Display results if set to 1 otherwise results are returned silently. Defaults to 1.

Raises
IndexError	Index must contain embeddings but is currently empty.
ValueError	Uncalibrated model: run model.calibration()")

Args
x	Examples to be matched against the index.
y	Label associated with the examples supplied.
retrieval_metrics	List of - [RetrievalMetric()](retrieval_metrics/overview.md) to compute. verbose (int, optional): Display results if set to 1 otherwise results are returned silently. Defaults to 1.

Args
x	Samples to index.
y	class ids associated with the data if any. Defaults to None.
store_data	store the data associated with the samples in the key value store. Defaults to True.
build	Rebuild the index after indexing. This is needed to make the new samples searchable. Set it to false to save processing time when calling indexing repeatidly without the need to search between the indexing requests. Defaults to True.
verbose	Output indexing progress info. Defaults to 1.

Args
x	Sample to index.
y	class id associated with the data if any. Defaults to None.
data	store the data associated with the samples in the key value store. Defaults to None.
build	Rebuild the index after indexing. This is needed to make the new samples searchable. Set it to false to save processing time when calling indexing repeatidly without the need to search between the indexing requests. Defaults to True.
verbose	Output indexing progress info. Defaults to 1.

Args
path	Directory where the checkpoint is located.
verbose	Be verbose. Defaults to 1.

Args
x	Samples to match.
k	Number of nearest neighboors to lookup. Defaults to 5.
verbose	display progress. Default to 1.

Args
x	Batch of examples to be matched against the index.
cutpoint	Which calibration threshold to use. Defaults to 'optimal' which is the optimal F1 threshold computed using calibrate().
no_match_label	Which label value to assign when there is no match. Defaults to -1.
k	How many neighboors to use during the calibration. Defaults to 1.
matcher	'match_nearest', 'match_majority_vote' or ClassificationMatch object. Defines the classification matching, e.g., match_nearest will count a True Positive if the query_label is equal to the label of the nearest neighbor and the distance is less than or equal to the distance threshold. verbose. Be verbose. Defaults to 0.

Args
path	directory where to save the index
compression	Store index data compressed. Defaults to True.

Args
x	Sample to match.
k	Number of nearest neighboors to lookup. Defaults to 5.

Files

ContrastiveModel.md

Latest commit

History

ContrastiveModel.md

File metadata and controls

TFSimilarity.models.ContrastiveModel

Args

Example:

Methods

calibrate

create_index

evaluate_classification

evaluate_retrieval

index

index_single

index_size

index_summary

load_index

lookup

match

Notes:

reset_index

save_index

single_lookup

to_data_frame