`api.types` not found when using pytest

[smokestacklightnin]

Please go to Stack Overflow for help and support:

System information

• Have I written custom code (as opposed to using a stock example script
  provided in TensorFlow Model Analysis): No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 22.04
• TensorFlow Model Analysis installed from (source or binary): source
• TensorFlow Model Analysis version (use command below): 0.47.0.dev
• Python version: 3.9, 3.10
• Jupyter Notebook version:
• Exact command to reproduce: $ pytest

Describe the problem

Before PR #183, running pytest yields several errors. One such repeated error is

FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

This error is repeated for many tests (listed below).

I have included a temporary fix in #183 by including

from tensorflow_model_analysis.api import types

__all__ = [
    "types",
]

to tensorflow_model_analysis/api/__init__.py. This gets rid of the error, but not of the deeper underlying issue.

Source code / logs

The full traceback for the particular example (tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData) above is

encoded = b'QlpoOTFBWSZTWfGpBdoACmP/8P//////////v+//qv////Zr5A4AYAAAAOAJJwfYhkBQAUkKKWWKQFOGiKek0T1T09U9NGmgCDZJ5T2qYamm0Jj1TI8o...Wo108/nlI02xnupRhNuthTljTSAfIjSd7o2REZ4IzMws1nNJkh21YUjBjs4aY3SFDGN1qTL4LmU7aWzo2d/p5Hr3GDG5YT9Oj+LL/8XckU4UJDxqQXaA=='
enable_trace = True, use_zlib = False

    def loads(encoded, enable_trace=True, use_zlib=False):
      """For internal use only; no backwards-compatibility guarantees."""
    
      c = base64.b64decode(encoded)
    
      if use_zlib:
        s = zlib.decompress(c)
      else:
        s = bz2.decompress(c)
    
      del c  # Free up some possibly large and no-longer-needed memory.
    
      with _pickle_lock:
        try:
>         return dill.loads(s)

.venv310/lib/python3.10/site-packages/apache_beam/internal/dill_pickler.py:415: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv310/lib/python3.10/site-packages/dill/_dill.py:275: in loads
    return load(file, ignore, **kwds)
.venv310/lib/python3.10/site-packages/dill/_dill.py:270: in load
    return Unpickler(file, ignore=ignore, **kwds).load()
.venv310/lib/python3.10/site-packages/dill/_dill.py:472: in load
    obj = StockUnpickler.load(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

import_name = 'tensorflow_model_analysis.api.types', safe = False

    def _import_module(import_name, safe=False):
        try:
            if '.' in import_name:
                items = import_name.split('.')
                module = '.'.join(items[:-1])
                obj = items[-1]
            else:
                return __import__(import_name)
>           return getattr(__import__(module, None, None, [obj]), obj)
E           AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

.venv310/lib/python3.10/site-packages/dill/_dill.py:827: AttributeError

During handling of the above exception, another exception occurred:

self = <tensorflow_model_analysis.api.model_eval_lib_test.EvaluateTest testMethod=testAnalyzeRawData>

    def testAnalyzeRawData(self):
    
      # Data
      # age language  label  prediction
      # 17  english   0      0
      # 30  spanish   1      1
      dict_data = [
          {'age': 17, 'language': 'english', 'prediction': 0, 'label': 0},
          {'age': 30, 'language': 'spanish', 'prediction': 1, 'label': 1},
      ]
      df_data = pd.DataFrame(dict_data)
    
      # Expected Output
      expected_slicing_metrics = {
          (('language', 'english'),): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 1.0},
                  }
              }
          },
          (('language', 'spanish'),): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 1.0},
                  }
              }
          },
          (): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 2.0},
                  }
              }
          },
      }
    
      # Actual Output
      eval_config = text_format.Parse(
          """
        model_specs {
          label_key: 'label'
          prediction_key: 'prediction'
        }
        metrics_specs {
          metrics { class_name: "BinaryAccuracy" }
          metrics { class_name: "ExampleCount" }
        }
        slicing_specs {}
        slicing_specs {
          feature_keys: 'language'
        }
      """,
          config_pb2.EvalConfig(),
      )
>     eval_result = model_eval_lib.analyze_raw_data(df_data, eval_config)

tensorflow_model_analysis/api/model_eval_lib_test.py:1483: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tensorflow_model_analysis/api/model_eval_lib.py:1609: in analyze_raw_data
    p
.venv310/lib/python3.10/site-packages/apache_beam/pvalue.py:138: in __or__
    return self.pipeline.apply(ptransform, self)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:681: in apply
    return self.apply(
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:692: in apply
    return self.apply(transform, pvalueish)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:754: in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:191: in apply
    return self.apply_PTransform(transform, input, options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:195: in apply_PTransform
    return transform.expand(input)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:1005: in expand
    return self._fn(pcoll, *args, **kwargs)
tensorflow_model_analysis/api/model_eval_lib.py:1216: in ExtractEvaluateAndWriteResults
    extracts = examples | 'BatchedInputsToExtracts' >> BatchedInputsToExtracts()
.venv310/lib/python3.10/site-packages/apache_beam/pvalue.py:138: in __or__
    return self.pipeline.apply(ptransform, self)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:681: in apply
    return self.apply(
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:692: in apply
    return self.apply(transform, pvalueish)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:754: in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:191: in apply
    return self.apply_PTransform(transform, input, options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:195: in apply_PTransform
    return transform.expand(input)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:1005: in expand
    return self._fn(pcoll, *args, **kwargs)
tensorflow_model_analysis/api/model_eval_lib.py:876: in BatchedInputsToExtracts
    return batched_inputs | 'AddArrowRecordBatchKey' >> beam.Map(to_extracts)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:2109: in Map
    pardo = FlatMap(wrapper, *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:2052: in FlatMap
    pardo = ParDo(CallableWrapperDoFn(fn), *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:1564: in __init__
    super().__init__(fn, *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:870: in __init__
    self.fn = pickler.loads(pickler.dumps(self.fn))
.venv310/lib/python3.10/site-packages/apache_beam/internal/pickler.py:50: in loads
    return desired_pickle_lib.loads(
.venv310/lib/python3.10/site-packages/apache_beam/internal/dill_pickler.py:419: in loads
    return dill.loads(s)
.venv310/lib/python3.10/site-packages/dill/_dill.py:275: in loads
    return load(file, ignore, **kwds)
.venv310/lib/python3.10/site-packages/dill/_dill.py:270: in load
    return Unpickler(file, ignore=ignore, **kwds).load()
.venv310/lib/python3.10/site-packages/dill/_dill.py:472: in load
    obj = StockUnpickler.load(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

import_name = 'tensorflow_model_analysis.api.types', safe = False

    def _import_module(import_name, safe=False):
        try:
            if '.' in import_name:
                items = import_name.split('.')
                module = '.'.join(items[:-1])
                obj = items[-1]
            else:
                return __import__(import_name)
>           return getattr(__import__(module, None, None, [obj]), obj)
E           AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

.venv310/lib/python3.10/site-packages/dill/_dill.py:827: AttributeError

The list of tests with this error is

FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testBytesProcessedCountForRecordBatches - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testBytesProcessedCountForSerializedExamples - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysis - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisMultiMicroAggregationNoHistogram - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisMultiMicroAggregationWithHistogram - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithDeterministicConfidenceIntervals - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithExplicitModelAgnosticPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModelrubber_stamp - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_js - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_keras - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_keras_custom_metrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_lite - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasMultiOutputModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithQueryBasedMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithSchema - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithUncertainty - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithAttributions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithJackknifeAndDiffMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasAndDiffMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasModelWithInGraphMetricscompiled_metrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasModelWithInGraphMetricsevaluate - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testMetricsSpecsCountersInModelAgnosticMode - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_non_cf_multiple_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_non_cf_single_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorwith_example_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorwithout_example_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/features_extractor_test.py::FeaturesExtractorTest::test_features_extractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/features_extractor_test.py::FeaturesExtractorTest::test_features_extractor_no_features - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorwith_label - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorwithout_label - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/materialized_predictions_extractor_test.py::MaterializedPredictionsExtractorTest::test_rekey_predictions_in_features - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testBatchSizeLimitWithKerasModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithKerasModelModelSignaturesDoFnInferenceCallableModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithKerasModelModelSignaturesDoFnInferenceServingDefault - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithSequentialKerasModelModelSignaturesDoFnInferenceCallableModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithSequentialKerasModelModelSignaturesDoFnInferenceServingDefault - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testRekeyPredictionsInFeaturesForPrematerializedPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithCrossSlices - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithEmptySqlConfig - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithMultipleSchema - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelmulti_model_multi_output_batched_examples_batched_inputs - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelmulti_model_single_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelsingle_model_multi_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelsingle_model_single_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel0 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel1 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel10 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel11 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel12 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel13 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel14 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel15 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel2 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel3 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel4 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel6 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel7 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel8 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel9 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractorkeras_custom - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractorkeras_defaults - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractortf_custom - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractortf_defaults - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/example_count_test.py::ExampleCountEnd2EndTest::testExampleCountsWithoutLabelPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_max_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_precision - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_precision_at_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_threshold_at_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_plot_test.py::ObjectDetectionConfusionMatrixPlotTest::testConfusionMatrixPlot - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_ave - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_iou0.5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_iou0.75 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_arlarge - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_armedium - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_arsmall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet1 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet10 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet100 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithSplittedData_average_precision_iou0.5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/score_distribution_plot_test.py::ScoreDistributionPlotTest::testScoreDistributionPlot - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_fp_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_tp_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_two_class - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetricsWithClassWeights_precision_with_class_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetricsWithClassWeights_recall_with_class_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_precision - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_precision_top_k - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall_top_k - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall_top_k_with_threshold_set - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/stats_test.py::MeanEnd2EndTest::testMeanEnd2End - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/stats_test.py::MeanEnd2EndTest::testMeanEnd2EndWithoutExampleWeights - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/utils/example_keras_model_test.py::ExampleModelTest::test_example_keras_model - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteAttributionsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteAttributionstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsNoThresholdsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsNoThresholdstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

[Maheshingle984]

Possible Causes & Solutions

1. Check the TFMA Installation

Since you're installing from source, it's possible that your build is incomplete or dependencies are missing. Try reinstalling TFMA properly:

pip uninstall tensorflow_model_analysis
pip install tensorflow_model_analysis

If you want to install from source again, ensure you're following the correct steps:

git clone https://github.com/tensorflow/model-analysis.git
cd model-analysis
pip install -e .

Then, verify the installation:

import tensorflow_model_analysis as tfma
print(tfma.__version__)

---

2. Check Import Structure in tensorflow_model_analysis.api

Your error message suggests that tensorflow_model_analysis.api.types is missing. Normally, TFMA exposes types through tensorflow_model_analysis.api.types, but if it's not included in the __init__.py, it might not be available.

To confirm:

import tensorflow_model_analysis.api as api
print(dir(api))  # Check if 'types' exists

If types is missing, your fix (adding from tensorflow_model_analysis.api import types to __init__.py) is reasonable, but we need to understand why it’s missing.

---

3. Confirm if types.py Exists in the Expected Location

Go to the tensorflow_model_analysis/api/ directory and check if types.py is present:

ls tensorflow_model_analysis/api/

If it’s missing, you may need to pull the latest code from the repository:

git pull origin main

Then, rebuild and reinstall.

---

4. Run Tests in an Isolated Environment

Conflicting installations of TFMA might cause issues. Run tests in a clean virtual environment:

python -m venv tfma_env
source tfma_env/bin/activate  # On Windows: tfma_env\Scripts\activate
pip install -e .
pytest

If the issue persists, try installing a specific stable version of TFMA instead of the latest development version.

---

5. Investigate PR #183

Since you mentioned PR #183, check its details on GitHub:
🔗 [PR #183 on GitHub](#183)

If this PR introduced breaking changes, consider checking out an earlier version of the repo before PR #183:

git checkout <commit-before-PR-183>

Then, reinstall TFMA and rerun tests.

---

Final Debugging Questions

1. Does tensorflow_model_analysis.api.types exist in dir(tfma.api)?
2. Did PR Use pytest and make GitHub Workflow for tests #183 modify any core API imports?
3. Is the issue happening in a fresh environment?