Normalize tf record io

[derrickaw]

Add TFRecordIO normalization for java and python.
This fixes #28692.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[derrickaw]

When running the yaml integration tests, I get this error that I haven't figured out yet how to solve -
FAILED apache_beam/yaml/integration_tests.py::TfrecordTest::test_WriteToTFRecord_InlineProvider_0 - RuntimeError: AssertionError: (BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107(record2=b'foo'), <class 'apache_beam.typehints.schemas.BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107'>) [while running 'WriteToTFRecord/Write/WriteImpl/WriteBundles']

[damccorm]

Because WriteToTFRecord takes in a PCollection of bytes, but yaml returns a PCollection of rows, we need a small conversion layer here - similar to

beam/sdks/python/apache_beam/yaml/yaml_io.py

Line 70 in d030e3f

def write_to_text(pcoll, path: str):

We will probably need a similar construct for reading to map it to rows

[damccorm]

Once this is ready for full review (IMO, this is probably now if we can add a few integration tests). I'd recommend doing a separate PR for read/write since they're not really tied together at all. That will make it easier to review/iterate. It also will hopefully unblock the read review while dealing with the issues you're seeing on write

[damccorm]

When running the yaml integration tests, I get this error that I haven't figured out yet how to solve - FAILED apache_beam/yaml/integration_tests.py::TfrecordTest::test_WriteToTFRecord_InlineProvider_0 - RuntimeError: AssertionError: (BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107(record2=b'foo'), <class 'apache_beam.typehints.schemas.BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107'>) [while running 'WriteToTFRecord/Write/WriteImpl/WriteBundles']

If you have integration test code, could you include that in the draft PR (or whatever follow up PR you have here)? I think it will help to see the full thing.

Noting the InlineProvider piece means that this is an issue with the python provider, which shouldn't be too bad to figure out since its less code.

[derrickaw]

When running the yaml integration tests, I get this error that I haven't figured out yet how to solve - FAILED apache_beam/yaml/integration_tests.py::TfrecordTest::test_WriteToTFRecord_InlineProvider_0 - RuntimeError: AssertionError: (BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107(record2=b'foo'), <class 'apache_beam.typehints.schemas.BeamSchema_c8a75360_f2fc_4d80_9173_2d839bf21107'>) [while running 'WriteToTFRecord/Write/WriteImpl/WriteBundles']

This error is fixed.