[Data] Remove get_internal_block_refs

[kyuds]

Description

get_internal_block_refs has been marked deprecated for over a year. We should remove it.

Related issues

N/A

Additional information

N/A

[gemini-code-assist]

Code Review

This pull request removes the deprecated get_internal_block_refs method from the Dataset class and updates the test suite to use iter_internal_ref_bundles as the replacement. I have no feedback to provide.

[kyuds]

@bveeramani daft seems to use this deprecated api so tests are failing...

[bveeramani]

@richardliaw how should we proceed here?

• get_internal_block_refs has been deprecated for two years (with a warning), so we want to remove it.
• Daft's to_ray_dataset depends on the deprecated API https://github.com/Eventual-Inc/Daft/blob/a23a1b722de5db2b5808b6591e290c9a35419d41/daft/runners/ray_runner.py#L451.

[kyuds]

blocked on Daft: Eventual-Inc/Daft#6870

[github-actions]

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[kyuds]

Daft PR is merged, but holding this for now because we need to update Daft dependency for our tests. @richardliaw what do you think about skipping the daft tests for now? Happy to hold this until Daft releases a new version and update dependencies accordingly as well.

[bveeramani]

Holding off seems like the responsible move vs. skipping tests.

@kyuds would you mind pinging the Eventual team and asking them when they're planning to ship their next release? IIRC they release every two weeks or something

[kyuds]

@elliot-barn I need to update Daft to v0.7.15 (their latest version) to pass tests on this deprecation. Could you guide me on how to do this? cc @bveeramani