Plan for releasing non-free models?

[samarth4149]

Hi,

The README on non-free models mentions that its current restricted access would be relaxed in the future. Is there a plan for that release sometime soon? A project, Task2Sim that I worked on with IBM cannot be fully replicated without these models.

Thanks!

[alters-mit]

@samarth4149 I'm not working on TDW full time anymore so yes, all models will be free at some point, but not any time soon.

We originally made the models private due to legal concerns that ended up being unfounded. The main reason that the models are still in the private bucket is that they haven't been audited like the ones in tdw-public.

You, specifically, should have the required AWS credentials. You can therefore do something like this:

1. Download all private models using a script like this one:

from pathlib import Path
import boto3
from tdw.librarian import ModelLibrarian

lib_full = ModelLibrarian("models_full.json")
session = boto3.Session(profile_name="tdw")
s3 = session.resource("s3")
for record in lib_full.records:
    # Filter out bad models.
    if record.do_not_use:
        continue
    # Filter for private models.
    is_private = any(["tdw-private" in url for url in record.urls.values()])
    if is_private:
        for platform in record.urls:
            # Download the model.
            key = record.urls[platform].split("https://tdw-private.s3.amazonaws.com/")[1]
            resp = s3.meta.client.get_object(Bucket='tdw-private', Key=key)
            # Write the model to disk.
            directory = Path(record.name).joinpath(platform)
            if not directory.exists():
                directory.mkdir(parents=True)
            directory.joinpath(record.name).write_bytes(resp["Body"].read())

2. Create a ModelLibrarian that copies from models_full.json but with the local asset bundle paths. The paths must be prepended with file://
3. Use that custom library + models to replicate the simulation.

Let me know if you have any problems setting that up.