Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
path: coverage
merge-multiple: true
- name: Upload coverage
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
fail_ci_if_error: true
token: ${{ secrets.CODECOV_TOKEN }}
220 changes: 220 additions & 0 deletions docs/how_tos/migrate_v1_libraries_to_v2.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
Merging V1 Library Exports into a V2 Archive

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@irfanuddinahmad Could you share how you generated these details and what sources or references you used? Also, how can I verify that all of this information is accurate?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the question, @salman2013!

This document was generated with Claude Code (AI-assisted), and the primary source I relied on was the V2 backup/restore format specification in openedx/openedx-core#492, which documents the package.toml, per-entity TOML layout, and component_versions/v1/block.xml directory structure. The V1 side (OLX export format: library.xml, <type>/<block_id>/definition.xml, static/) is the existing well-known export produced by Studio's Export feature.

How to verify:

  1. Against the format spec — Cross-check the TOML structure shown in steps 3 and 4e against the schema defined in Document backup/restore format openedx-core#492 (or its merged docs once landed).
  2. End-to-end on a local instance — Follow the test plan in the PR description: export a real V1 library from a local Studio, walk through the merge steps, run lp_load, and confirm the library appears in the V2 library list with all expected components. This is the most reliable verification.
  3. Spot-check the TOML files against an actual V2 backup ZIP exported from a working V2 library instance — the structure should match exactly.

I'm happy to update any steps if you find discrepancies during testing.

=============================================

V1 (legacy) content libraries exported from Studio produce an XML-based OLX
archive. V2 libraries use the ``openedx-core`` backup/restore format — a ZIP
of TOML metadata files plus the same XBlock XML.

This guide explains how a savvy operator can combine one or more V1 OLX
exports into a single V2 backup ZIP, so the content can be imported into a
V2 library without running a migration script.

.. contents:: Contents
:local:
:depth: 2

Background: Format Differences
-------------------------------

.. list-table::
:header-rows: 1
:widths: 20 40 40

* - Aspect
- V1 OLX export
- V2 backup ZIP
* - Container format
- ``.tar.gz`` or ``.zip``
- ``.zip``
* - Library metadata
- ``library.xml`` (XML)
- ``package.toml`` (TOML)
* - Component files
- ``<type>/<block_id>/definition.xml``
- ``entities/xblock.v1/<type>/<uuid>/component_versions/v1/block.xml``
* - Static assets
- ``static/<filename>``
- ``entities/xblock.v1/<type>/<uuid>/component_versions/v1/static/<filename>``
* - Identifiers
- Short ``block_id`` strings
- UUIDs (assigned during merge)
* - Version history
- Not preserved
- Single ``v1`` entry per component after merge
* - Collections
- Not supported
- Can be added manually (optional)

.. note::

After a merge restore, **course content that previously referenced V1 library
blocks via** ``usage_key`` **will not automatically point at the new V2
components**. Those references must be updated separately in each course.

Prerequisites
-------------

* The V1 library export ZIP(s) produced by Studio's *Export* feature.
* Access to the target Open edX instance with ``lp_load`` permissions.
* Python 3.9+ and the ``tomlkit`` package (``pip install tomlkit``) if you
want to generate the TOML files with a script instead of by hand.
* A basic familiarity with ZIP archives (the standard ``zip`` / ``unzip``
command-line tools or any GUI archive manager).

Step-by-Step Merge
------------------

1. Extract all V1 archives
~~~~~~~~~~~~~~~~~~~~~~~~~~

Unzip each V1 library export into its own directory::

unzip v1_library_A.zip -d v1_library_A/
unzip v1_library_B.zip -d v1_library_B/

Each directory will contain at minimum a ``library.xml`` file and one
subdirectory per XBlock type (e.g. ``html/``, ``problem/``, ``video/``).

2. Create the V2 directory skeleton
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

::

mkdir -p v2_library/collections
mkdir -p v2_library/entities/xblock.v1

3. Write ``package.toml``
~~~~~~~~~~~~~~~~~~~~~~~~~

Create ``v2_library/package.toml`` using the metadata from one of the
``library.xml`` files (or supply new values for the merged library):

.. code-block:: toml

[meta]
format_version = 1
created_by = "operator_username"
created_at = 2025-01-01T00:00:00Z

[learning_package]
title = "Merged Library"
key = "lib:MyOrg:MergedLib"
description = "Combined from V1 library A and V1 library B."
created = 2025-01-01T00:00:00Z
updated = 2025-01-01T00:00:00Z

``key`` must be unique on the target instance. Use the pattern
``lib:<organization>:<library_code>``.

4. Convert each V1 component
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For every block in the V1 archives, do the following.

a. **Assign a UUID** — generate one per component, e.g. with Python::

python3 -c "import uuid; print(uuid.uuid4())"

b. **Create the version directory**::

UUID=e32d5479-9492-41f6-9222-550a7346bc37 # use your generated UUID
TYPE=html # e.g. html, problem, video
mkdir -p "v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/static"

c. **Copy the block XML** — the V1 export stores each block's XML at
``<type>/<block_id>/definition.xml`` (or sometimes directly as
``<type>/<block_id>.xml``). Copy it to the V2 path::

cp "v1_library_A/${TYPE}/${BLOCK_ID}/definition.xml" \
"v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/block.xml"
Comment thread
irfanuddinahmad marked this conversation as resolved.
Outdated

The XML content itself is unchanged — V2 uses the same XBlock XML format.

d. **Copy static assets** — any files from ``v1_library_A/static/`` that are
referenced in this block's XML (look for ``/static/<filename>`` or
``static/<filename>``):

cp "v1_library_A/static/diagram.png" \
"v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/static/"

e. **Write the entity TOML** at
``v2_library/entities/xblock.v1/<type>/<uuid>.toml``:

.. code-block:: toml
Comment thread
irfanuddinahmad marked this conversation as resolved.

[entity]
can_stand_alone = true
key = "xblock.v1:html:e32d5479-9492-41f6-9222-550a7346bc37"
created = 2025-01-01T00:00:00Z

[entity.draft]
version_num = 1

[entity.published]
version_num = 1

[[version]]
title = "Untitled"
version_num = 1

Set ``title`` to the ``display_name`` attribute from the block XML if one
is present.

Repeat steps (a)–(e) for every block across all V1 archives.

5. (Optional) Create collections
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want to group blocks from different source libraries, create a TOML
file in ``v2_library/collections/``:

.. code-block:: toml

[collection]
title = "From Library A"
key = "from-library-a"
description = ""
created = 2025-01-01T00:00:00Z
entities = [
"xblock.v1:html:e32d5479-9492-41f6-9222-550a7346bc37",
"xblock.v1:problem:256739e8-c2df-4ced-bd10-8156f6cfa90b",
]

6. ZIP the result
~~~~~~~~~~~~~~~~~~

The ZIP must be created from *inside* the ``v2_library/`` directory so that
``package.toml`` sits at the archive root (not nested under a
``v2_library/`` prefix)::

cd v2_library/
zip -r ../merged_library.zip .
cd ..

Verify the root entry is correct::

unzip -l merged_library.zip | head -5
# Should show: package.toml (not v2_library/package.toml)

7. Load the archive
~~~~~~~~~~~~~~~~~~~~

Use the ``lp_load`` management command on the target instance::

python manage.py lp_load merged_library.zip <username>

Or via the Python API::

from openedx_content.api import load_learning_package
result = load_learning_package("merged_library.zip")

On success, the library will appear in Studio's library list under the key
specified in ``package.toml``.

Further Reading
---------------

* :ref:`backup-restore-format` — full reference for the V2 archive schema
(in the ``openedx-core`` documentation).
* `Legacy Libraries Deprecation <https://openedx.atlassian.net/wiki/spaces/COMM/pages/>`_
Comment thread
irfanuddinahmad marked this conversation as resolved.
Outdated
— deprecation timeline for V1 (legacy) content libraries.