-
Notifications
You must be signed in to change notification settings - Fork 202
docs: add how-to for merging V1 library OLX exports into a V2 ZIP #3087
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
irfanuddinahmad
wants to merge
4
commits into
openedx:master
Choose a base branch
from
irfanuddinahmad:irfanuddinahmad/document-v1-v2-export-format-2877
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+230
−1
Open
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
144b301
docs: add how-to for merging V1 library OLX exports into a V2 ZIP
214bb68
chore: bump codecov action to v7
172a6d5
docs: fix Copilot review issues in V1-to-V2 migration guide
25b344b
docs: replace Sphinx-only :ref: with plain text cross-reference
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,220 @@ | ||
| Merging V1 Library Exports into a V2 Archive | ||
| ============================================= | ||
|
|
||
| V1 (legacy) content libraries exported from Studio produce an XML-based OLX | ||
| archive. V2 libraries use the ``openedx-core`` backup/restore format — a ZIP | ||
| of TOML metadata files plus the same XBlock XML. | ||
|
|
||
| This guide explains how a savvy operator can combine one or more V1 OLX | ||
| exports into a single V2 backup ZIP, so the content can be imported into a | ||
| V2 library without running a migration script. | ||
|
|
||
| .. contents:: Contents | ||
| :local: | ||
| :depth: 2 | ||
|
|
||
| Background: Format Differences | ||
| ------------------------------- | ||
|
|
||
| .. list-table:: | ||
| :header-rows: 1 | ||
| :widths: 20 40 40 | ||
|
|
||
| * - Aspect | ||
| - V1 OLX export | ||
| - V2 backup ZIP | ||
| * - Container format | ||
| - ``.tar.gz`` or ``.zip`` | ||
| - ``.zip`` | ||
| * - Library metadata | ||
| - ``library.xml`` (XML) | ||
| - ``package.toml`` (TOML) | ||
| * - Component files | ||
| - ``<type>/<block_id>/definition.xml`` | ||
| - ``entities/xblock.v1/<type>/<uuid>/component_versions/v1/block.xml`` | ||
| * - Static assets | ||
| - ``static/<filename>`` | ||
| - ``entities/xblock.v1/<type>/<uuid>/component_versions/v1/static/<filename>`` | ||
| * - Identifiers | ||
| - Short ``block_id`` strings | ||
| - UUIDs (assigned during merge) | ||
| * - Version history | ||
| - Not preserved | ||
| - Single ``v1`` entry per component after merge | ||
| * - Collections | ||
| - Not supported | ||
| - Can be added manually (optional) | ||
|
|
||
| .. note:: | ||
|
|
||
| After a merge restore, **course content that previously referenced V1 library | ||
| blocks via** ``usage_key`` **will not automatically point at the new V2 | ||
| components**. Those references must be updated separately in each course. | ||
|
|
||
| Prerequisites | ||
| ------------- | ||
|
|
||
| * The V1 library export ZIP(s) produced by Studio's *Export* feature. | ||
| * Access to the target Open edX instance with ``lp_load`` permissions. | ||
| * Python 3.9+ and the ``tomlkit`` package (``pip install tomlkit``) if you | ||
| want to generate the TOML files with a script instead of by hand. | ||
| * A basic familiarity with ZIP archives (the standard ``zip`` / ``unzip`` | ||
| command-line tools or any GUI archive manager). | ||
|
|
||
| Step-by-Step Merge | ||
| ------------------ | ||
|
|
||
| 1. Extract all V1 archives | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Unzip each V1 library export into its own directory:: | ||
|
|
||
| unzip v1_library_A.zip -d v1_library_A/ | ||
| unzip v1_library_B.zip -d v1_library_B/ | ||
|
|
||
| Each directory will contain at minimum a ``library.xml`` file and one | ||
| subdirectory per XBlock type (e.g. ``html/``, ``problem/``, ``video/``). | ||
|
|
||
| 2. Create the V2 directory skeleton | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| :: | ||
|
|
||
| mkdir -p v2_library/collections | ||
| mkdir -p v2_library/entities/xblock.v1 | ||
|
|
||
| 3. Write ``package.toml`` | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Create ``v2_library/package.toml`` using the metadata from one of the | ||
| ``library.xml`` files (or supply new values for the merged library): | ||
|
|
||
| .. code-block:: toml | ||
|
|
||
| [meta] | ||
| format_version = 1 | ||
| created_by = "operator_username" | ||
| created_at = 2025-01-01T00:00:00Z | ||
|
|
||
| [learning_package] | ||
| title = "Merged Library" | ||
| key = "lib:MyOrg:MergedLib" | ||
| description = "Combined from V1 library A and V1 library B." | ||
| created = 2025-01-01T00:00:00Z | ||
| updated = 2025-01-01T00:00:00Z | ||
|
|
||
| ``key`` must be unique on the target instance. Use the pattern | ||
| ``lib:<organization>:<library_code>``. | ||
|
|
||
| 4. Convert each V1 component | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| For every block in the V1 archives, do the following. | ||
|
|
||
| a. **Assign a UUID** — generate one per component, e.g. with Python:: | ||
|
|
||
| python3 -c "import uuid; print(uuid.uuid4())" | ||
|
|
||
| b. **Create the version directory**:: | ||
|
|
||
| UUID=e32d5479-9492-41f6-9222-550a7346bc37 # use your generated UUID | ||
| TYPE=html # e.g. html, problem, video | ||
| mkdir -p "v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/static" | ||
|
|
||
| c. **Copy the block XML** — the V1 export stores each block's XML at | ||
| ``<type>/<block_id>/definition.xml`` (or sometimes directly as | ||
| ``<type>/<block_id>.xml``). Copy it to the V2 path:: | ||
|
|
||
| cp "v1_library_A/${TYPE}/${BLOCK_ID}/definition.xml" \ | ||
| "v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/block.xml" | ||
|
irfanuddinahmad marked this conversation as resolved.
Outdated
|
||
|
|
||
| The XML content itself is unchanged — V2 uses the same XBlock XML format. | ||
|
|
||
| d. **Copy static assets** — any files from ``v1_library_A/static/`` that are | ||
| referenced in this block's XML (look for ``/static/<filename>`` or | ||
| ``static/<filename>``): | ||
|
|
||
| cp "v1_library_A/static/diagram.png" \ | ||
| "v2_library/entities/xblock.v1/${TYPE}/${UUID}/component_versions/v1/static/" | ||
|
|
||
| e. **Write the entity TOML** at | ||
| ``v2_library/entities/xblock.v1/<type>/<uuid>.toml``: | ||
|
|
||
| .. code-block:: toml | ||
|
irfanuddinahmad marked this conversation as resolved.
|
||
|
|
||
| [entity] | ||
| can_stand_alone = true | ||
| key = "xblock.v1:html:e32d5479-9492-41f6-9222-550a7346bc37" | ||
| created = 2025-01-01T00:00:00Z | ||
|
|
||
| [entity.draft] | ||
| version_num = 1 | ||
|
|
||
| [entity.published] | ||
| version_num = 1 | ||
|
|
||
| [[version]] | ||
| title = "Untitled" | ||
| version_num = 1 | ||
|
|
||
| Set ``title`` to the ``display_name`` attribute from the block XML if one | ||
| is present. | ||
|
|
||
| Repeat steps (a)–(e) for every block across all V1 archives. | ||
|
|
||
| 5. (Optional) Create collections | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| If you want to group blocks from different source libraries, create a TOML | ||
| file in ``v2_library/collections/``: | ||
|
|
||
| .. code-block:: toml | ||
|
|
||
| [collection] | ||
| title = "From Library A" | ||
| key = "from-library-a" | ||
| description = "" | ||
| created = 2025-01-01T00:00:00Z | ||
| entities = [ | ||
| "xblock.v1:html:e32d5479-9492-41f6-9222-550a7346bc37", | ||
| "xblock.v1:problem:256739e8-c2df-4ced-bd10-8156f6cfa90b", | ||
| ] | ||
|
|
||
| 6. ZIP the result | ||
| ~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| The ZIP must be created from *inside* the ``v2_library/`` directory so that | ||
| ``package.toml`` sits at the archive root (not nested under a | ||
| ``v2_library/`` prefix):: | ||
|
|
||
| cd v2_library/ | ||
| zip -r ../merged_library.zip . | ||
| cd .. | ||
|
|
||
| Verify the root entry is correct:: | ||
|
|
||
| unzip -l merged_library.zip | head -5 | ||
| # Should show: package.toml (not v2_library/package.toml) | ||
|
|
||
| 7. Load the archive | ||
| ~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Use the ``lp_load`` management command on the target instance:: | ||
|
|
||
| python manage.py lp_load merged_library.zip <username> | ||
|
|
||
| Or via the Python API:: | ||
|
|
||
| from openedx_content.api import load_learning_package | ||
| result = load_learning_package("merged_library.zip") | ||
|
|
||
| On success, the library will appear in Studio's library list under the key | ||
| specified in ``package.toml``. | ||
|
|
||
| Further Reading | ||
| --------------- | ||
|
|
||
| * :ref:`backup-restore-format` — full reference for the V2 archive schema | ||
| (in the ``openedx-core`` documentation). | ||
| * `Legacy Libraries Deprecation <https://openedx.atlassian.net/wiki/spaces/COMM/pages/>`_ | ||
|
irfanuddinahmad marked this conversation as resolved.
Outdated
|
||
| — deprecation timeline for V1 (legacy) content libraries. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@irfanuddinahmad Could you share how you generated these details and what sources or references you used? Also, how can I verify that all of this information is accurate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the question, @salman2013!
This document was generated with Claude Code (AI-assisted), and the primary source I relied on was the V2 backup/restore format specification in openedx/openedx-core#492, which documents the
package.toml, per-entity TOML layout, andcomponent_versions/v1/block.xmldirectory structure. The V1 side (OLX export format:library.xml,<type>/<block_id>/definition.xml,static/) is the existing well-known export produced by Studio's Export feature.How to verify:
lp_load, and confirm the library appears in the V2 library list with all expected components. This is the most reliable verification.I'm happy to update any steps if you find discrepancies during testing.