New request: NDLA - Norwegian Digital Learning Arena

[tronba]

Please use the following format for a ZIM creation request (and delete unnecessary information)

• Website URL: https://ndla.no/
• License: Text in articles is CC-BY or CC-BY-SA
  Many pictures are CC-BY, and some are CC BY-NC or Copyright (metadata shows the license for each object).
  The separate movie section at https://ndla.no/subject:20 is copyrighted.
• Desired ZIM Title: NDLA - Nasjonal digital læringsarena (or Norwegian Digital Learning Arena )
• Desired ZIM Description: NDLA is a Norwegian joint county enterprise offering open digital learning assets for upper secondary education. NDLA offers learning resources in around 150 exam subjects, and is the largest provider of open learning resources in Norway. Resources are primarily in the Norwegian languages, Bokmål and Nynorsk.
• Desired ZIM Icon –png (URL or attach one): https://ndla.no/static/ndla-favicon.png or https://ndla.no/static/logo.png
• Language (ISO 639-3): All articles are both in "nob" and "nno"
• Is this a MediaWiki?: no

[Popolechien]

@tronba Thanks. There's no off-the-shelf way for us to parse out copyrighted images. Would the lessons still work without these illustrations (much like wikipedia without images is still 99% Wikipedia)?

[tronba]

@Popolechien Good day; the copyrighted images are almost exclusively stock photos at the top /middle of articles. There to make things pretty, but not crucial for the content. Where illustrations are needed for the texts, these are mainly custom-made and CC-lisenced, a lot of the "pretty" images are also CC.
The site with no pictures would work (have value), but it would not be the best since it's teaching resources that often refer to things shown in images. The site without the copyrighted pictures removed would work well.

[tronba]

@Popolechien I did a spot-check of 20 random articles from the site. 17 used CC images only. The last 3 had one copyright picture per article (non of them essential for article content).
The site has about 35 000 articles (if I'm not mistaken).

[Popolechien]

@RavanJAltaie can you please make a first pass, and add https://ndla.no/subject:20 in the zimit exclude parameter?

[RavanJAltaie]

https://farm.openzim.org/recipes/ndla_nno_all

[RavanJAltaie]

This is the updated recipe and it's failing
https://farm.openzim.org/recipes/ndla.no_no_all

[RavanJAltaie]

@benoit74 the recipe is failing with error:

node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "crashed".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

Node.js v18.17.1
Traceback (most recent call last):
  File "/usr/bin/zimit", line 566, in <module>
    zimit()
  File "/usr/bin/zimit", line 437, in zimit
    raise subprocess.CalledProcessError(crawl.returncode, cmd_args)
subprocess.CalledProcessError: Command '['crawl', '--failOnFailedSeed', '--waitUntil', 'load', '--title', 'Norwegian Digital Learning', '--description', 'NDLA is a Norwegian joint county enterprise.', '--depth', '-1', '--timeout', '90', '--exclude', 'https://ndla.no/subject:20 ', '--lang', 'nno', '--behaviors', 'autoplay,autofetch,siteSpecific', '--behaviorTimeout', '90', '--diskUtilization', '90', '--url', 'https://ndla.no/', '--userAgent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15 +Zimit contact+zimfarm@kiwix.org', '--cwd', '/output/.tmptk15y3ym', '--statsFilename', '/output/crawl.json']' returned non-zero exit status 1.

Any idea if we can fix this?

[benoit74]

Issue is upstream: openzim/zimit#266