Skip to content

Fix and export DataProvider in line with lwmdb spec #12

@griff-rees

Description

@griff-rees

Ideally sources get renamed in line with the spec @mialondon provides for lwmdb. Following issues managing this in alto2txt, adding a process here to fix those fields in line for import into lwmdb is preferred.

Tasks:

  • Add a means of only processing these components without needing all other Newspapers data
  • Refactor if necessary
  • Add tests for these results in tests.test_newspapers.py
  • Generate both csv and json results
  • Provide that json fixture for lwmdb

Spec to be implemented:

Fixes

`bna` -> `fmp`
`hmd` -> `bl_hmd`
`lwm` -> `bl_lwm`
  • fmp: FindMyPast-funded digitised newspapers provided by the British Newspaper Archive
  • jisc: JISC-funded digitised newspapers provided by the British Newspaper Archive
  • bl_hmd: British Library-funded digitised newspapers provided by the British Newspaper Archive
  • bl_lwm: Living with Machines-funded digitised newspapers provided by the British Newspaper Archive

Potential final output

[
{"pk": 1, "model": "newspapers.dataprovider", 
"fields": {
    "name": "FindMyPast",
    "code": "fmp",
    "legacy_code": "bna",
    "collection": "newspapers", 
    "source_note": "FindMyPast-funded digitised newspapers provided by the British Newspaper Archive", 
    "created_at": "2023-05-23 15:19:57.795573+00:00", 
    "updated_at": "2023-05-23 15:19:57.795573+00:00"}},
{"pk": 2, "model": "newspapers.dataprovider",
"fields": {
    "name": "hmd",
    "code": "bl_hmd",
    "legacy_code": "hmd"
    "collection": "newspapers", 
    "source_note": "British Library-funded digitised newspapers provided by the British Newspaper Archive", 
    "created_at": "2023-05-23 15:19:57.795573+00:00", 
    "updated_at": "2023-05-23 15: 19:57.795573+00:00"}}, 
{"pk": 3, "model": "newspapers.dataprovider", 
"fields": {
    "name": "Joint Information Systems Committee (JISC)",
    "code": "jisc",
    "legacy_code": "jisc",
    "collection": "newspapers",
    "source_note": "JISC-funded digitised newspapers provided by the British Newspaper Archive",
    "created_at": "2023-05-23 15:19:57.795573+00:00",
    "updated_at": "2023-05-23 15:19:57.795573+00:00"}}, 
{"pk": 4, "model": "newspapers.dataprovider", 
"fields": {
    "name": "Living with Machines",
    "code": "bl_lwm",
    "legacy_code": "lwm",
    "collection": "newspapers",
    "source_note": "Living with Machines-funded digitised newspapers provided by the British Newspaper Archive", 
    "created_at": "2023-05-23 15:19:57.795573+00:00", 
    "updated_at": "2023-05-23 15:19:57.795573+00:00"}}
]

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions