Merge pull request #28 from AmericanPhilosophicalSociety/merge-fillable

Merge wb-fillable and wb-blank, fix formatting issues, etc.

Author: ydshiroma

README.md

@@ -81,7 +81,7 @@ cd C:/Users/username/Desktop/aspace-wb-scripts
 
 ## Prepare necessary files
 
-### Prepare media files (required)
+### Prepare media files (highly recommended)
 
 + Copy your media files into ```files_to_upload```. Do NOT run the script directly on files in the Digital Library Staging Area.
 + Your files should be named according to [CDS guidelines](https://americanphilosophicalsociety.github.io/APS_digitization/metadata/#file).
@@ -101,38 +101,39 @@ If the collection you're ingesting already has a finding aid, you can follow the
 
 This code provides you with a set of command line utilities that can be used to perform various Workbench-related tasks. These are designed to be used in sequence.
 
-Some of these commands take optional **flags**, which give the scripts additional information about how to process your data. Flags are parameters like ```--fields``` or ```--filefolder``` that you can include in your commands, usually followed by some other piece of information such as a file name or variable. For more information and examples, see the sections below.
+Some of these commands take optional **flags**, which give the scripts additional information about how to process your data. Flags are parameters like ```--fields``` or ```--files``` that you can include in your commands, usually followed by some other piece of information such as a file name or variable. For more information and examples, see the sections below.
 
-### Create fillable spreadsheet (```wb-fillable``` or ```wb-blank```)
+### Create fillable spreadsheet (```wb-fillable```)
 
 Creates a simplified version of a Workbench spreadsheet, with some data prepopulated. Data is pulled from media files, with the option to add additional data from an ArchivesSpace bulk update spreadsheet.
 
 | Information | Acceptable input | Required? | Flag | Example
 | --- | --- | --- | --- | --- |
-| Workbench upload type | ```book``` (an object with multiple pages) or ```single``` (a graphic, audio, or video object) | Yes | |  book |
+| Workbench upload type | ```book``` (an object with multiple pages) or ```single``` (a graphic, audio, video, or PDF object) | Yes | |  book |
 | Fields to include | Name of a .csv file containing a list of fields to include. Omit the .csv extension. You can create your own custom list (see below) or use one of the preset options: ```example_minimum_book```, ```example_minimum_single```, ```cnairaudio```, ```cnairbook```, or ```cnairimage```. If no list of fields is specified, all valid Workbench fields will be included. | Recommended | ```--fields```  | example_minimum_book |
 | Bulk update spreadsheet | Name (with .xlsx extension) of an adapted ArchivesSpace bulk update spreadsheet file | Recommended | ```--AS```  | update.xlsx
-| Path to media folder | Location of the folder containing your media files. Only necessary if you haven't copied these files into ```/files_to_upload```. Use forward slashes and if any directory names contain spaces, surround them in quotes. | No | ```--filefolder```  | C:/Users/yshiroma/Desktop/"Files to Upload" |
+| Path to media folder | Location of the folder containing your media files. Only necessary if you haven't copied these files into ```/files_to_upload```. Use forward slashes and if any directory names contain spaces, surround the path in quotes. | No | ```--files```  | "C:/Users/yshiroma/Desktop/Files to Upload" |
+| Output blank sheet? | Use this flag if you want to create a blank Workbench sheet using only the specified fields and field descriptions. This flag does not take any input. | No | ```--blank```  | |
 
 Example command with minimum required information:
 
 ```bash
 wb-fillable book
 ```
 
-Example command with all flags:
+Example command to use a custom set of fields, ArchivesSpace data, and file path:
 
 ```bash
-wb-fillable book --fields fields_file --AS archivesspace_file.xlsx --filefolder C:/Users/username/Desktop/"Folder Name"
+wb-fillable book --fields fields_file --AS archivesspace_file.xlsx --files "C:/Users/username/Desktop/Folder Name"
 ```
 
-**To create a blank spreadsheet with a particular set of fields, without drawing any information from media files** run the following command:
+Example command to create a blank spreadsheet from a particular set of fields, without drawing any information from media files or ArchivesSpace:
 
 ```bash
-wb-blank book --fields fields_file 
+wb-fillable book --fields fields_file --blank
 ```
 
-Check in ```/metadata``` that your output has been created successfully. It should be named ```output_wb-fillable``` or ```output_wb-blank```, depending on which script you used (with 'output' replaced by the name of your ArchivesSpace bulk update sheet if you provided one).
+Check in ```/metadata``` that your output has been created successfully. It should be named ```output_wb-fillable``` (with 'output' replaced by the name of your ArchivesSpace bulk update sheet if you provided one).
 
 ### Manually fill spreadsheet
 
@@ -143,7 +144,7 @@ Fill out the remaining fields according to standard Workbench guidelines, with a
 + ```id```, ```parent_id```, ```field_weight```, ```field_display_hints```, and ```field_metadata_title``` are omitted in ```output_wb_fillable``` because they will be filled in automatically later by ```wb-to-wb```
 + For any field marked "Fills down," you can fill in a value only once and it will be auto-filled to any blank cell below it in that column
 + ```field_language``` can be entered as an ISO639 language name or code
-+ ```field_linked_agent``` is broken out into ```field_linked_agent_NAME```, ```field_linked_agent_ROLE```, and ```field_linked_agent_TYPE```, making it possible to enter these pieces of information separately. If there are multiple linked agents, entries should be pipe-separated.
++ ```field_linked_agent``` is broken out into ```field_linked_agent_NAME```, ```field_linked_agent_RELATOR```, and ```field_linked_agent_TYPE```, making it possible to enter these pieces of information separately. If there are multiple linked agents, entries should be pipe-separated.
 
 For example, consider the following ```field_linked_agent``` entry from a standard Workbench sheet:
 
@@ -154,7 +155,7 @@ The examples below show how this same data would be entered into ```output_wb-fi
 | Field | Description | Example |
 | --- | --- | --- |
 | ```field_linked_agent_NAME``` | Name, in Library of Congress subject heading format | Nussbaum, Martha, 1947-\|American Philosophical Society |
-| ```field_linked_agent_ROLE``` | Relator code, from list of Default Relationship Types | cre\|pbl |
+| ```field_linked_agent_RELATOR``` | Relator code, from list of Default Relationship Types | cre\|pbl |
 | ```field_linked_agent_TYPE``` | Type of linked agent (```person```, ```corporate_body```, or ```family```), or an abbreviation (```p```, ```c```, or ```f```). If left blank, all entires are assumed to be persons. | person\|corporate_body |
 
 
@@ -256,13 +257,6 @@ Testing/sample data is currently lacking. This should include sample media for v
 - There is currently no way to validate field entry against Islandora controlled vocabularies themselves unless they are explicitly downloaded into /vocabularies/ and code added to validate them (in ```wb-validate``` calling a function in utils/validate.py)
 - Agents from ArchivesSpace must currently come from a custom report where the maximum (50k rows, we need something close to 80k) is overridden using the browser inspect tool. An API call could be an improvement. There is currently [a ticket](https://archivesspace.atlassian.net/browse/ANW-2376) with ArchivesSpace to include agents (and all subjects and all other fields) in the spreadsheet.
 
-# To do
-
-- Replace language vocabulary with one that uses ISO639-2B where different to ISO639-3 (~20 cases). Probably necessary to replace the json extraction with something that uses the ISO639 library. Alternatively, have an ISO639-2B vocabulary of these differences and reference that, as they are unlikely to change. Temporary fix was to just edit the iso639.csv file to use 2B for a few languages.
-- To avoid duplication, the functionality of ```wb-blank``` (which is ```wb-fillable``` but bypassing any file metadata) could be incorporated into ```wb-fillable``` using a flag, e.g. --blank. This would require putting the file checking and metadata extraction into dedicated functions.
-- After upgrade to ArchivesSpace v4, adapt Bulk Update Spreadsheet instructions and check for any discrepency between old and new sheets.
-- reorder output of ```wb-create-dos``` so all AS-entry fields are to the right
-
 # Contributing
 
 This library is under active development and welcomes contributions. Please work from the existing issues before submitting a pull request.

aspace_wb/cli/create_blank_fillable.py

@@ -8,12 +8,12 @@
 from aspace_wb.utils import default_specs as c
 from aspace_wb.utils import convert_data
 from aspace_wb.utils import use_CSVs
+import aspace_wb.utils.extract_file as extract_file
 
 '''
 Parse command line arguments
     required, positional: nodes file (csv)
 '''
-print('Checking command line arguments - expected: [nodes csv file] ...')
 
 cl_parser = argparse.ArgumentParser()
 cl_parser.add_argument('nodes_file', type=str, help="Name of your Workbench output CSV (with .csv extension)")
@@ -22,9 +22,7 @@
 
 # check existence of file
 if not os.path.exists(os.path.join(c.METADATA_DIR, NODES_FILE)):
-    raise OSError(f"Workbench output CSV {str(NODES_FILE)} not found in folder {str(c.METADATA_DIR)}. Check file name and location and try again.")
-
-print("... command line arguments okay ...")
+    raise FileNotFoundError(f"Workbench output CSV {str(NODES_FILE)} not found in folder {str(c.METADATA_DIR)}. Check file name and location and try again.")
 
 '''
 Load the nodes
@@ -83,19 +81,9 @@
 output_pandas_DF = pandas.DataFrame(data=output_dict)
 
 # create output filename
-OUTPUT_FILENAME = f"{os.path.splitext(NODES_FILE)[0]}_wb-create-dos"
-OUTPUT_EXTENSION = ".xlsx"
-
-# if file already exists, append a counter to prevent overwriting
-while os.path.exists(os.path.join(c.METADATA_DIR, f"{OUTPUT_FILENAME}{OUTPUT_EXTENSION}")):
-    filename_split = OUTPUT_FILENAME.split("_")
-    if filename_split[-1].isdigit():
-        counter = int(filename_split[-1]) + 1
-        OUTPUT_FILENAME = f"{'_'.join(filename_split[:-1])}_{counter}"
-    else:
-        OUTPUT_FILENAME += "_2"
-
-filename = f"{OUTPUT_FILENAME}{OUTPUT_EXTENSION}"
-output_pandas_DF.to_excel(os.path.join(c.METADATA_DIR, filename), index=False)
-
-print(f'Done. Created file: {os.path.join(c.METADATA_DIR, filename)}')
+FILE_PREFIX = os.path.splitext(NODES_FILE)[0]
+FILE_NAME = extract_file.construct_output_filename(FILE_PREFIX, ".xlsx", "wb-create-dos")
+
+output_pandas_DF.to_excel(os.path.join(c.METADATA_DIR, FILE_NAME), index=False)
+
+print(f'Done. Created file: {os.path.join(c.METADATA_DIR, FILE_NAME)}')