You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<p><strong>How it works:</strong> The vessel trip report data was received as a ZIP folder with several CSV files, a separte file for each year of interest. This function reads these files and combines them into a single data frame. For consistency across each file, <code>vtrserno</code> is converted into a character vector, while <code>year</code>, <code>sub_trip_id</code>, <code>calc_lat_deg</code>, <code>calc_lat_min</code>, <code>calc_lat_sec</code>, <code>calc_lon_deg</code>, <code>calc_lon_min</code>, <code>calc_lon_sec</code>, and <code>calc_inshr_area</code> are all forced as numerics.</p>
2316
2316
<p><code>calc_lat_deg</code>, <code>calc_lat_min</code>, <code>calc_lat_sec</code> and converted and combined to create a decimal degree measurement for latitude, and the same is done for longitude. From there, any states outside GARFO jurisdiction are removed (Texas, Alabama, etc.) and trips are filtered to the area between -60 and -80 degrees west and 20 degrees north.</p>
2317
2317
<p>The data that is returned includes all that is shown above, and <code>sub_trip_id</code> as the unique trip identifier.</p>
<p>*Trip ID information removed from demo to maintain confidentiality.</p>
2403
-
<p><strong>How it works:</strong> The fisheries observer information was received from the **Northeast Fisheries Science Center* as a Microsoft Excel Workbook. The data spans 1989 to 2024 and contains trip, haul and catch data from that time frame. The haul data is broken up into two separate sheets within the excel workbook. Catch data is stored within individual sheets for each year of the data. All trip data is contained within one sheet.</p>
2404
+
<p><strong>How it works:</strong> The fisheries observer information was received from the <strong>Northeast Fisheries Science Center</strong> as a Microsoft Excel Workbook. The data spans 1989 to 2024 and contains trip, haul and catch data from that time frame. The haul data is broken up into two separate sheets within the excel workbook. Catch data is stored within individual sheets for each year of the data. All trip data is contained within one sheet.</p>
2404
2405
<p>From the <code>readxl</code> package, <code>excel_sheets()</code> to read in the entire workbook, and map <code>read_excel()</code> to read in each sheet. The two haul data sheets are combined to create a singular data frame. Catch data from 1989-1995 are missing a year column, and so one is created from the link column in order to combine these years with subsequent years of data. Haul data and catch data are joined by the <code>link3</code> column.</p>
2405
2406
<p>Because coordinates are recorded at different points depending on the type of gear used on the trip, an intermediate data set is created to capture the proper recorded coordinates to each trip. This list is then joined back to the combined catch-haul data set to ensure accurate coordinates based on trip type.</p>
<p><strong>How it works:</strong> The trawl data is received from the Northeast Fisheries Science Center as a .Rdata file pulled from an SQL database. <code>survdat</code> is extracting from the larger list of data and is the basis of the data used here. <code>survdat</code> is combined with a previously built species list to add the common names of survey species.</p>
2487
+
<p><strong>How it works:</strong> The trawl data is received from the <strong>Northeast Fisheries Science Center</strong> as a .Rdata file pulled from an SQL database. <code>survdat</code> is extracting from the larger list of data and is the basis of the data used here. <code>survdat</code> is combined with a previously built species list to add the common names of survey species.</p>
2486
2488
<p>From there, a unique tow ID is built based on the cruise, station, and strata surveyed. A date column is added based on the estimate month and day of the survey.</p>
2487
2489
<p>Observations where there is a mismatch in abundance and biomass are revised so that when biomass is 0 but abundance is greater than 0, the biomass is recorded as 0.0001 kg and when abundance is 0 but biomass is greather than 0, the abundance is recorded as 1.</p>
2488
2490
<p>Strata and species not regularly sampled are then filtered out from the data, and the data subset to begin at 1970, when the trawl began to run consistently.</p>
<h3 class="anchored" data-anchor-id="garfo-federal-permits">GARFO Federal Permits</h3>
@@ -2570,6 +2573,11 @@ <h3 class="anchored" data-anchor-id="garfo-federal-permits">GARFO Federal Permit
2570
2573
</table>
2571
2574
</div>
2572
2575
</div>
2576
+
<p><strong>How it works:</strong> Federal permits data is received as a series of individual Excel spreadsheets, one for each year. This function is designed to read in multiple Excel files from a single source folder and combine them into one data frame.</p>
2577
+
<p>Permits are grouped by species, in that each species is represented by a single column and each permits type represented by a number, letter, or some combination of both. If an individual holds multiple permits types/endorsements, a single entry is made and those values are separated by a comma within a single cell. As such, this function breaks up each column into the respective number of permits types, and renamed with the target species name and permit type (ex: <code>black_sea_bass_1</code> and <code>black_sea_bass_2</code>). The all permit type columns are then pivoted, so that each permit application number has a corresponding permit type and value <code>1</code> for that permit.</p>
2578
+
<p>From there, the names of target species are cleaned and added as a separate column, for later species-level grouping. An additional column is added based on whether a permit type is categorized as <code>commercial</code>, <code>for-hire/charter</code> or <code>recreational</code>. Because we are interested in also accessing species-level trends, permits that cover multiple species (<code>multispecies</code> and <code>squid/mackerel/butterfish</code>) have been parsed out so that each specific permit type corresponds with it’s target species.</p>
2579
+
<p>Lastly, permits are geocoded using <code>tidygeocoder</code> to their reported principal port and grouped to their respective management council region. Roughly 1% of entries are lost due to misspelled principal ports.</p>
<p><strong>How it works:</strong> Confidential landings data was received along with the vessel trip reports as an Excel spreadsheet. The first 9 lines contained metadata as provided by <strong>GARFO</strong>. As such, this function uses <code>readxl</code> to read in the file and requires a <code>skip</code> argument to skip the first few rows that contain metadata. <code>SwimmeR</code> is used to clean the species names and <code>tidygeocoder</code> to geocode the ports associated with landings.</p>
<p><strong>How it works:</strong> The function to pull in and clean the MRIP directed trips is arguably the most complex of the data cleaning functions. The data is hosted online and is downloadable as a ZIP file for individual years. This function pulls in the zipped files, unzips them and saves them locally in intermediate files, which are required for the trip estimation code. Once the data has been pulled in and saved to the intermediary files, the trip estimates are calculated using <a href="gary.nelson@state.ma.us">Gary Nelson</a>’s <a href="https://www.fisheries.noaa.gov/recreational-fishing-data/recreational-fishing-data-downloads">MRIP directed trip estimate R template</a>. This code estimates the number of directed trips by species, wave, domain, trip type and year.</p>
2764
+
<p>For this analyses, we used all waves, primary target trip type, all states along the east coast, and calculate estimates for Atlantic croaker, Atlantic mackerel, black sea bass, blueline tilefish, bluefish, gray triggerfish, king mackerel, Spanish mackerel, striped bass, summer flounder, scup, spiny dogfish, goosefish (monkfish) and tilefish (golden tilefish).</p>
<p><strong>How it works:</strong> Catch estimates are publically available on the <a href="https://safis.accsp.org/accsp_prod/f?p=1490:1:1685966511435:::::">ACCSP Data Warehouse</a> and downloaded as a CSV File. This function reads in the CSV and cleans the species names to be uniform with other data sets.</p>
\*Trip ID information removed from demo to maintain confidentiality.
112
114
113
-
**How it works:** The fisheries observer information was received from the **Northeast Fisheries Science Center* as a Microsoft Excel Workbook. The data spans 1989 to 2024 and contains trip, haul and catch data from that time frame. The haul data is broken up into two separate sheets within the excel workbook. Catch data is stored within individual sheets for each year of the data. All trip data is contained within one sheet.
115
+
**How it works:** The fisheries observer information was received from the **Northeast Fisheries Science Center** as a Microsoft Excel Workbook. The data spans 1989 to 2024 and contains trip, haul and catch data from that time frame. The haul data is broken up into two separate sheets within the excel workbook. Catch data is stored within individual sheets for each year of the data. All trip data is contained within one sheet.
114
116
115
-
From the `readxl` package, `excel_sheets()` to read in the entire workbook, and map `read_excel()` to read in each sheet. The two haul data sheets are combined to create a singular data frame. Catch data from 1989-1995 are missing a year column, and so one is created from the link column in order to combine these years with subsequent years of data. Haul data and catch data are joined by the `link3` column.
117
+
From the `readxl` package, `excel_sheets()` to read in the entire workbook, and map `read_excel()` to read in each sheet. The two haul data sheets are combined to create a singular data frame. Catch data from 1989-1995 are missing a year column, and so one is created from the link column in order to combine these years with subsequent years of data. Haul data and catch data are joined by the `link3` column.
116
118
117
-
Because coordinates are recorded at different points depending on the type of gear used on the trip, an intermediate data set is created to capture the proper recorded coordinates to each trip. This list is then joined back to the combined catch-haul data set to ensure accurate coordinates based on trip type.
119
+
Because coordinates are recorded at different points depending on the type of gear used on the trip, an intermediate data set is created to capture the proper recorded coordinates to each trip. This list is then joined back to the combined catch-haul data set to ensure accurate coordinates based on trip type.
**How it works:** The trawl data is received from the Northeast Fisheries Science Center as a .Rdata file pulled from an SQL database. `survdat` is extracting from the larger list of data and is the basis of the data used here. `survdat` is combined with a previously built species list to add the common names of survey species.
136
+
**How it works:** The trawl data is received from the **Northeast Fisheries Science Center** as a .Rdata file pulled from an SQL database. `survdat` is extracting from the larger list of data and is the basis of the data used here. `survdat` is combined with a previously built species list to add the common names of survey species.
134
137
135
-
From there, a unique tow ID is built based on the cruise, station, and strata surveyed. A date column is added based on the estimate month and day of the survey.
138
+
From there, a unique tow ID is built based on the cruise, station, and strata surveyed. A date column is added based on the estimate month and day of the survey.
136
139
137
-
Observations where there is a mismatch in abundance and biomass are revised so that when biomass is 0 but abundance is greater than 0, the biomass is recorded as 0.0001 kg and when abundance is 0 but biomass is greather than 0, the abundance is recorded as 1.
140
+
Observations where there is a mismatch in abundance and biomass are revised so that when biomass is 0 but abundance is greater than 0, the biomass is recorded as 0.0001 kg and when abundance is 0 but biomass is greather than 0, the abundance is recorded as 1.
138
141
139
142
Strata and species not regularly sampled are then filtered out from the data, and the data subset to begin at 1970, when the trawl began to run consistently.
**How it works:** Federal permits data is received as a series of individual Excel spreadsheets, one for each year. This function is designed to read in multiple Excel files from a single source folder and combine them into one data frame.
165
+
166
+
Permits are grouped by species, in that each species is represented by a single column and each permits type represented by a number, letter, or some combination of both. If an individual holds multiple permits types/endorsements, a single entry is made and those values are separated by a comma within a single cell. As such, this function breaks up each column into the respective number of permits types, and renamed with the target species name and permit type (ex: `black_sea_bass_1` and `black_sea_bass_2`). The all permit type columns are then pivoted, so that each permit application number has a corresponding permit type and value `1` for that permit.
167
+
168
+
From there, the names of target species are cleaned and added as a separate column, for later species-level grouping. An additional column is added based on whether a permit type is categorized as `commercial`, `for-hire/charter` or `recreational`. Because we are interested in also accessing species-level trends, permits that cover multiple species (`multispecies` and `squid/mackerel/butterfish`) have been parsed out so that each specific permit type corresponds with it's target species.
169
+
170
+
Lastly, permits are geocoded using `tidygeocoder` to their reported principal port and grouped to their respective management council region. Roughly 1% of entries are lost due to misspelled principal ports.
**How it works:** Confidential landings data was received along with the vessel trip reports as an Excel spreadsheet. The first 9 lines contained metadata as provided by **GARFO**. As such, this function uses `readxl` to read in the file and requires a `skip` argument to skip the first few rows that contain metadata. `SwimmeR` is used to clean the species names and `tidygeocoder` to geocode the ports associated with landings.
**How it works:** The function to pull in and clean the MRIP directed trips is arguably the most complex of the data cleaning functions. The data is hosted online and is downloadable as a ZIP file for individual years. This function pulls in the zipped files, unzips them and saves them locally in intermediate files, which are required for the trip estimation code. Once the data has been pulled in and saved to the intermediary files, the trip estimates are calculated using [Gary Nelson](gary.nelson@state.ma.us)'s [MRIP directed trip estimate R template](https://www.fisheries.noaa.gov/recreational-fishing-data/recreational-fishing-data-downloads). This code estimates the number of directed trips by species, wave, domain, trip type and year.
217
+
218
+
For this analyses, we used all waves, primary target trip type, all states along the east coast, and calculate estimates for Atlantic croaker, Atlantic mackerel, black sea bass, blueline tilefish, bluefish, gray triggerfish, king mackerel, Spanish mackerel, striped bass, summer flounder, scup, spiny dogfish, goosefish (monkfish) and tilefish (golden tilefish).
**How it works:** Catch estimates are publically available on the [ACCSP Data Warehouse](https://safis.accsp.org/accsp_prod/f?p=1490:1:1685966511435:::::) and downloaded as a CSV File. This function reads in the CSV and cleans the species names to be uniform with other data sets.
Once the data has been pulled and saved to the local environment, each data set has a respective `plot_()` or `map_()` function. More about those functions [here](https://carlylovas.github.io/species.shifts/articles/plotting_functions.html).
0 commit comments