arc.write error

[Saadi4469]

I am getting the following error while writing an sf object dataframe to a GDB. The dataframe imported is a feature class from a GDB brought in via the sf package. After some data manipulation I am trying to write the dataframe back into the same GDB as a new feature class/layer.

I read in another post that sf does not support writing a feature class to a gdb so I had to use arcgisbinding. However, when I use sf::st_write to write the df as a shapefile, it works just fine.

The arc.write works on the sample nc dataset in the sf package but does not work on my dataset. I don't know if it matters but my sf object has more than 700,000 rows and 82 columns.

How can I fix this?

Error

Error in .call_proxy("arc_write", path, pairlist(data = data, coords = coords,  : 
  insert row failed

Code

library(sf)
 library(arcgisbinding)

arc.check_product()

product: ArcGIS Pro (12.8.0.29751)
license: Advanced
version: 1.0.1.244 
 
 arc.write("path/GDB.gdb/Feature_Class_Name", data = df, overwrite = TRUE)

[Saadi4469]

Also, it would be really helpful to add a arcgisbindings tag on stackoverflow.

[orhuna]

@Saadi4469 there can be numerous causes to this issue. Here are some thoughts:

1. Missing values containing special characters
2. Reserved characters in row entries
3. Missing crs (what do you get when you run st_crs on your df?)

If possible, sharing a few rows of your data will allow me to provide more direct help.

[Saadi4469]

@orhuna how can I attach a zipped shapefile?

[orhuna]

@Saadi4469 the easiest way would be emailing it to oaydin@esri.com. If the file is too big, I can set up a OneDrive folder.

[scdub]

@Saadi4469 If the file is smaller, you can also make a .zip archive and attach it as a comment here

[Saadi4469]

@orhuna I have sent you an email with the shapefile, thank you.

[Saadi4469]

@orhuna please note that the columns showing NAs in the sample_df are not completely empty in the original dataset.

[orhuna]

@Saadi4469 I tried writing the dataset to all fgdbs we support using R v 4.1.3 with arcgisbinding v244:

• Current
• 9.2
• 9.3
• 10.0

arc.write writes the data frame you shared into these without any issues. Below is the code:

library(arcgisbinding)
library(sf)

arc.check_product()
root.dir <- "<my_path_to_data>"
dir <- file.path(root.dir,"sample_df.shp")

#### Reading Your DF into a SF Data Frame ####
sf_data <- st_read(dir)

arc.write(<gdb_loc>, sf_data, overwrite=T)

[Saadi4469]

@orhuna thank you, could the big dataset be an issue then?

[Saadi4469]

Also is arcgisbinding v244 same as 1.0.1.244?

[Saadi4469]

@orhuna I am beginning to think that the dataset is so big that that might be causing the issue.

[orhuna]

@Saadi4469 the data volume is not the issue as the cursor that inserts data from your R dataframe to the feature class in the GDB throws an error and exits.

I think we can debug this further and get you going with your workflow. Below are some checks on the R side, that I recommend you perform on your R data frame:

Steps to Take on the R/R Studio side

Checking for Different Types of missing values

I recommend applying the below separately on the dataset to investigate if there are more than one type of missing data. Where we can handle nas in writint NaN and infinite values need to be represented with a place holder such as -999

Would you please check for different types of missing/out of bound values with is.na , is.infinite, is.nan ?

Note that if you wanted to assign a value for any na, nan or inifinte you can use the following pattern is.na(x) <- value

Trying to find the problematic row

I suspect that you have one row that causes a problem in writing. After trying the subset you given me, I was able to write it out successfully. Here is what I recommend:

Create a for loop that loops through only up to a row and try to write it.

library(arcgisbinding)
library(sf)

arc.check_product()
root.dir <- <your data loc>

write.dir <- file.path(root.dir, 'gdb_current.gdb')
dir <- file.path(root.dir,"sample_df.shp")

#### Reading Your DF into a SF Data Frame ####
sf_data <- st_read(dir)
n <- dim(sf_data)[1]
for (row in seq(2, n))
{
  data_write <- sf_data[1:row,]
  arc.write(write.dir, data_write, overwrite=T)
}
 

Note that this takes a minute with the 10 rows you gave me and you may want to search for the row may be first with a jump of 100 and then a jump of 10 etc. Searching every row range one by one may take a while for 800,000 rows.

Steps to Take on the ArcGIS Pro/Desktop side

Creating a New GDB

We see write errors like to one you encountered on corrupt gdbs. One step I recommend is running the Create File Geodatabase tool to create a new gdb to write into. There were times where this solved pretty complex problems.

Visualizing the feature class that resulted in the error

This step might be a replacement for the for loop I recommended above. Here you can, simply run your code that results in the error. Then in Pro Go to Catalog, right click on the gdb you wrote to and click refresh. If you can see the feature class there add it to your map and look at the Attribute Table. If the Attribute Table is populated the bottom-most row will be the row right before the problematic row that you can either remove or fix in R.

I hope this helps and please reach out if it does not.

[Saadi4469]

@orhuna thank you for the detailed guidance.