ArgumentError: Cannot open 'cities.csv': not a file

[s-celles]

Hello,

The following example:

using TableSchema
import TableSchema: read, validate

using DataPackage

fname = joinpath(dirname(pathof(DataPackage)), "..", "data/cities/datapackage.json")
println(fname)
p = Package(fname)

println("Loaded a Data Package, resources: ", length(p.resources))

r = DataPackage.get_resource(p, "cities")

data = DataPackage.read(r)
println( "The number of cells is ", length(data[:,1]) )
println( "Sum of column 1 is ", sum([ row for row in data[1] ]) )

table = DataPackage.get_table(r)
if validate(table); println("The table is valid according to the Schema"); end

raises

ERROR: LoadError: ArgumentError: Cannot open 'cities.csv': not a file

DataPackage.jl should look at cities.csv in the same directory as datapackage.json

Kind regards

[loleg]

@scls19fr I'm a little confused by this issue (and apologies for the slow reply) - cities.csv is in the same directory.. and the library does try to load resources relative to the Data Package. See https://github.com/frictionlessdata/DataPackage.jl/tree/master/data/cities

[s-celles]

I think we shouldn't confuse:

• relative path from current working directory (cwd)
• relative path from datapackage directory

I think package p should keep a track (ie internally) of absolute path of datapackage so /Users/myuser/.julia/dev/DataPackage/src/../data/cities/ should (need) to be stored in a field of Package.

After adding this, absolute path of cities.csv could be calculated using datapackage absolute path of directory and ressource relative path (relative from datapackage... not from current working directory)

My opinion is that it's better to use absolute path internally than relative path from current working directory because getting resource will fail if you change cwd after creating a Package struct.

Note that it works fine if your working directory is .julia/dev/DataPackage/

$ cd ~/.julia/dev/DataPackage/
(base) ✔ ~/.julia/dev/DataPackage [master|✚ 3]
17:53 $ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using TableSchema

julia> import TableSchema: read, validate

julia> using DataPackage

julia> fname="data/cities/datapackage.json"
"data/cities/datapackage.json"

julia> p = Package(fname)
Package(Dict{String,Any}("name"=>"datapackage","resources"=>Any[Dict{String,Any}("profile"=>"tabular-data-resource","name"=>"cities","dialect"=>Dict{String,Any}("quoteChar"=>"|"),"schema"=>Dict{String,Any}("fields"=>Any[Dict{String,Any}("name"=>"id","type"=>"integer"), Dict{String,Any}("name"=>"city","type"=>"string")]),"path"=>"cities.csv")]), false, PackageError[], Resource[Resource(Dict{String,Any}("profile"=>"tabular-data-resource","name"=>"cities","dialect"=>Dict{String,Any}("quoteChar"=>"|"),"schema"=>Dict{String,Any}("fields"=>Any[Dict{String,Any}("name"=>"id","type"=>"integer"), Dict{String,Any}("name"=>"city","type"=>"string")]),"path"=>"cities.csv"), false, "cities", "cities.csv", "tabular-data-resource", Dict{String,Any}("quoteChar"=>"|"), Schema(TableSchema.SchemaError[], Dict{String,Any}("fields"=>Any[Dict{String,Any}("name"=>"id","type"=>"integer"), Dict{String,Any}("name"=>"city","type"=>"string")]), String[], Dict[], String[], TableSchema.Field[Field(Dict{String,Any}("name"=>"id","type"=>"integer"), "id", "integer", "default", Constraints(false, false, nothing, nothing)), Field(Dict{String,Any}("name"=>"city","type"=>"string"), "city", "string", "default", Constraints(false, false, nothing, nothing))]), PackageError[])])

julia> println("Loaded a Data Package, resources: ", length(p.resources))
Loaded a Data Package, resources: 1

julia> r = DataPackage.get_resource(p, "cities")
Resource(Dict{String,Any}("profile"=>"tabular-data-resource","name"=>"cities","dialect"=>Dict{String,Any}("quoteChar"=>"|"),"schema"=>Dict{String,Any}("fields"=>Any[Dict{String,Any}("name"=>"id","type"=>"integer"), Dict{String,Any}("name"=>"city","type"=>"string")]),"path"=>"cities.csv"), false, "cities", "cities.csv", "tabular-data-resource", Dict{String,Any}("quoteChar"=>"|"), TableSchema.Schema(TableSchema.SchemaError[], Dict{String,Any}("fields"=>Any[Dict{String,Any}("name"=>"id","type"=>"integer"), Dict{String,Any}("name"=>"city","type"=>"string")]), String[], Dict[], String[], TableSchema.Field[Field(Dict{String,Any}("name"=>"id","type"=>"integer"), "id", "integer", "default", Constraints(false, false, nothing, nothing)), Field(Dict{String,Any}("name"=>"city","type"=>"string"), "city", "string", "default", Constraints(false, false, nothing, nothing))]), PackageError[])

[s-celles]

To be clearer...

in this expression

p = Package(fname)

if fname is a relative path it's a relative path of datapackage.json from current working directory

but in datapackage.json, relative path of ressources are relative path from datapackage.json directory

[loleg]

Thanks @scls19fr I think the issue is clearer now.