Issue scanning files with time units as days since 0000-00-00

[pochedls]

This issue results from this issue: CDAT/cdms#334

One way to avoid this issue is to retrieve the file time units and if they are relative to the year 0 we could invoke cdscan with the flag: -e "time.units='days since 1900-01-01'"

We probably do not want to check the units on every file, but maybe could check units on files with particular error messages and rerun cdscan with the -e flag if they are tied to year zero.

[durack1]

@pochedls thanks for pushing on this, we could also invoke the time units = 'days since 0001-01-01 which will get us much closer to the noted time, with a single year offset

[pochedls]

@durack1 - yeah - this is actually what I added to our xmlWrite function. I think the data might be off by a year? See comment on: CDAT/cdms#334.

[pochedls]

This code produces a scan, but is off by a year:

def xmlWrite(inpath, outfile):
    """
    xmlWrite(inpath, outfile)

    Function calls cdscan to create an xml file (outfile) for a given
    directory of CMIP data (inpath).

    Inputs:
        inpath (string): directory containing input files
        outfile (string): xml file to write

    Returns:
        out, err: command output and error message strings

    """
    cmd = 'cdscan -x ' + outfile + ' ' + inpath + '/*.nc'
    cmd = cmd.replace('//', '/')
    p = Popen(cmd,shell=True,stdout=PIPE,stderr=PIPE)
    out,err = p.communicate()

    # Check for relative time unit issue
    # https://github.com/pochedls/xagg/issues/6
    relTimeIssue = str(err).find('Invalid relative time units') >= 0
    yearZero = str(out).find(' 0000-') >= 0
    if relTimeIssue & yearZero:
        cmd = "cdscan -x " + outfile + " -e time.units=\"days since 0001-01-01\" " + inpath + "/*.nc"
        cmd = cmd.replace('//', '/')
        p = Popen(cmd,shell=True,stdout=PIPE,stderr=PIPE)
        out,err = p.communicate()

    return out, err

Holding off implementing, because many of the runs are historical simulations in which the year does matter (year probably doesn't matter much for piControl and abrupt4xCO2).