Scan for orphaned $FILE_NAME attributes of attribute list MFT entries for path reconstruction

[joachimmetz]

When reading a $MFT scan for orphaned $FILE_NAME attributes of attribute list MFT entries for path reconstruction. See log2timeline/plaso#3840 for more context.

• how to best represent the path hint and indicate it was deleted e.g. $Deleted\Folder1\File1.txt or $Orpan\Folder1\File1.txt ?
• how to handle a Folder1 created after File2

[joachimmetz]

Changes to https://github.com/dfirlabs/ntfs-specimens to generate test data dfirlabs/ntfs-specimens@41e93ad

Looks like the Linux NTFS implementation re-uses the first available (user) MFT entry, by default

The changes to the script do not have the desired behavior but highlights another interesting case. Looks like the Linux NTFS3g 2021.8.22 implementation allows the creation of files named testdir1:ads1

MFT entry: 66 information:
        Is allocated                    : true
        File reference                  : 66-1
        Base record file reference      : Not set (0)
        Journal sequence number         : 0
        Number of attributes            : 4

Attribute: 1
        Type                            : $STANDARD_INFORMATION (0x00000010)
        Creation time                   : Sep 25, 2021 05:48:18.003678400 UTC
        Modification time               : Sep 25, 2021 05:48:18.003761700 UTC
        Access time                     : Sep 25, 2021 05:48:18.003678400 UTC
        Entry modification time         : Sep 25, 2021 05:48:18.003761700 UTC
        File attribute flags            : 0x00000020
                Should be archived (FILE_ATTRIBUTE_ARCHIVE)

Attribute: 2
        Type                            : $FILE_NAME (0x00000030)
        Parent file reference           : 5-5
        Creation time                   : Sep 25, 2021 05:48:18.003678400 UTC
        Modification time               : Sep 25, 2021 05:48:18.003678400 UTC
        Access time                     : Sep 25, 2021 05:48:18.003678400 UTC
        Entry modification time         : Sep 25, 2021 05:48:18.003678400 UTC
        File attribute flags            : 0x00000020
                Should be archived (FILE_ATTRIBUTE_ARCHIVE)
        Name space                      : POSIX (0)
        Name                            : testdir1:ads1

Adding streams_interface=windows to the mount command causes NTFS3g to mimic Windows like behavior

[joachimmetz]

Looks like the NTFS3g implementation actually removes the $FILE_NAME attribute from the list on removal:

before removal

MFT entry: 67 information:
        Is allocated                    : true
        File reference                  : 67-1
        Base record file reference      : 64-1
        Journal sequence number         : 0
        Number of attributes            : 6

Attribute: 1
        Type                            : $FILE_NAME (0x00000030)
        Parent file reference           : 5-5
        Creation time                   : Sep 25, 2021 06:26:15.329177000 UTC
        Modification time               : Sep 25, 2021 06:26:15.329177000 UTC
        Access time                     : Sep 25, 2021 06:26:15.329177000 UTC
        Entry modification time         : Sep 25, 2021 06:26:15.329177000 UTC
        File attribute flags            : 0x10000020
                Should be archived (FILE_ATTRIBUTE_ARCHIVE)
                Unknown (0x10000000)
        Name space                      : POSIX (0)
        Name                            : testdir1

Attribute: 2
        Type                            : $DATA (0x00000080)
        Data size                       : 6 bytes
        Data flags                      : 0x0000
        Name                            : ads12

after removal:

MFT entry: 67 information:
        Is allocated                    : false
        File reference                  : 67-2
        Base record file reference      : 64-1
        Journal sequence number         : 0
        Number of attributes            : 5

Attribute: 1
        Type                            : $DATA (0x00000080)
        Data size                       : 6 bytes
        Data flags                      : 0x0000
        Name                            : ads12

Attribute: 2
        Type                            : $DATA (0x00000080)
        Data size                       : 6 bytes
        Data flags                      : 0x0000
        Name                            : ads13

[joachimmetz]

Having a look at Windows

before:

MFT entry: 39 information:
        Is allocated                    : true
        File reference                  : 39-1
        Base record file reference      : 37-1
        Journal sequence number         : 1078483
        Number of attributes            : 13

Attribute: 1
        Type                            : $FILE_NAME (0x00000030)
        Parent file reference           : 5-5
        Creation time                   : Sep 25, 2021 11:53:22.326620600 UTC
        Modification time               : Sep 25, 2021 11:53:22.326620600 UTC
        Access time                     : Sep 25, 2021 11:53:22.326620600 UTC
        Entry modification time         : Sep 25, 2021 11:53:22.326620600 UTC
        File attribute flags            : 0x10000000
                Unknown (0x10000000)
        Name space                      : POSIX (0)
        Name                            : testdir1

Attribute: 2
        Type                            : $DATA (0x00000080)
        Data size                       : 9 bytes
        Data flags                      : 0x0000
        Name                            : ads1

after:

MFT entry: 39 information:
        Is allocated                    : false
        File reference                  : 39-2
        Base record file reference      : 37-1
        Journal sequence number         : 2115106
        Number of attributes            : 13

Attribute: 1
        Type                            : $FILE_NAME (0x00000030)
        Parent file reference           : 5-5
        Creation time                   : Sep 25, 2021 11:53:22.326620600 UTC
        Modification time               : Sep 25, 2021 11:53:22.326620600 UTC
        Access time                     : Sep 25, 2021 11:53:22.326620600 UTC
        Entry modification time         : Sep 25, 2021 11:53:22.326620600 UTC
        File attribute flags            : 0x10000000
                Unknown (0x10000000)
        Name space                      : POSIX (0)
        Name                            : testdir1

Attribute: 2
        Type                            : $DATA (0x00000080)
        Data size                       : 9 bytes
        Data flags                      : 0x0000
        Name                            : ads1

So Windows does not remove the $FILE_NAME attribute from the list on removal

[joachimmetz]

An additional factor to account for in the scenario is if a Folder1 created after File2 (test scenario in ntfs-specimens needs a bit of tweaking to prevent testfile1 from being overwritten)

MFT entry: 38 information:
        Is allocated                    : true
        File reference                  : 38-2
        Base record file reference      : Not set (0)
        Journal sequence number         : 3163616
        Number of attributes            : 3

Attribute: 1
        Type                            : $STANDARD_INFORMATION (0x00000010)
        Creation time                   : Sep 25, 2021 11:53:39.899168700 UTC
        Modification time               : Sep 25, 2021 11:53:39.899168700 UTC
        Access time                     : Sep 25, 2021 11:53:39.899168700 UTC
        Entry modification time         : Sep 25, 2021 11:53:39.899168700 UTC
        Owner identifier                : 0
        Security descriptor identifier  : 263
        Update sequence number          : 5024
        File attribute flags            : 0x10000000
                Unknown (0x10000000)

Attribute: 2
        Type                            : $FILE_NAME (0x00000030)
        Parent file reference           : 5-5
        Creation time                   : Sep 25, 2021 11:53:39.899168700 UTC
        Modification time               : Sep 25, 2021 11:53:39.899168700 UTC
        Access time                     : Sep 25, 2021 11:53:39.899168700 UTC
        Entry modification time         : Sep 25, 2021 11:53:39.899168700 UTC
        File attribute flags            : 0x10000000
                Unknown (0x10000000)
        Name space                      : POSIX (0)
        Name                            : testdir1

Which is not the same testdir1 as the one that contained testfile1

[joachimmetz]

Made changes to test data scripts. With the larger $INDEX_ROOT attribute Windows seems to prefer to store the $FILE_NAME attribute in the "main" MFT entry (not in an attribute list MFT entry). Forcing $INDEX_ROOT to be non-resident has pushed $FILE_NAME back to the attribute list MFT entry.

So the a new Folder1 can be created after File2, and it would likely be good to provide for means to distinguish between the deleted Folder1 and the existing Folder1 in the path hint.