Understanding the $ATTRIBUTE_LIST in NTFS

Question

I'm examining the NTFS (New Technology File System) and have been stuck in a loop trying to figure out the $ATTRIBUTE_LIST attribute. From this documentation, it is unusual to come across an $ATTRIBUTE_LIST and they're only used if the MFT table is running out of room. However, from looking at the following parsers, I've found they do parse it:

• NtfsFileExtractor
• ntfs by icaleo
• zenwinx
• JkDefrag v3.36 source code (found using some Googling)

From looking at these, I've come up with the following flowchart:

(There should be a yes to the right of "Has $ATTRIBUTE_LIST")

I would like to refer to the 2 processes on the right side of the flow chart. Is it correct that:

1. The attribute is only parsed if it's FRN is different than the file containing the attribute lists FRN?
2. The attribute is added to the file with the FRN listed in the attribute and not the FRN containing the attribute list? Or, is the FRN listed in the attribute only used for attributes for this file record (and not really a file)?

Answer 1

The attribute headers are always part of MFT records in the MFT, they are never non-resident; only the body of an attribute can be non-resident and described by data run entries. When a body of an attrbute becomes too big (e.g. the regular body of the DATA attribute of the file when the file is larger than around 800 B), the body is made non resident and is described by a resident attribute body consisting of data runs. When there are too many data runs to fit in the MFT record, the data runs themselves cannot be made non-resident and there is nothing built into the data header to indicate this 2nd level of indirection. When the attribute headers, which can never be made non-resident, and certain attribute bodies like the data runs themselves, which can't be made non-resident, no longer fit in the MFT record (which is limited to 1024 B), you need an $ATTRIBUTE_LIST attribute to refer to secondary MFT records that contain the rest of the attribute headers and bodies that are not allowed to be non-resident.

If the $ATTRIBUTE_LIST's resident attribute body that describes the list of attributes that cant fit into the MFT record then it can be made non-resident. The remaining resident attribute body then contains data runs, which can't in itself also be made non-resident if there are too many data runs (when the $ATTRIBUTE_LIST regular body pointed to by the runs becomes too fragmented and non-contiguous as opposed to too long), and if this remaining resident $ATTRIBUTE_LIST data-run body becomes too large, then the $ATTRIBUTE_LIST data runs are moved to 2 other MFT records and the $ATTRIBUTE_LIST now contains a resident regular data (not data run body) describing the 2 records that contain $ATTRIBUTE_LISTs with non-resident data runs.

The regular $ATTRIBUTE_LIST body that is either resident or pointed to by an $ATTRIBUTE_LIST header with data runs, contains a list of variable length entries that describe where to find each attribute header that couldn't fit in the MFT record. It contains the sector of the sector-aligned MFT entry that contains the particular attribute header that is described (data, filename etc.), and if it's an attribute that contains its own data runs then it inicates the range of VCNs covered by the data runs in the attribute by giving a starting VCN (the data runs in the data attribute body themselves have LCNs, where each run has a different separate contiguous range of LCNs, but virtually the data run entries themselves describe one contiguous block with a VCN starting from 0 to the total number of clusters made up by all the LCN ranges; the VCN is assigned based on the run entry). LCN is the actual cluster where the data is relative to the start of the NTFS volume and the VCN is the cluster number in the sequence of clusters that form the data trying to be accessed, which is a virtually contiguous unit i.e. a file or an attribute list of pointers to MFT residing attribute headers.

Here is an example of a very fragmented file with a MFT entry containing a $STANDARD_INFORMATION, $FILE_NAME, $OBJECT_ID, and an $ATTRIBUTE_LIST that is too long because there are so many data runs due to fragmentation that it requires too many MFT records to be referenced in the list that the $ATTRIBUTE_LIST body (the list) doesn't fit in the main MFT record with the other attributes (like how an 800 B file is too big a body for the$DATA attribute to fit), so the resident body of $ATTRIBUTE_LIST consists of data runs that point to the non resident body (the actual list). The list contains 228 secondary MFT entries containing 4878 fragments (data runs) between them (averaging 21.39 data runs each), totalling 325106 clusters (VCN 0-325105):

Metadata
Name: /img_PhysicalDrive0/vol_vol3/$Recycle.Bin/S-1-5-21-4290757439-1660816946-3063094355-1000/$RTY6UTY.dmp
Type: File System
MIME Type: application/octet-stream
Size: 1331634176
File Name Allocation: Allocated
Metadata Allocation: Allocated
Modified: 2021-04-14 10:03:04 BST
Accessed: 2021-04-14 09:20:10 BST
Created: 2021-04-14 09:20:10 BST
Changed: 2021-04-14 10:03:23 BST
MD5: Not calculated
SHA-256: Not calculated
Hash Lookup Results: UNKNOWN
Internal ID: 791240

From The Sleuth Kit istat Tool: MFT Entry Header Values:
Entry: 132670 Sequence: 217
$LogFile Sequence Number: 100415212298
Allocated File
Links: 1

$STANDARD_INFORMATION Attribute Values:
Flags: Archive
Owner ID: 0
Security ID: 487 (S-1-5-32-544)
Last User Journal Update Sequence Number: 13948449880
Created: 2021-04-14 09:20:10.280000000 (BST)
File Modified: 2021-04-14 10:03:04.474412800 (BST)
MFT Modified: 2021-04-14 10:03:23.403412900 (BST)
Accessed: 2021-04-14 09:20:10.280000000 (BST)

$FILE_NAME Attribute Values:
Flags: Archive
Name: $RTY6UTY.dmp
Parent MFT Entry: 15659 Sequence: 2
Allocated Size: 1331634176 Actual Size: 1331634176
Created: 2021-04-14 09:20:10.280000000 (BST)
File Modified: 2021-04-14 10:03:04.474412800 (BST)
MFT Modified: 2021-04-14 10:03:04.474412800 (BST)
Accessed: 2021-04-14 09:20:10.280000000 (BST)

$OBJECT_ID Attribute Values:
Object Id: 33aa2e2f-9cfa-11eb-b407-3cf862d1bc6b

$ATTRIBUTE_LIST Attribute Values:
Type: 16-0 MFT Entry: 132670 VCN: 0
Type: 48-8 MFT Entry: 132670 VCN: 0
Type: 64-6 MFT Entry: 132670 VCN: 0
Type: 128-0 MFT Entry: 278896 VCN: 0
Type: 128-0 MFT Entry: 89688 VCN: 6599
Type: 128-0 MFT Entry: 89687 VCN: 6636
Type: 128-0 MFT Entry: 89681 VCN: 6674
Type: 128-0 MFT Entry: 309001 VCN: 6689
Type: 128-0 MFT Entry: 309000 VCN: 6730
Type: 128-0 MFT Entry: 308997 VCN: 6797
Type: 128-0 MFT Entry: 308963 VCN: 6881
Type: 128-0 MFT Entry: 308546 VCN: 6979
Type: 128-0 MFT Entry: 231348 VCN: 7097
Type: 128-0 MFT Entry: 191228 VCN: 7104
Type: 128-0 MFT Entry: 191204 VCN: 7226
Type: 128-0 MFT Entry: 191187 VCN: 7232
Type: 128-0 MFT Entry: 191172 VCN: 7354
Type: 128-0 MFT Entry: 280179 VCN: 7360
Type: 128-0 MFT Entry: 280124 VCN: 7484
Type: 128-0 MFT Entry: 270567 VCN: 7489
Type: 128-0 MFT Entry: 270562 VCN: 7613
Type: 128-0 MFT Entry: 270530 VCN: 7616
Type: 128-0 MFT Entry: 254584 VCN: 7741
Type: 128-0 MFT Entry: 280286 VCN: 7750
Type: 128-0 MFT Entry: 89695 VCN: 26559
Type: 128-0 MFT Entry: 89685 VCN: 26575
Type: 128-0 MFT Entry: 308995 VCN: 26607
Type: 128-0 MFT Entry: 231361 VCN: 26626
Type: 128-0 MFT Entry: 231360 VCN: 26642
Type: 128-0 MFT Entry: 231063 VCN: 26658
Type: 128-0 MFT Entry: 231055 VCN: 26674
Type: 128-0 MFT Entry: 231024 VCN: 26690
Type: 128-0 MFT Entry: 230932 VCN: 26706
Type: 128-0 MFT Entry: 230931 VCN: 26722
Type: 128-0 MFT Entry: 230930 VCN: 26738
Type: 128-0 MFT Entry: 230929 VCN: 26774
Type: 128-0 MFT Entry: 230752 VCN: 26790
Type: 128-0 MFT Entry: 230748 VCN: 26836
Type: 128-0 MFT Entry: 216009 VCN: 26848
Type: 128-0 MFT Entry: 215970 VCN: 26864
Type: 128-0 MFT Entry: 215918 VCN: 26880
Type: 128-0 MFT Entry: 215917 VCN: 26898
Type: 128-0 MFT Entry: 215877 VCN: 26907
Type: 128-0 MFT Entry: 214453 VCN: 26923
Type: 128-0 MFT Entry: 211852 VCN: 26939
Type: 128-0 MFT Entry: 191813 VCN: 26955
Type: 128-0 MFT Entry: 191769 VCN: 26969
Type: 128-0 MFT Entry: 191737 VCN: 26987
Type: 128-0 MFT Entry: 191719 VCN: 26993
Type: 128-0 MFT Entry: 191650 VCN: 27009
Type: 128-0 MFT Entry: 191644 VCN: 27025
Type: 128-0 MFT Entry: 191616 VCN: 27041
Type: 128-0 MFT Entry: 191489 VCN: 27057
Type: 128-0 MFT Entry: 191472 VCN: 27089
Type: 128-0 MFT Entry: 191427 VCN: 27121
Type: 128-0 MFT Entry: 191375 VCN: 27137
Type: 128-0 MFT Entry: 191276 VCN: 27153
Type: 128-0 MFT Entry: 191246 VCN: 27169
Type: 128-0 MFT Entry: 169413 VCN: 27185
Type: 128-0 MFT Entry: 169410 VCN: 27467
Type: 128-0 MFT Entry: 169402 VCN: 27564
Type: 128-0 MFT Entry: 169400 VCN: 27565
Type: 128-0 MFT Entry: 169392 VCN: 27566
Type: 128-0 MFT Entry: 169378 VCN: 27567
Type: 128-0 MFT Entry: 169377 VCN: 27568
Type: 128-0 MFT Entry: 169354 VCN: 27569
Type: 128-0 MFT Entry: 169187 VCN: 27572
Type: 128-0 MFT Entry: 169175 VCN: 27580
Type: 128-0 MFT Entry: 169138 VCN: 27581
Type: 128-0 MFT Entry: 169132 VCN: 27582
Type: 128-0 MFT Entry: 169125 VCN: 27584
Type: 128-0 MFT Entry: 169106 VCN: 27586
Type: 128-0 MFT Entry: 169097 VCN: 27587
Type: 128-0 MFT Entry: 169076 VCN: 27709
Type: 128-0 MFT Entry: 168705 VCN: 27716
Type: 128-0 MFT Entry: 168682 VCN: 27722
Type: 128-0 MFT Entry: 168143 VCN: 27723
Type: 128-0 MFT Entry: 168136 VCN: 27859
Type: 128-0 MFT Entry: 168117 VCN: 27860
Type: 128-0 MFT Entry: 168087 VCN: 27864
Type: 128-0 MFT Entry: 168083 VCN: 27867
Type: 128-0 MFT Entry: 168079 VCN: 27874
Type: 128-0 MFT Entry: 168059 VCN: 27875
Type: 128-0 MFT Entry: 168057 VCN: 27879
Type: 128-0 MFT Entry: 168054 VCN: 27882
Type: 128-0 MFT Entry: 168005 VCN: 27884
Type: 128-0 MFT Entry: 167999 VCN: 27894
Type: 128-0 MFT Entry: 167987 VCN: 27901
Type: 128-0 MFT Entry: 167977 VCN: 27903
Type: 128-0 MFT Entry: 167956 VCN: 27906
Type: 128-0 MFT Entry: 167928 VCN: 27909
Type: 128-0 MFT Entry: 167926 VCN: 27910
Type: 128-0 MFT Entry: 167925 VCN: 27913
Type: 128-0 MFT Entry: 167918 VCN: 28043
Type: 128-0 MFT Entry: 167913 VCN: 28046
Type: 128-0 MFT Entry: 167908 VCN: 28645
Type: 128-0 MFT Entry: 167895 VCN: 29851
Type: 128-0 MFT Entry: 167866 VCN: 29856
Type: 128-0 MFT Entry: 167849 VCN: 31066
Type: 128-0 MFT Entry: 167660 VCN: 31072
Type: 128-0 MFT Entry: 282782 VCN: 32384
Type: 128-0 MFT Entry: 282774 VCN: 33760
Type: 128-0 MFT Entry: 282770 VCN: 35161
Type: 128-0 MFT Entry: 282757 VCN: 35169
Type: 128-0 MFT Entry: 282749 VCN: 35497
Type: 128-0 MFT Entry: 282738 VCN: 35595
Type: 128-0 MFT Entry: 282732 VCN: 35603
Type: 128-0 MFT Entry: 282705 VCN: 35623
Type: 128-0 MFT Entry: 282701 VCN: 35625
Type: 128-0 MFT Entry: 231299 VCN: 35627
Type: 128-0 MFT Entry: 282697 VCN: 36582
Type: 128-0 MFT Entry: 282637 VCN: 38146
Type: 128-0 MFT Entry: 282614 VCN: 38166
Type: 128-0 MFT Entry: 282594 VCN: 38195
Type: 128-0 MFT Entry: 282574 VCN: 38196
Type: 128-0 MFT Entry: 282568 VCN: 38199
Type: 128-0 MFT Entry: 282564 VCN: 38205
Type: 128-0 MFT Entry: 282563 VCN: 38207
Type: 128-0 MFT Entry: 282522 VCN: 38210
Type: 128-0 MFT Entry: 282520 VCN: 38216
Type: 128-0 MFT Entry: 282476 VCN: 38221
Type: 128-0 MFT Entry: 282465 VCN: 38224
Type: 128-0 MFT Entry: 282411 VCN: 38225
Type: 128-0 MFT Entry: 282401 VCN: 38228
Type: 128-0 MFT Entry: 282361 VCN: 38234
Type: 128-0 MFT Entry: 282287 VCN: 38237
Type: 128-0 MFT Entry: 282265 VCN: 38243
Type: 128-0 MFT Entry: 282247 VCN: 38254
Type: 128-0 MFT Entry: 282218 VCN: 38255
Type: 128-0 MFT Entry: 282216 VCN: 38269
Type: 128-0 MFT Entry: 282212 VCN: 38272
Type: 128-0 MFT Entry: 282208 VCN: 38275
Type: 128-0 MFT Entry: 282204 VCN: 38278
Type: 128-0 MFT Entry: 282196 VCN: 38281
Type: 128-0 MFT Entry: 282188 VCN: 38282
Type: 128-0 MFT Entry: 282184 VCN: 38292
Type: 128-0 MFT Entry: 282180 VCN: 38306
Type: 128-0 MFT Entry: 282161 VCN: 38307
Type: 128-0 MFT Entry: 282144 VCN: 38322
Type: 128-0 MFT Entry: 282143 VCN: 38325
Type: 128-0 MFT Entry: 282141 VCN: 38326
Type: 128-0 MFT Entry: 282070 VCN: 38335
Type: 128-0 MFT Entry: 282054 VCN: 38337
Type: 128-0 MFT Entry: 282048 VCN: 38339
Type: 128-0 MFT Entry: 282045 VCN: 38342
Type: 128-0 MFT Entry: 282034 VCN: 38345
Type: 128-0 MFT Entry: 282006 VCN: 39795
Type: 128-0 MFT Entry: 281587 VCN: 41502
Type: 128-0 MFT Entry: 281569 VCN: 41504
Type: 128-0 MFT Entry: 281518 VCN: 43266
Type: 128-0 MFT Entry: 281386 VCN: 45032
Type: 128-0 MFT Entry: 281109 VCN: 45033
Type: 128-0 MFT Entry: 281035 VCN: 45039
Type: 128-0 MFT Entry: 281023 VCN: 45042
Type: 128-0 MFT Entry: 280815 VCN: 45043
Type: 128-0 MFT Entry: 280544 VCN: 45053
Type: 128-0 MFT Entry: 280483 VCN: 45054
Type: 128-0 MFT Entry: 277351 VCN: 45058
Type: 128-0 MFT Entry: 277113 VCN: 45061
Type: 128-0 MFT Entry: 276372 VCN: 45067
Type: 128-0 MFT Entry: 276332 VCN: 45071
Type: 128-0 MFT Entry: 276221 VCN: 45072
Type: 128-0 MFT Entry: 276198 VCN: 45077
Type: 128-0 MFT Entry: 276172 VCN: 45078
Type: 128-0 MFT Entry: 276149 VCN: 45090
Type: 128-0 MFT Entry: 275969 VCN: 45092
Type: 128-0 MFT Entry: 275883 VCN: 45094
Type: 128-0 MFT Entry: 275557 VCN: 45097
Type: 128-0 MFT Entry: 275339 VCN: 45099
Type: 128-0 MFT Entry: 275304 VCN: 45102
Type: 128-0 MFT Entry: 275227 VCN: 45105
Type: 128-0 MFT Entry: 275119 VCN: 45108
Type: 128-0 MFT Entry: 275079 VCN: 45111
Type: 128-0 MFT Entry: 274921 VCN: 45114
Type: 128-0 MFT Entry: 274919 VCN: 45120
Type: 128-0 MFT Entry: 274904 VCN: 45123
Type: 128-0 MFT Entry: 274769 VCN: 45126
Type: 128-0 MFT Entry: 274765 VCN: 45156
Type: 128-0 MFT Entry: 274763 VCN: 45159
Type: 128-0 MFT Entry: 274740 VCN: 45161
Type: 128-0 MFT Entry: 274722 VCN: 45162
Type: 128-0 MFT Entry: 274704 VCN: 45172
Type: 128-0 MFT Entry: 274654 VCN: 45177
Type: 128-0 MFT Entry: 274650 VCN: 45182
Type: 128-0 MFT Entry: 274638 VCN: 45205
Type: 128-0 MFT Entry: 274602 VCN: 45228
Type: 128-0 MFT Entry: 274594 VCN: 45232
Type: 128-0 MFT Entry: 274579 VCN: 45235
Type: 128-0 MFT Entry: 274575 VCN: 45237
Type: 128-0 MFT Entry: 274574 VCN: 45241
Type: 128-0 MFT Entry: 274571 VCN: 45243
Type: 128-0 MFT Entry: 249096 VCN: 45244
Type: 128-0 MFT Entry: 274537 VCN: 45258
Type: 128-0 MFT Entry: 274530 VCN: 46810
Type: 128-0 MFT Entry: 274484 VCN: 46819
Type: 128-0 MFT Entry: 274470 VCN: 48643
Type: 128-0 MFT Entry: 274467 VCN: 50492
Type: 128-0 MFT Entry: 274448 VCN: 50497
Type: 128-0 MFT Entry: 274444 VCN: 52443
Type: 128-0 MFT Entry: 274087 VCN: 52448
Type: 128-0 MFT Entry: 274000 VCN: 54496
Type: 128-0 MFT Entry: 273761 VCN: 56644
Type: 128-0 MFT Entry: 273756 VCN: 59331
Type: 128-0 MFT Entry: 273716 VCN: 62554
Type: 128-0 MFT Entry: 273705 VCN: 62563
Type: 128-0 MFT Entry: 273556 VCN: 65903
Type: 128-0 MFT Entry: 273396 VCN: 69263
Type: 128-0 MFT Entry: 273046 VCN: 73343
Type: 128-0 MFT Entry: 272903 VCN: 73344
Type: 128-0 MFT Entry: 272874 VCN: 78295
Type: 128-0 MFT Entry: 272853 VCN: 78304
Type: 128-0 MFT Entry: 272695 VCN: 86746
Type: 128-0 MFT Entry: 272562 VCN: 86755
Type: 128-0 MFT Entry: 272502 VCN: 86819
Type: 128-0 MFT Entry: 272495 VCN: 95291
Type: 128-0 MFT Entry: 272491 VCN: 95300
Type: 128-0 MFT Entry: 231993 VCN: 101950
Type: 128-0 MFT Entry: 272485 VCN: 106721
Type: 128-0 MFT Entry: 271915 VCN: 125351
Type: 128-0 MFT Entry: 161789 VCN: 129446
Type: 128-0 MFT Entry: 191166 VCN: 135133
Type: 128-0 MFT Entry: 216022 VCN: 139843
Type: 128-0 MFT Entry: 216027 VCN: 147209
Type: 128-0 MFT Entry: 216034 VCN: 154135
Type: 128-0 MFT Entry: 271789 VCN: 156638
Type: 128-0 MFT Entry: 271517 VCN: 156640
Type: 128-0 MFT Entry: 230790 VCN: 202097
Type: 128-0 MFT Entry: 230792 VCN: 228045
Type: 128-0 MFT Entry: 230804 VCN: 239215
Type: 128-0 MFT Entry: 271200 VCN: 252608
Type: 128-0 MFT Entry: 230805 VCN: 269592
Type: 128-0 MFT Entry: 230806 VCN: 285774

Attributes: 
Type: $STANDARD_INFORMATION (16-0) Name: N/A Resident size: 72
Type: $ATTRIBUTE_LIST (32-9) Name: N/A Non-Resident size: 7424 init_size: 7424
Starting address: 8799623, length: 1 
Starting address: 9219487, length: 1 
Type: $FILE_NAME (48-8) Name: N/A Resident size: 90
Type: $OBJECT_ID (64-6) Name: N/A Resident size: 16
Type: $DATA (128-10) Name: N/A Non-Resident size: 1331634176 init_size: 1331634176

In this case there are 325106 virtually contiguous clusters mapped onto 4878 separate fragments (contiguous runs of LCNs).

Answer 2

they're only used if the MFT table is running out of room

This is not correct. They are used whenever the MFT entry is too large to hold all the attributes.

The attribute is only parsed if it's FRN is different than the file containing the attribute lists FRN?

It depends on the OS/software, I guess, but it kinda makes sense. While $ATTRIBUTE_LIST must contain a list of all attributes, you can enumerate "local" attributes by simply parsing the whole MFT entry. For instance, my software RecuperaBit does it that way.

Conversely, you need the list to figure out in which other MFT entries the "remote" attributes are stored.

Or, is the FRN listed in the attribute only used for attributes for this file record (and not really a file)?

The MFT entry whose number is contained in the $ATTRIBUTE_LIST attribute does not contain a $DATA attribute and doesn't have a $FILE_NAME attribute either. It is not a file, it's just an additional MFT entry.

---

Note: I edited the answer because I was using the word "resident" in a confusing way to refer to attributes inside the base MFT entry. However, the concept of resident attribute is a different thing.