The Microsoft ReFS On-Disk Layout

With the public beta release of Windows Server 8, Microsoft introduced an implementation of its Resiliant File System (ReFS). This page links to ReFS resources that include Microsoft documentation, forensic images of ReFS volumes, and disk structures. Please join me in reversing the on-disk layout of ReFS.

Microsoft Documentation

• Building the Next Generation File System for Windows by Surendra Verma - The initial description of ReFS and explanations of its design decisions.
• Application Compatibility with ReFS - “Provides an introduction to ReFS and an overview of changes that are relevant to developers”.
• File Directory Volume Support - The File Directory Volume Support spreadsheet documents ReFS support for existing file system APIs.

Sample Images

The archive available here [zip] contains eight forensic images created by FTK Imager Lite 2.9.0 on Window Server 8 Beta build 8250 running on VirtualBox 4.1.8. There are seven physical images and one logical image of a local volume formatted with ReFS. The size of the archive is 492 MB and the uncompressed size is approximately 8 GB. The archive MD5 sum is 2c67d770e7473a516d05ded1432432b1. You may use the contents of the archive without restriction.

I envision the sample images could be used as initial test cases by those interested in ReFS. For example, when describing an interesting structure, the images provide reference offsets (like, the “ReFS VBR starts at offset 201:0000h in Physical Image 1”). The images can also be used by those who do not have immediate access to a working instance of Windows Server 8. Grab them and get reversing!

I created the target volume on March 3, 2012 at 2:50 EST by attaching a 1024 MB VirtualBox virtual disk and formatting a 512 MB volume using the Windows Volume Management utility. I selected the default values at each prompt.

I generated the set of test images by systematically manipulating the file system and immediately acquiring an image. The file system manipulations included creating files, deleting files, creating nested directories and files, and completely filling the file system. I performed all actions using Windows Explorer. See the section Sample Image Sequence for a detailed timeline of the image acquisition sequence.

Disk Structures

“Under construction”!

Based on data gleaned from the sample images referenced in the previous section, the file system may use structures described in this section. A pseudo-C/010 Editor template formats each structure in this section. Of course, the contents of this section are subject to change pending additional research.

Boot Structures

Microsoft Windows Server 8 will not support booting from a ReFS volume when it is released. By default, Windows Server 8 partitions a disk using a standard GUID Partition Table. A description of GPT follows.

A legacy MBR table fills the first sector (Physical Image 1 offset :0000h). The GPT immediately follows the MBR (PI-1 offset :0200h). The first entry of the GPT is reserved, while the second entry points to the first usable partition (PI-1 offset :0480h). A GPT entry specifies the type of the partition using a well-defined GUID, the volume identifier as a GUID, and the first and last sectors of the partition. The final few sectors of the disk hold a backup copy of the GPT (PI-1 offset 3FFF:FE00h). A 010 Editor template is available [here][4].

GPT Header

GPT Entry

ReFS Volume Boot Record

In Physical Image 1, the ReFS Volume Boot Record (VBR) begins at offset 201:000h. The fourth through eigth bytes of the VBR contain the NULL terminated string “ReFS”. The VBR is at least 65 bytes long. The remainder of the sector is filled with zeros.

File System Metadata

File system metadata is spread throughout a ReFS formatted partition. This is in contrast to file systems like NTFS, where the majority of metadata is stored in the MFT. ReFS metadata structures follow a consistent top level layout. The contents of a ReFS metadata block are determined either implicitly (eg. the root metadata block) or via a header at a known offset.

A ReFS metadata block is 0x4000 bytes in size. The first DWORD of a ReFS metadata block equals the 0x4000 byte cluster number of the block. For example, in Physical Image 5, the root metadata block is located at volume offset 0x7:8000. In terms of 0x4000 byte clusters, this is offset 0x1E. Note that the first DWORD of this metadata structure is 0x1E:

1
2
3

208:8000h: 1E 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................ 
208:8010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................ 
208:8020h: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

File Contents

Contents of small files in a ReFS volume may be stored at offsets mod 0x1:0000 (for example, 0x1:0000, 0x2:0000, etc.). The contents of a file begin at the first byte in a ReFS data cluster. Small files are not stored “resident” in a metadata structure. In Physical Image 5, the contents of “desktop.ini” are located at volume offset 0x221:0000 (0x4000 cluster 0x84):

1
2
3
4
5
6
7
8

222:0000h: 5B 2E 53 68 65 6C 6C 43 6C 61 73 73 49 6E 66 6F  [.ShellClassInfo 
222:0010h: 5D 0D 0A 43 4C 53 49 44 3D 7B 36 34 35 46 46 30  ]..CLSID={645FF0 
222:0020h: 34 30 2D 35 30 38 31 2D 31 30 31 42 2D 39 46 30  40-5081-101B-9F0 
222:0030h: 38 2D 30 30 41 41 30 30 32 46 39 35 34 45 7D 0D  8-00AA002F954E}. 
222:0040h: 0A 4C 6F 63 61 6C 69 7A 65 64 52 65 73 6F 75 72  .LocalizedResour 
222:0050h: 63 65 4E 61 6D 65 3D 40 25 53 79 73 74 65 6D 52  ceName=@%SystemR 
222:0060h: 6F 6F 74 25 5C 73 79 73 74 65 6D 33 32 5C 73 68  oot%system32sh 
222:0070h: 65 6C 6C 33 32 2E 64 6C 6C 2C 2D 38 39 36 34 0D  ell32.dll,-8964.

Upcase table

In Physical Image 1, volume offsets 19:0000h through 1B:0000h (0x4000 clusters 0x64 through 0x6B) list many WORD values that generally increase. Jeff Hamm suggests this may be an Upcase table similar to that found in NTFS and exFAT. For example, at volume offset 19:0060h:

1
2
3
4
5
6
7
8
9
10

21A:0060h: 00 30 00 31 00 32 00 33 00 34 00 35 00 36 00 37  .0.1.2.3.4.5.6.7 
21A:0070h: 00 38 00 39 00 3A 00 3B 00 3C 00 3D 00 3E 00 3F  .8.9.:.;..? 
21A:0080h: 00 40 00 41 00 42 00 43 00 44 00 45 00 46 00 47  .@.A.B.C.D.E.F.G 
21A:0090h: 00 48 00 49 00 4A 00 4B 00 4C 00 4D 00 4E 00 4F  .H.I.J.K.L.M.N.O 
21A:00A0h: 00 50 00 51 00 52 00 53 00 54 00 55 00 56 00 57  .P.Q.R.S.T.U.V.W 
21A:00B0h: 00 58 00 59 00 5A 00 5B 00 5C 00 5D 00 5E 00 5F  .X.Y.Z.[..].^._ 
21A:00C0h: 00 60 00 41 00 42 00 43 00 44 00 45 00 46 00 47  .`.A.B.C.D.E.F.G 
21A:00D0h: 00 48 00 49 00 4A 00 4B 00 4C 00 4D 00 4E 00 4F  .H.I.J.K.L.M.N.O 
21A:00E0h: 00 50 00 51 00 52 00 53 00 54 00 55 00 56 00 57  .P.Q.R.S.T.U.V.W 
21A:00F0h: 00 58 00 59 00 5A 00 7B 00 7C 00 7D 00 7E 00 7F  .X.Y.Z.{.|.}.~.

Misc. and Tidbits

• There are about 25 FILETIME timestamps similar to the creation date of the volume in Physical Image 1.
• There are about four instances of the volume name in Unicode in Physical Image 1.

Given that ReFS metadata blocks begin with their 0x4000 cluster number and that ReFS data blocks begin with content, the following script identifies ReFS structures:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# ReFS 0x4000 Cluster Usage Mapper
# By Willi Ballenthin  2012-03-25
import sys, struct

REFS_VOLUME_OFFSET = 0x2010000

def main(args):
    with open(args[1], "rb") as f:
        global REFS_VOLUME_OFFSET
        offset = REFS_VOLUME_OFFSET
        cluster = 0
    while True:
        f.seek(offset %2B cluster * 0x4000)
            buf = f.read(4)
            if not buf: break
            magic = struct.unpack("<I", buf)[0]
            if magic == cluster:
                print "Metadata cluster %s (%s)" % \
                  (hex(cluster), hex(offset + cluster * 0x4000))
            elif magic != 0:
                print "Non-null cluster %s (%s)" % \
                  (hex(cluster), hex(offset + cluster * 0x4000))
            cluster += 1

if __name__ == '__main__':
    main(sys.argv)

Running the script against Physical Image 5 yields the following layout. Note that I’ve annotated the 0x4000 clusters with comments that begin with #.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

Non-null cluster 0x0 (0x2010000) # VBR
Metadata cluster 0x1e (0x2088000) # Root ReFS metadata cluster
Metadata cluster 0x20 (0x2090000)
Metadata cluster 0x21 (0x2094000)
Metadata cluster 0x22 (0x2098000)
Metadata cluster 0x28 (0x20b0000)
Metadata cluster 0x29 (0x20b4000)
Metadata cluster 0x2a (0x20b8000)
Metadata cluster 0x2b (0x20bc000)
Metadata cluster 0x2c (0x20c0000)
Metadata cluster 0x2d (0x20c4000)
Metadata cluster 0x2e (0x20c8000)
Metadata cluster 0x2f (0x20cc000)
Metadata cluster 0x30 (0x20d0000)
Metadata cluster 0x31 (0x20d4000)
Metadata cluster 0x32 (0x20d8000)
Metadata cluster 0x33 (0x20dc000)
Metadata cluster 0x54 (0x2160000)
Metadata cluster 0x55 (0x2164000)
Metadata cluster 0x56 (0x2168000)
Metadata cluster 0x57 (0x216c000)
Metadata cluster 0x58 (0x2170000)
Metadata cluster 0x5c (0x2180000)
Metadata cluster 0x60 (0x2190000)
Metadata cluster 0x61 (0x2194000)
Metadata cluster 0x62 (0x2198000)
Metadata cluster 0x63 (0x219c000)
Non-null cluster 0x64 (0x21a0000) # \
Non-null cluster 0x65 (0x21a4000) # |
Non-null cluster 0x66 (0x21a8000) # |
Non-null cluster 0x67 (0x21ac000) # +- $UPCASE table
Non-null cluster 0x68 (0x21b0000) # |
Non-null cluster 0x69 (0x21b4000) # |
Non-null cluster 0x6a (0x21b8000) # |
Non-null cluster 0x6b (0x21bc000) # /
Metadata cluster 0x70 (0x21d0000)
Metadata cluster 0x71 (0x21d4000)
Metadata cluster 0x72 (0x21d8000)
Metadata cluster 0x73 (0x21dc000)
Metadata cluster 0x74 (0x21e0000)
Metadata cluster 0x78 (0x21f0000)
Metadata cluster 0x79 (0x21f4000)
Metadata cluster 0x7a (0x21f8000)
Metadata cluster 0x7b (0x21fc000)
Metadata cluster 0x7c (0x2200000)
Metadata cluster 0x7d (0x2204000)
Metadata cluster 0x7e (0x2208000)
Metadata cluster 0x80 (0x2210000)
Metadata cluster 0x81 (0x2214000)
Non-null cluster 0x84 (0x2220000) # Contents of "desktop.ini"
Non-null cluster 0x88 (0x2230000) # Contents of "eicar.com"
Non-null cluster 0x8c (0x2240000)
Non-null cluster 0x90 (0x2250000) # Contents of "eicar2.zip"
Non-null cluster 0x94 (0x2260000) # Contents of "eicar.com"
Non-null cluster 0x98 (0x2270000) # Contents of "eicar.zip"
Metadata cluster 0x9c (0x2280000)
Metadata cluster 0x167 (0x25ac000)
Metadata cluster 0xf7c (0x5e00000)
Metadata cluster 0x7ffd (0x22004000)
Metadata cluster 0x7ffe (0x22008000)

Sample Image Sequence

Process started: 2012/03/03 at 13:52:XX EST.

• Create new volume, using the New Volume Wizard.

Settings:

Disk 2 Capacity 1.00GB
Disk will be partitioned with GPT
Volume size 512 MB
Drive Letter F
ReFS File System
Default Allocation Unit Size
Volume Label: Willi Ballenthin Test Volume 1
Not short filenames

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 1
Unique Description Physical Drive Image 1
Examiner Willi Ballenthin
Notes 1
Filename Physical_Image_1
Not Fragmented

Logical Drive Image:

Type dd
Case Number 1
Evidence Number 2
Unique Description Logical Drive Image 1
Examiner Willi Ballenthin
Notes 2
Filename Logical_Image_1
Not Fragmented

FTK creates an image of size 1019.75, similar to the Physical Drive Image. Only Physical Drive images from this point on.

• Download eicar.com.
• Download eicar.zip.
• Download eicar2.zip from www.rexwain.com/eicar.html.
• Copy eicar.com to F:\ on 2012/03/03 at 14:20:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 3
Unique Description Physical Drive Image 3
Examiner Willi Ballenthin
Notes 3
Filename Physical_Image_3
Not Fragmented

• Delete F:\eicar.com on 2012/03/03 at 14:25:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 4
Unique Description Physical Drive Image 4
Examiner Willi Ballenthin
Notes 4
Filename Physical_Image_4
Not Fragmented

• Copy eicar.com, eicar.zip, and eicar2.zip to F:\ on 2012/03/03 at 14:30:xx EST.
• Create directory F:\test directory on 2012/03/03 at 14:31:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 5
Unique Description Physical Drive Image 5
Examiner Willi Ballenthin
Notes 5
Filename Physical_Image_5
Not Fragmented

• Download Linux Kernel 3.2.9 archive from Kernel.org.
• Delete eicar.com, eicar.zip, eicar2.zip, and test directory from F:\ on 2012/03/03 at 14:38:xx EST.
• Copy linux-3.2.9.tar.bz2 to F:\ on 2012/03/03 at 14:39:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 6
Unique Description Physical Drive Image 6
Examiner Willi Ballenthin
Notes 6
Filename Physical_Image_6
Not Fragmented

• Use 7zip to extract linux-3.2.9.tar.bz2 to directory linux-3.2.9.tar on 2012/03/03 at 14:46:xx EST.
• Ran out of space after 304 MB at 2012/03/03 at 14:48:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 7
Unique Description Physical Drive Image 7
Examiner Willi Ballenthin
Notes 7
Filename Physical_Image_7
Not Fragmented

• Download python-registry v0.2.4.1.
• Delete linux-3.2.9.tar.bz2 and linux.3.2.9.tar/ from F:\ on 2012/03/03 at 14:59:xx EST.
• Copy williballenthin-python-registry-v0.2.4.1-1-g1e78a95.zip to F:\ on 2012/03/03 at 15:00:xx EST.
• Extract williballenthin-python-registry-v0.2.4.1-1-g1e78a95.zip to F:\williballenthin-python-registry-v0.2.4.1-1-g1e78a95\ on 2012/03/03 at 15:00:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 8
Unique Description Physical Drive Image 8
Examiner Willi Ballenthin
Notes 8
Filename Physical_Image_8
Not Fragmented

• Delete F:\williballenthin-python-registry-v0.2.4.1-1-g1e78a95.zip and F:\williballenthin-python-registry-v0.2.4.1-1-g1e78a95\ on 2012/03/03 at 15:12:xx EST.

Physical Drive Image:

Type dd
Case Number 1
Evidence Number 9
Unique Description Physical Drive Image 9
Examiner Willi Ballenthin
Notes 9
Filename Physical_Image_9
Not Fragmented