Application Compatibility with ReFS

Microsoft Corporation

Published: March 2012

Abstract

This document provides an introduction to Resilient File System (ReFS) and an overview of changes that are relevant to developers interested in ensuring application compatibility with ReFS. This document is not final and will be updated at subsequent release milestones with more information. We are making it available now to provide initial documentation and encourage the ecosystem to start thinking about application compatibility with ReFS.

Copyright information

This document is provided “as-is”.Information and views expressed in this document, including URL and other Internet Web site references, may change without notice.

Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes.

© 2012 Microsoft. All rights reserved.

Microsoft, Windows, and Windows Server are trademarks of the Microsoft group of companies.

All other trademarks are property of their respective owners.

Contents

1.1Overview

1.2Introduction

1.3Windows Server “8” Capabilities

1.4Application Compatibility

1.5New APIs

1.5.1FSCTL_GET_INTEGRITY_INFORMATION

1.5.2FSCTL_SET_INTEGRITY_INFORMATION

1.5.3FSCTL_REPAIR_COPIES

1.5.4MARK_HANDLE_READ_COPY/MARK_HANDLE_NOT_READ_COPY

1.6New File Attributes

1.7Salvage

1.8File System Recognition

1.9Certification Kit

1.10Other considerations

1.10.1Semantic and functional behavior of ReFS metadata

1.10.2Integrity Streams and Conventional Streams

1.10.3128-bit File IDs

1.10.4Distinguishing between a ReFS volume and an NTFS volume

1.11Deployment Considerations

1.12Configuring a ReFS volume

1.12.1From Disk Management

1.12.2From CMD

1.13Summary

1.1Overview

This document provides an introduction to Resilient File System (ReFS) and an overview of changes that are relevant to developers interested in ensuring application compatibility with ReFS. This document is not final and will be updated at subsequent release milestones with more information. We are making it available now to provide initial documentation and encourage the ecosystem to start thinking about application compatibility with ReFS.

Note that the information in this document is subject to change.

1.2Introduction

Resilient File System (ReFS) is a new local file system introduced in Windows Server “8”, immediately addressing critical server customer needs, and providing the foundation for future platform evolution, for all Windows customers.

1.3Windows Server “8” Capabilities

The following is an overview of the capabilities that ReFS will deliver in Windows Server “8”:

  • Integrity: ReFS stores data in a way that it is protected from many of the common errors that can cause data loss. File system metadata is always protected. Optionally, user data can be protected on a per-volume, per-directory, or per-file basis. If corruption occurs, ReFS can detect and, when configured with Storage Spaces, automatically correct the corruption. In the event of a system error, ReFS is designed to recover from that error rapidly, with no loss of user data.
  • Availability: ReFS is designed to prioritize the availability of data. With ReFS, if corruption occurs, and it cannot be repaired automatically, the online salvage process is localized to the area of corruption, requiring no volume down-time.
  • Scalability: As the amount and the size of the data we store in computers increases, it is critical to have a system that is designed with such data sets in mind. ReFS is designed for data sets sizes of today and the data set sizes of tomorrow, optimized for high scalability.
  • Application Compatibility:To maximize AppCompat, ReFS supports a subset of NTFS features and Win32 APIs that are widely adopted.
  • Proactive Error Identification: The integrity capabilities of ReFS are leveraged by a data integrity scanner (commonly known as a “scrubber”) that periodically scans the volume, attempting to identify latent corruption and then proactively triggering a repair of that corrupt data.
  • Architectural Evolution: A new architecture allows ReFSto evolve in conjunction with new storage devices, new data types, and new access patterns, providing a file system platform for the future.

For more information on ReFS, see Building the next generation file system for Windows: ReFS

1.4Application Compatibility

Architecturally, ReFS includes many of the same features and APIs as NTFS. This is an intentional decision to promote application compatibility.

The design of NTFS includes:

•an upper layer that implements APIs/semantics

•a lower layer that implements the on-disk store

ReFS reuses the upper-layer to provide API compatibility to applications while providing a new on-disk engine, named “Minstore” that enables new capabilities.

1.4.1.1Differences between NTFS and ReFS

There are, however, certain NTFS features that ReFS does not support. Some of these features are unsupported intentionally, for long-term architectural reasons, while others are unsupported in the Windows Server “8” version of ReFS.

The following list describes current (as of January 2012) feature differences between NTFS and ReFS; this list may change later in the Windows Server “8” release to include additional features:

  • Hard Links
  • Named Streams
  • Extended Attributes (EA’s)
  • Object ID’s
  • Volume Shrink
  • Short Names
  • Fast MFT-like Enumeration
  • NTFS Compression
  • NTFS Encryption
  • NTFS Quotas
  • TxF
  • Reparse Index

The following is a current (as of January 2012) list of unsupported APIs; the list may change later in the Windows Server“8” release to include additional APIs:

  • AddUsersToEncryptedFile
  • CloseEncryptedFileRaw
  • CopyFileTransacted
  • CreateFileTransacted
  • CreateHardLink
  • CreateHardLinkTransacted
  • CreateSymbolicLinkTransacted
  • DecryptFile
  • DeleteFileTransacted
  • DuplicateEncryptionInfoFile
  • EncryptFile
  • EncryptionDisable
  • FindFirstFileNameTransactedW
  • FindFirstFileTransacted
  • FindFirstStreamTransactedW
  • FindNextFileNameW
  • FindNextStreamW
  • FreeEncryptionCertificateHashList
  • GetCompressedFileSizeTransacted
  • GetFileAttributesTransacted
  • GetFullPathNameTransacted
  • GetLongPathNameTransacted
  • MoveFileTransacted
  • OpenEncryptedFileRaw
  • QueryRecoveryAgentsOnEncryptedFile
  • QueryUsersOnEncryptedFile
  • ReadEncryptedFileRaw
  • RemoveUsersFromEncryptedFile
  • SetFileAttributesTransacted
  • SetFileShortName
  • SetUserFileEncryptionKey
  • WriteEncryptedFileRaw

The following is a current (as of January 2012) list of unsupported FSCTLs; the list may change later in the Windows Server “8” release to include additional FSCTLs:

  • FSCTL_CORRUPTION_HANDLING
  • FSCTL_CREATE_OR_GET_OBJECT_ID
  • FSCTL_DELETE_OBJECT_ID
  • FSCTL_ENCRYPTION_FSCTL_IO
  • FSCTL_ENUM_USN_DATA
  • FSCTL_FILE_LEVEL_TRIM
  • FSCTL_FILE_PREFETCH
  • FSCTL_FILE_TYPE_NOTIFICATION
  • FSCTL_FIND_FILES_BY_SID
  • FSCTL_GET_NTFS_FILE_RECORD
  • FSCTL_GET_NTFS_VOLUME_DATA
  • FSCTL_GET_OBJECT_ID
  • FSCTL_GET_REPAIR
  • FSCTL_INITIATE_REPAIR
  • FSCTL_LOOKUP_STREAM_FROM_CLUSTER
  • FSCTL_MARK_VOLUME_DIRTY
  • FSCTL_OFFLOAD_READ
  • FSCTL_OFFLOAD_WRITE
  • FSCTL_QUERY_ALLOCATED_RANGES
  • FSCTL_QUERY_FILE_LAYOUT
  • FSCTL_QUERY_FILE_REGIONS
  • FSCTL_QUERY_RETRIEVAL_POINTERS
  • FSCTL_READ_RAW_ENCRYPTED
  • FSCTL_RESET_VOLUME_ALLOCATION_HINTS
  • FSCTL_SD_GLOBAL_CHANGE
  • FSCTL_SET_BOOTLOADER_ACCESSED
  • FSCTL_SET_COMPRESSION
  • FSCTL_SET_ENCRYPTION
  • FSCTL_SET_OBJECT_ID
  • FSCTL_SET_OBJECT_ID_EXTENDED
  • FSCTL_SET_REPAIR
  • FSCTL_SET_SHORT_NAME_BEHAVIOR
  • FSCTL_SET_SPARSE
  • FSCTL_SET_ZERO_ON_DEALLOCATION
  • FSCTL_SHRINK_VOLUME
  • FSCTL_TXFS_CREATE_MINIVERSION
  • FSCTL_TXFS_CREATE_SECONDARY_RM
  • FSCTL_TXFS_GET_METADATA_INFO
  • FSCTL_TXFS_GET_TRANSACTED_VERSION
  • FSCTL_TXFS_LIST_TRANSACTION_LOCKED_FILES
  • FSCTL_TXFS_LIST_TRANSACTIONS
  • FSCTL_TXFS_MODIFY_RM
  • FSCTL_TXFS_QUERY_RM_INFORMATION
  • FSCTL_TXFS_READ_BACKUP_INFORMATION
  • FSCTL_TXFS_READ_BACKUP_INFORMATION2
  • FSCTL_TXFS_ROLLFORWARD_REDO
  • FSCTL_TXFS_ROLLFORWARD_UNDO
  • FSCTL_TXFS_SAVEPOINT_INFORMATION
  • FSCTL_TXFS_SHUTDOWN_RM
  • FSCTL_TXFS_START_RM
  • FSCTL_TXFS_TRANSACTION_ACTIVE
  • FSCTL_TXFS_WRITE_BACKUP_INFORMATION
  • FSCTL_TXFS_WRITE_BACKUP_INFORMATION2
  • FSCTL_VOLUME_IS_DIRTY
  • FSCTL_WAIT_FOR_REPAIR
  • FSCTL_WRITE_RAW_ENCRYPTED

1.5New APIs

1.5.1FSCTL_GET_INTEGRITY_INFORMATION

Purpose:

On a ReFS volume, gets the integrity status of a file or directory, including the type of checksum algorithm, the checksum chunk size, and the cluster size.

typedefstruct_FSCTL_GET_INTEGRITY_INFORMATION_BUFFER {

USHORTChecksumAlgorithm;

USHORTReserved;

ULONGFlags;

ULONGChecksumChunkSizeInBytes;

ULONGClusterSizeInBytes;

} FSCTL_GET_INTEGRITY_INFORMATION_BUFFER, *PFSCTL_GET_INTEGRITY_INFORMATION_BUFFER;

Member Name / Description
ChecksumAlgorithm / ChecksumAlgorithm (2 bytes): MUST be one of the following standard values.
Value / Meaning
CHECKSUM_TYPE_NONE
0x0000 / The file or directory should not be configured to use integrity.
CHECKSUM_TYPE_CRC64
0x0002 / The file or directory is configured to use a CRC64 checksum to provide integrity.
All other values
0x0002 — 0xFFFF / Reserved for future use and MUST NOT be used.
Reserved / A 16-bit reserved value. This field MUST be set to zero and MUST be ignored.
Flags / Flags (4 bytes): A 32-bit unsigned integer that contains zero or more of the following flag values. Flag values that are unspecified in the following table SHOULD be set to 0 and MUST be ignored.
Value / Meaning
FSCTL_INTEGRITY_FLAG_CHECKSUM_ENFORCEMENT_OFF
0x00000001 / When set, checksums are disabled on this file.
ChecksumChunkSizeinBytes / A 32-bit unsigned integer specifying the size of each chunk in a stream that is configured with integrity.
ClusterSizeinBytes / A 32-bit unsigned integer specifying the size of a cluster for this volume in bytes. ClusterSize MUST be a power of 2, and MUST be greater than or equal to SectorSizeand a power of 2 multiple of SectorSize

Examples:

BOOLEAN GetIntegrity(

_In_ HANDLE Handle,

_Out_opt_ PUSHORT Algorithm,

_Out_opt_ PULONG Flags

)

{

FSCTL_GET_INTEGRITY_INFORMATION_BUFFER IntegrityInformationBuffer;

ULONG Length;

if ( !DeviceIoControl(Handle, FSCTL_GET_INTEGRITY_INFORMATION, NULL,

0, &IntegrityInformationBuffer, sizeof(IntegrityInformationBuffer), &Length, FALSE ) ) {

return FALSE;

}

*Algorithm = IntegrityInformationBuffer.ChecksumAlgorithm;

*Flags = IntegrityInformationBuffer.Flags;

return TRUE;

}

1.5.2FSCTL_SET_INTEGRITY_INFORMATION

On a ReFS volume, sets the integrity state of a file or directory on a ReFS volume.

typedefstruct_FSCTL_SET_INTEGRITY_INFORMATION_BUFFER {

USHORTChecksumAlgorithm;

USHORTReserved;

ULONGFlags;

} FSCTL_SET_INTEGRITY_INFORMATION_BUFFER, *PFSCTL_SET_INTEGRITY_INFORMATION_BUFFER;

Parameter Name / Description
ChecksumAlgorithm / MUST be one of the following standard values.
Value / Meaning
CHECKSUM_TYPE_NONE
0x0000 / The file or directory should not be configured to use integrity.
CHECKSUM_TYPE_CRC64
0x0002 / The file or directory is configured to use a CRC64 checksum to provide integrity.
CHECKSUM_TYPE_UNCHANGED
0xFFFF / The integrity status of the file or directory should be unchanged.
All other values
0x0002 — 0xFFFE / Reserved for future use and MUST NOT be used.
Reserved / A 16-bit reserved value. This field MUST be set to zero and MUST be ignored.
Flags / A 32-bit unsigned integer that contains zero or more of the following flag values. Flag values that are unspecified in the following table SHOULD be set to 0 and MUST be ignored.
Value / Meaning
FSCTL_INTEGRITY_FLAG_CHECKSUM_ENFORCEMENT_OFF
0x00000001 / When set, checksums are disabled on this file, allowing for the data to be returned, even if there is a checksum error.

Remarks:

  • The integrity status of a file can be changed only for empty files. For example, if a file is created and then data is written to the file, its integrity status cannot be changed once data is written.
  • If ReplaceFile is called on a file, there is a specific case where the integrity status of the original file will not be persisted. If the backup file created by ReplaceFile is specified with a location different from the location of the original file and that location has an integrity status different than the original file, the integrity status of the original file will not be persisted.
  • Writes to integrity streams are always cluster-sized. Reads from integrity streams are always 0x4000 bytes. The implication of this is that if 0x1000 bytes are read from offset 0, but there’s a corruption at offset 0x3000, the read will fail with ERROR_DATA_CHECKSUM_ERROR. Similarly, if 0x1000 bytes are written to offset 0, but there’s a corruption at offset 0xf000, the write will fail with ERROR_DATA_CHECKSUM_ERROR.
  • If FSCTL_INTEGRITY_FLAG_CHECKSUM_ENFORCEMENT_OFF is specified and a file is opened with permissions such that subsequent open operations are allowed to succeed, there is the potential for an application to read corrupt data from the disk or introduce corruption to data on the disk. FSCTL_INTEGRITY_FLAG_CHECKSUM_ENFORCEMENT_OFF disables verification of checksums on the data, allowing all reads or writes to succeed, even if there is a checksum error. In a situation where there are multiple processes interacting with data, if any one of the processes specifies FSCTL_INTEGRITY_FLAG_CHECKSUM_ENFROCEMENT_OFF, all processes may inadvertently become susceptible to either reading data from the disk that may have become corrupt or introducing corruption.

Examples:

BOOLEAN SetIntegrity(

_In_ HANDLE Handle,

_In_ USHORT Algorithm,

_In_ ULONG Flags

)

{

FSCTL_SET_INTEGRITY_INFORMATION_BUFFER IntegrityInformationBuffer;

ZeroMemory(&IntegrityInformationBuffer, sizeof(IntegrityInformationBuffer));

IntegrityInformationBuffer.ChecksumAlgorithm = Algorithm;

IntegrityInformationBuffer.Flags = Flags;

return (BOOLEAN)DeviceIoControl(Handle, FSCTL_SET_INTEGRITY_INFORMATION, &IntegrityInformationBuffer, sizeof(IntegrityInformationBuffer), NULL, 0, NULL, FALSE );

}

1.5.3FSCTL_REPAIR_COPIES

Purpose:

The data integrity scanner (aka scrubber) is a background process designed to identify and, when possible, correct latent silent corruption.

•NTFS: Scrubber will trigger a read such that bad sectors are identified, if they exist, and re-mapped by Spaces.

•ReFS: The data integrity features provided by ReFS are designed to ensure that when data is read that data is identical to the last time the data was written. However, there is a gap with this model – not all data will be read all the time. In other words, if data is not read, then its checksums are not verified, introducing an opportunity for latent silent corruption. To complete the value proposition of the data integrity features offered by ReFS, we will provide a data scrubber. The primary goal of the scrubber is to periodically read all allocated data, including both metadata and user data (if configured with checksums), and verify that the checksum associated with that data is correct.

Ultimately, scrubber is another part of the solution to provide data integrity for Windows’ customers.

FSCTL_REPAIR_COPIES CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 173, METHOD_BUFFERED, FILE_READ_DATA | FILE_WRITE_DATA)
typedefstruct _REPAIR_COPIES_INPUT {
ULONG Size;
ULONG Flags;
LARGE_INTEGER FileOffset;
ULONG Length;
ULONG SourceCopy;
ULONG NumberOfRepairCopies;
ULONG RepairCopies[ANYSIZE_ARRAY];
} REPAIR_COPIES_INPUT, *PREPAIR_COPIES_INPUT;
typedefstruct _REPAIR_COPIES_OUTPUT {
ULONG Size;
ULONG Status;
LARGE_INTEGER ResumeFileOffset;
} REPAIR_COPIES_OUTPUT, *PREPAIR_COPIES_OUTPUT;

Parameter Name / Description
Size / Must be sizeof(REPAIR_COPIES_INPUT). sizeof(REPAIR_COPIES_OUTPUT) will be set on the output buffer
Flags / Reserved. Must be 0.
FileOffset / The file position at which to start the repair requestupto the value specified in Length.
Length / Number of bytes to repair.
SourceCopy / The copy number of the source copy.
NumberOfRepairCopies / The number of copies that will be repaired.
RepairCopies[ANYSIZE_ARRAY] / The copy numbers of all the copies that will be repaired.
ResumeFileOffset / File position that can be used as a FileOffset in the subsequent call to resume the repair operation, skipping the range where errors were found.
Status / Operational status of the repair.

Return Value:

DeviceIOControl.

Remarks:

On ReFS, only allowed on non-integrity streams.

Examples:

BOOL Success;
// Change the handle info to read from a copy
MARK_HANDLE_INFO HandleInfo;
ZeroMemory( &HandleInfo, sizeof(MARK_HANDLE_INFO) );
HandleInfo.HandleInfo = MARK_HANDLE_READ_COPY;
HandleInfo.CopyNumber = 1; /* or 0, 1, ... */
DWORD BytesReturned;
Success = DeviceIoControl(NonCachedFileHandle, FSCTL_MARK_HANDLE, &HandleInfo, sizeof(MARK_HANDLE_INFO), NULL, 0, &BytesReturned, NULL);
// Normal read file call
Success = ReadFile(NonCachedFileHandle, Buffer, BytesToRead, &BytesReturned, NULL);
// Repair call
union {
REPAIR_COPIES_INPUT RepairInput;
UCHAR RepairInputBuffer[FIELD_OFFSET(REPAIR_COPIES_INPUT, RepairCopies) + sizeof(ULONG) * 2];
};
REPAIR_COPIES_OUTPUT RepairOutput;
ZeroMemory( &RepairInput, sizeof(RepairInputBuffer) );
RepairInput.Size = sizeof(REPAIR_COPIES_INPUT);
RepairInput.Flags = 0;
RepairInput.FileOffset = 0x1000;
RepairInput.Length = 0x4000;
RepairInput.SourceCopy = 1;
RepairInput.NumberOfRepairCopies = 2;
RepairInput.RepairCopies[0] = 0;
RepairInput.RepairCopies[1] = 2;
Success = DeviceIoControl(FileHandle, FSCTL_REPAIR_COPIES, &RepairInput, (RepairInputBuffer), &RepairOutput, (REPAIR_COPIES_OUTPUT), &BytesReturned, NULL);

1.5.4MARK_HANDLE_READ_COPY/MARK_HANDLE_NOT_READ_COPY

#define MARK_HANDLE_READ_COPY (0x00000080)

#define MARK_HANDLE_NOT_READ_COPY (0x00000100)

typedefstruct {

union {

ULONG UsnSourceInfo;

ULONG CopyNumber;

};

HANDLE VolumeHandle;

ULONG HandleInfo;

} MARK_HANDLE_INFO, *PMARK_HANDLE_INFO;

//

// 32/64 Bit thunking support structure

//

typedefstruct {

union {

ULONG UsnSourceInfo;

ULONG CopyNumber;

};

UINT32 VolumeHandle;

ULONG HandleInfo;

} MARK_HANDLE_INFO32, *PMARK_HANDLE_INFO32;

Purpose:

MARK_HANDLE_READ_COPY is a type of HandleInfo of MARK_HANDLE_INFO structure which is used as an input parameter of FSCTL_MARK_HANDLE. It marks the handle to read data from a specified copy.

The handle is subject to these restrictions:

•MARK_HANDLE_READ_COPY is only allowed to non-cached I/O handles (such as handles created with a FILE_FLAG_NO_BUFFERING flag.)

Member Name / Description
USNSourceInfo /
CopyNumber / The number that indicates the corresponding copy of the data, e.g., copy 0, copy 1, etc.
VolumeHandle / Not used for MARK_HANDLE_READ_COPY and MARK_HANDLE_NOT_READ_COPY
HandleInfo /

Remarks:

On ReFS, only allowed on non-integrity streams.

1.6New File Attributes

In support of the new integrity capabilities of ReFS, there are two new file attributes.

FILE_ATTRIBUTE INTEGRITY STREAM

This attribute indicates that the user data stream is configured with integrity. The following table describes the semantics of the attribute.

Copy / Target dir has integrity set / Target dir without integrity set
File with Integrity set / No change to the file / Persist integrity
File without Integrity / Inherit integrity / No change to the file
Rename / Target dir has integrity set / Target dir without integrity set
File with Integrity set / No change / No change
File without Integrity / No change / No change
Cross-device / Target dir has integrity set / Target dir without integrity set
File with Integrity set / No change / Persist integrity
File without Integrity / Inherit integrity / No change

One implication of the above is that once Integrity is set on a non-empty file, it cannot be removed without copying to a file system that does not support Integrity, such as NTFS. (Note that in ReFS deployments, there will always be an NTFS volume available: the system volume.) For empty files, integrity can be set and unset using the FSCTL/integrity.exe.

FILE ATTRIBUTE NO SCRUB DATA

This attribute indicates that the file should be skipped by the scrubber. This attribute is useful for those files that maintain their own integrity information. The following table describes the semantics of the attribute.

Copy / Target dir has NoScrub set / Target dir without NoScrub set
File with NoScrub set / No change / Persist NoScrub
File without NoScrub / Inherit NoScrub / No change
Rename / Target dir has NoScrub set / Target dir without NoScrub set
File with NoScrub set / No change / No change
File without NoScrub / No change / No change
Cross-device / Target dir has NoScrub set / Target dir without NoScrub set
File with NoScrub set / No change / Persist NoScrub
File without NoScrub / Inherit NoScrub / No change

Notes:

  • Both the NoScrub and Integrity attribute can be set on the same file.
  • NoScrub can be set on directories, but it provides inheritance only.
  • CreateFile2 will set Integrity if the attribute is specified by the caller or the target directory has the attribute set. NoScrub can be set using File_Basic_Information or other API that provides access to file attributes.product
  • In CMD, the letter ‘V’ will indicate integrity; ‘X’ will be used to indicate NoScrub.

1.7Salvage

To maximize the availability of data on a volume, ReFS implements “salvage”, a feature that will remove corrupt data from the namespace in the event that the corruption cannot be automatically repaired. The intention behind this feature is to ensure that non-repairable corruption does not adversely affect the availability of non-corrupt data. If, for example, a single file in a directory were to become corrupt and could not be automatically repaired, salvage will remove that file from the file system namespace. A corrupt file cannot be opened or deleted by the file system, making it impossible for an Administrator to respond. With salvage, an Administrator can recover that file from a backup or have the application re-create it. Salvage is compatible with user files, directories, and other file system metadata.

For file system filter drivers, you must implement code that can gracefully react to a file or directory disappearing suddenly.

Note that ReFS does not require chkdsk and chkdsk will never run on a ReFS volume.

1.8File System Recognition

ReFS implements the File System Recognition feature, ensuring that ReFS volumes will be identified on Windows 7 and Windows Server 2008 R2 operating systems. As appropriate, filter drivers should query for FILE_SYSTEM_RECOGNITION_STRUCTURE using FSCTL_QUERY_FILE_SYSTEM_RECOGNITION. On ReFS volumes, file system filter drivers must not overwrite or alter logical disk sector zero, which is used to store the file system recognition data.