Single-instance storage
- 3 min read
- 2 chapters
Related on Lably
Through File system, Data and Backup
Concept
In the case of an e-mail server, single-instance storage would mean that a single copy of a message is held within its database while individual mailboxes access the content through a reference pointer. However, there is a common misconception that the primary benefit of single-instance storage in mail servers is a reduction in disk space requirements. The truth is that its primary benefit is to greatly enhance delivery efficiency of messages sent to large distribution lists. In a mail server scenario disk space savings from single-instance storage are transient and drop off very quickly over time.
When used in conjunction with backup software, single-instance storage can reduce the quantity of archive media required since it avoids storing duplicate copies of the same file. Often identical files are installed on multiple computers, for example operating system files. With single-instance storage, only one copy of a file is written to the backup media therefore reducing space. This becomes more important when the storage is offsite and on cloud storage such as Amazon S3. In such cases, it has been reported that deduplication can help reduce the costs of storage, costs of bandwidth and backup windows by up to 10:1.
Novell GroupWise was built on single-instance storage, which accounts for its large capacity.
ISO CD/DVD image files can be optimized to use SIS to reduce the size of a CD/DVD compilation (if there are enough duplicated files) to make it fit into smaller media.
SIS is related to system wide file duplication search and multiple file instance detection tools such as the P2P application BearShare (5.n Versions and below) but differs in that SIS reduces storage utilization automatically and creates and retains symbolic linkages, whereas Bearshare allows for manual deletion of duplicates and associated user-level file system, Windows Explorer type of icon links.
Sign in to keep reading
One more chapter is free with a Lably account.
Microsoft
SIS was introduced with the Remote Installation Services feature of Windows 2000 Server. A typical server might hold ten or more unique installation configurations (perhaps with different device drivers or software suites) but perhaps only 20% of the data may be unique between configurations. Microsoft states that "SIS works by searching a hard disk volume to identify duplicate files. When SIS finds identical files, it saves one copy of the file to a central repository, called the SIS Common Store, and replaces other copies with pointers to the stored versions." Files are compared solely by their hash functions; files with different names or dates can be consolidated so long as the data itself is identical. Windows Server 2003 Standard Edition has SIS capabilities but is limited to OEM OS system installs.
The file-based Windows Imaging Format introduced in Windows Vista also supported single-instance storage. Single-instance storage was a feature of Microsoft Exchange Server since version 4.0 and is also present in Microsoft's Windows Home Server. It is deduplicating attachments only in Exchange 2007 and was dropped completely in Microsoft Exchange Server 2010. Microsoft announced Windows Storage Server 2008 (WSS2008) with Single Instance Storage on June 1, 2009, and states this feature is not available on Windows Server 2008.
The feature is officially deprecated since Windows Server 2012, when a new, more powerful chunk-based data deduplication mechanism was introduced. It allows files with similar content to be deduplicated as long as they have stretches of identical data. This mechanism is more powerful than SIS. Since Windows Server 2019, the feature is fully supported on ReFS.