This article is rated Start-class on Wikipedia's
content assessment scale. It is of interest to the following WikiProjects: | |||||||||||
|
The contents of the Can't extend page were merged into File system fragmentation on April 10, 2015. For the contribution history and old versions of the redirected page, please see its history; for the discussion at that location, see its talk page. |
To-do list for File system fragmentation:
Priority 6
|
Consider copy on write filesystems, which use immutable allocated blocks (like ZFS) This allocation will just waste time, and will not change the characteristics of the filesystem —Preceding unsigned comment added by 129.65.102.1 ( talk) 17:49, 27 February 2008 (UTC)
Wasn't there an attempt at a file system that attempted to avoid that by writing data in layers, from top to bottom, letting the old data drop? Could someone tell me its name please? Thanks! 84.129.170.44 ( talk) —Preceding undated comment added 04:23, 5 April 2009 (UTC).
For the record, I created this article as I was dissatisfied with the current defragmentation article that:
I have attempted to mitigate these somewhat, but ultimately decided to write this article. I don't know if I can get it into a good enough shape to be merged with "defragmentation" (if at all), but I will try, and I will cite genuine research in the progress. It may or may not be considered a "rewrite" at this point. Any criticisms and comments are very welcome. -- intgr 03:53, 14 December 2006 (UTC)
While I myself added this to the article, I'm not sure it is fair to consider "related file fragmentation" a type of fragmentation. While research dealing with fragmentation very often also touches the topic of keeping related files together (e.g, files in a single directory), I don't think I can recall any instances where it's actually referred to as "fragmentation" per se.
However, consider when an archive is unpacked. As all files are decompressed in order, they will likely be laid out sequentially. But when time goes on, and files are added and deleted, the directory tree in which the files were decompressed into, becomes less and less "contiguous", e.g., can be considered "fragmented". -- intgr 14:21, 19 December 2006 (UTC)
I struck the Mac OS X note, since it isn't what actually happens. Mac OS X/HFS+ do not defrag at idle. What happens is that when a fragmented file is opened, it is defragged (unless the system has been up less than some specific time, and I forget what that time is). Thus, there's no "at idle" about it. Now if there's a seperate "at idle" process, by all means put the claim back in (but please reference it). Thanks. :) -- Steven Fisher 18:58, 19 December 2006 (UTC)
As the article has been tagged with {{ merge| Defragmentation}}, does anyone have ideas how to do that (e.g., what to merge, what not to merge, and what to throw away)? -- intgr 14:32, 30 January 2007 (UTC)
Although it would seem logical to merge the "Defragmentation" and the "File system fragmentation" articles, the first will naturally focus on practical aspects of dealing with the problem, and the second on a more theoretical understanding of the root cause of the problem. Combined into one article, there is a danger that it will get overly complex -- or that important material will be deleted to keep the article simple.-- 69.87.193.53 19:22, 18 February 2007 (UTC)
I totally disagree...defragging is the way by which the natural order is kept...extra work that users have to do is good for their character. —The preceding
unsigned comment was added by
68.249.171.240 (
talk)
I also disagree, merging the two articles would lead to one gigantic overly complex article. As stated earlier, information would probably be cut for the sake of simplicity, resulting in an incomplete article overall. Just link from the fragmentation article to the defragmentation article. -- Rollerskatejamms 13:07, 13 March 2007 (UTC)
A much better method, only merges the needless copy of File system fragmentation#Cause.
Spitfire ( talk) 01:35, 25 September 2008 (UTC)
There is POSIX fallocate API for allocating predermined size for file (or part of it). It have two motivations: make sure there is enaugh free space for file (so write operations will not file becuase of no space), and file can be preallocated on single extent on disk if possible in optimal manner.
Some tools already uses it to decress fratgmentation. —Preceding unsigned comment added by 149.156.67.102 ( talk) 21:50, 7 December 2009 (UTC)
User:Codename Lisa recently changed some content in this article and introduced a concept of "infinite fragmentation", which I reverted. He/she started a discussion at User talk:Intgr#File system fragmentation, which I have moved here:
It’s also somewhat of a misconception that fragmentation is not a problem on SSDs. If an SSD gets too fragmented you can hit maximum file fragmentation (when the metadata can’t represent any more file fragments) which will result in errors when you try to write/extend a file. Furthermore, more file fragments means more metadata to process while reading/writing a file, which can lead to slower performance.
Additionally, there is a maximum level of fragmentation that the file system can handle. [... Read on]
Sorry, I was perhaps too blunt. I was mainly ticked off by the weird term "infinite fragmentation" (that it appears you made up, hence "original research") and the claim that filesystems can't "sustain" some level of fragmentation.
The only source you're citing is a post on the personal blog of Scott Hanselman. Looking at his publications, he doesn't appear to be an expert on file systems. I don't claim that he's incompetent, maybe just makes oversimplifications, which I think are too misleading for an encyclopedia.
You (and/or this source) are confusing the fragmentation within the SSD device itself (in the flash translation layer, or FTL) and fragmentation of file systems. Those are two separate layers; fragmentation occurs at both layers, but FTL fragmentation is of no concern to the file system. The difference here is that SSDs must make guarantees about how much user data they're capable of storing. Perhaps it's possible for very fragmented SSDs to violate that guarantee. It sounds dubious to me, but that's not really relevant. It doesn't apply to file systems because FSes don't make such guarantees.
On most regular file systems, file data and overflow extent (fragment) mappings are allocated from the same pool of storage. What this means is, if your files are very fragmented, you'll run out of disk space slightly quicker. But the filesystems will "handle" and "sustain" it fine, nothing will break. Out of space is a well-defined state for file systems to be in. File systems don't make any guarantees about how much of the disk space will be available for file content. The "limits" of file system fragmentation are bound by disk space. -- intgr [talk] 09:07, 1 December 2015 (UTC)
he doesn't appear to be an expert on file systems". Skipping over the questionable "appear" part, I didn't appeal to his expertise (even though 16 published books goes long way to establish expertise). Scott Hanselman, in this case, is a secondary source in good standing for relying information on behalf of the storage team.
You (and/or this source) are confusing the fragmentation [...]". The subject of discussion is file system fragmentation and the source is quite clear that it is Windows Defragmenter that is involved, that the operation is on volume level and that the fragment limit is an issue on both traditional and SSD storages. Furthermore, judging the source from a technical standpoint without another source is exactly original research. Finally, I have a purely unofficial FYI to add: Frankly, I think the last two paragraph are bits and peaces of random facts with highly technical words, sewn together like a Chimera, to frighten.
@ Codename Lisa: Ok, in some respects you are right, I have been unreasonable in my communication and I apologise for that. In other points, I think you're misrepresenting what I said and inverting burden of proof. But quarreling about that won't move this discussion forward, let's stop arguing about who-said-what-when.
I looked into this and it appears that NTFS indeed has issues with hitting file fragmentation limits. If I understand correctly, this is what you're talking about. These pages go into some depth about what's going on: [2] and [3]. Based on these, I think it's fair to state in the article that NTFS has a limit on the amount of fragments per file. But that doesn't extend to generalizations like "file systems cannot sustain unlimited fragmentation", or the more extreme "no file system can sustain unlimited fragmentation" — if by "not sustaining" you mean something other than degraded performance. Can you find any sources about fragmentation limits on other common file systems, such as FAT, ext4, UFS, ZFS, XFS?
I believe these pages even confirm that it is not an inherent limitation of all file systems, but an NTFS implementation detail: "A heavily fragmented file in an NTFS file system volume may not grow beyond a certain size caused by an implementation limit in structures that are used to describe the allocations", "In the newer versions of Windows, NTFS will stop fragmenting compressed and sparse files before the attribute list reaches 100% of its maximum size. This should put the issue to rest once and for all."
There are books written on the design and administration of Windows and many of them talk about NTFS. Can you find any that cover this issue? I gave it a shot ( [4], [5], [6]. [7], [8]) and sadly they barely cover fragmentation at all. For the sake of argument (please bear with me): if it applies only to NTFS, it's not documented in reliable secondary sources, and the primary source that I linked says that it's a rare occurrence, is it really important enough to cover it in the article? -- intgr [talk] 23:15, 2 December 2015 (UTC)
Based on these [sources], I think it's fair to state in the article that NTFS has a limit on the amount of fragments per file. But that doesn't extend to generalizations like "file systems cannot sustain unlimited fragmentation", or the more extreme "no file system can sustain unlimited fragmentation"-- intgr [talk] 09:57, 7 December 2015 (UTC)
And finally this NTFS distraction. I asked "so what?" and you ignored it."
I looked into this and it appears that NTFS indeed has issues with hitting file fragmentation limits. If I understand correctly, this is what you're talking about". A simple "no, I don't think that's the same thing because [...]" would have sufficed.
It is hard to figure out what this discussion is about. Beginning with the title, "Infinite fragmentation", there is no such thing: Try imagining how one might appoximate unbounded fragmentation: Even if 128 GB of ram were fragmented into 128 billion single bytes of storage, it is far from infinite.
The subject of SSDs not suffering from fragmentation appears to have two points-of-view. Of course a fragmented file cannot be read into memory with a single i/o operation ("read 123 clusters beginning with disk cluster 765 into address 1000"); instead many i/o operations are needed, at least one per fragment, though the i/o would not suffer from seek delays. But the on-disk overhead of tracking all those fragments affects SSDs, hard disks, thumb drives, floppies, etc. equally.
NTFS has what I think is an elegant way of describing stream cluster locations. If the data content is less than about 800 bytes (the actual value depends on the length of the filename and the number of aliases, security descriptor complexity, and number of streams in the file), the data is stored in the FRS (the $Mft structure which describes all aspects of the file) in the place where, for a larger file, the "run list" would be. The run list is a list data structure where each element corresponds to a "fragment": that is a starting cluster number and the number of consecutive clusters. The run list is cleverly compacted so that most elements are 3 to 5 bytes long. This means that 200 to 230 fragments can be held in one FRS (which are 1024 bytes long). I once examined a file (an exchange server database) with more than 22,000 fragments. It took only 101 extension FRSs to describe it all. 100 K of filesystem overhead to describe a 75 GiB file. Of course, had the file been defragmented, the overhead would have been 1 K.
NTFS manages all allocations by cluster units. There is no possibility of block fragmentation. 4 K (8 sectors) is the usual cluster size for NTFS due to NTFS-provided compression limitations. Any discovered bad sectors are managed by the cluster they are in.
All filesystems are inherently fragmentation limited: For a file of N clusters, the upper bound is N fragments. It is that simple. Likewise the filesystem's upper bound of freespace fragmentation is the number of free clusters. This is true of NTFS, FAT, UFS, ext2-4, etc. Of these, FAT is unique in that it does not provide a cluster count representation, but reasonable drivers look for sequential cluster numbers in the allocation table with the goal of treating it as a [cluster number, cluster count] structure.
— EncMstr ( talk) 09:26, 3 December 2015 (UTC)
@ EncMstr: Well, it appears that Codename Lisa is out. Can you please chime in, do you agree with the changes I made to the article? -- intgr [talk] 22:46, 10 December 2015 (UTC)
@ Intgr: TL;DR. It is obvious that the conversation above has lead to nowhere. Please explain to me: Why did you delete this:
However, regardless of the performance issue, no file system can sustain unlimited fragmentation. For each fragment, there needs to be one additional piece of metadata that records its location and affiliation to its file. Each piece of metadata itself occupies space and requires processing power and processor time. When the maximum fragmentation limit is reached, write requests will fail.<ref name=":0" />
Also this:
Hence Microsoft engineers recommend regularly defragmenting NTFS even when used on SSDs.<ref name="hanselman" />
...fails verification. Period. You did read the blog posts I hope? Fleet Command ( talk) 15:13, 12 December 2015 (UTC)
regardless of the performance issue, no file system can sustain unlimited fragmentation.". First, it's not really clear what it means — the word "sustain" is not a technically accurate term. Obviously, a file system can run out of space faster if it needs to write out more metadata about file fragments.
For each fragment, there needs to be one additional piece of metadata that records its location and affiliation to its file. Each piece of metadata itself occupies space and requires processing power and processor time.is correct and accurate. I removed it because it's not directly related to the NTFS limitation I was describing. In retrospect, I probably should have moved it instead of deleting.
When the maximum fragmentation limit is reached, write requests will fail" was not deleted, but merely rephrased as "very fragmented files on NTFS can reach this limit, whereby writes to the file will fail even if there is free space available". The reference also wasn't deleted.
fails verification. Period. You did read the blog posts I hope?"
Is it unreasonable to connect these dots?Yes, not only it is unreasonable but also forbidden by the policy. Abandon anything that has NTFS in it. Windows supports FAT12, FAT16, FAT32, NTFS, ReFS, ExFAT and EFS. (I excluded file systems that cannot be defragged.) Abandon all other assumptions too, such as "Obviously, a file system can run out of space faster [~snip~]" and "If this fragmentation limit was a common problem [~snip~]". You are invoking false dichotomies while there are zillion other options. These assumptions are so dangerous that if you take them for granted and discuss, it can quickly alienate the other party.
the word "sustain" is not a technically accurate term. "Sustain", "support", "handle", "allow", "permit", "function properly with". Take your pick. Looking at the very top of the discussion, "handle" is chosen by S.H. I advise "support".
I got the impression that that's not what it means. What did you think it meant?
Abandon anything that has NTFS in it" — but why?
So now we've established that this fragmentation limit does not necessarily affect all file systems.' No! On the contrary, I made a point of being completely silent about whether it does or does not.
You say "Abandon anything that has NTFS in it" — but why?' Because we have insufficient data. You struggle to make up for it by bringing in sources from the Core team and Microsoft Support but you cannot fill this void without resorting to argumentum ad ignorantiam or other fallacies which are not tried yet. As long as you don't have a source that says "File System X has no fragmentation limit" or a source that says "all file systems have a hard fragmentation limit", any attempt to read between the line is non-scientific prejudice.
[~snip~] does not seem to pass WP:RS'. I investigated. Hanselman seems an author of good standing; we can trust him acting as a secondary source. Wikipedia articles cite the notoriously biased Paul Thurrott! Hanselman is in much better standing in comparison and in this specific post, he is receiving critical review of another author in good standing. Fleet Command ( talk) 13:03, 14 December 2015 (UTC)
First thing, first: You didn't answer my one question."
What did you think it meant?"? It was about this quote from the article: "regardless of the performance issue, no file system can sustain unlimited fragmentation." I found that the original was vague enough that it could mean multiple things. I didn't answer because it's a moot point now anyway: you rewrote this sentence in two edits and I agree with your new version, so I don't see any reason to continue discussing this.
Hanselman seems an author of good standing; we can trust him acting as a secondary source" You've done nothing to convince me that it meets the criteria set in WP:RS (your argument is a red herring: RS doesn't require "
good standing" and comparing it to another bad source has no bearing on RS). But I don't want to argue about this, for now I agree that the Hanselman source can be kept in the article.
I made a point of being completely silent about whether it does or does not." You're misunderstanding me here. I agree entirely with this point and that's exactly what I was trying to say. when I said "
does not necessarily affect all file systems" I meant: it might or it might not affect them all, we don't know for sure.
As long as you don't have a source that says "File System X has no fragmentation limit" or a source that says "all file systems have a hard fragmentation limit", any attempt to read between the line is non-scientific prejudice.
we have insufficient data"?
Although the phrase goes back 10 years to the initial article, What does fragmentation have to do with "allow in-place modification" mentioned in the very beginning of the article? DGerman ( talk) 01:12, 29 November 2016 (UTC)
Hello fellow Wikipedians,
I have just modified 2 external links on File system fragmentation. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
This message was posted before February 2018.
After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than
regular verification using the archive tool instructions below. Editors
have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the
RfC before doing mass systematic removals. This message is updated dynamically through the template {{
source check}}
(last update: 18 January 2022).
Cheers.— InternetArchiveBot ( Report bug) 16:31, 30 September 2017 (UTC)
This article is rated Start-class on Wikipedia's
content assessment scale. It is of interest to the following WikiProjects: | |||||||||||
|
The contents of the Can't extend page were merged into File system fragmentation on April 10, 2015. For the contribution history and old versions of the redirected page, please see its history; for the discussion at that location, see its talk page. |
To-do list for File system fragmentation:
Priority 6
|
Consider copy on write filesystems, which use immutable allocated blocks (like ZFS) This allocation will just waste time, and will not change the characteristics of the filesystem —Preceding unsigned comment added by 129.65.102.1 ( talk) 17:49, 27 February 2008 (UTC)
Wasn't there an attempt at a file system that attempted to avoid that by writing data in layers, from top to bottom, letting the old data drop? Could someone tell me its name please? Thanks! 84.129.170.44 ( talk) —Preceding undated comment added 04:23, 5 April 2009 (UTC).
For the record, I created this article as I was dissatisfied with the current defragmentation article that:
I have attempted to mitigate these somewhat, but ultimately decided to write this article. I don't know if I can get it into a good enough shape to be merged with "defragmentation" (if at all), but I will try, and I will cite genuine research in the progress. It may or may not be considered a "rewrite" at this point. Any criticisms and comments are very welcome. -- intgr 03:53, 14 December 2006 (UTC)
While I myself added this to the article, I'm not sure it is fair to consider "related file fragmentation" a type of fragmentation. While research dealing with fragmentation very often also touches the topic of keeping related files together (e.g, files in a single directory), I don't think I can recall any instances where it's actually referred to as "fragmentation" per se.
However, consider when an archive is unpacked. As all files are decompressed in order, they will likely be laid out sequentially. But when time goes on, and files are added and deleted, the directory tree in which the files were decompressed into, becomes less and less "contiguous", e.g., can be considered "fragmented". -- intgr 14:21, 19 December 2006 (UTC)
I struck the Mac OS X note, since it isn't what actually happens. Mac OS X/HFS+ do not defrag at idle. What happens is that when a fragmented file is opened, it is defragged (unless the system has been up less than some specific time, and I forget what that time is). Thus, there's no "at idle" about it. Now if there's a seperate "at idle" process, by all means put the claim back in (but please reference it). Thanks. :) -- Steven Fisher 18:58, 19 December 2006 (UTC)
As the article has been tagged with {{ merge| Defragmentation}}, does anyone have ideas how to do that (e.g., what to merge, what not to merge, and what to throw away)? -- intgr 14:32, 30 January 2007 (UTC)
Although it would seem logical to merge the "Defragmentation" and the "File system fragmentation" articles, the first will naturally focus on practical aspects of dealing with the problem, and the second on a more theoretical understanding of the root cause of the problem. Combined into one article, there is a danger that it will get overly complex -- or that important material will be deleted to keep the article simple.-- 69.87.193.53 19:22, 18 February 2007 (UTC)
I totally disagree...defragging is the way by which the natural order is kept...extra work that users have to do is good for their character. —The preceding
unsigned comment was added by
68.249.171.240 (
talk)
I also disagree, merging the two articles would lead to one gigantic overly complex article. As stated earlier, information would probably be cut for the sake of simplicity, resulting in an incomplete article overall. Just link from the fragmentation article to the defragmentation article. -- Rollerskatejamms 13:07, 13 March 2007 (UTC)
A much better method, only merges the needless copy of File system fragmentation#Cause.
Spitfire ( talk) 01:35, 25 September 2008 (UTC)
There is POSIX fallocate API for allocating predermined size for file (or part of it). It have two motivations: make sure there is enaugh free space for file (so write operations will not file becuase of no space), and file can be preallocated on single extent on disk if possible in optimal manner.
Some tools already uses it to decress fratgmentation. —Preceding unsigned comment added by 149.156.67.102 ( talk) 21:50, 7 December 2009 (UTC)
User:Codename Lisa recently changed some content in this article and introduced a concept of "infinite fragmentation", which I reverted. He/she started a discussion at User talk:Intgr#File system fragmentation, which I have moved here:
It’s also somewhat of a misconception that fragmentation is not a problem on SSDs. If an SSD gets too fragmented you can hit maximum file fragmentation (when the metadata can’t represent any more file fragments) which will result in errors when you try to write/extend a file. Furthermore, more file fragments means more metadata to process while reading/writing a file, which can lead to slower performance.
Additionally, there is a maximum level of fragmentation that the file system can handle. [... Read on]
Sorry, I was perhaps too blunt. I was mainly ticked off by the weird term "infinite fragmentation" (that it appears you made up, hence "original research") and the claim that filesystems can't "sustain" some level of fragmentation.
The only source you're citing is a post on the personal blog of Scott Hanselman. Looking at his publications, he doesn't appear to be an expert on file systems. I don't claim that he's incompetent, maybe just makes oversimplifications, which I think are too misleading for an encyclopedia.
You (and/or this source) are confusing the fragmentation within the SSD device itself (in the flash translation layer, or FTL) and fragmentation of file systems. Those are two separate layers; fragmentation occurs at both layers, but FTL fragmentation is of no concern to the file system. The difference here is that SSDs must make guarantees about how much user data they're capable of storing. Perhaps it's possible for very fragmented SSDs to violate that guarantee. It sounds dubious to me, but that's not really relevant. It doesn't apply to file systems because FSes don't make such guarantees.
On most regular file systems, file data and overflow extent (fragment) mappings are allocated from the same pool of storage. What this means is, if your files are very fragmented, you'll run out of disk space slightly quicker. But the filesystems will "handle" and "sustain" it fine, nothing will break. Out of space is a well-defined state for file systems to be in. File systems don't make any guarantees about how much of the disk space will be available for file content. The "limits" of file system fragmentation are bound by disk space. -- intgr [talk] 09:07, 1 December 2015 (UTC)
he doesn't appear to be an expert on file systems". Skipping over the questionable "appear" part, I didn't appeal to his expertise (even though 16 published books goes long way to establish expertise). Scott Hanselman, in this case, is a secondary source in good standing for relying information on behalf of the storage team.
You (and/or this source) are confusing the fragmentation [...]". The subject of discussion is file system fragmentation and the source is quite clear that it is Windows Defragmenter that is involved, that the operation is on volume level and that the fragment limit is an issue on both traditional and SSD storages. Furthermore, judging the source from a technical standpoint without another source is exactly original research. Finally, I have a purely unofficial FYI to add: Frankly, I think the last two paragraph are bits and peaces of random facts with highly technical words, sewn together like a Chimera, to frighten.
@ Codename Lisa: Ok, in some respects you are right, I have been unreasonable in my communication and I apologise for that. In other points, I think you're misrepresenting what I said and inverting burden of proof. But quarreling about that won't move this discussion forward, let's stop arguing about who-said-what-when.
I looked into this and it appears that NTFS indeed has issues with hitting file fragmentation limits. If I understand correctly, this is what you're talking about. These pages go into some depth about what's going on: [2] and [3]. Based on these, I think it's fair to state in the article that NTFS has a limit on the amount of fragments per file. But that doesn't extend to generalizations like "file systems cannot sustain unlimited fragmentation", or the more extreme "no file system can sustain unlimited fragmentation" — if by "not sustaining" you mean something other than degraded performance. Can you find any sources about fragmentation limits on other common file systems, such as FAT, ext4, UFS, ZFS, XFS?
I believe these pages even confirm that it is not an inherent limitation of all file systems, but an NTFS implementation detail: "A heavily fragmented file in an NTFS file system volume may not grow beyond a certain size caused by an implementation limit in structures that are used to describe the allocations", "In the newer versions of Windows, NTFS will stop fragmenting compressed and sparse files before the attribute list reaches 100% of its maximum size. This should put the issue to rest once and for all."
There are books written on the design and administration of Windows and many of them talk about NTFS. Can you find any that cover this issue? I gave it a shot ( [4], [5], [6]. [7], [8]) and sadly they barely cover fragmentation at all. For the sake of argument (please bear with me): if it applies only to NTFS, it's not documented in reliable secondary sources, and the primary source that I linked says that it's a rare occurrence, is it really important enough to cover it in the article? -- intgr [talk] 23:15, 2 December 2015 (UTC)
Based on these [sources], I think it's fair to state in the article that NTFS has a limit on the amount of fragments per file. But that doesn't extend to generalizations like "file systems cannot sustain unlimited fragmentation", or the more extreme "no file system can sustain unlimited fragmentation"-- intgr [talk] 09:57, 7 December 2015 (UTC)
And finally this NTFS distraction. I asked "so what?" and you ignored it."
I looked into this and it appears that NTFS indeed has issues with hitting file fragmentation limits. If I understand correctly, this is what you're talking about". A simple "no, I don't think that's the same thing because [...]" would have sufficed.
It is hard to figure out what this discussion is about. Beginning with the title, "Infinite fragmentation", there is no such thing: Try imagining how one might appoximate unbounded fragmentation: Even if 128 GB of ram were fragmented into 128 billion single bytes of storage, it is far from infinite.
The subject of SSDs not suffering from fragmentation appears to have two points-of-view. Of course a fragmented file cannot be read into memory with a single i/o operation ("read 123 clusters beginning with disk cluster 765 into address 1000"); instead many i/o operations are needed, at least one per fragment, though the i/o would not suffer from seek delays. But the on-disk overhead of tracking all those fragments affects SSDs, hard disks, thumb drives, floppies, etc. equally.
NTFS has what I think is an elegant way of describing stream cluster locations. If the data content is less than about 800 bytes (the actual value depends on the length of the filename and the number of aliases, security descriptor complexity, and number of streams in the file), the data is stored in the FRS (the $Mft structure which describes all aspects of the file) in the place where, for a larger file, the "run list" would be. The run list is a list data structure where each element corresponds to a "fragment": that is a starting cluster number and the number of consecutive clusters. The run list is cleverly compacted so that most elements are 3 to 5 bytes long. This means that 200 to 230 fragments can be held in one FRS (which are 1024 bytes long). I once examined a file (an exchange server database) with more than 22,000 fragments. It took only 101 extension FRSs to describe it all. 100 K of filesystem overhead to describe a 75 GiB file. Of course, had the file been defragmented, the overhead would have been 1 K.
NTFS manages all allocations by cluster units. There is no possibility of block fragmentation. 4 K (8 sectors) is the usual cluster size for NTFS due to NTFS-provided compression limitations. Any discovered bad sectors are managed by the cluster they are in.
All filesystems are inherently fragmentation limited: For a file of N clusters, the upper bound is N fragments. It is that simple. Likewise the filesystem's upper bound of freespace fragmentation is the number of free clusters. This is true of NTFS, FAT, UFS, ext2-4, etc. Of these, FAT is unique in that it does not provide a cluster count representation, but reasonable drivers look for sequential cluster numbers in the allocation table with the goal of treating it as a [cluster number, cluster count] structure.
— EncMstr ( talk) 09:26, 3 December 2015 (UTC)
@ EncMstr: Well, it appears that Codename Lisa is out. Can you please chime in, do you agree with the changes I made to the article? -- intgr [talk] 22:46, 10 December 2015 (UTC)
@ Intgr: TL;DR. It is obvious that the conversation above has lead to nowhere. Please explain to me: Why did you delete this:
However, regardless of the performance issue, no file system can sustain unlimited fragmentation. For each fragment, there needs to be one additional piece of metadata that records its location and affiliation to its file. Each piece of metadata itself occupies space and requires processing power and processor time. When the maximum fragmentation limit is reached, write requests will fail.<ref name=":0" />
Also this:
Hence Microsoft engineers recommend regularly defragmenting NTFS even when used on SSDs.<ref name="hanselman" />
...fails verification. Period. You did read the blog posts I hope? Fleet Command ( talk) 15:13, 12 December 2015 (UTC)
regardless of the performance issue, no file system can sustain unlimited fragmentation.". First, it's not really clear what it means — the word "sustain" is not a technically accurate term. Obviously, a file system can run out of space faster if it needs to write out more metadata about file fragments.
For each fragment, there needs to be one additional piece of metadata that records its location and affiliation to its file. Each piece of metadata itself occupies space and requires processing power and processor time.is correct and accurate. I removed it because it's not directly related to the NTFS limitation I was describing. In retrospect, I probably should have moved it instead of deleting.
When the maximum fragmentation limit is reached, write requests will fail" was not deleted, but merely rephrased as "very fragmented files on NTFS can reach this limit, whereby writes to the file will fail even if there is free space available". The reference also wasn't deleted.
fails verification. Period. You did read the blog posts I hope?"
Is it unreasonable to connect these dots?Yes, not only it is unreasonable but also forbidden by the policy. Abandon anything that has NTFS in it. Windows supports FAT12, FAT16, FAT32, NTFS, ReFS, ExFAT and EFS. (I excluded file systems that cannot be defragged.) Abandon all other assumptions too, such as "Obviously, a file system can run out of space faster [~snip~]" and "If this fragmentation limit was a common problem [~snip~]". You are invoking false dichotomies while there are zillion other options. These assumptions are so dangerous that if you take them for granted and discuss, it can quickly alienate the other party.
the word "sustain" is not a technically accurate term. "Sustain", "support", "handle", "allow", "permit", "function properly with". Take your pick. Looking at the very top of the discussion, "handle" is chosen by S.H. I advise "support".
I got the impression that that's not what it means. What did you think it meant?
Abandon anything that has NTFS in it" — but why?
So now we've established that this fragmentation limit does not necessarily affect all file systems.' No! On the contrary, I made a point of being completely silent about whether it does or does not.
You say "Abandon anything that has NTFS in it" — but why?' Because we have insufficient data. You struggle to make up for it by bringing in sources from the Core team and Microsoft Support but you cannot fill this void without resorting to argumentum ad ignorantiam or other fallacies which are not tried yet. As long as you don't have a source that says "File System X has no fragmentation limit" or a source that says "all file systems have a hard fragmentation limit", any attempt to read between the line is non-scientific prejudice.
[~snip~] does not seem to pass WP:RS'. I investigated. Hanselman seems an author of good standing; we can trust him acting as a secondary source. Wikipedia articles cite the notoriously biased Paul Thurrott! Hanselman is in much better standing in comparison and in this specific post, he is receiving critical review of another author in good standing. Fleet Command ( talk) 13:03, 14 December 2015 (UTC)
First thing, first: You didn't answer my one question."
What did you think it meant?"? It was about this quote from the article: "regardless of the performance issue, no file system can sustain unlimited fragmentation." I found that the original was vague enough that it could mean multiple things. I didn't answer because it's a moot point now anyway: you rewrote this sentence in two edits and I agree with your new version, so I don't see any reason to continue discussing this.
Hanselman seems an author of good standing; we can trust him acting as a secondary source" You've done nothing to convince me that it meets the criteria set in WP:RS (your argument is a red herring: RS doesn't require "
good standing" and comparing it to another bad source has no bearing on RS). But I don't want to argue about this, for now I agree that the Hanselman source can be kept in the article.
I made a point of being completely silent about whether it does or does not." You're misunderstanding me here. I agree entirely with this point and that's exactly what I was trying to say. when I said "
does not necessarily affect all file systems" I meant: it might or it might not affect them all, we don't know for sure.
As long as you don't have a source that says "File System X has no fragmentation limit" or a source that says "all file systems have a hard fragmentation limit", any attempt to read between the line is non-scientific prejudice.
we have insufficient data"?
Although the phrase goes back 10 years to the initial article, What does fragmentation have to do with "allow in-place modification" mentioned in the very beginning of the article? DGerman ( talk) 01:12, 29 November 2016 (UTC)
Hello fellow Wikipedians,
I have just modified 2 external links on File system fragmentation. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
This message was posted before February 2018.
After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than
regular verification using the archive tool instructions below. Editors
have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the
RfC before doing mass systematic removals. This message is updated dynamically through the template {{
source check}}
(last update: 18 January 2022).
Cheers.— InternetArchiveBot ( Report bug) 16:31, 30 September 2017 (UTC)