Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years...
But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives.
I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
At least on iPhone, it basically doesn't do anything.
Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that.
Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps.
But the switcher shows every app ever opened and tries to pretend these apps aren't being closed.
Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months
You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
>Otherwise the claimed defragmentation here is not actually resulting in sequential data.
I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes.
> Fragmentation and extent allocation were adding measurable variance, even on NVMe,
Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort of coincidence due to other factors and that this defragmentation won’t achieve much (except for wearing out your SSD even faster in certain cases).
SSD still have slower reads when the data isn't sequential. Not slow enough to matter, especially with extents meaning the data is probably in a handful of locations, not 10000, but it's absolutely measurable if your blocks are small enough (<1MB typically, though disk benchmarks usually use 64KB random reads and sometimes 4KB).
Except most SSD also don't store pages sequentially internally (hidden flash wear leveling is even in microsd cards now), and the dram/SLC buffer areas are finite especially on low end budget hardware. Don't defrag your SSD dude. =3
It's more complex than that... Sequential reads are faster than random reads across most mediums (including system ram), but if there's enough prefetching, you might get better throughput on an SSD with the data fragmented than contiguous because SSDs can often read from multiple regions in parallel and contiguous data might not allow for that.
If you're at the point where you're optimizing for this, you've got some really high performance requirements though. And you'll have to do your own testing, because rules of thumb won't do.
Indeed, most Linux setups supporting trim, already defer these operations to a weekly schedule to reduce wear, and most fs will optimize in 10MB or 25MB chunks given unlike HDD... the SSD seeks are nearly constant time. Logging fs like f2fs, are content aware so will auto re-locate hot and cold (rarely modified) file types, and despite the log-structure... on an SSD performance losses are often surprisingly negligible.
Most modern NVMe with dram cache and SLC buffer areas also defer committing pages to low-endurance flash areas. And most kernel tweakers will set swapiness to 1 on SSD/NVMe machines to try to keep stuff buffered in dram as long as reasonably possible. It is a space-time tradeoff that can boost a desktop machine performance especially with preload daemon active.
If people want ludicrous speed... than just run ext4 with a separate 128GB journal NVMe drive on a split PCIe x4 bus.
Defrag on most modern drives usually just fills these buffer areas full, and things grind to the slowest i/o choke point. =3
This cool, but can we get an option to emulate HDD noises please? I really miss the whirrs, clicks and creaks. And the little LED that indicated disk activity... but that'd be a separate project.
Coil whine is common in budget setups, but often it is also just cheap MLCC Piezoelectric acoustic harmonic noise. Most EE impassioned pleas for large solid polymer capacitors to knock down the noise completely... are often ignored for cost reasons, or factory replaced with liquid-electrolytic type that slowly degrade over time until they literally explode on occasion.
The "Fast, Cheap, or Good... choose any two..." joke is very real. =3
This made me think of https://e4rat.sourceforge.net/, which moves boot files into sequential runs, which was helpful on spinning disks where seeks were slow but sequential reads were fast.
Just as an idea for a feature your defragmenter could have.
(Also this is a side thought; but I wonder if AWS AMIs and EBS volumes will boot faster if also organized sequentially.)
> Also this is a side thought; but I wonder if AWS AMIs and EBS volumes will boot faster if also organized sequentially.
Probably a measurable, but small, difference. I would expect it's not worth the effort unless you're spending a lot of time booting, and even then, there's probably better things to work on in the boot process, such as reducing the amount of things that run or reducing the size of them.
This is the most useless thing I've seen on here lately. It's not necessary on hard drives, as the page cache did a decent job of improving disk access even in the 90s, let alone with today's fast HDDs. And if you're on an SSD... it'll just burn that out faster.
Page caches are nice, but you would see meaningful reduction in load times for the OS and applications by getting data in order on spinning disks. Especially games that used most of your ram and loaded reasonably large level files.
fat seemed particularly prone to spreading files into very small chunks across the disk surface; it's not so big of a deal when fragmented files are made up of large contiguous chunks.
I have a question about disk defragmenter. I've asked a couple of people before and they've said it is not possible but iirc if I stated disk defragmenter and then even so much as moved the mouse after it started it would start with the first boxes again. Do you remember this too?
Beautiful.
Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years...
But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives.
I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
> My partner complains about me closing apps on my phone consistently.
I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?
At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that.
Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps.
But the switcher shows every app ever opened and tries to pretend these apps aren't being closed. Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months
I do this also - but wonder because I'll have some app open, phone gets warm. Close the app and it cools down.
This is literally always how iOS has worked. You can refer to the application lifecycle documentation: https://developer.apple.com/documentation/uikit/managing-you...
When you die, you will leave behind nice orderly contiguous files. It's thoughtful. hehe
Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?
Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
Is it possible to determine which sectors are physically sequential given remapping for wear-leveling?
Otherwise the claimed defragmentation here is not actually resulting in sequential data.
You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
>Otherwise the claimed defragmentation here is not actually resulting in sequential data.
I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
Prefetching applies to SSDs too, and that's generally sequential in some sense.
only HDD drives I use are in a ZFS pool which has a very usable fragmentation value from zpool.
I like the graphical interface and the movement of the blocks. Brings back memories of defragmenting hard drives once a month and seeing noticeable performance improvements sometimes.
> Fragmentation and extent allocation were adding measurable variance, even on NVMe,
Why exactly would there be a measurable variance on NVMe? I understand there could be some impact on magnetic hard drives. This sounds like some sort of coincidence due to other factors and that this defragmentation won’t achieve much (except for wearing out your SSD even faster in certain cases).
SSD still have slower reads when the data isn't sequential. Not slow enough to matter, especially with extents meaning the data is probably in a handful of locations, not 10000, but it's absolutely measurable if your blocks are small enough (<1MB typically, though disk benchmarks usually use 64KB random reads and sometimes 4KB).
Except most SSD also don't store pages sequentially internally (hidden flash wear leveling is even in microsd cards now), and the dram/SLC buffer areas are finite especially on low end budget hardware. Don't defrag your SSD dude. =3
It's more complex than that... Sequential reads are faster than random reads across most mediums (including system ram), but if there's enough prefetching, you might get better throughput on an SSD with the data fragmented than contiguous because SSDs can often read from multiple regions in parallel and contiguous data might not allow for that.
If you're at the point where you're optimizing for this, you've got some really high performance requirements though. And you'll have to do your own testing, because rules of thumb won't do.
>defragmentation won’t achieve much
Indeed, most Linux setups supporting trim, already defer these operations to a weekly schedule to reduce wear, and most fs will optimize in 10MB or 25MB chunks given unlike HDD... the SSD seeks are nearly constant time. Logging fs like f2fs, are content aware so will auto re-locate hot and cold (rarely modified) file types, and despite the log-structure... on an SSD performance losses are often surprisingly negligible.
Most modern NVMe with dram cache and SLC buffer areas also defer committing pages to low-endurance flash areas. And most kernel tweakers will set swapiness to 1 on SSD/NVMe machines to try to keep stuff buffered in dram as long as reasonably possible. It is a space-time tradeoff that can boost a desktop machine performance especially with preload daemon active.
If people want ludicrous speed... than just run ext4 with a separate 128GB journal NVMe drive on a split PCIe x4 bus.
Defrag on most modern drives usually just fills these buffer areas full, and things grind to the slowest i/o choke point. =3
This cool, but can we get an option to emulate HDD noises please? I really miss the whirrs, clicks and creaks. And the little LED that indicated disk activity... but that'd be a separate project.
Be in a very quiet room and I can still hear my SSD.
I suspect the noise is caused by the power supply emitting a tiny bit of 'coil whine' when put under more load
Coil whine is common in budget setups, but often it is also just cheap MLCC Piezoelectric acoustic harmonic noise. Most EE impassioned pleas for large solid polymer capacitors to knock down the noise completely... are often ignored for cost reasons, or factory replaced with liquid-electrolytic type that slowly degrade over time until they literally explode on occasion.
The "Fast, Cheap, or Good... choose any two..." joke is very real. =3
As long as we keep trying to bring windows mainstream shit for penguins, I am happy with that. Btw I am building similar tools for linux hahaha.
This made me think of https://e4rat.sourceforge.net/, which moves boot files into sequential runs, which was helpful on spinning disks where seeks were slow but sequential reads were fast.
Just as an idea for a feature your defragmenter could have.
(Also this is a side thought; but I wonder if AWS AMIs and EBS volumes will boot faster if also organized sequentially.)
> Also this is a side thought; but I wonder if AWS AMIs and EBS volumes will boot faster if also organized sequentially.
Probably a measurable, but small, difference. I would expect it's not worth the effort unless you're spending a lot of time booting, and even then, there's probably better things to work on in the boot process, such as reducing the amount of things that run or reducing the size of them.
This is the most useless thing I've seen on here lately. It's not necessary on hard drives, as the page cache did a decent job of improving disk access even in the 90s, let alone with today's fast HDDs. And if you're on an SSD... it'll just burn that out faster.
I love it!
Page caches are nice, but you would see meaningful reduction in load times for the OS and applications by getting data in order on spinning disks. Especially games that used most of your ram and loaded reasonably large level files.
fat seemed particularly prone to spreading files into very small chunks across the disk surface; it's not so big of a deal when fragmented files are made up of large contiguous chunks.
So now we are using vibe coded disk fragmenters that actually touch the disk state on your machine. It has little to no tests.
Now HNers here don't even question the soundness of this software and just assume it works as it was suspiciously built in 4 days.
One side-effect of the AI mania is that developers have lost the ability to reason around the verification stage of vibe coded software.
Oh dear.
I have a question about disk defragmenter. I've asked a couple of people before and they've said it is not possible but iirc if I stated disk defragmenter and then even so much as moved the mouse after it started it would start with the first boxes again. Do you remember this too?