Microsoft Redesigns chkdsk For Windows 8, Improves NTFS Health Model 219
MojoKid writes "Microsoft can't do anything to magically make hard drives stop failing when parts go bad, but Redmond is rolling out a new NTFS health model for Windows 8 with a redesigned chkdsk tool for disk corruption detection and fixing. In past versions of the chkdsk and NTFS health model, the file system volume was either deemed healthy or not healthy. In Windows 8, Microsoft is changing things up. Rather than hours of downtime, Windows 8 splits the process into phases that include 'Detect Corruption,' 'Online Self-Healing,' 'Online Verification,' 'Online Identification & Logging,' and 'Precise & Rapid Correction.'"
New options? (Score:2)
Why is Online Self Healing different and from "Make the damn FS work properly"?
WTF FS is that has problems that can be fixed online?
Re:New options? (Score:5, Informative)
Re: (Score:2)
In other words big ass journals and lots of buffering. Sounds like a Microsoft solution; just give us more RAM and drive space.
Re:New options? (Score:4, Insightful)
RAM is cheap. Why not use it?
Re: (Score:2)
Re: (Score:2)
Linux install = OS + full working set of applications.
Try comparing like with like.
Re: (Score:3)
Linux uses free memory for additional buffers, but drops them immediately if that memory is needed for anything else. This memory is counted as "used" even though it's available for allocation. The ideal system designed on this principle would never show any free memory once the total amount of data read from storage exceeds the physical RAM size.
Re: (Score:3)
there is no "even though"... almost all used memory is available for use. only difference is some allocated memory is never sent to swap..
There is a fundamental difference between a throwavay buffer and everything else -- buffer can be re-used at any time except while I/O on it is in progress.
ofcource.. first you need to deal with the broken unix design of overcomiting mem in the first place.
Overcommitting is neither Unix-originated, nor mandatory in Linux.
Re: (Score:3)
RAM is cheap. Why not use it?
That kind of thinking is why whenever you buy a computer or even an Android phone, you have a ton of things loaded in memory at startup that you don't even like, let alone actually use. It's also why Pac-Man on an iPod is a 50-megabyte app when the game used to fit in like 3k of ram.
Re: (Score:2)
Pac-Man is from an era when RAM was not cheap. According to a quick googling, the game fit on a 16k ROM, and had only 2k of RAM available for actual program use.
To be equivalent, pac-man on iPod would have to be a a 1-gigabyte app...
That kind of thinking is why whenever you buy a computer or even an Android phone, you have a ton of things loaded in memory at startup that you don't even like, let alone actually use.
The problem with that isn't those things are occupying memory. Memory unused is wasted. The problem is that those things have to be loaded from a disk. RAM is cheap, but disk is slow.
Re: (Score:3)
Er, because any filesystem can have nonfatal minor errors? For example a filesystem that uses a bitmap to track free space (like NTFS does) can have that bitmap corrupted by bad sectors or whatever. Things like that are sufficiently simple that you can attempt a repair while the volume is online, without major risk of failure.
source leaked!!! (Score:5, Funny)
here the highlight.
if disk.mbr.has_grub
for part in disc.partitions
if part.type.not_ours
chair.throw() # dammit... let's do something about it
part.raw_write(offset=random(1,part.size),data=random(1,255)) # voila'
end if
end for
end if
Re: (Score:2)
Wait.. why would low level Microsoft utilities be written in vimscript?
Online (Score:2)
Marketing dept. (Score:5, Funny)
Given the other phase names, I surprised the marketing department didn't call this "Detect Awesomeness!".
v8 chkdsk on windows 7? (Score:4, Interesting)
chkdsk is a standalone app. Can I use v8 on my v7 OS?
Re:v8 chkdsk on windows 7? (Score:5, Interesting)
Part of what the team has been doing since Windows 7 is to refactor large monolithic DLLs (like kernel32 and advapi) into smaller DLLs that are layered more properly and quicker to load (due to reduced size). Windows 8 continues this work. This is part of the whole "minwin" effort that lots of people in the (external to MS) rumor mill got excited about a few years back. (At least minwin is what they used to call it. Core system is another term used later.)
As a result of this work, in Win7 and Win8, most binaries you find in system32 depend on newfangled DLLs not present in a downlevel system and will thus not load on an older version of the OS. So I doubt it. (Not to mention that this new thing in particular, since it's about modifying online filesystems, might depend on new ioctls or other hooks in ntfs.sys or maybe some other driver - though I can't say I know that for sure.)
-Former Windows dev.
Re: (Score:2)
In the past it has been possible to run newer versions of chkdsk on older versions of the filesystem, just not on the older version of the OS. Chkdsk for Windows 7 requires the NTFS driver from Windows 7 so won't work on XP, but you can boot a Windows 7 install disc and run chkdsk on an XP partition.
Next Gen File system (Score:5, Informative)
I was curious as to why MS is continuing on with NTFS, surely there must be something newer coming out of their R&D labs. So a quick google turned up this from the same blog, but earlier this year: building the next generation file system for windows refs [msdn.com]
Re:Next Gen File system (Score:5, Informative)
In fact the whole blog is interesting Building Windows 8 [msdn.com]
Re: (Score:3)
surely there must be something newer coming out of their R&D labs.
Yes, they have something newer that they will release with GNU Hurd, when it comes out . . .
Re: (Score:2)
I believe ReFS is only going to be included in the Server version of Windows 8, not the regular one, which will stick with NTFS.
Re: (Score:2)
Right?
Re: (Score:2)
You want something new out of MS labs on your computer?
Re: (Score:2)
I said it many time, MS Research is not MS Product... :
Go read the best paper on Monads called
Tackling the Awkward Squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell
or a recent one named :
A monad for deterministic parallelism
, those are world acclaimed pure CS research papers. A part of it went into F# and another went to LINQ. But the rest of it is still not productivized...
A similar thing could be said about the series of papers on the Courrier device but like almost everything else from MS research they failed to productivizeit.
For the Professional Edition (Score:2)
The Professional Edition will include a DVD of Daily Affirmation with Stuart Smalley, in case the user blames himself for the computer's failure during stressful times.
Re: (Score:2)
To bad it will not be able to play the DVD ... [slashdot.org]
Back in '99 (Score:2)
Disk corruption? (Score:2)
What they are actually doing is classifying 18 different forms of filesystem corruptions and are building the OS and filesystem drivers in such a way tha
Error correction? (Score:2)
Wake me up when there's pervasive Reed-Solomon error correction everywhere.
Re: (Score:2)
ZFS!
#1 thing I want - block-level checksums (Score:5, Interesting)
The number 1 feature I want in current filesystems is block-level checksums.
I've had to perform data recovery for a number of people recently (yes, backups help, but sometimes having them just 24 hours out of date means there are advantages to attempting to recover the data off the failed or failing drive or array)
Now, using a combination of tools I've been able to get the faulty drive to give me back data, but I've got no way whatsoever of knowing if the data it's given back to me is actually the data that was stored on it in the first place.
Having end-to-end checksums would easily allow me to assign a confidence level to data recovery procedures, letting me know that the data I have retrieved is what was stored - it would also allow better control over operations like fsck or chkdsk if the blocks that hold metadata are also checksummed, that way it would be possible to tell if a block has been randomly corrupted somehow, or if it's stored as intended.
Re: (Score:3, Interesting)
You must spend more time actually working with non-Windows systems. Multiple filesystems, most free, some commercial have been doing these sorts of things, and more for YEARS.
Re: (Score:2)
Yeah, like zfs or btrfs with checksumming of all [meta]data and built-in RAID.
Re: (Score:3)
Kindly have your own reading comprehension checked.
Non-Windows does NOT automatically and exclusively mean Linux.
Also, once that reactionary rash of yours stops flaring up, go investigate what ZFS and its tools, such as scrub, can do for people who care about data integrity.
Also, in your detailed reading of the article, did you note, for example, "NTFS detects", "NTFS attempts", "NTFS validates", "healing feature built into NTFS",
"introduced a new file system (emphasis added) ReFS"
Here, have another loo
Re: (Score:2)
I just hope both Apple and Linux developers would add something similar
You mean like btrfs? (which has many additional advantages that Microsoft can't simply "add" to NTFS without replacing it entirely; it's like how ext4 is a good improvement on the old filesystem design, but overall it's limited in very fundamental ways. NTFS is similar; it needs to be thrown out entirely)
Re: (Score:2)
It seems like Microsoft is really going out of it's way to innovate in Windows 8.
No Windows 8 is Microsoft implementing a lot of things that Linux/BSD already have. This would include an attempt to force a first gen "Duel OS" onto its users.
Duel OS? (Score:4, Funny)
En garde!!!
Re: (Score:2)
"I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems."
It's called SMART and has been around for a while already.
Warns you when your hard drive starts acting outside normal parameters or throwing soft errors (which it usually does before it actually fails).
Re: (Score:2)
Re: (Score:2)
Are you really buying into these marketing names? Are you really implying that Linux and OSX dont have something similar to a journaling filesystem?
We have no details on what this means, whether there is a background filesystem check going on at all times, whether it might increase wear and tear on the drive, or what; its a little early to start calling this a great development.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
How can blatant astroturf posts be +5 intredasting ?
Seriously, it read like a lazy marketing blurb...
This OR people never heard aboot S.M.A.R.T. reporting...
Re:No more hours of downtime (Score:5, Informative)
sane precautions with your data such as RAID and/or backing up your information
RAID and backing up should never be considered an "OR".
.. "RAID is not a backup strategy".
Repeat after me
Re:No more hours of downtime (Score:5, Insightful)
If there is one thing you can count on in the world, it's someone screaming "RAID is not a backup!" at the top of their lungs in any conversation dealing with RAID.
Yes, thank you. We get it. RAID does not protect against deleted files, etc. You can go back to shouting other contrarian favorites in other threads.
In the mean time, if and when one of the drives in my RAID-1 mirror fails, I'll be sure to throw its working partner straight into the garbage can. I certainly wouldn't use it to restore my entire filesystem that would have otherwise been obliterated.
I don't know about you, but I'm constantly deleting files by accident, and getting personal data destroying viruses (via a time machine from the 90s) where as my drives never, ever fail.
Re: (Score:2)
Raid is also not a full proof protection against disk corruption. The classic RAID model assumes only one disk in an array will fail at a time. With today's disk sizes, this assumption simply cannot be relied on. There are various, newer, RAID configurations that are more resilient than that, but they are rather expensive for casual use (you, obviously, don't).
The chances of one disk having latent errors that only show up during the intense operation of a disk sync are not insignificant. Such a problem took
Re:No more hours of downtime (Score:4, Interesting)
If anyone wants an example of why RAID should always have a backup solution and not just and/or solution. Please check http://dslreports.com/ [dslreports.com] , as they just recovered from a powerloss at nac.net which took their entire array system with it, and fudged 2 years worth of data, which had to be sent off for recovery. That was on April16th, the site is just starting to come back up in the last two days.
Some info here: https://docs.google.com/document/d/1kll86bDn_MgWoo6Ja7oHo_yvI0SCqggEvNWwPWIcrHY/edit?pli=1 [google.com]
Re:No more hours of downtime (Score:4, Insightful)
Right up until your primary gets some corruption and proceeds to mirror it to the other.
Re: (Score:2)
Not if you use ZFS. Corrupted files get fixed in a mirror.
Re: (Score:3)
Unless Im mistaken, there is no "primary" in RAID 1, nor does one drive "mirror to the other". Data writes sent to the controller are mirrored to both drives simultaneously. If for some reason on-disk data gets corrupted, that will not be "mirrored over" to the other in any scenario I can think of. If you got a bad sector on one disk for example, any errors induced by it would not appear on the second drive (how the array would behave in that scenario I dont know, but it likely depends on the raid contro
Re: (Score:3)
You should be fired. (Score:3, Insightful)
Seriously. Your strategy puts at risk all the writes during that rebuild process.
Re:No more hours of downtime (Score:4, Insightful)
That only "backs up" against drive failure. What happens if something gets deleted? What happens if a process goes mad and scribbles all over something important? What if someone breaks in?
Comment removed (Score:4, Informative)
Re: (Score:2)
What happens if something gets deleted?
Not that its a good idea to go without a backup....
But windows has since at least XP had a concept of "shadow copies", whereby you can view the state of the drive at various checkpoints, and recover data. Right click your C: volume, properties, shadow copies-- click one of the checkpoints and click to view the data. You can restore individual files quite quickly doing this.
Re: (Score:2)
Certainly doesn't replace a backup, but that's good to know about.
I assume these shadow files are counted as free space? Eg, when you actually do need the room they "make way" for the new data?
Re:No more hours of downtime (Score:4, Insightful)
Just delete an important file or directory on a RAID1 and see how much that "backup" protected you. Or install a virus. Or have data corruption on a disk.
Re: (Score:2)
you can back up a virus with a non RAID 1 back up system.
Re: (Score:2)
And if you are smart and have previous backups and/or incrementals, you can get what you need without bringing the virus along for the fun.
Re: (Score:2)
That's what removable media and rotation schemes are for.
Re: (Score:3)
No.
What happens if, say, you get a virus, or your app goes haywire, and The Critical Irreplaceable File gets corrupted or deleted?
RAID protects from hardware failures. It does nothing against software or human errors. And given my history with hardware, software and people, I'd say the first is generally the most reliable.
Re: (Score:3)
Re: (Score:2)
RAID protects against hardware malfunctions, but not user malfunctions. Backups protects against both.
Re: (Score:2)
I've always wondered about this, and I can see it for RAID 5 (also, I've seen RAID 5 setups fail irrecoverably), but what about RAID 1, where you're just mirroring. Wouldn't that essentially be the same as backing up to a different hard drive?
Drives don't need to fail for a backup to be needed. You might think you are formatting an SD card and end up formatting an essential partition. You could do like I did once while setting up an OS on a VM and told it format the wrong drive. Say what you will about Linux, but it continued to run even though the root partition had been reformatted out from underneath it. Or, it might be something as simple as someone goes in and deletes your LDF files trying to save hard drive space. It might even be a s
Re:No more hours of downtime (Score:5, Funny)
It seems like Microsoft is really going out of it's way to innovate in Windows 8. I just hope both Apple and Linux developers would add something similar, as it's hard drive failure can lead to huge problems. Yes, you can use RAID or something, but that will bring costs significantly up. It's better to see these things before failure actually happens.
Rather than take sane precautions with your data such as RAID and/or backing up your information, you want a warning 1 minute before your drive fails?
1 minute should be more than enough for anybody.
Re: (Score:2)
Can I have both? Can I have a drive that, on that one minute warning, immediately flushes it's cache and goes offline?
It would make recovery after replacement much smoother. Clone and go, no worries about incomplete writes etc.
Re: (Score:2)
SSD? Sure. NTFS has such a vibrant disk activity life, it is amazing. I mean, how is it even possible to constantly write something to the drive, like every second, all the time, even during idle. There is a huge gap for improvement. The current NTFS behaviour can only be called a calamity.
Re: (Score:2)
Re: (Score:2)
Defrag and indexing is automatically disabled on a SSD
Re: (Score:2)
Re:No more hours of downtime (Score:4, Interesting)
Because there is basically no point to it. The point of indexing is that rotational media has substantial access time penalties for fetching various info from random locations on the drive. By indexing it, you place all that metadata in one place so you can get a sequential read.
But with an SSD, there is not a substantial difference between random and sequential reads, and no significant penalty, so the indexing is not terribly useful. Additionally, indexing means more writes as well as increased background reads, which not only impacts performance (for little gain) but also wears out the drive.
Ditto for defragmenting; in fact defragging and indexing attack the same core problem, but from different angles, which is why neither is necessary on an SSD.
Re: (Score:2)
happily wearing my SSD in the process.
You don't have to worry about SSD wear any more than of a HDD's. It is only an issue if you use a memory card as your SSD.
Re: (Score:2)
Please stop spreading false information. Flash is flash. It has a limited number of write cycles. Straight [intel.com] from Intel [intel.com]:
Re: (Score:2)
Re: (Score:3)
You are essentially saying "yeah, if you don't write very much to it, the SSD will not wear out", which is exactly the obvious corollary to what I pointed out. It just is not so that the user doesn't have to worry about wearing out his SSD. It's a failure mode that has to be considered. You can't just wave it away.
1) You can write an HDD CONTINUOUSLY for its entire expected mechanical and electronic lifetime without ANY wearout due to the writing - it essentially has an infinite design number of cycles with
Re: (Score:3)
This "logging" you speak of is a feature of basically every modern filesystem used by basically any OS you might stick on a computer. HFS+, ext3+, btrfs, NTFS, all of them journal, and hence are slightly slower and slightly more active than non-journaling filesystems.
But most people with a clue recognize that those slight disadvantages are more than made up for by having a FS that remains consistent even after an unclean shutdown from ie a power outage.
As for the amount of activity you talk about, you are
Re: (Score:2)
"Speedup" features that waste time writing stuff to disk are very poorly named....
Re: (Score:2)
Windows, since XP, defragments the filesystem when idle.
You can turn this off via TweakUI for XP, I know this. Not sure how to do so on Vista/7.
Re: (Score:2)
Only Vista and above do that. XP did not allow you to schedule a defrag, IIRC.
Re: (Score:2)
I didn't say schedule.
To turn it off (try it in an XP VM, if you refuse to believe truth):
1. Run TweakUI
2. Click on "General" (the second item in the left hand pane)
3. Scroll the "Settings" list in the right hand pane.
4. The next to last item is "Optimize hard disk when idle".
Re: (Score:3)
Re: (Score:3)
Task manager can be tweaked to display the disk writes (I/O) of individual processes.
Processes --> View --> Select Columns
You can do something similar with Process Explorer,
which was an awesome program that got bought by Microsoft,
but has avoided the meddling that usually follows being aquired.
Process Explorer will show you the programs attached to processes like svchost.exe
so you can pinpoint exactly what's kicking up problems.
Re: (Score:2)
Open up task manager, go to performance, click on resource monitor. Use this to track down what is running on your system that generates so much disk activity. On my machine, the disk housing the C partition is at this moment at 0.00% activity, and has averaged about that for quite some time. Average IO for C: over the past few seconds was around 800 bytes/sec.
If what you posted is accurate for your system, you need to get it fixed.
Re: (Score:2)
Ever wonder why it's *always* turned on?
I just set up 2 servers that had 128GB RAM, running MS Server 2008 Enterprise. Guess what the boot drives had? Yep, a 128GB swap file.
Re: (Score:3)
I don't think it is. I think it's another of those far-too-obvious ones like "googlewatch" and "apple-fan" which I think have just been set up to troll.
If it's being paid for (as we;re meant to believe has been going on for ages) then the subtlety level has dropped an order of magnitude.
Also, the names of the accounts are all highly suspect - this one just so happens to be named after the programming language used in OS X and iOS? Come on!
Re: (Score:2)
Re:Improves chkdsk? Heh. (Score:4, Interesting)
Considering that the current chkdsk is actually capable of causing massive logical damage [slashdot.org] , Microsoft has a LONG way to go to make it function as intended.
You mean it's suspected of causing additional damage in a couple of comments.
It's very possible that there are long standing bugs. It's also possible that it just tried very hard on a hopelessly borked drive and failed.
Re: (Score:2)
A FAT USB drive (or Android phone) doesn't need to be 'safely removed'. You can just yank the thing and it's fine (as long as it's finished its r/w operations).
These two statements are mutually exclusive. The translation of your post, once only the facts remain, is "I hate windows." Why didnt you just say that you hate windows?
Re: (Score:2)
I thought making sure that all read/write operations have finished was the point of "safely remove".
No blinking light (Score:2)
You can just yank the thing and it's fine
This is true if you've set the drive's caching policy to "optimize for quick removal" rather than "optimize for performance". (Names are from memory and may not be exact.) "Optimize for quick removal", which Windows turns on automatically for removable media drives, syncs the file system continuously.
(as long as it's finished its r/w operations).
The trouble is figuring out when this has happened, as a lot of USB mass storage devices don't have a blinking access light. Only one of my USB flash drives has one, and my Android device does not. As maxwell d
Re: (Score:2)
Re: (Score:2)
and you can't boot from it or
Q) Can ReFS be used on removable media or drives?
No, this is not implemented or supported.
Re: (Score:2)
ReFS is the filesystem of the future -- and always will be?
ReFS will be coming in the next +1 version of Windows, and always will be?
Re: (Score:2)
... ... ...
I don't know if this is the real APK or not. Wow.
I wonder what entries he adds or removes from his hosts file when his car doesn't start.
Re: (Score:2)
It's called fsck, not chkdsk.
And the fact that ext4 doesn't support online fsck is a major annoyance for a lot of sysadmins.
It's not about running well or not, it's about the system being shut down in a middle of a write operation.
btrfs can do online fsck, and I'm looking forward to it just for that.
Re: (Score:2)
Yeah. e4fsck is a much more modern name.
Re: (Score:2)
Ah, finally, all those Online-something goodies for chkdsk. I've always wanted to have Windows Activation Wizard popping up before my chkdsk session, just in case I was in doubt was my copy legitimate. (It is btw)
I assume they meant "online" as in "in the background while the rest of the system is operational", rather than "connected to the Internet and phoning home to Microsoft", but I could be wrong.
Re: (Score:2)
Re: (Score:3)
Cutting the power on a modern file system also results in it having to finish what it was up to when the lights went out.