SSDs Cause Crisis For Digital Forensics 491
rifles only writes "Firmware built into many solid state drives (SSDs) to improve their storage efficiency could be making forensic analysis at a later date by police forces and intelligence agencies almost impossible to carry out to legally safe standards, Australian researchers have discovered. They found that SSDs start wiping themselves within minutes after a quick format (or a file delete or full format) and can even do so when disconnected from a PC and rigged up to a hardware blocker." So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
Good. (Score:5, Insightful)
Deleted, should mean deleted.
Re:Good. (Score:4, Informative)
Re:Good. (Score:5, Informative)
The drive's firmware is responsible for remapping blocks, and so the OS can't really control what happens whenever the OS tries to permanently delete things by asking the drive nicely to do so. Consequently, if the drive decides it's best to remap the logical block silently while not deleting the cell contents, then the OS doesn't realise the data is still there. That's what the other SSD study noted.
On the other hand, in this situation, the thing doing the purging is the drive's firmware itself, not the OS, and the firmware knows for sure where the data is in the cells, and furthermore it's on a mission - to purge cells that it knows the filesystem is no longer using for data.
The purging that is being done here, is taking place with the specific intention of getting cells freshened up and ready to be written to in future without delay. Consequently we can reasonably expect that the drive firmware really *wants* to nuke cells that contain real data that is no longer needed according to the filesystem metadata since that's the only way to boost performance, and that's the GC's job.
If you want to check for yourself, try carrying out reads at the sector level yourself after running the experiment with the experimental setup described in the paper. You won't get much.
I guess what I'm saying is that both studies are right despite the apparent contradiction, since they're dealing with completely different circumstances and motivations for 'purging' - OS or drive GC controlled.
Graeme.
p.s. Does anyone know of a postdoc position in AI, robotics, or bioinformatics in Grenoble, France? I'm currently looking for a new job.
Re:Good. (Score:4, Informative)
These drives are oblivious to the file-system, which is why there needs to be a TRIM command which allows the OS to say "hey, I don't care about that page (4 KB) any longer."
Some SSDs actually do understand a few filesystem formats (maybe just NTFS?), and used this to GC unused blocks before TRIM was implemented.
Re: (Score:3)
That's mostly correct, but incomplete. The TRIM command marks a range of logical flash pages as unused, but it's not correct to say that pages are not wiped except when a TRIM occurs. When the flash controller needs to write a new copy of a flash page, it may leave around an older copy of that page. Those older copies of the pages may later be wiped on an as-time-permits basi
Re: (Score:3)
Inaccurate.
If the TRIM command is used, from the firmware's perspective, wouldn't the original statement be completely accurate? The device knows which cells contain data. Using the TRIM command, the device now knowns which cells contain data and which cells the filesystem is no longer using for data.
Unless you're advocating everyone stop using the TRIM command, it appears you're being overly pedantic.
Re:Good. (Score:4, Interesting)
SSDs do wear-leveling. If one logical block is busy, it will migrate from physical block to physical block over time to balance activity counts. The physical blocks left behind have data that the OS cannot purge by any means. The drive's firmware is certainly aware of these blocks, and will eventually wipe them for re-use (though given this is a wear-leveling scheme, reclamition isn't necessarily that urgent).
This is all separate from file system activity: in addition to the above, the firmware purges TRIMmed blocks, and presumably does so in preference to blocks rotated out do to wear-leveling, so a block that did get TRIMmed would be agressively purged.
Re:Good. (Score:5, Interesting)
Inaccurate. These drives are oblivious to the file-system
Hey there. Unfortunately, what you've written isn't correct, and I encourage you to read into modern SSD garbage collection. These drives really do open up the filesystem by themselves, and look for deleted files and purge them, without being asked to do so by the OS. Otherwise, can you explain how file deletion using a non-TRIM OS, followed by a drive being connected to a non-TRIM OS with a write-blocker, would result in data being purged? (we proved this experimentally, and you can reproduce it in your own home using the supplied software and experimental parameters).
But you are wrong that the SSD is doing the purging outside of OS intervention.. the OS must specifically mark pages for purging.
The GC on the drive in question is specifically designed to provide performance improvement for *OS that do not have TRIM*. If you don't believe me, look up the drive model in the paper and google it for forum discussions of its behaviour (or buy one for yourself, and watch it happen in realtime using the probe program! It's freaky to see a drive rapidly purge it's supposedly recoverable data when connected to a non-TRIM OS)
Graeme.
Re: (Score:3)
Re:Good. (Score:4, Informative)
If you take a look at: http://www.jdfsl.org/subscriptions/JDFSL-V5N3-Bell.pdf [jdfsl.org] , you'll see the drive model we used was the Corsair P64 (see page 16).
There's also support for GC on the OCZ vertex 2 drives, I think. However, some bad news and good news for you:
The bad news is, the firmware GC implementation we looked at only supported GC on NTFS partitions.
The good news is, you can use NTFS with macos if you like (e.g. through FUSE), or you can wait for MacOS Lion, which is about to be released, and which is rumoured to support TRIM directly. Shouldn't be long!
Graeme.
Re:Good. (Score:4, Informative)
"To clarify - is this deletion occurring because the drive analyzed the filesystem metadata and decided it would be a good idea to zero some sectors, or is it an asynchronous response to a fast TRIM command applied to an extent covering the whole drive?"
It's not TRIM. We used a non-TRIM OS and a write blocker for this reason. The manufacturer has mentioned a garbage collector that works with NTFS, and the only way it could work is by zero-ing areas of the SSD using information gleaned from the metadata. Hope this clarifies the situation for you! :-)
Graeme.
trim/discard (Score:3)
At a guess this is caused by mounting with the discard option, or trim as its called in Windows. It tells the drive you don't need the data stored where a deleted file used to be.
Maybe it's still there if you look with a microscope but who really does that?
Re:trim/discard (Score:4, Insightful)
Deleting a file should tell the OS that I don't need that data. That's what deleting is.
Re:trim/discard (Score:5, Informative)
> Deleting a file should tell the OS that I don't need that data. That's what deleting is.
It does, and that is, in fact, what it is. So Windows unlinks the file from the directory and removes the blocks from the in-use map.
The drive, however, doesn't know any of this. It updates the requested sectors representing the directory and volume info, and that's about it. It has no idea that blocks somewhere else on the drive are no longer needed, so it will dutifully copy and maintain that data while rewriting blocks.
The idea of the trim command is that it tells the drive that the data is deleted so the drive doesn't have to worry about maintaining it. This was never needed in old drives because the data was basically ignored after being written. In SSDs, the data needs to be continuously copied around to facilitate the erasing (which wipes several thousand blocks at once).
Re:trim/discard (Score:5, Informative)
"The drive, however, doesn't know any of this. It updates the requested sectors representing the directory and volume info, and that's about it. It has no idea that blocks somewhere else on the drive are no longer needed, so it will dutifully copy and maintain that data while rewriting blocks."
Actually, this is no longer correct. SSDs (such as the one in this study) are quite capable of examining the filesystem stored on the drive, independently, and the concept of 'dutifully' and ignorantly maintaining deleted data goes out of the window as a result.
The main point of the paper is not that this 'analyse-and-purge' drive GC behaviour happens (which is well known among SSD enthusiasts), but that it fouls up long-accepted court and forensics assumptions about what computer drives are. It can result in loss of evidence (of guilt or innocence) very easily in court cases in a manner which might be incorrectly attributed to malicious human intention.
Graeme.
Re:trim/discard (Score:4, Informative)
"How would it do this without maintaining firmware for all possible types of file systems?"
It wouldn't. But, how many different filesystems are in realworld use on 95%+ of systems? HFS+, ext2/3, NTFS, FAT32?
I can imagine that it does this for simple FAT16 or FAT32, but I doubt they deal with NTFS or EXT3 or HFS.
It does work on NTFS, and if you set up an experimental rig similar to the one we describe at the back of the paper (page 16), hopefully you can satisfy your doubts :-)
Re:trim/discard (Score:4, Interesting)
2 - Defragging: similarly, if you're moving data around in dead space without safely duplicating it or having a filename pointing to the blocks in use at any given time, you're not being careful. Also, which defraggers have random 3-minute gaps in operation that would even allow GC to kick in?
I think it is time to start bringing the discussion to a close, as it appears that we do share at least some common ground.
I will comment only on this one question. Your implication that the 3-minute rule somehow makes the GC "safe" is missing my point entirely. Yes, in practice there are checks and balances such as you describe, that make the GC unlikely to screw up. But, in my view, "unlikely" is not good enough. I want, and need, perfect (logical) block storage and retrieval. This should and must be the design goal. Of course, this goal is impossible to achieve in practice. For example, if the firmware (such as older, pre-SSD firmware) is designed with the goal of providing logical block storage, but fails in this task because of some honest bug, then I can understand that. At least, in this case, the programming code was written unambiguously with the correct goal (and no other directly conflicting goals) in mind, even if this goal was not achieved in practice due to an unintentional bug.
However, when a manufacturer deliberately designs firmware with the goal of deleting logical sectors, no matter how well-intentioned or well-implemented, this design goal (by definition) must come into conflict with the original, core goal of reliable (logical) data retrieval. I do not care what happens in the underlying physical layer, but I do care very greatly about data accuracy at the logical layer. The existence of certain checks and balances to prevent data loss is better than no checks and balances, but it is not better than the REAL alternative, namely, firmware that stores and retrieves logical blocks correctly, and is designed for this and only this purpose, without any other directly contradictory design goals.
No one, not even rocket scientists, has ever succeeded in writing bug-free software. But one should make an effort to minimize the number of opportunities for data loss bugs to arise. Firmware-based logical-sector garbage collection fundamentally and irreconcilably contradicts every reliability design principle known to man. That is why I consider the idea to be so abhorrent.
Re: (Score:3)
Re: (Score:2)
That is, of course, true.
The new part is that in ages past, the OS would just mark the blocks free and that was that. The data tended to remain there until the space was needed for another file, then it would be overwritten.
Because flash has to be erased before it is written (directly overwriting doesn't work), erase can take some time, and the drive is not just a dumb device, it is now necessary to actually let the drive know that the data is no longer needed as well rather than just making a note somewher
Re: (Score:3)
"Actual erasing things would slow down your computer quite a bit."
True for spinning magnetic hard drives. Not true for SSDs with some idle time to erase unused blocks in the background due to how flash memory works. Hence the creation of the ATA TRIM command, which lets the OS inform the SSD that a block is no longer being used.
Re: (Score:3)
Maybe it's still there if you look with a microscope but who really does that?
The forensics team mentioned in TFS?
Re:trim/discard (Score:5, Informative)
At a guess this is caused by mounting with the discard option, or trim as its called in Windows. It tells the drive you don't need the data stored where a deleted file used to be.
Maybe it's still there if you look with a microscope but who really does that?
Hello there, I'm one of the authors of the paper (Graeme).
This finding isn't related to TRIM in any way, though TRIM poses another nightmare for forensic investigators and may make the idea of examining deleted data/metadata redundant in a few years.
What's happening here is that the drive itself has some code, a garbage collector, that reads the NTFS filesystem metadata and wipes any cells it thinks aren't needed any more, so that the next set of writes over those cells can take place more quickly.
Normally this GC has seemed to take a while to kick in (various benchmarking sites suggest e.g. 30-60 minutes and unpredictable behaviour, e.g. not always kicking in when expected); here, we found that after a quick format has taken place, it reliably can be seen to kick in within minutes and also purges the drive within minutes. This is just one example of a case when the GC can kick in, of course.
Current court-accepted forensics practice is to stick a write blocker between the drive and computer, under the assumption that the drive only modifies data when a PC tells it to: but that doesn't help you when the drive itself nukes all it's data cells within minutes of being powered on.
I can imagine a situation where someone connects up the drive to power, (and possibly a write blocker), but not to the PC and makes a cup of coffee for a few minutes - by the time they connect it up for data transfer, it's too late, and they wouldn't even realise it had happened - the drive doesn't exactly make any whirrs or clicks as it wipes itself.
Alternatively, the circumstances we found: in less than the time that would be taken to read an entire image off the disk for forensic study, the GC is racing ahead of you and purging the disk! Yikes. So you can imagine... a forensic investigator takes an image, examines it, presents it to the court, and when the court verifies the original disk that the copy was taken from, nothing is there any more, and the forensic investigator seems to be making it all up....
So this feature has the potential to make it look like the police or forensic investigator themselves tampered with the original disk, as well as potentially destroying evidence that could help establish guilt or help establish innocence. We've put a number of 'interesting legal grey areas' at the back of the paper that we hammered out with reviewers, which are worth being aware of.
Graeme.
Re:trim/discard (Score:5, Insightful)
Re:trim/discard (Score:5, Informative)
Yep, it's the answer that immediately springs to mind when you approach the problem as a techie, and we touched on in the paper (see point 16, page 13).
Keep in mind though that this is extremely non-trivial - e.g. not many court officers will know how to do it - and that a court may have real trouble with a forensics guy telling them that for 'this particular case', they need to not use the standard procedure to preserve the data unmodified, but instead to rip open the disk and muck about with how it processes data.
Courts have accepted practices that are used worldwide; what you propose is not accepted practice; therefore courts will have a problem, at least in the short term.
Also, given the number of variations of controllers and firmware out there too, I'm sure you'll agree there's tremendous potential for a court forensics officer to get it wrong and accidentally nuke the drive data. e.g. how do you find out which firmware was in use, without powering on the drive (which activates the GC)...
Graeme.
Re: (Score:3)
If such becomes accepted practice, I imagine the demand will go up for drives that are designed to frustrate exactly that sort of approach. I use Flagstone drives [stonewood.co.uk] that are designed to resist such attacks.
Re: (Score:3)
De-soldering and accessing the flash directly would allow access to flash sectors that where previously used, and are currently on the list of sectors available for erasing but that have not yet been erased. These are the sectors that a good GC in the SSD firmware will quickly erase as long as power is available.
Also erasing is done in pages which contain multiple sectors. So a previously used sector that is free to recycle may remain unerased if the GC algorithm does not consider the page it is in a suitab
Re: (Score:3)
The problem is with desoldering chips, the defense can state that the drive was completely mangled by the prosecution that there is no proof that the data has not been fabricated.
This is why in conventional forensics, there is a rule about using a hardware write blocker and make an image before going anywhere else. Otherwise, a good defense attorney could get the case thrown out of court.
How this can be solved? Who knows. There is always a tug-of-war underway between one group that wants unfettered acces
Re: (Score:3)
Mirror'd it here for you Andy.
http://graemebell.net/downloads/JDFSL-V5N3-Bell.pdf [graemebell.net]
Graeme.
Re: (Score:3)
Also, how could a forensic investigator show a court that his image 'really' was the original version?
There's no authoritative copy representing the data at time of capture to refer to any more, if the original was being modified as he took the copy.
There won't be, that was my point with the analogy to a biological specimen. The laws regarding hard drives will become much closer to the laws regarding biologicals which can alter themselves between collection and trial, and can be altered by the investigative process.
There is no more "original version". There is the version that was analyzed in the lab, and maybe a stable version that gets created several hours into the lab process. Much less can be known. This is why I agree your insight is really
Re: (Score:3)
"The laws regarding hard drives will become much closer to the laws regarding biologicals which can alter themselves between collection and trial, and can be altered by the investigative process."
I agree, that's one way it could go; other options might include 'ignorance is bliss, except for defendants' and 'attempt to legislate ineffectively against a moving target'.
Broadly speaking, I think a can of worms will be opened in the next few years in regard to the legal idea of volition and intention, as co
Re:trim/discard (Score:4, Informative)
Solution is simple, but not easy (Score:2, Informative)
You need to disassemble the drive and read the memory chips independently of the controller. I believe I read this is how one of the major drive recovery companies is handling SSDs.
Re:Solution is simple, but not easy (Score:5, Insightful)
Bingo. Forensic tech are used to being able to just plug in a write-blocker and assume the disk will remain intact and unchanged, which is the "legally safe" part they are complaining about. Since SSDs do lots more under the hood than spinning rust drives, they can't guarantee the device is unchanged unless they disassemble it, which might be considered tampering and leaves room for the other side's lawyers to ask "and then you took a soldering iron to a delicate IC?".
It also requires a lot more know-how than "Use this magic cable when copying drives for investigation".
Re: (Score:3)
Yep, I bet this has a lot to do with money as well. With that write-blocker attached to the drive, you can just use software to clone the disk bits and reconstruct the filesystem and recent deletes.
Sure it's possible (and more expensive) to do a complete magnetic scan of a disk platter and reconstruct that past few layers of content written (and rewritten) to the drive... the kind of thing that the secure wipe utilities are supposed to defeat with multiple layers of white noise. But I doubt law enforcemen
Re:Difficult to create data with soldering iron .. (Score:5, Funny)
Plaintiff's Attorney: "Sir, what are the chances of the drive automatically generating the exact sequence of bits required to form this email?"
Expert Witness: "Billions to one, certainly."
Defendant's Attorney: "And how many times will that this 2KB email fit on the drive?"
Expert Witness: "Well, it's a 2TB drive, so... about a billion, give or take."
Defendant's Attorney: "So, assuming the data on the drive is random, then it's safe to say there are at least two billion opportunities on this drive to produce this email?"
Expert Witness: "That's not what I meant..."
Defendant's Attorney: "Yes or no?"
Expert Witness: "Well, yes, but..."
Defendant's Attorney: "No further questions"
Re: (Score:3)
You're off by so many orders of magnitude, it hurts.
The chance of generating that particular e-mail, if it's 2 KiB, is 1 in 2^16384. (That's not really the number you want -- you want the chance of generating a similarly-incriminating e-mail. They're roughly equally improbable, though.)
So that's 1 in 2^16384 compared to 2^30 copies of the e-mail that would fit on the drive.
If the prosecution scoured every drive ever made for a random sequence of bits that looked like that e-mail, they'd never find it.
Plus,
Re: (Score:3, Informative)
-----
Hey there
Yep, it's the answer that immediately springs to mind when you approach the problem as a techie, and we touched on in the paper (see point 16, page 13).
Keep in mind though that this is extremely non-trivial - e.g. not many court officers will know how to do it - and that a court may have real trouble with a forensics guy telling them that for 'this particular case', they need to not use the standard procedure to preserve the data unmod
Encrypt your data (Score:5, Insightful)
Re: (Score:2)
Re: (Score:3)
This property of modern SSDs is neither positive nor negative by itself. It's a property you'd be thankful for if you were organizing protests in Iran and stored contacts on an SSD. You'd curse it if it were exploited to destroy incriminating photos by someone who'd stalked you or a loved one.
Re: (Score:2)
So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
It is really hard to prove that data from SSDs was really deleted, and it is also hard to prove that data from SSDs was really not deleted.
Re: (Score:2)
That article was bunk. The correct way to get an SSD to erase something properly is to "trim" the data – i.e. tell the SSD that the cell is no longer needed. This (a) causes the SSD to erase it so that the next write cycle is faster, and (b) lets the SSD know it can overwrite that cell without explicit instructions from the OS, which allows it to wear level better.
What the people in that article did instead was try to write over data with crap. The SSD then did a good job of wear levelling and inst
When it absolutely, positively, has to be.. (Score:3)
..destroyed overnight, go with the SSDs. The melting point of a surface mount IC is a lot less than that of a spinning platter.
Re: (Score:2)
Re: (Score:2)
.357 at 10 yards does it for me.
Re: (Score:2)
Yes but thermite is so much fun to play with.
Well... (Score:5, Insightful)
So either SSDs are really hard to erase, or really hard to recover. I'm so confused.
All I know is that if SSDs were really hard to erase, and I was in the business of recovering data that other people didn't want recovered, this is exactly the kind of story that I would tell them so that they would continue using SSDs.
Not that I'm paranoid or anything.
Re:Well... (Score:4, Insightful)
What wear levelling gives with one hand (performance and life) at the expense of the OS never knowing in which memory cell the data has -actually- been stored; making targeted deletion runreliable..
and, most importantly:
$ cat /dev/urandom >> /dev/ssdX
Just needs to be run once to -really- bollox an investigation..
Re:Well... (Score:4, Funny)
What wear levelling gives with one hand (performance and life) at the expense of the OS never knowing in which memory cell the data has -actually- been stored; making targeted deletion runreliable..
Ruh-roh, Raggy! It's runreliable!
Re: (Score:3)
Depends on if it's the SSD controller that is hiding the space.
I really need to get one to test but...
Example:
you've got a 64Gb ssd on SDB it say's it's 50Gb partition
If you just run a "dcfldd" command "dcfldd if=/dev/zero/ of=/dev/sdb" (the whole device not just the partition)
Will it wipe 50Gb or 64Gb?
Re:Well... (Score:5, Interesting)
It's a good joke, but with a grain of truth in it. If you're concerned, you can buy the drive we used, flash it to the firmware we used, (optionally) buy a write blocker if you like, and run the programs I placed at the back of the paper and see what happens. We carefully documented as many of the experimental setup parameters as we could so it should be possible to reproduce the results exactly.
Skepticism is a good thing - so I hope you'll reproduce the experiment at home and tell the world afterwards if you still think we're running PsyOps for the forensics community :-)
Graeme. p.s. I'm currently looking for a postdoctoral/research engineer/research scientist position in Grenoble.
Re: (Score:2)
As opposed to HDDs, which aside from being comparatively slow, supposedly allow recovering information after it's been overwritten multiple times?
If you're still using a 20MB hard drive from 1993, perhaps.
Given how hard current drives have to work to recover information that _hasn't_ been overwritten, you can be pretty sure that no-one's going to be recovering information which has.
Also the end of data recovery (Score:2)
platters in a clean environment into another drive with working heads.
For a few hundred to some thousands, your poor choice of having no backup media
can be resolved.
On SSD I can desolder the chips, dump them and then tell you there's nothing recoverable.
For a few hundred to some thousands, your poor choice of having no backup media
can be resolved.
Hard to Erase AND Hard to Recover (Score:2)
Ultimately since the Flash Translation Layer goes and does things under-the-hood that are not externally visible, it is hard to be sure your data were erased, and it's also hard to be sure they were not erased... Essentially since there is an opaque interface at the logical-block level and the device is internally free to behave as it chooses so long as that interface is maintained, it makes it tricky to guess how the internal implementation will behave.
Plain old magnetic disks used a fairly predictable im
really hard to erase, or really hard to recover? (Score:5, Funny)
Why the confusion, dear editor? This should be well understood.
If you want to recover, you can't. If you want to erase, you can't. It's Murphy's Law of Data Storage.
Huh? (Score:2)
Forgive my ignorance, but how is this possible? Does this mean that the drives understand NTFS and are actually zeroing out data on the drive when the OS simply deletes the entry from the FAT table? How can the SSD second guess what the OS is doing? I thought that SSD's use the same interface as regular HD's and should behave the same.
Re:Huh? (Score:5, Informative)
There is an ATA command called "TRIM [wikipedia.org]". If a device supports it, the OS can tell it that a group of sectors is no longer needed, and should be wiped.
Re: (Score:2)
I would think that the command would also be useful for wear-leveling.
The more space that the device can understand as "free" the more opportunity it has to avoid writing to places that are wearing. if the whole drive is 100% free the best it can do is try to move data back and forth if it notices one spot getting written to more. That wastes writes and time. If it knows that a spot is now free, the drive can take the opportunity to optimize more.
Re: (Score:2)
they use the same interface tween the computer and the device but tween that device controller and the actual raw memory is a whole different story
Well, I guess.. (Score:2)
So what's the problem? (Score:2)
Re:So what's the problem? (Score:5, Insightful)
Well to be fair, that's only part of the problem. Say you're a stupid criminal. You store all your records of your criminal acts on your computer which happens to have an SSD drive. You've never deleted your records of your criminal acts, but just before the police busted through the door you were deleting some old pictures of your cat, Fluffy. So in order to maintain evidence chain of custody, the police immediately turn off you computer and turn it over to a tech. The tech's first two actions should always be the same. He should plug your drive into a a special read only device that will first do an MD5sum or other fingerprint of the whole contents of the drive, and then do a bit for bit copy of the drive.
The idea here is that the police can prove that nothing was done to alter the data present at the time of seizure. Power was removed from the system and the drive was immediately fingerprinted and copied. Assuming the fingerprints match, there should be no question that the copy on which the actual analysis is done is identical to the original drive. The problem is that you were deleting pictures of Fluffy when the Police came in. It's highly likely that as the drive does its self cleaning routine the fingerprint of the data will change. They fingerprint, get a value, but while they're copying the drive self cleans the bits associated with your cat pics. Bam, the copy has a different fingerprint. Now there's reasonable doubt about the usefulness of the evidence you stupidly left unencrypted in your desktop folder.
Now I'm not saying this is a problem, and that they need to modify the design of SSDs. I didn't get the impression that article said that either. What they are saying is "Hey, we need to come up with another way to do this, becasue what we've been doing will no longer stand up in court."
Both hard and easy are true (Score:5, Informative)
The drives have internal overprovisioning and perform internal garbage collection. This means that marked for deletion data has an unknown lifetime and may disappear at any point without interaction from a controller.
The hard to erase bit means that you really can't be sure something is totally erased without a full specific erase command to all flash blocks. Without that a page marked unused but not erased may be nestled in with a bunch of valid pages. As all pages in a block are erased together that marked unused page can hang around for a wile.
On the other side the firmware does garbage collection it actively looks for blocks with many erased pages and then tries to consolidate things so it can create more free blocks. This means if the drive is powered but not connected to a host machine it can still be doing data moves for consilidation and erasing marked for deletion pages.
There are thresholds for the garbage collection so it won't overwork and try for 100% consolidation. Thus you get both the presence of some really sticky stale marked unused pages and some active erasing of others.
Not necessarily a bad thing.... (Score:3)
I'm on the fence about this, and it's possible neither pasture is green. On the one hand, I might be the victim of a genuine crime, evidence of which happens to be hiding in an SSD drive. On the other hand, these techniques are just as routinely abused now to go after people for political noncriminal reasons that don't serve the Common Good at all, people and organizations like Julian Assange, Wikileaks, Bradley Manning, the U.S. Chamber of Commerce opponents... you name it.
These techniques are like nuclear physics: just as easily applied for Bad Things as Good. If we can't selectively prevent the abuses, maybe we should err on the side of caution and ban the techniques altogether. They aren't being universally applied to serve justice.
Elementary my dear Watson (Score:3)
I thought that this was particularly telling. In the article it said:
... the state of the drive cannot be taken to indicate that its owner did or did not interact with it in ways that allow prosecutors to infer guilt or innocence. The fact that data has been purged does not mean a human knowingly did it (e.g. accidental guilt)...
So in other words, until SSDs came along, evidence of purged data was evidence of guilt... at least in Austrailia.
Re: (Score:3)
Destroying evidence is a crime in the US, too.
Re: (Score:2)
Destroying evidence is a crime in the US, too.
Yes, but destroying data is not. The point is that evidence that something was destroyed is not itself evidence of crime.
Re: (Score:3)
Oh, wait I forgot that I am supposed to feel sorry
This is bad news (Score:2)
"Firmware built into many solid state drives (SSDs) to improve their storage efficiency could be making forensic analysis at a later date by police forces and intelligence agencies almost impossible to carry out to legally safe standards, Australian researchers have discovered..."
So expect some government intervention on matters concerning which firmware should be built into the devices we use.
I cannot see any government worth its credibility endorse a product which if employed in crime and confiscated (by police), it is almost impossible to use it to prosecute the perpetrators by government agencies and the FBI in the case of these United States.
You might wonder how a government might endorse a product:
By allowing its importation or production and subsequent collection of taxes fro
Wasn't this... (Score:5, Interesting)
...a foregone conclusion ever since ATA Secure Erase and TRIM were introduced?
Secure Erase basically tells the SSD that all of its cells are now blank (AFAIK implementations actually zero the drive as well but I'm happy to be corrected on that); therefore as soon as anything is written to the disc, it will be written here, there and everywhere. It took about 30s to run on my first vertex and I couldn't find any trace of
TRIM support in the ATA spec, along with kernel/filesystem support, tells the disc that when file A is deleted, cells X, Y, and ABQ are now officially "empty" and that if the controller feels like it, it can zero them out, shunt other data in there, or have a mardi gras for all it cares. The same happens when a drive is formatted; OS tells drive controller "I've just formatted you" and for the sake of preserving performance the controller goes "Brilliant! I can chuck out all this shit I've been saddled with."
As soon as hard drives start intelligently erasing/shuffling bits of themselves about so that cells are utilised to their utmost efficiency this was bound to happen. Unlike spinning platters where bad blocks were reallocated only if a) the hard disc knew about it and b) the data could actually be read/recovered, it becomes terribly obvious that data on SSD's is going to be read and written and deleted completely and utterly all over the place, without sequential series of sector found in slackspace like you would on a magnetic drive.
Magnetic drives have no performance penalties for not actually erasing the data, so if you work your way around that double negative you'll see that one of the staples of digital forensics (e.g. recovering files from slack) is a by-product of people trying to make magnetic platters as fast as possible by not actually erasing stuff, because as long as the controller knows that sector is blank then it'll just be overwritten as needed. Technology has now changed sufficiently that the performance gains from new solid state tech are helped by a drive controller that erases stuff as soon as possible, since writing over an occupied cell is slower than writing over a blank one.
I'm sure there'll be new methods to mitigate the change in tech, we're just somewhat on the cusp of a completely new tech. They'll probably come to an agreement that TRIM doesn't actually delete stuff until the amount of free space in the cells reaches a certain threshold or something like that.
Disclaimer: I'm not a digital forensic scientist, but am friends with one and we discussed this problem over some exquisite cocktails a few months back. And I don't think TRIM instructions follow the exact specifications I laid out above (e.g. using Brilliant! as an ACK).
Re: (Score:2)
To be honest, most data recovery from a hard drive is foiled by a simple zero anyway. All this "must wipe several times with certain patterns" thing is cobblers. Yes, if I was the military, I'd be doing it too, just to make sure, but no-one has yet provided convincing proof that "magnetic history" from drives can actually be recovered in anywhere near a cost-effective or reliable way.
Digital forensics consists of taking an image of the drive and seeing what you can get from that. Any half-decent implemen
Right to easily spy on you? (Score:3)
Why does the government have this expectation that technology should be built in order to make it easy to spy on citizens?
Re: (Score:3)
Why does it always come down to the "government spying on citizens". There are plenty of uses for digital forensics that do not involve tin-foil hats. The only time the government is going to doing forensics on a drive is during a criminal investigation. Those investigations are well documented.
Look at it from the other side of the coin. "Well your honor, we know that those bastards at Enron cooked the books, and we have the data... but the MD5 fingerprint does not match, so all of that evidence is inad
"So either SSDs are really hard to erase, or..." (Score:5, Informative)
Hello, I'm one of the authors of the paper. To explain the apparent paradox in rough terms:
Drive data was traditionally purged manually, by having the computer tell the drive to write something else over the top of the old data. In the absence of such an overwrite, magnetically stored data persists. However, if you try that trick on an SSD, it may not work. The logical address you try to overwrite may be remapped on the fly, so that your 'overwrite' goes to some other physical cell rather than the one which stored the data. From a logical viewpoint, it looks like the overwrite worked - you can't access the data any more through your computer's OS. But from the drives point of view, the data is still there, lurking in some physical cell that is presently out of use as far as the logical sector list is concerned. A cunning firmware or a hacker with a soldering iron might still get at it.
However, separately to this, modern SSD drives use tricks to try and automatically improve their performance, and one of these tricks is to pre-empetively wipe data cells that contain data no longer referenced by the filesystem. Here, the drive is actively attempting to permanently purge everything it can from the drive, all of it's own accord, in the interests of accelerating future writes by having a pool of completely unused cells available.
Summary:
- If you're a computer telling a drive to zero over some data, the drive may lie to you a bit, and not bother to zero it.
- If you're a drive, you do whatever the heck you like, and you see the physical layer directly (unlike the computer). That means the drive can open up the NTFS metadata, looking for data cells which could be preemptively reset, and nuking that data out of existence (when it might traditionally have been recoverable to an expert).
In summary. If your drive wants to nuke something, (and we've shown, they really DO want to nuke everything they can at a few minutes notice), it gets nuked. If your PC wants to nuke something, it may or may not get nuked by attempting an overwrite.
Finally, separate to this is TRIM, which is a hybrid of the two situations - an ATA command by which the OS can signal to the drive that it would like the corresponding physical cell for a particular logical sector address to be nuked, thank you very much.
Hope that clears things up.
Graeme.
Re:And the downside here is... (Score:5, Funny)
When Mindy the undergrad accidentally deletes her term paper and would be really REALLY grateful for a super smart and kinda cute geek to go in and recover the file with Backtrack... then you'll see the downside.
What? I reject your reality and substitute my own!
Re: (Score:2, Informative)
When Mindy the undergrad accidentally deletes her term paper and would be really REALLY grateful for a super smart and kinda cute geek to go in and recover the file with Backtrack...
OH NOES, what will I ever do without being told "thank you" and about what a nice guy I am.
Yeah it never goes any further than that outside of geek fantasies.
Not even in pornos. <-- business opportunity
Re: (Score:3)
Business Tip #1: Get payment up front.
Re: (Score:2)
Nice guys don't get laid as much either....
Re:And the downside here is... (Score:4, Funny)
You often get laid for being a dick. Hell, if you're in the right place at the right time, with low enough standards, you can get laid just for *having* a dick.
Re: (Score:3)
Err...we're talking about getting laid often here...not long term relationships.
Re:And the downside here is... (Score:4, Insightful)
I think it's about impoliteness when asking favors. Friends help each other: I'm quite ok fixing my non-technical friend's WLAN, just as he's ok with giving me a hand when I'm moving houses. That's the social norm, and thus asking someone for a favor also indicates how you feel about them.
So if you don't like someone, if you would under normal circumstances not want to spend time with them - then you don't ask them for favors. That would be plain rude.
If you don't have an actual friend to provide tech services - just purchase said services.
Re: (Score:3)
Most girls that would dangle that sort of carrot know that teasing is just as effective as giving where geeks are concerned.
Re: (Score:3, Funny)
Re:Why can't they make up their minds (Score:5, Insightful)
It's not hard really. The drives autoclean themselves. So when you delete the inode reference to a given data file, it will be completely much more quickly that on a normal magnetic drive (which won't reclaim the space till it's needed). On the other hand it won't respond to commands that would FORCE a magnetic drive to completely wipe the file.
So if you're in your secret lab and you hear that the evil enemy is an hour away, you just type "rm /*" and wander off to escape. By the time they get there all your data will be completely wiped. On the other hand if they are breaking down the door to your secret lab and you have only seconds left you can't type "shred /home/joeari/secretfile" and expect it to be perma-deleted like you could with a magnetic drive.
They recover deleted sectors more quickly, but can't be forced to do so in a controlled manner.
Re: (Score:2)
Re: (Score:3)
Actually the man page states that on ext3 file systems running in the default manor (data=ordered) shred works fine. If you journal data as well, not so good.
The underlying file system in use is no guarantee either. Software, such as word processors often write out a new file each time you save, and if that is successful, they then delete the old one. This defeats shred because shred requires a file to exist to delete it. (It does not shred un-allocated space).
Re:Why can't they make up their minds (Score:5, Interesting)
You wrote: "So if you're in your secret lab and you hear that the evil enemy is an hour away, you just type "rm /*" and wander off to escape. By the time they get there all your data will be completely wiped. On the other hand if they are breaking down the door to your secret lab and you have only seconds left you can't type "shred /home/joeari/secretfile" and expect it to be perma-deleted like you could with a magnetic drive."
Actually, we found something even more interesting when we prepared this paper.
An evil criminal could run a quick format (about 8 seconds effort) if police were at the door, or they could set up a dead man switch to do the same. When the investigator powers up the drive, the drive's internal GC runs quickly (we couldn't get any formal documentation about why this is - presumably it begins so quickly since the filesystem metadata is nice and simple to process) and so after about 3 minutes it begins wiping itself.
Just a few minutes later it has finished, and data and old filesystem metadata are gone (at least, gone as far as logical access to drive sectors is concerned).
I'd encourage anyone who finds the result interesting to take a glance at the original paper, it's written to be accessible to people outside the traditional drive forensics community and there are some fun, free script tools you can use to monitor drive GC behaviour in near-realtime.
Graeme.
Re: (Score:3)
yep, at least on my drive it does.
(note I emulated this by removing the sata cable and not the power cable, as opposed to a nifty diode steered backup battery).
re-connected the drive after 5 min, and re-enumerated the drive, and it was rather blank.
-nB
Re:Why can't they make up their minds (Score:4, Insightful)
On the other hand it won't respond to commands that would FORCE a magnetic drive to completely wipe the file.On the other hand it won't respond to commands that would FORCE a magnetic drive to completely wipe the file.
So why isn't a "wipe everything, including the spares" command part of the SATA standard command set? I can think of many times I'd like to completely erase a drive. Basically, any time I run the equivalent of "fdisk, o" in whatever OS I'm using, I'm explicitly telling the drive, "I value nothing here". That would be a prime time to optionally tell a drive to totally erase itself.
There are even non-security-related reasons for doing this. IIRC, the reiserfs recovery tools would scan a partition to look for data laid out in reiserfs-formatted chunks and undelete them. That's normally exactly what you want. However, suppose you're using reiserfs and running a VM that also uses reiserfs on its virtual disk, which lives inside a file on your main filesystem. The recovery tools would see those structures, too, and merge them back into your directory structure. This is typically disastrous.
When programming in low-level languages, it's nice to know that the only thing in your process's heap is data you've explicitly put there. The same holds true for hard drives. While we're obviously pretty good at keeping the wheat separated from the chaff, it'd be nice if you could guarantee that there was no chaff to start with.
Re: (Score:3)
I think you missed the point. Say you've installed Ubuntu on a reiserfs partition inside a VM. The VM's harddisk file is stored at /var/tmp/ubuntu.vdi. For some reason, you need to fsck /var. It reads each block in /var's partition and looks for things that look like inodes and merges them back into the directory structure tree. Guess what happens when it starts scanning blocks that are inside ubuntu.vdi? It finds a lot of things that look like inodes and merges them back into the directory structure tree.
Re:Why can't they make up their minds (Score:5, Insightful)
The do not completely erase when you hit them with a commercial erase tool because the commercial erase tool does not give the drive a chance to "take a breath". If you hit it with a DOD wipe instead of overwriting old blocks the drive will give you new until it runs out of spares and after that will slowly overwrite some old. At the end some data will remain.
You need to hit it with short write bursts which are alternated by rest periods so that the controller can reorganise the flash especially if you use the TRIM ATA command to mark stuff as "really deleted". As a result the data will be wiped to a standard which no hard drive can achieve.
I can write a perl 10-liner that feeds DD to do that in about half an hour. So any geek can erase one. However the commercial and most importantly certified by banking and govt tools cannot just yet. They definitely will as there is no rocket science here. Until this happens you will hear both sides of the story depending who you talk to.
By the way the same will apply to overlapping recording and drives with flash cache. A "dumb" wipe will not wipe them. At the same time a script any sysadmin can write in his spare time can wipe them so effectively that no forensics can get the data back.
Re:Why can't they make up their minds (Score:4, Interesting)
Trust me, dd can really make a mess of things, if by intention or mistake.
I was cloning one drive (A) to a larger one (B). It needed to preserve the partition table, MBR, etc.
dd of=/dev/sda if=/dev/sdb bs=1024
About 10 seconds after I hit enter, I saw the mistake, and aborted it. Ya, I dd'd an empty drive over the populated one. How much harm could that possibly do, right? :)
I've done plenty of drive recoveries, with about an 80% success rate. The only ones I can't do are drives that don't spin. I don't have the facilities to transfer the platters.
Two weeks and several recovery attempts, I had recovered about 25% of what I wanted. The partition table, and the MFT were destroyed, and files were on four partitions with two different filesystems (NTFS and ext3). That was in 10 seconds. If you were to say" dd if=/dev/random of=/dev/sda bs=1024", it's rather unlikely even the best would find anything.
From what I understand, the best chance at recovering data from an overwritten media is to look at sectors marked as bad. They are automatically skipped, so they don't get overwritten by anything. Who knows what will be in them though. Most likely on most users machines, it will be pieces of the browser cache, or Windows Update files.
That's not to say if some agency named "No Such Agency", were really bent on finding out what I had on there, they couldn't recover something. Many agencies can't go beyond files that currently exist, or are in the "Recycle Bin". Some go as far as undeleting from an otherwise functional filesystem. Is it worth weeks and thousands of dollars for whatever organization to recover what you have hiding on your desktop? Probably not. If you have secrets, don't hide them on your PC at home, even if you are protected by a nice wood front door with fancy security locks. One brick through the window, and a fast getaway, will defeat any door lock and alarm system. If you kept all of your important information on some random colocated server in another country, AND you have a second person with access to clear your data, it would be untouchable by most agencies. Say I had a server colocated in China, with the normal protections (encrypted virtual filesystem, etc, etc.), and only used my desktop to manipulate them through a terminal session (VNC, SSH, etc) the most that could be found is that I connected to a server in an untouchable place, and by the time they managed to get to it, my 2nd party could have removed it.
Re: (Score:3)
Lately all you have heard is the complete opposite. That they are impossible to completely erase so it's unsafe to store company/secure data on them. Because even if you erase the file its still left on the disk and just marked as empty. Now they say they erase them self.
you've discovered the paradox that exists in the gulf between "in theory" and "in the real world". In the real world, a lot of shit that people get lathered up about is completely impractical and for all real-world purposes "impossible". But that doesn't stop some tech "editor" on a slow news day from hype pimping some theory as if it's the pending zombie apocalypse.
Re: (Score:2)
Ignition of the magnesium slab mounted to each SSD and memory module works 100% of the time.
Don't forget your backups!
TRIM (Score:5, Interesting)
It used to be the drive remapping meant that if you deleted something (or even overwrote it), there was no guarantee it would be gone - SSD controllers do wear leveling to avoid having part of the drive get used excessively. Forensic analysts could go pull the raw data from the NAND flash.
The problem was that the wear leveling algorithms need a "free block" pool to work well. Drives that have been used heavily deplete the free block pool, and the drive slows down. For a long time, SSDs would have no knowledge of whether a file had been deleted or not.
The ATA TRIM command was added for just this purpose - with TRIM, when the OS deletes a file that references a block, it can tell the SSD controller that those blocks are free. The SSD controller will then begin erasing those blocks in its free time. (SSDs can be written one block at a time, but must be erased one page at a time. A page consists of multiple blocks. Oh, and I may have page/block swapped here.) So you get a lot of performance improvement by having a bunch of pre-erased pages - these can just have individual blocks written without a read/erase/modify/write on a whole page.
Pre-TRIM, SSDs were probably great for forensic analysts. Post-TRIM, SSDs are not. Oh, and I think the latest ATA standard added a "sanitize" command to make life easier for information assurance types, for whom SSDs have always been a pain.
Re: (Score:3, Informative)
Re:This is great news (Score:5, Informative)
Not likely in this situation - here, the GC isn't killing the data for the hell of it, it's killing it because it needs to reset the cell preemptively in order to maintain drive performance at a high level. If you preserve the data in the manner of an old HDD, you nuke performance by forcing the controller to do a reset before future writes.
Graeme.
Re: (Score:3)
Re: (Score:3)
A few drives offer it, with varying names and degrees of marketing fanfare, but there is little documentation as to the precise workings of the GC features. In the paper PDF you'll see the drive make/firmware for the drive we used in these experiments. You can buy that one if you like :-)
Graeme.
Re: (Score:3)
Graeme.