Data Corruption Issues

In spite of hard disk drive reliability improving and file systems more resilient, data corruption remains a serious issue. The most reliable way to guard against losing data through corruption is through the use of backup strategy.

If you suffer corruption of your data, the best option for recovering the files is to send your hard disk to a professional data recovery company, such as DiskEng. Attempts to recover the data yourself, especially if the file system has become corrupt, may result in cause further damage, resulting an even greater loss of data.

What is Data Corruption?

Data corruption is when file data or important file system structures are altered to contain data they shouldn’t. This results in bad or damaged data being returned when the file system is accessed or an attempt is made to access a file.

Such corruption can lead to software or operating system crashes, driver damage, further data or file system damage. Data corruption is a serious problem, as it will not fix itself unless caught early enough.

What Causes Data Corruption?

There are several reasons for data corruption to occur, which can range from physical damage, damage caused by malware through to damage caused by a sudden disruption of a data transfer. Physical damage usually the result of bad sectors appearing as the result of overuse or some other failure causing damage to the surface of the platters.

A sudden disruption during the transfer of data to the disk, for instance as the result of power loss or disconnecting a drive can cause some data not to be written to disk. While most modern file systems contain journaling, there are rare instances where this is unable to resolve the problem.

Malware can cause the worst corruption, either to the contents of data files or the data system structures. If you are not running any anti-virus software, you are putting your data at serious risk.

Data Recovery and Data Corruption

Data corruption can make the process of data recovery complex, which is why it should not be allowed to become worse. No risks should be taken, such as attempting to fix the issue yourself, as they result in making the corruption becoming worse.

It is best left to the professional data recovery experts to recover the data, which for the worst instances may require manual inspection of the data structures in order to determine the solution. The use of do-it-yourself software could result in the incorrect assumptions being made, which could lead to the problem becoming worse.

SSD the Truths and Myths

With the massive increase in data transfer speeds, combined with the drop in price per gigabyte, the uptake of the SSD drive has unsurprisingly been rapid. They appeared to be the answer we had been looking for, touted as super reliable, faster data transfer rates and use less power than traditional hard disk drives.

Much has been made of how data recovery from SSD, where development is moving rapidly, being a complex problem during technology changes, making it most sensible to use them for installing the operating system and applications, rather than storing any critical user data. The high duty cycle numbers suggest that even when used in for intensive IO applications they should not fail before the end of the average computer lifespan.

SSD Lifetime

Contrary to the popular belief, even though each memory cell is only good for a certain number of write cycles, this is not the significant factor governing the lifespan of an SDD. In research published earlier this year, it appears that an SSD is not damaged through usage, but primarily through its age.

Even more worrying is that the research has not thus far reached a conclusion on the causes of these failures. For most users it is also alarming how many read errors an SSD has, most of which are transparently corrected by the drive.

Data Loss Through Sudden Power Loss

There were many stories of data being lost as the result of a power surge or sudden power outage. Some of the earliest first and second generation drives did not employ enough protection to write any pending data to the memory chips. Such issues, unless you are still using an old SSD which had this problem, should no longer be an issue.

SSD and Long Term Storage

Alarming stories appeared last year, stemming from research which showed an enterprise level SDD stored at a high temperature with no power, lost data in only 7 days. While this is significant research, in most real world situations, loss of data from an unpowered SSD is extremely unlikely.

However, if they should not be used for long term storage media, as the average data retention time for an enterprise level SSD is about a year while consumer drives this is two years. This makes them a poor choice for storing data which may not be required for several years, making other storage solutions a wiser choice.

SSD and Data Recovery

The research earlier about SSD failures, was largely ignored as most people seemed to ignored the significant issue, only focussing on the fact that the number of failures has increased with more drives of this type being used. What should have been the focal point, is how a much higher proportion of SSD failures resulted significant levels of data loss, when compared to traditional hard disk drives.

Although data recovery from an SSD is a much more complex, it does not account for increased occurrence of data loss. It appears that when an SSD fails, it is usually associated with multiple read failures which cannot be corrected by the drive or during the data recovery process.

Cheap and Free Recovery Utilities

There are many free or cheap data recovery tools available on the internet, which are tempting to use in the event of a failure, due to the potential for saving money. There is however, a risk especially if your hard disk drive is suffering a physical failure, as the problem will only get worse.

It is for this reason that at DiskEng we secure an image of your drive first, to ensure that we have harvest the maximum amount number of sectors, which is extremely important on a failing drive. We prefer that the drive is sent for data recovery without any attempts being made at recovering the data yourself, as the problem may have been made worse.

Taking a Risk is a Gamble

Accessing the disk directly to fix the problem or recover the files can be a bit like playing Russian Roulette, especially if the hard drive is starting to fail, or there is severe corruption of the file system metadata structures. While many people have had success using such utilities, we have seen many examples where further damage has occurred to the drive or important data structures have been deleted overwritten with incorrect information.

If it works, the gamble has paid off, but when it doesn’t, the data recovery process may be considerably more complex than it would have been and can result in data loss. In the worst cases we have seen, the file system data structures been completely destroyed, leaving a data trawl the only available option, which can only recover some files, which will not have their original file names.

Dealing with Errors

It is important to understand that if your data has become inaccessible, it means that an error has occurred, whether it’s physical damage or logical data corruption. The reputation of any data recovery software depends on well it is able to cope with errors and if an unreadable is encountered, how it deals with that.

Repeatedly attempting to read an unrecoverable bad sector can lead to further damage occurring to the surface of the hard disk, which if it happens can very quickly destroy large areas of the disk surface. Any process which attempts to alter the contents of a hard disk drive is a serious risk, as a mistake could severely decrease the chances of recovering the files stored on the drive.

Impact on Data Recovery

When a hard disk drive arrives at our laboratory, with the information that attempts have already been made to recover the data, our data recovery specialists tend to suffer a sinking feeling. In order to avoid any further damage once the physical state of the drive has been assessed and remedial work untaken, a sector-by-sector image of the drive is secured.

All further work is undertaken using the image of the drive, with any modifications required, stored separately to ensure data integrity. Unfortunately, we cannot predict how successful the data recovery from a hard disk where previous attempts to recover the data will be. These can range from recovering all the files intact through serious data loss to a total catastrophic failure, such as a head crash resulting from repeated attempts to read unrecoverable sectors.

Upgrading Computing Systems

Before upgrading your operating system, whether it be a Windows system, OS X, Linux or other Unix derivative, it is always advisable to back you data up first. Often however, this only happens the very first time you do an upgrade, if at all, especially if nothing untoward happens the first time.

It is easy to become complacent, because the chances of a problem occurring which leads to loss of data is low. However, when a problem happens, the only course of action is to send the hard disk for data recovery.

Preparing For OS Upgrade

The preparation for an upgrade is important, especially if the operating system is update being installed over an existing version. It is this situation when problems are most likely to happen, such as incompatibility problems or unexpected file system corruption. With user areas often located by default on the same partition as the operating system, important data could be rendered inaccessible.

If the operating system is being installed to a new partition, or overwriting an existing one, it is important to double check that correct disk and partition have been selected. A simple mistake, especially when deciding to overwrite an existing partition which is being reformatted, is easy to make. It could lead to data being lost, which could be completely unrecoverable.

Operating System Downgrade

The desire to downgrade back to previous operating occurs many times, whether the user dislikes the newer version of the operating, it has compatibility issues, or the operating runs slow on the hardware. Although downgrading is technically possible, in many cases the complications, particularly if the operating system has been running for a while, can be insurmountable.

Should you decide the new version of the operating system does not meet your needs, either due to the interface or it runs too slow, it is imperative that any data not already securely backed up is made safe. In order to downgrade the operating it may be necessary to reinstall the previous operating from scratch.

Data Recovery Following Upgrade Failure

It is important that if something does go wrong with your upgrade or downgrade that you do not panic, particularly if you have not previously backed up all your data. The worst possible case is reformatting a partition which contained data, onto which the new operating system was installed, as the chances of recovering the previous data reduces, the more applications you have installed.

Always be sure what you are doing before you start, as mistakes are very easy to make, as it is not always easy to know which drives contains what data when installing a new version of an operating system. All it takes is a momentary lapse of concentration to delete or reformat the wrong partition, which would require a complex data recovery, the results of which cannot be easily predicted without a full analysis.

Dangers of Complacency

When we talk to people who have suffered a failure of their storage device we often hear a comment along the lines of “I didn’t know hard disks fail” or “I’d been meaning to make a backup.” Sometimes it is naivety, a belief that modern technology won’t let you down, but in many cases the issue is down to complacency, assuming that everything will be okay. Just because everything is currently working and you intend to back the data up, is no protection at all.

Failure of a storage device which has not been backed up could lead to a critical loss of important data and at the very least cause a loss of time and money, through recovering the data and lack of productivity. Such a failure, especially if attempts at data recovery are unsuccessful could result in the continuity of your business being put at risk.

Storage Devices Fail

All storage devices will fail at some point, unless they are taken out of use before they reach this point, which is the case for a majority hard disk drives. With the financial issues which have developed over the last decade, many home users and businesses stretching the overall lifespan of their computers, leading to a higher chance of any given storage device failing.

For any storage device, a Mean Time Before Failure (MTFB) is usually quoted, which is given as guidance, but should does not mean the device is continue to operate that long before it fails. This can often be a source of misplaced complacency. Even a brand new device has the potential, although rare to fail, usually the result of a handling or manufacturing issue.

Logical Data Issues

Many occurrences of data loss due to logical problems are down to complacency, such as malware corrupting the contents of the drive or reformatting a partition without double checking that it is the corrupt volume. Any machine with access to the internet or has software installed from an external source may in danger if it has been infected by malware.

Historically Windows computers have been most at risk, with many users still being complacent. For Apple Mac users the problems of malware were relatively low, which has caused many to be complacent and even arrogant about the situation, which has caught many out over the last year, as the number of malware attacks on OS X based computers has increased alarmingly.

Avoid Taking a Risk

It is important that you install an anti-virus software suite, which can help to avoid these problems, but again you should not be complacent and think it can’t happen to you, as many incidents where malware is installed manage to bypass many security precautions. It is therefore essential that you have a backup plan in place, to ensure that all important data can be accessed and restored in the event of a failure.

It has become common to use cloud storage space as a means of backing up important data, but if you produce large quantities of data, these may prove inadequate when it comes to restoring the data in a timely manner. We have undertaken a number of recoveries from RAID storage arrays holding tens of terabytes of data, which was also stored in the cloud, but it was found to be faster to send it for recovery, than wait for the files to be downloaded over the internet.

Apple Mac Computer and Malware

For many years Apple ran ad campaigns based on the idea that Mac computers were not vulnerable to virus infection or malware. While it was true that there were very few viruses written with the Mac operating systems as a target, there were still instances.

When Apple moved away from using the Motorola processors to using Intel and the introduction of operating system independent languages, the situation changed as it was easier to design cross-platform malware. At DiskEng we have seen a rapid rise in the number of Apple Mac computers and laptops arriving for data recovery which have suffered a virus or malware infection.

The Driver Behind Apple Targeted Viruses

A major reason for the initial interest in creating viruses targeted at Apple Mac computers was Apple’s original viewpoint that their operating system was more secure than others and therefore not vulnerable. Hackers and virus writers took this as a challenge, putting resources into finding security holes and how to take advantage of them.

The second reason behind the increased interest in targeting Apple users, is that their systems are in general much more expensive than an equivalent PC. This suggests that there is more money to be made illegally from taking control of an Apple Mac, to either obtain online banking details or install ransomware in an attempt to directly take money.

Main Virus and Malware Issues

There are two basic types of malware which can cause serious issues for the user. The first type is intended to solely be malicious just for the sake of causing damage, often deleting or moving important areas of the file system, such that it can no longer be mounted by the operating system.

The second type is much more insidious, either taking control of the computer to use as part of a botnet, steal user information, or even worse install ransomware which demands payment from the user before they can gain access to their files. Once ransomware has been installed, the only way to gain access to your files again without paying up, is to reinstall the operating system. Reinstalling the operating system will most likely cause the user files to be deleted, making it essential to send your disk for data recovery.

Complacency and Data Recovery

The main reason for the alarming rise in malware attacks on Apple Mac computers is down to complacency, leading to many users not even installing any security software. This situation needs to be addressed properly before it becomes an epidemic, causing mass data loss.

Our experience of recovering data from Apple Macs which have suffered a virus attack, reveals a very high data recovery success rate. If your Apple Mac becomes infected with malware which is not dealt with by your security software, you should turn it off and send your drive for data recovery.

The Future of Storage: SSD and HDD

For many years the press has been predicting that hard disk drives are about to be superseded by solid state drives, but these appear to have been premature predictions, despite the slowly down in drive capacity increases. Although solid state drives provide a significant boost in data transfer speeds, there still exists a large price differential, which is about a factor of ten. It is also interesting to note that the first type of solid state device pre-dates the first spinning drum hard disk technology, which was introduced in 1956.

Although hard disk drives recording techniques are undergoing constant development, the technology is well known in terms of data recovery. In contrast solid state drive development is changing significantly, meaning that new data recovery techniques and proprietary hardware may need to be developed each time a different iteration of the technology is seen.

Important Data Storage

Humans tend to be inherently lazy, so that even when we store new that is extremely important, we often don’t think about whether it will be backed up. If your storage device fails before it is backed up, the storage technology will determine how difficult and costly it will be to recover the files.

Professional data recovery companies, such as DiskEng have a wealth of experience in recovering data from a multitude of hard disks drive, having seen thousands of cases. It is only in the worst cases of damage that data recovery will either take a long time or be a failure. For a solid state drive the success rate is slightly lower and the situation may call for new hardware and techniques to be developed, which can be expensive.

Use of Traditional Hard Drives

While solid state drives are perfect for the operating system, within the data recovery industry the consensus of opinion is that all important data is stored on a hard disk drive, as the method of data recovery is a well-known quantity. It is only in data centres will multiple redundancy built into the storage systems, where it makes sense to use large quantities of solid state drives, in order to provide high speed storage space.

These arrays of solid state drives tend to be much more expensive and not necessarily more energy efficient, than the traditional hard drive. However, in a system which may be prone to vibrations, is likely to be the more sensible choice. Whichever technology is used, a backup strategy should be in place, rather than trusting to luck.

The Future of Storage

While Samsung have released the largest ever drive in the form of a 16TB SSD, the device is very expensive and aimed at the enterprise market. Such a device is intended for data centres requiring high capacity storage combined with the fastest possible data transfer speeds. Traditional enterprise level hard disk drives are however competing more directly with arrays comprising smaller SSD drives, where the difference in power consumption and space required for the larger number of solid state drives, mean there is little difference apart from data transfer speeds.

With Western Digital acquiring SanDisk along with Fujitsu and Samsung also owning a large proportion of the solid state drive manufacturing equipment, the price differential is likely to remain in place. It will be difficult for an independent solid state drive manufacturer to muscle in and change the current status quo. While this may sound counter-productive for the consumer, in the long run with future hard disk drive technology promising faster and larger drives than SSD’s are capable of, it ensures a controlled development cycle, while minimising the risks of data loss.

Brand New Hard Disk Won’t Fail

Even though the reliability of hard disk drives has improved immeasurably over the last two decades, it would be a mistake to think that just because you’ve purchased a brand new drive, it cannot fail. In spite of the years of experience the manufacturers have, the odd model of drives may contain a design flaw, which may affect the firmware, electronics or in the worst case be a physical problem.

It is also possible that an incident may occur during shipping or installation, such as an impact event or static shock damage when fitting the hard drive. If such an issue is present, these are quite likely to present themselves as a failure within the first few weeks of the drive being used and in almost all cases will require professional data recovery.

Shipping and Installation Issues

Checking online reviews for any hard disk drive will usually reveal at least one incident of a drive being classified dead on arrival, either as the result of an impact event, such as the drive or box containing it being dropped, or through poor installation procedure, whereby a static shock has damaged the electronic circuitry. It is all too common now to buy computer components and find that the packaging is sub-standard or the hard disk drive is packaged only in an anti-static bag and placed at the edge of the box, where it may be hit during shipping.

An impact event may however, not cause an immediate failure in all cases, causing the drive to operate in a degrade state before the problem finally causes a catastrophic failure. Maintaining another copy of your important could save you having to send the drive for data recovery.

Manufacturing and Design Flaws

The drive manufacturers are always pushing the boundaries of the technology in an attempt to stay ahead of the competition. This does however result in the occasional family of drives being produced with an inherent fault, some of which may be revealed early in the life of the drive, although many only become apparent after many months or even years of use.

Reviews from online retailers may reveal drives with an issue, but for a new drive model any problems may not be revealed for some time. It is only a small subset of drives which tend of have these issues, but these are often some of the most popular models. Manufacturing issues are now rare, although these still crop up occasionally.

Hard Disks Will Fail

The harsh reality is, that all hard disk drives will fail, ranging from those dead on arrival upwards. The majority of hard disk drives will remain working for their intended operational lifespan, depending upon how often you upgrade your computer systems.

You have little or no control over manufacturing or designs flaws and the shipping of the drive, but the handling and installation procedure should only be entrusted to personnel who will take all the necessary anti-static precautions. Having to send an almost brand new hard disk drive for data recovery is a problem the end user can do without.

It Was Working Yesterday

The phrase “it was working yesterday” or something similar is regularly muttered every day. Just because an electronic machine was working yesterday, or even just minutes ago, it may fail for no apparent reason at any moment. Even if it’s not visible, there will always be a reason for the failure.

Computers have an annoying habit of failing at the most injudicious moments, which can lead to panic decisions being made. It is important to determine the cause of the issue, but not put your valuable data risk. Any hard disk which fails should be sent for data recovery and repeated attempts at booting a computer with a failing or failed hard drive can result in further damage occurring.

Warning Signs

There are often warning signs that a problem is imminent, but we as humans are often poor at noticing these. These can range from hearing unusual sounds coming from the computer, such scraping or clicking noises or fans becoming louder or noticeably running faster.

The computer becoming less responsive, such as slower to open an application or for example load an office document. Running software to monitor the computer may prove useful, but again it is easy to ignore any warnings it may give.

Power Surges

Power surges are still a fairly common occurrence, often noticeable by a slight flickering from the lights. In general, these are not bad enough to cause any harm, but once in a while, especially when a power cut occurs, these can be quite severe, especially if they are the result of a lightning strike.

There are many power boards which provide surge protection, but even the best ones may not be able to protect against the severest power spike. Most modern motherboards also have surge protection built in, but again this can only protect to a certain level especially the power supply is damaged, which may lead to damage occurring to the hard disk drives or other components.

Data Recovery

At DiskEng we have seen a vast array of problems following a sudden unexpected failure. A sudden failure is less likely to have caused damage to the file system stored on the disk. However, if the underlying problem has existed for a long time without being noticed, the damage could be quite severe.

Following a computer system failure, it is important to seek professional advice, rather than attempt to recover the data yourself, which could result in more damage and in the worst case a total loss of data. At DiskEng our engineers are trained not take any risks which could lead to further unnecessary damage occurring.

Disaster Recovery Plan

It is easy to think that we will never be subject to a disaster or lose any data, but every day the future of companies are put at risk through the lack of an adequate disaster recovery plan. A fire or a flood is an extreme example of a disaster which could lead to data loss, right through to a hard disk drive failure.

Knowing how to react to disaster, such as acquiring new equipment and which company to send any storage media for data recovery are important aspects of disaster recovery planning. It is important that not only is the data recovered as fast as possible, but also that when that data is returned, you can immediately become operational again.

Company Facilities

It is important that you know who is responsible for your disaster recovery plan. This plan should cover purchasing new equipment, temporary office space if required and communications, both phone and internet. Another important aspect is the key software applications, such as the accounts package, some of which may require newer versions and how these can be operational as soon as possible.

Data Backups

Data backups are too often overlooked, or after a period of time, the procedures may become lax due to them not being used. It is of the utmost importance that key data files are located and regularly backed up, which needs to be regularly reviewed. Of equal importance is having a procedure in place for ensuring that the backups have successfully completed. How often these backups are verified and media rotated are also important, along with sending media off-site, so that in the event of a disaster, such as a fire, it is not destroyed.

If you are using a cloud service provider, or any other third party company for data access, it is important to review their procedures, to ensure they are not a weak link.

Security and Protection

The security of data backups and company assets should be reviewed, to ensure that these cannot be tampered with. Data security extends to any third party data storage provider you use, to ensure that they are not putting your company at risk.

The protection of your computers and any servers is important. Such protection includes battery backups such as UPS, security software such as antivirus/malware and spyware. These must always be maintained ensuring UPS’s work correctly and all security software is updated.

Data Recovery Plan

Even if you have a complete disaster recovery plan, with all aspects operating correctly, it may still be necessary for data recovery to be required. Recently changed data may not have been backed up or the most recent data backup may have been destroyed in the disaster.

A number of companies using cloud servers to back up data have encountered the problem where it could take weeks to restore the data to their computers. With companies storing multiple terabytes of data cases such as these are quite common, making data recovery the only suitable solution to recovering the data in a timescale that will avoid the company going out of business.