Veritas NetBackup Archive

Veritas NetBackup, an enterprise level backup and disaster recovery suite, acts as a heterogeneous cross-platform solution, was formerly known as Symantec NetBackup. Veritas NetBackup utilises a central master backup server, upon which one of many operating system may be installed, such as Solaris, HP-UX, AIX, Tru64, Linux and Windows. Data is streamed between the client computers and the backup media via the central master server testtest.

NetBackup is supported by a wide variety of hardware devices, providing the ability to back up live databases and virtual machines or create snapshots. Veritas NetBackup is mainly used in the enterprise market, which means the need for data recovery is rare, with most examples sent for data conversion. The Veritas NetBackup underlying data structure is based upon a variation of the well-known TAR archive format.

Veritas NetBackup History

In 1987 Chrysler Corporation hired Control Data Corporation to create a backup software solution, who later used it for their own data archiving purposes. The Automated Workstation Backup System (AWBUS) was created in 1990, for which the first version on the SGI IRIX OS supported twin tape drives which utilised a robotic carousel. The product was renamed to BackupPlus three years later, by which time additional support for media Volume Management and Hierarchical Storage Management had also been added.

OpenVision Technologies the acquired this product and the developers later in 1993, renaming the product again, this time to NetBackup. When Veritas acquired OpenVision Technologies in 1997, they also took control of the NetBackup product family. In 2005, Symantec acquired Veritas and despite their own existing backup solutions, continued to market NetBackup. Symantec split their Information Management Business in 2005, which includes the NetBackup products, creating a new company called Veritas Technologies Corporation.

Features of NetBackup

While the main function of NetBackup is the ability to simultaneously backup multiple client computers, many other features as also present, including data encryption, data deduplication and image replication. Another additional functionality of NetBackup is the ability of creating snapshots of systems with purpose of securing or replicating another server.

The management functionality is provided through a Java based administration console, through which activity can be monitored. The creation and viewing of management reports can also be done using a web server.

Conversion and Recovery of Veritas NetBackup

Due to NetBackup mainly being used in an enterprise environment, the requirement for data recovery is rare. Requests for converting data from tapes containing NetBackup archives are much more common. The main use of Veritas NetBackup is within the enterprise environment, where multiple backups and other security measure are usually in place, making backup failures to be rare.

NetBackup archives often contain several multiplexed backup streams from several computer systems, which is the main complexity when it comes to recovering or converting data from an archive tape. Each backup must be de-multiplexed in order that the data from each computer can be processed correctly.

So far, we have not seen any corrupt data streams, with all data recovery examples featuring media damage. This requires an expert knowledge of the backup structure, in order to overcome the damage resulting from lost data blocks. Our tape data recovery at DiskEng have the experience and expertise to de-multiplex NetBackup data streams allowing data to be converted or recovered.

Hard Drive Platter Storage Architecture

One of the misconceptions about the low-level data storage is that the 0’s and 1’s are physically stored on the drive platter. The data is however encoded before being written to the platter as a wave form testtest.

Prior to writing the data, it is randomised, which eliminates repeated patterns, which could cause problems whereby the Error Correction Code (ECC) becomes confused. As the drive firmware handles this automatically, in most cases it has no bearing on the data recovery procedure.

Physical Data Layout and Servo Data

Current hard drive technology lays the data down in a set of concentric tracks, around which servo tracking information is also stored, which the read/write head uses to ensure the alignment it correctly maintained. Each sector of store data also has an overhead associated with it, including Sync data, Address Marker, CRC and ECC, all of which are essential for correctly storing and accessing the data.

In the future, this tracking information may however be stored on a separate layer, allowing more data to be stored while improving the efficiency of maintaining the read/write head alignment. Future improvements may come from not only storing the track servo data as a separate layer, but also creating additional data layers on a single platter surface, which are accessed by focusing the read/write head to a different depth as required.

Drive Error Codes and Diagnostics

An extremely important part of the operation of a hard disk drive is how it handles any errors and how it interacts with the host computer system. Although called error codes, they also indicate if the drive is ready, busy or received a data request to or from the host computer.

In most cases, any real errors will be handled by the drive firmware, such as automatically mapping an unreadable sector to the spare area of the drive. If an error code is passed back from the drive to the host computer, it is essential that the operating system handles this in a sensible manner, without causing a system crash, which could result in unsaved worked being lost.

Data Recovery and ECC

At one time, it was possible to read the data sectors without invoking error correction, but even for data recovery purposes this proved to be of little use and is only available for drives less than 137GB in volume. By default, the drive will make several attempts to read each sector using different methods before it will timeout and return an error.

There are several techniques which our data recovery specialists can employ to read a previous unreadable bad sector, although each attempt may risk causing further damage to the drive if the problem caused a physical issue. Therefore, our data recovery engineers will maximise the yield of good data sectors before making attempts to read sectors which failed to be read correctly during the imaging process. In a number of cases our data recovery engineers have been able to recover all the sectors from a failing hard drive, despite a sector being unreadable during the sequential imaging process.

Hard Disk Drive System Area

An extremely important, but rarely mentioned part of a hard disk is the System Area (SA) which contains vital information required for the normal operation of the drive. Any damage to the System Area can cause problems, ranging from small levels of erratic behaviour to the drive being totally inaccessible.

If the System Area on a hard drive becomes damaged, the drive must be sent for professional data recovery, in order for the data to be accessed. Making attempts to recover the data yourself may, depending upon the problem, make the situation worse, which could result in a total loss of data.

System Area Information

The importance of the System Area becomes apparent by examining the data stored in it, which includes the system logs, smart data, drive serial and model numbers, the defect lists, firmware, test routines, recalibration code, translation data, security information and other important information essential for the correct operation of the drive.

The corruption of even a single part of the data within the System Area may cause the drive to malfunction. Depending upon the exact nature of the corruption, it may be possible that the drive will still, function behave in an erratic manner, or it may total fail operate at all.

Important Information About the System Area

In most drives, there is at least two copies of the System Area, which is usually located on different platters, in most cases at the extreme out edge. This method of storing one or more backups of the System Area, should allow the drive to overcome corruption of the primary copy, which allows the disk to continue operating correctly.

There is no unified or standardised format used for storing the System Area information, which can be completely different for between drives in the same family. Often, even a difference in the firmware version for a particular make, model and capacity of drive will lead to the System Area data being different. This can cause problems in sourcing the correct donor drive when the firmware or other System Area information has been damaged, causing the drive not to function correctly.

System Area Damage and Data Recovery

In the event of System Area damage, the drive will require professional data recovery to gain access to the data. The recovery process requires another drive of the same model, capacity and firmware to be sourced as a donor.

The process requires complex electronic work to complete the hardware modifications essential to allow the drive to be made operational again. This work should only be undertaken by a qualified data recovery hardware specialist, in order to ensure the drive is not damaged further.

Hard Disk Drive Startup

The sound a hard disk drive makes when you first turn your computer or external storage device on is well-known, with the motor starting to spin, closely followed by a few clicking sounds. If all is fine with the drive, it will quickly settle down with the platters continuing to spin, waiting for the drive to be accessed.

Any change in these sounds, such a different pitch to the spinning or extra clicks from the read/write head should be taken as warnings that the drive may not be completely healthy. If the drive is heard to spin down or it makes a lot of clicking sounds without the computer being able to boot up, the computer should be turned off and the drive sent for data recovery.

The Disk Drive Boot Sequence

When power is applied to a hard disk drive, it checks the status of each chip contained on the controller board, in order to ensure all the electronics are functioning correctly. The drive then performs a self-check of its other components.

If the controller board chips check and the self-test of the other components passes, the drive spindle motor is started, which spins the platters causing debris to be removed from the surface. The spinning platters causes movement of the air or gas contained with the drive to flow, which creates the air bearing, essential for keeping the read/write heads flying at the required height. This airflow causes the guard protecting the read/write heads, which are parked when the drive is powered off, is moved once the air is moving fast enough to ensure they will not come into contact with the platters. When the drive is powered off, the spinning of the platters is used to ensure the read/write heads are moved back to the parking position.

The read/write heads check the servo timings which allows them to located the exact position of the system area and other sections of the drive. The system area is then read from the drive platters, which can include additional firmware and overlays, which is most often stored at the other edge of the platters.

Problems During the Boot Sequence

If the drive servo timings, the system area or the read/write heads are damaged it will result in repeated clicking noises as an attempt is made to access the data. It is also possible that if a hard disk drive is powered up and powered of several times in a very short period of time, that the read/write heads can fail to move back to their parking position, instead becoming stuck to the surface of the platters, which will most likely damage the read/write heads as well as the magnetic recording layer.

Corruption of the firmware or system area information can result in a failure of the drive to boot up. These may result in the drive clicking repeatedly or the platters may spin down. In either case, the drive will probably not be detected by the computer BIOS.

Disk Boot Failure and Data Recovery

If your hard disk drive fails to start correctly and makes repeated clicking noises, or the platters can be head to spin down, you will require professional data recovery. The drive will require the use of donor parts to overcome the issues before a sector-by-sector image of the drive can be secured.

In either situation is important that the drive is powered off as continually attempting to boot the drive up can result in further damage occurring, which could ultimately result in your data files being lost.

Half Inch Open Reel Tapes

In 1951 the first magnetic tape media for storing computer data was introduced, using the half inch tape media, used in the UNISERVO tape drive used on the UNIVAC I computer. IBM introduced the 7 track recording format the following year with half inch media wound onto open reel spools, which used the linear recording format.

Half inch open reel media was the standard backup and near line storage medium of choice within the enterprise computer industry. Half inch open reel tape drives have featured in many movies over the years, in some cases long after being obsolete. Half inch open reels are rarely used, although many half inch tape reels are held in storage, containing archived data, which may require data conversion or data recovery when the stored data is required.

History of Half Inch Open Reel Tapes

IBM’s 7 track recording format stored 6 bit values plus a parity bit, adequate for storing textual data until the early 60’s when a replacement was required. The IBM System/360 was introduced in 1964 a 9 track format, which could store 8 bit values in parallel with the calculated parity bit.

Many advances were made during the next three decades such as increasing tape speed and the recording density. These techniques included the use of phase encoding (PE), group code recording (GCR) and non-return-to-zero, inverted (NRZI). Originally a zero bit was indicated by a zero voltage being returned sometimes causing issues with detecting recorded data bits. NRZI introduced the idea of returning a negative voltage making it easier to detect each recorded data bit. Although these advances had enabled more data to be stored with an increased reliability, by the 90s newer tape media using smaller self-contained cartridges were capable of storing much larger quantities of data, leading to the decline of the half inch open reel tape.

Issues with Half Inch Open Reels

Until auto loading tape drives were introduced, loading a half inch open reel could be a complicated process. The tape media is threaded around several guide wheels and across the read/write head, which has proved robust, with few tapes ever suffering an alignment issues or being damaged by the drive mechanism.

The deterioration of the lubrication layer deposited on the surface of the media, is the source of the biggest problems that are seen with half inch open tape media. The lubrication layer will be sticky when it deteriorates which can cause it to stick to the read/write head. All half inch open reel drives feature a safety cut-off which will stop the tape when the tension becomes too high, ensuring no further damage can result. The read/write head will usually become clogged and dirty, with damage resulting to the part of the tape which stuck to it. Even if the media does not stick, if the lubrication layer starts to clog the read/write head it can cause read errors to occur.

Data Recovery and Conversion from Half Inch Open Reels

Many examples of half inch open reel tapes which are seen, even if stored correctly are now suffering the deterioration of the lubrication layer. This means that most half inch open reel tapes will require data recovery. Depending on the state of the media the recovery process can be extremely complex. When the media is in poor condition it will usually require manual intervention by the data recovery engineer to ensure the media streams smoothly.

A well-known but last resort method for treating the lubrication breakdown issue is to back the tape media. Once this process has been undertaken there is only one chance to successfully read the data from the media, as the magnetic recording is destroyed. This process is therefore rarely done until all other non-destructive data recovery techniques have been explored.

JBOD and Windows 10 Storage Spaces

Many motherboards, storage controllers and even standalone multi-disk units support JBOD (Just a Bunch of Disks) which allows multiple disks to be attached to a computer and presented to the host operating system as a single drive. Windows 10 has introduced a facility called Storage Spaces, which allows unallocated storage space on any attached disks to be combined together to create a single data volume.

Neither of these include any fault tolerance or redundancy and may complicated the data recovery process in the event of even one drive failing, particularly when Storage Spaces has been used in simple mode. In concept these appear to be good ideas, as they allow large volumes to be created, but it puts any data stored on them at risk. Storage Spaces has replaced a similar system which was available in some older versions of Windows.

Data Spanned Across Disks

Apart from being able to create a single large data volume, using a JBOD configuration or Storage Spaces in simple mode does not provide any advantages over any other method, even setting up a set of hard disks as a RAID 0 array.

If you wish to use such a configuration it is important that all data being stored is regularly backed up to ensure that in the event of failure, your data can be restored without resorting to data recovery, which may be a complex process. Determining the order of spanned sections of drives can sometimes be difficult to determine.

Storage Spaces Mirroring and Parity

Storage Spaces does provide the possibility to configure mirroring and even parity, which uses a RAID 5 scheme. These configurations are preferable to using the simple mode, which spans a volume across different drives.

Mirroring allows the use of two or even three drives, which provides the maximum possible redundancy, but as it is controlled by the operating system will use some CPU. The use of RAID 5, especially through the operating system poses a much higher risk, but still preferable to no redundancy at all.

Data Recovery from JBOD or Storage Spaces

The use of the RAID 5 configuration or mirroring through Storage Spaces provides a high level of recoverability in the event of failure. The use of simple mode or a JBOD provides no fault tolerance and the chances of successful data recovery is determined by the level of damage which has been suffered to the failed drive.

If you require a larger data volume a better option to Storage Spaces is to install a dedicated RAID controller. If you are using a standalone unit, it is much better to use a RAID level which provides fault tolerance, thereby increasing the chances of a successful data recovery in the event of a failure.

Exabyte Tape Cartridges

Exabyte Corporation pioneered the 8mm magnetic tape data storage backup format for use on computer systems. The format was also known as Data8, in many cases abbreviated to D8 and also sometimes written as D-Eight. The cartridges are mechanically the same as tapes used in camcorders and 8mm video format recorders.

Until the development of the AIT data cartridge, Exabyte were the sole vendor of 8mm format tape drives. Exabyte Corporation was created with the aim developing the 8mm video format to be suitable for storing digital data. Using the available 8mm video tape technology they built a reliable mechanism and data format. The last Exabyte data cartridge, the Mammoth-2 was developed in 1999, making them obsolete and usually only seen for data conversion or data recovery when old archive tapes fail to restore.

Exabyte Data Cartridges

Exabyte cartridges used Metal Particle (MP) tape, first developed in 1987 with the EXB-8200 drive range was capable of storing up to 7GB of data using the EXB-8205XL drive using software compression on a 112m tape cartridge. The earliest models did not feature high-speed search, an important feature introduced with the EXB-8200SX drive. The EXB-8500 drive released in 1990, improved the data transfer speed and added hardware compression with the 8500c model.

The EXB-8505 drive introduced in 1992, further improved the data transfer speed, also increasing the capacity using a 112m tape to 10GB. Two years later the EXB-8505XL was introduced to allow data to be written to the newly developed 160m XL data cartridge, allowing 14GB of data to be stored. The EXB-8700 introduced a desktop top loader in 1995 and the Eliant 820 in 1998 failed to add any new features, probably due to the introduction of the Mammoth tape cartridge, leaving the Exabyte range of drives behind the competition.

Exabyte Mammoth Tape Cartridges

The Exabyte Mammoth tape cartridges use Advanced Metal Evaporated (AME) tape. The Mammoth drives provided read-only capability for data write to MP tape cartridges. The EXB-8900 drive was introduced in 1996 was capable of storing 20GB of data in native format on a 170m data cartridge. The EXB-8900 tape drive featured an LCD for displaying the status of the drive.

The Exabyte Mammoth-2 (M2) introduced in 1999 also made use of AME tape cartridges, which introduced an integrated 2m cleaning tape header, named Smart Clean. Three new cartridges were introduced, the 75m AME (20GB Native), the 150m AME (40GB Native) and the 225m AME (60GB Native) for use in the drive. The Mammoth-2 drive was capable of writing to AME media used in the original Mammoth drive, but unable to read the original Exabyte data cartridges.

Exabyte and Mammoth Data Recovery and Conversion

The Exabyte and Mammoth data cartridges are rarely used, except for some legacy systems. Most data cartridges of these types contain archive data, normally only required when specific archived data is required, often as the result of an E-Disclosure order. Such data should be transferred to new media on a regular basis, as the tape media will slowly deteriorate over time.

With the lifespan of most Exabyte and Mammoth tape cartridges having expired, it is all too common for a cartridge sent for data conversion to require data recovery, due to the presence of a media flaw. Such tapes should only be handled by a professional data recovery company who can handle the extraction of data blocks from damaged media and have the required knowledge to abstract the data files from the backup format held on the tape cartridge.

Tape Has Unspooled

In general tape data cartridges have a high level of reliability, but when problems do occur it is often when attempting to restore your data. Fortunately, the tape media becoming unspooled leads to the data cartridge being unusable and for cartridges using a single spool, the tape drive will need to be repaired.

If the tape media unspools into the tape drive and the data stored on it is important, the tape drive should be sent for data recovery, before it sent to be check for any damage which may have occurred. It is also best to send the data cartridge, even if you believe it is empty, as it may allow us to determine the cause of the problem.

Unspooled Tape Media

Fortunately, it is rare for the media used in cartridges such as DLT, Super DLT and LTO data cartridges to unspool into the drive. No attempt should be made to remove the tape media from the drive, as it must be removed carefully to avoid any damage occurring. This is a job best left to a professional data recovery specialist.

There are two reasons for tape media to become unspooled from a cartridge which uses a single spool, either a fault in the tape drive “end of tape” detection circuit or a problem with the media itself. The most common method for determining the end of the media is using sets of holes, which are detected during the movement of the tape, but a failure to detect them will result in the media spooling off the cartridge.

One set of media which uses two spools, such as Quarter Inch and Mini-QIC data cartridges detects the end of the media using the same method of holes in the media, with light shining through a prism fitted in the cartridge. For cartridges such as DAT and AIT cartridges the media is fixed each spool and relies on the detecting information in the tracking area. When rewinding the tape, it will often wind tight using media fixed to the empty spool to stop the tape. A failure of either will result in all the media being spooled on only one of the spools.

Unspooled Tape Data Recovery

The data written to a tape which has unspooled should be unharmed, although if it occurs when writing a backup, the dataset will be incomplete, but all files backup up to that point should be safe. If the media itself has been overused, there may be damage to the data stored on the media, which is why media rotation is extremely important.

Never attempt to fix the problem yourself, as it may result in further damage. In many tape data cartridges, the end of the media is not affixed to the spool, which is an important safety feature. Using sticky tape to the fix the media to the spool could result in the further damage occurring to the media, particularly if the end of tape detection mechanism has failed, either in the data cartridge or the tape drive. Such actions could result in the media becoming tangled. Always send any unspooled data cartridge for professional tape data recovery.

Data Corruption Issues

In spite of hard disk drive reliability improving and file systems more resilient, data corruption remains a serious issue. The most reliable way to guard against losing data through corruption is through the use of backup strategy.

If you suffer corruption of your data, the best option for recovering the files is to send your hard disk to a professional data recovery company, such as DiskEng. Attempts to recover the data yourself, especially if the file system has become corrupt, may result in cause further damage, resulting an even greater loss of data.

What is Data Corruption?

Data corruption is when file data or important file system structures are altered to contain data they shouldn’t. This results in bad or damaged data being returned when the file system is accessed or an attempt is made to access a file.

Such corruption can lead to software or operating system crashes, driver damage, further data or file system damage. Data corruption is a serious problem, as it will not fix itself unless caught early enough.

What Causes Data Corruption?

There are several reasons for data corruption to occur, which can range from physical damage, damage caused by malware through to damage caused by a sudden disruption of a data transfer. Physical damage usually the result of bad sectors appearing as the result of overuse or some other failure causing damage to the surface of the platters.

A sudden disruption during the transfer of data to the disk, for instance as the result of power loss or disconnecting a drive can cause some data not to be written to disk. While most modern file systems contain journaling, there are rare instances where this is unable to resolve the problem.

Malware can cause the worst corruption, either to the contents of data files or the data system structures. If you are not running any anti-virus software, you are putting your data at serious risk.

Data Recovery and Data Corruption

Data corruption can make the process of data recovery complex, which is why it should not be allowed to become worse. No risks should be taken, such as attempting to fix the issue yourself, as they result in making the corruption becoming worse.

It is best left to the professional data recovery experts to recover the data, which for the worst instances may require manual inspection of the data structures in order to determine the solution. The use of do-it-yourself software could result in the incorrect assumptions being made, which could lead to the problem becoming worse.

SSD the Truths and Myths

With the massive increase in data transfer speeds, combined with the drop in price per gigabyte, the uptake of the SSD drive has unsurprisingly been rapid. They appeared to be the answer we had been looking for, touted as super reliable, faster data transfer rates and use less power than traditional hard disk drives.

Much has been made of how data recovery from SSD, where development is moving rapidly, being a complex problem during technology changes, making it most sensible to use them for installing the operating system and applications, rather than storing any critical user data. The high duty cycle numbers suggest that even when used in for intensive IO applications they should not fail before the end of the average computer lifespan.

SSD Lifetime

Contrary to the popular belief, even though each memory cell is only good for a certain number of write cycles, this is not the significant factor governing the lifespan of an SDD. In research published earlier this year, it appears that an SSD is not damaged through usage, but primarily through its age.

Even more worrying is that the research has not thus far reached a conclusion on the causes of these failures. For most users it is also alarming how many read errors an SSD has, most of which are transparently corrected by the drive.

Data Loss Through Sudden Power Loss

There were many stories of data being lost as the result of a power surge or sudden power outage. Some of the earliest first and second generation drives did not employ enough protection to write any pending data to the memory chips. Such issues, unless you are still using an old SSD which had this problem, should no longer be an issue.

SSD and Long Term Storage

Alarming stories appeared last year, stemming from research which showed an enterprise level SDD stored at a high temperature with no power, lost data in only 7 days. While this is significant research, in most real world situations, loss of data from an unpowered SSD is extremely unlikely.

However, if they should not be used for long term storage media, as the average data retention time for an enterprise level SSD is about a year while consumer drives this is two years. This makes them a poor choice for storing data which may not be required for several years, making other storage solutions a wiser choice.

SSD and Data Recovery

The research earlier about SSD failures, was largely ignored as most people seemed to ignored the significant issue, only focussing on the fact that the number of failures has increased with more drives of this type being used. What should have been the focal point, is how a much higher proportion of SSD failures resulted significant levels of data loss, when compared to traditional hard disk drives.

Although data recovery from an SSD is a much more complex, it does not account for increased occurrence of data loss. It appears that when an SSD fails, it is usually associated with multiple read failures which cannot be corrected by the drive or during the data recovery process.