Many commercial backup products are available on the market today. The leader amongst them on UNIX platforms is VERITAS Software's NetBackup DataCenter. We will use this product, which we refer to as just NetBackup, in our explanations and examples of setting up a backup domain. We start with an introduction to NetBackup, including an explanation of the unique architecture of the product, and then we define the terms that it uses. Most of the examples use the latest release, 4.5, but we will mention when there is a significant difference with older releases.
NetBackup Tiered Architecture
NetBackup uses a four-tiered architecture for backup domains.
The tiers are as follows:
-
Client. Any system that contains data that needs to be backed up.
-
Media server. Any system that has physically connected storage devices to be used for backups. These can be robotic devices, standalone tape drives, or optical storage devices.
-
Master. The NetBackup server that provides administration and control for backups and restores for all clients and servers. It is also the system that contains all the catalog information for the backup domain.
-
Global Data Manager. A master of masters that can monitor and facilitate management of multiple master servers and multiple backup domains.
All systems within a NetBackup domain fall within at least one of these tiers and can actually fit into more than one. The first three tiers are always found, even if on the same system. The fourth tier, the Global Data Manager tier, is usually found when there are multiple NetBackup domains that are monitored and administered from a single location. This tiered architecture is one of the things that make NetBackup so scalable and flexible. As you start out, you can have a single master server that gives you a single point of administration, and at the same time, you can have as many media servers as are needed to support your configuration. As you grow, you can add more tape devices and just add more media servers without having a great impact on your overall configuration. If your enterprise continues to grow, you can simply add another master server with media servers as needed. At this point, you might add the fourth tier. The first tier, clients, can be added or deleted easily, since the configuration is kept on the master server.
Explanation of Specific NetBackup Concepts
The NetBackup product can be thought of as being made up of two major components: netbackup and media manager. The netbackup component is responsible for the who, what, when, where, and how aspects of the backup jobs:
-
Who needs to be backed up? The client.
-
What needs to be backed up? The file or file list.
-
When do the backups run? The schedule.
-
Where should they be stored? The logical storage unit.
-
How should this policy be handled? Specific attributes of the backup policy.
It also tracks and manages all the backups and all of the backup images.
The media manager component is responsible for managing all the physical media and all the devices. In general, the netbackup component deals with logical devices, and the media manager deals with physical devices.
The netbackup component tracks and manages all the data that is backed up by using the unique backup identifier assigned to each backup image when it is created. It also manages the overall catalog and the scheduling of new tasks. It selects the appropriate media server to match each backup job.
The media manager component manages all the physical storage devices and the physical media. It is through the media manager that the physical tape libraries and drives are configured, and the volume database is populated. The media manager controls the tape libraries and maintains the inventory of all the volumes.
Layout NetBackup Domain
Now the fun begins. You have gathered tons of data and know more about your enterprise than you ever thought was possible. It is time to put all of this knowledge to use. If this is the first time an actual backup and recovery strategy has been implemented, you will be able to tailor the backup domain. If this is an upgrade or application change, you will probably have to work within the confines of the existing layout, making changes as required.
Using NetBackup as the application in this domain, you first want to list all the systems that will be backed up as clients. This will give you an idea of the number of systems that need to be backed up and the distribution of data. Any systems that have a large amount of data, over 100 GB for example, should be noted, as you might want to make them media servers. The other important thing to track with the clients is their network connectivity. If it looks like there are a lot of network-based clients on slow networks, you should consider installing a high-speed backup network. This gives you increased backup and recovery performance, as well as keeping backup and recovery traffic off the production network. It is now common to install a 100Base-T or Gigabit Ethernet network just as a backup and recovery network.
A NetBackup domain requires at least one master server. In most situations, there will be only one; however, in a later chapter we discuss some reasons to have more than one master. The system that you choose for the master will depend on the size of your enterprise-the number of clients, the total number of files being backed up, and the number of storage units you will need. In a smaller environment, the master server can be a system that is already being used for other work or could be a combined master and media server if it is attached to a backup device. Figure 3.2 shows an example of a configuration where the master server is also a media server and all the client backups are basically LAN-based backups.
In larger environments, the master server is usually a dedicated NetBackup server, although it could still be a media server. This server must have enough disk capacity to handle the NetBackup catalogs and, potentially, the debug logs. Most of the debug logs are located in /usr/openv/ netbackup/logs. If this directory is not located in a separate partition, you must make sure you do not allow the logs to grow and fill the disk. The largest part of the catalog is the image database, which is located in /usr/ openv/netbackup/db/images. It is not uncommon for this directory to be a separate partition. All of the meta data for all the backups are sent to the master and stored in this image database portion of the catalog. The maximum amount of disk space that NetBackup requires at any given time varies according to the following factors:
-
Number of files that you are backing up
-
Frequency of full and incremental backups
-
Number of user backups and archives
-
Retention period of backups
-
Average length of full pathname of files
-
File information (such as owner permissions)
-
Average amount of error log information existing at any given time
-
Whether you have enabled the master catalog compression option
To estimate the disk space required for the image database portion of the NetBackup catalog:
-
Estimate the maximum number of files that each schedule for each policy backs up during a single backup of all its clients.
-
Determine the frequency and retention period of the full and incremental backups for each policy.
-
Use the information from Steps 1 and 2 to calculate the maximum number of files that exist at any given time.
-
Assume you schedule full backups every seven days with a retention period of four weeks and differential incremental backups daily with a retention period of one week. The number of file paths you must allow space for is four times the number of files in a full backup plus one week's worth of incrementals.
-
The following formula expresses the maximum number of files that can exist at any given time for each type of backup (daily, weekly, etc.):
Files per Backup × Backups per Retention Period = Maximum Number of Files
-
If a daily differential incremental schedule backs up 1200 files for all its clients and the retention period is seven days, the maximum number of files resulting from these incrementals that can exist at one time are as follows:
-
If a weekly full backup schedule backs up 3000 files for all its clients and the retention period is four weeks, the maximum number of files due to weekly full backups that can exist at one time are as follows:
3000 × 4 weeks = 12,000
-
Obtain the total for a server by adding the maximum files for all the schedules together. The maximum number of files that can exist at one time due to the preceding two schedules is the sum of the two totals, which is 20,400.
Note For policies that collect true image restore information, an incremental backup collects catalog information on all files (as if it were a full backup). This changes the preceding calculation for the incremental from 1200 × 7 = 8400 to 3000 × 7 = 21,000. After adding 12,000 for the fulls, the total for the two schedules is 33,000, rather than 20,400.
-
-
Obtain the number of bytes by multiplying the number of files by the average length of the file's full pathnames and file information.
-
Determining the space required for binary catalogs: If you are unsure of the average length of a file's full pathname, use 100. Using the results from the examples in Step 3 yields the following:
(8400 × 100) + (12,000 × 100) = 1992 KB (1024 bytes in a kilobyte)
-
Determining the space required for ASCII catalogs: If you are unsure of the average length of a file's full pathname, use 150. (Averages from 100 to 150 are common.) Using the results from the examples in Step 3 yields the following:
(8400 × 150) + (12,000 × 150) = 2988 KB (1024 bytes in a kilobyte)
Note If you have ASCII catalogs and use catalog indexing, multiply the number in Step 4 by 1.5 percent.
-
-
If you are running with debug logging, add 10 to 15 MB to the total calculated in Step 4. This is the average space for the error logs. Increase the value if you anticipate problems.
-
Allocate space so all this data remains in a single partition.
You must take many factors into account when determining what kind of system to use for the master server. It can be any type of system, any UNIX system or Windows system from the supported systems list. If the master is a dedicated system, you will need enough computing power to support the network adapters plus the NetBackup processes, as well as enough memory to support each. If the system is also a media server, the system resource requirements are higher. With NetBackup, it is not uncommon to share a tape library among multiple media servers. In many of these cases, the robotic control is handled by the master, while media servers share the tape drives, either directly connected or truly shared in a storage area network (SAN). (We look at using the Shared Storage Option in a SAN in a later chapter.) The following tables provide information about the number of CPUs and the amount of memory needed to support several hardware and software components, as well as the I/O adapter performance numbers. You should use the numbers listed in Tables 3.1 through 3.3 to design your master and media servers.
| COMPONENT | NUMBER OF CPUS PER COMPONENT |
|---|---|
| Network cards | 1 per 2-3 100Base-T cards 1 per 5-7 10Base-T cards 1 per 2-3 FDDI cards 1 per ATM card 1 per 1-2 Gb Ethernet card (preferably 1) |
| Tape drives | 1 per 2-3 DLT 8000 drives 1 per 2-3 DLT 7000 drives 1 per 3-4 DLT 4000 drives 1 per 2-4 8mm and 4mm drives |
| OS + NetBackup | 1 |
| COMPONENT | MEMORY NEEDED PER COMPONENT |
|---|---|
| Network cards | 16 MB per network card |
| Tape drives | 128 MB per DLT 8000 drives 128 MB per DLT 7000 drives 64 MB per DLT 4000 drives 32 MB per 8mm and 4mm drives |
| OS + NetBackup | 256 MB |
| OS + NetBackup + GDM | 512 MB |
| NetBackup multiplexing | 2 MB × no. of streams × no. of drives |
Figure 3.3 shows an example of a typical shared library configuration where there is a single master server and two media servers, each with two drives from a shared four-drive library. This would be a good option if the media servers either had a large amount of data or if you wanted to share the workload of backing up network clients.
If the amount of data is small enough, the drives could be directly connected to the master, making it the master and media server. When determining if you need a media server or multiple media servers, you should consider the following:
-
Amount of data
-
Location of data
-
Speed of networks
-
Backup window
Let's look at these in more detail.
| DRIVE CONTROLLER | THEORETICAL MB/SEC | THEORETICAL GB/HR |
|---|---|---|
| SCSI | 5 | 18 |
| Narrow SCSI-2 | 10 | 36 |
| Wide SCSI-2 | 20 | 72 |
| Ultra ATA | 33 | 118.8 |
| Ultra SCSI-3 | 40 | 144 |
| Ultra ATA 66 | 66 | 237.6 |
| Ultra2 SCSI-3 | 80 | 288 |
| Fibre Channel | 100 | 360 |
Amount of Data
The total amount of data that must be backed up when full backups are done is a good estimate for the maximum data that would be required to be managed. It is also very important to determine how much data is to be backed up on a daily basis. This is usually an estimation based on the amount of user or application data and the daily rate of change. If a filesystem contains 100 GB of data but only has a rate of change of 2 percent, you only have to worry about 2 GB of data for your daily backups. These two numbers, total data and changed data, are also used to determine how many tape drives are needed and are part of the media requirements formula.
Location of Data
If all the data is located on a couple of large file servers, you should make them media servers by physically connecting them to tape drives and maybe have one more to handle all the network-based clients. If the data is spread throughout your enterprise, you must decide how you want to configure the backup domain. You can configure a dedicated media server or servers and back up all the data over the LAN, or you can distribute media servers closer to the clients. The restriction here will be the SCSI cable length restrictions from the media servers to the libraries.
Speed of Networks
If a significant amount of data resides on clients on a slow network, you should consider either installing a high-speed backup network or, if there is enough data, making one of these clients a media server. The other consideration is the amount of traffic the backup and recovery requirements will add to the existing networks. If possible, you should put the backup and recovery traffic on a dedicated network. If this is not possible, you might have to throttle large backup clients on slow networks or they will dominate the network. Table 3.4 will help you determine how different networks will affect the overall backup performance.
Backup Window
The backup window can also come into play when you are determining media server requirements. Some straightforward formulas are used to calculate how many tape drives are required to back up a known amount of data in a fixed amount of time, assuming no other bottlenecks. We discuss these in the next chapter. If the amount of data to be backed up and the amount of time available result in too many drives required for a single media server, this would indicate another media server is needed. You must always stay within the system constraints when configuring media servers. It does no good to put more tape devices on a server than it has the I/O bandwidth to handle. You do not want to create any unnecessary bottlenecks.
| NETWORK TECHNOLOGY | THEORETICAL GB/HR |
|---|---|
| 10Base-T | 3.6 |
| 100Base-T | 36 |
| FDDI | 36 |
| Gigabit GbE | 360 |
| Quad FastEthernet QFE Trunked | 144 |
 |
-
Determine how much data needs to be backed up on a full schedule.
-
Determine the Window for the backups. How much time are you willing to allow for the backups to complete, for instance, 1 hour, 6 hours, 12 hours, and so on-'a weekend' is too nebulous. This needs to be a concrete number from start to finish to make this work out.
-
Once you have determined the amount of data and the window, you can easily determine how many drives are required to physically meet this challenge. This is all about simple numbers. A DLT 7000 drive can write about 8 MB/sec of data on average, so four DLT 7000 drives can write about 32 MB/sec. Therefore, 2 TB of data in a six-hour window will require 11.5 drives - (8 MB/sec × 60 seconds = 480 MB per minute × 60 = 28.8 GB per hour × 6 hours = 172.8 GB per drive per 6 hours. 2 TB / 172.8 GB per hour per drive = 11.57 drives to back up 2 TB in six hours to DLT 7000 tape drives). Drive selection is discussed more in the next chapter.
-
Now it's time to figure out how to provide enough bandwidth to the drives from the media server; a single F/W Diff SCSI card can write only 20 MB/sec, so each port on a card can handle only the bandwidth for two drives.
-
After you have determined the media server physical bandwidth, you need to determine LAN bandwidth to that media server, depending on how many drives it will be attached to. At 32 MB/sec, at least a Trunked QFE at 50 MB/sec or a GbE at 125 MB/sec will be needed. A media server backing up LAN traffic must be configured with something other than a 100-Mb network interface card (NIC).
-
So now that you have determined 4 and 5, you need to choose a media server that has the robustness to handle 4 and 5. A Sun Ultra 10 cannot have Quad FastEthernet (QFE) cards and won't handle a GbE; it is all about the numbers and is pure physics. Find out what you need to move across the copper/glass, and then build a system to accommodate this.
No comments:
Post a Comment