Backup for Workgroups - Blog
We recently had a user ask “Does Backup for Workgroups software use the Windows Archive bit?”
The answer is - No. Backup for Workgroups does not use the archive bit to determine if a file changes and it does not clear the bit after the backup. The reason why BFW does not use the archive bit is because it is unreliable.
First a little history – each file in Windows has a set of flags that software applications can set or use that are associated with each file on the computer. The most common bits or flags are the read-only and system flags. Windows also has a flag called the archive bit. The intention of the archive bit was that when an application changes a file, it should set this bit to tell backup programs the file has changed and needs to be backed up. Then a backup program can look for files that set the bit, backup the file and clear the bit.
The problem is that the archive bit very inconsistently implemented and unreliable. Most apps fail to set the bit making it useless. As a result backup programs cannot rely on the bit to accurately determine if a file has changed or not. When we first designed Backup for Workgroups over a decade ago, we determined that the archive bit gave little to no reliable information so Backup for Workgroups was designed to not look at it, touch it, nor change it.
So how does Backup for Workgroups determines whether a file has changed or not? Backup for Workgroups looks for a change in any of the following three items. The first is a change in the file size. The second is a change in the date / time. And the third is the most accurate of all - Windows has a feature that is called the Change Journal. The Change Journal records information about every file that is opened, read from, written to, and then closed. This information can be used to determine if a file has been altered or changed. It is very accurate and as a result Backup for Workgroups can detect when any software application alters a file anywhere on the computer being backed up.
Yes, Backup for Workgroups can store data on an ReFS formatted hard drive.
Let's spend a moment talking about "What is ReFS?" ReFS stands for "Resilient File System." Microsoft's goal with creating the ReFS file system is to support drives that are larger than what NTFS can handle AND, more importantly, to build-in error recovery. When you format a drive as ReFS, Windows will automatically check for bad sectors on the drive and label these bad sectors. And Windows will continuously look for bad spots on the drive and move your data away from these areas. It also tries to spread the use of the drive more evenly over the drive than simply reusing the same spots over and over. With drives formatted using ReFS, the use of the sectors on the drive is more evenly distributed than with NTFS and FAT32 formatted drives.
ReFS drives generally support the same features as an NTFS drive, but you will find that a couple of features are now missing, such as junction points. Microsoft's goal with ReFS is to streamline the features in order to focus on supporting larger drives and error handling.
From the BFW Backup Server's point of view, a drive formatted as ReFS is just fine to use for storage.
Backup for Workgroups software has earned the Windows Server 2012 Certification from Microsoft Corp. BFW Version 7.0 has passed Microsoft's Server 2012 tests.
When BFW writes files to disk, Windows caches the writes in memory in order to speed up drive access. Periodically, Windows takes the cache data from memory and writes it physically to your hard drive. If the process of taking cached data and writing to the drive fails, Windows reports "Delayed Write Error." In these cases, there is real data loss. Some information that BFW tried to write to the drive is gone. Unfortunately, Windows told BFW at the time the data was written to the cache that everything was OK. The data is lost after the fact and too late for BFW to take a corrective action.
So when you see this - what can you do? Most causes of delayed write errors stem from hardware errors. This can be a bad hard drive, a bad cable to your hard drive, a bad power supply, etc. This message indicates that some piece of hardware has a problem. You may experience intermittent delayed write errors - which makes tracking down the problem even harder.
We recommend replacing any drive that Windows reports a delayed write error on. Any time you receive this error, there is actual data loss on this drive. Avoid the temptation to think, "This is a big drive and other things seems OK with this drive." Beware this message and realize that this message is giving you the opportunity to ACT NOW and avoid even bigger problems in the future, just bite the bullet and replace the drive now.
We received several calls from BFW users letting us know that their SonicWALL Firewalls are reporting that the Backup for Workgroups Setup program contains the FakeSmoke_10 trojan and as a result their firewalls refuse to allow the BFW Setup program to be downloaded. We contacted SonicWALL, which is now owned by Dell, to get this false detect fixed. Backup for Workgroups DOES NOT contain the FakeSmoke_10 trojan. SonicWALL ran a test using their own equipment and stated that the Backup for Workgroups Setup program is clean and does not contain the FakeSmoke_10 trojan. Our developers spoke with SonicWALL to provide information that SonicWALL is using to try to fix this false detection. We will keep on this situation to make sure it gets resolved at SonicWALL.
Be sure to check out how easy it is to restore individual email messages
using the Backup for Workgroups Exchange Message Restore Addon. Scroll the
page down to access the link to the Video - it runs about 6 minutes.
How many businesses have heard this one from their CPAs? We are ready to file your corporate tax return but cannot until 2 new 2012 forms are approved by the IRS. This is proof that the TOO LARGE US Govt could learn a LOT from small business!
Attention all BFW Users! If your installation is using a v6 license key, you can upgrade your installation to Backup for Workgroups v7 for FREE! And, if your installation has older keys, make sure you purchase your upgrade key today - the cost to upgrade is $149 (per Backup Server). For more details - make sure you visit the BFW v7 Upgrade page today.
Today we received a call from a computer consultant who was using BFW to restore data for his customer, an accounting firm. The accounting firm has 350 GB of data stored on a device that failed. BFW restored the data at the clip of 1.2 GB per minute! That's the benefit of local backup - you restore at the speed of your network, which is always faster than the cloud. The accounting firm has their data back in under 5 hours, instead of weeks had they needed to restore from a cloud-based backup vendor.
Concurrent with the new BFW version 7 release, a new Addon is available. The Exchange Message Restore (EMR) Addon works with the version 7 Backup Client to restore individual Exchange email messages without taking your Exchange Server off-line. If you have ever needed to use EXMerge or an Exchange Recovery Group, you know how difficult and time-consuming it is to restore email messages! The new EMR Addon allows BFW version 7 users to restore email messages from a backup of your mailbox store without touching the running Exchange Server. Check out our webpage showing step-by-step screen shots of how easy this process is when you use Backup for Workgroups v7 with the EMR Addon.
What's new in BFW Version 7? Support for Windows 8, Windows Server 2012 and now you can send your backup status reports to email servers that require SSL. Concurrent with the new Version 7 release is the new Exchange Message Restore (EMR) Addon. When you use a BFW Version 7 Backup Client to backup your Exchange Server AND you have an EMR Addon, you can restore individual email messages, all without taking your Exchange Server offline.
We have received a couple of calls/emails indicating that the SonicWall firewall that uses the McAfee Antivirus is reporting that Backup for Workgroups Setup contains a Trojan. THIS IS A FALSE ALERT. McAfee Antivirus has reported the BFW Setup program as containing a Trojan several times in the past - and in every case, McAfee has been WRONG. Backup for Workgroups does not have and has never had a Trojan in its Setup program.
One of our trusted resellers believes that McAfee is making this false alert based on the file name only and not the file contents. If McAfee would actually check the contents of the file, they would find that there is no viruses inside the Backup for Workgroups Setup program.
Please be aware of McAfee's false reporting. Usually, when they make these errors, they will correct their error within a few days.
Backup for Workgroups is designed to co-mingle the backup files from all the computers that you backup into a SINGLE Data Repository. Co-mingling of data means that the data at the Backup Server (Data Repository) is a common repository for which all the Backup Clients store their data into the repository instead of having separate repositories for each Backup Client. The advantage of co-mingled data is that deduplication can occur across all of your Backup Clients. If the data was stored on a per-client basis (not co-mingled), you would not enjoy the benefit of deduplication - which helps you reduce overall storage requirements.
When you install Backup for Workgroups, we assume that the Backup Server and all the Backup Clients are under the control of the same company. In other words, let's say you have 4 computers to backup. Those 4 computers are under your control, and the data residing on those 4 computers is your data and therefore, there is no issue with co-mingling the backup data. This is why we do not recommend that you use Backup for Workgroups for a "service" in which you allow unrelated parties to backup to a single backup server. In this case, each party will have their data "co-mingled" with another, unrelated party's data which may be a surprise to the people backing up to your Backup Server!
We received a question from a current Backup for Workgroups user who had just upgraded to Version 6. He noticed that his backup reports included a section that he had not noticed before. The section he was looking is called Deduplication.
Deduplication is the term used to describe when Backup for Workgroups finds a file that has changed that is identical to a file that has already been backed up – either by this computer or another if you are backing up more than one computer to the Backup Server. In other words, if the same file exists in 2 locations, it is backed up once and “accounted for” multiple times.
In our customer's report, he noticed that his particular backup run had resulted in Zero files being listed as Duplicates. This means that all the new files to backup were unique. Seeing the Deduplication numbers as “zeroes” is common when you perform a daily backup of one computer, because a lot of times, the files that change on a daily basis are unique. Now, if a Windows Update gets applied, or you are collaborating with others you will probably see duplicate file detection, which results in the zeroes changing to the number of duplicate files found in a backup run.
The Block Delta engine was added to Backup for Workgroups version 6. The goal of the block delta engine is to look through the contents of a file and find duplicate portions of the file such that these smaller portions of the file do not need to be re-transmitted to the Backup Server. This process occurs by comparing the file to a previous version of the same file. The block delta engine has user-configurable settings (Tools>Preferences>Block Delta Engine) to allow you to choose which files are candidates for the process of “going through the engine.” Typically, the block engine is inefficient for small files and great for large files. So, for optimal results, the engine should be used on your largest files. This is why you will usually see very few files listed in this section of the Backup Report.
In a sample report that we received from a customer, they had 2 files that ran through the Block Engine. This does not sound like much, but the file size of these particular 2 files is over 6GB. From this user's report, we could see that the Block Engine determined that 99% of the contents of each of these files were identical to the previous backup version, of each file respectively. So, about 1% of each file changed from “backup to backup.” This is pretty common for large database files or Exchange data because you probably do not receive an equal of amount of mail in one day that is equal to all the mail you have received to-date. If you took the amount of mail that you receive in one day and compare that to your sum-total of email that you have received to date, you will find that a very small percent of your mail changes on a daily basis. And this is why the Block Engine is “interesting” – if the backup was going over a slow link (like the Internet) – the backup process only needs to transmit these small changes; as opposed to transmitting the whole file.
Please note that there is a cost of additional time to compute the block delta – which can elongate the time of the backup if the link between the link between the client and the server is very fast. However, compared to a slow link, the time to compute is small compared to the amount of time saved if you had to transmit the whole file. This time difference can sometimes be days saved; which is well-worth the extra time on the front end to run large files through the block engine.
The goal of RAID (Redundant Array of Independent Disks) is to keep your equipment operational, even if you have a hard drive failure. RAID allows you to continue normal operations during hard drive failure. This is attractive when you you need to avoid downtime on a critical computer that needs to be working 24/7. As a result, people put a RAID system in place so they can feel comfortable knowing that their important computer(s) will be accessible all the time, without interruption or surprises.
Some people think, Hey I have RAID, I don't need to backup because I already have redundancy. Here at Lockstep Systems, we know that this is a problem in the making. When you have RAID, it is probably MORE important to backup, because RAID introduces new risks in interruption of system operation and new risks of data loss.
The reality of RAID is that it takes a simple situation and unnecessarily makes it complicated. So think twice before you assume that RAID is going to solve all your problems. Why is RAID a source of the problem and not a solution to the problem? We frequently hear from our users that RAID was the cause of their problem due to:
There are LOTS of people in the world that are not using ANY backup solution - so of course we want to see those people backing up their Windows networks with Backup for Workgroups! And then there is the other end of the spectrum, people who are using 2 backup solutions. Why would someone do this?
Actually, double-backup coverage is a good idea. When you have a critical computer, you may want to consider double-backup. You may want to backup with BFW so you can perform your every-day, run-of-the-mill type restores, in which you just need to restore a file or folder that was accidentally deleted. And, then you may want to use the image backup program that comes with Windows, in which you are making complete images of the computer periodically. This gives you another copy of your entire computer. Of course, we know that it is overkill and time-consuming to restore a full-image just to get a couple of files, and that's one of the BIG limitations of image-based backup. The whole point of backup is give you the ability to restore, but it is no fun to restore selectively from image-based backup. But, you can use both backup strategies and you have lots of bases covered.
Of course you can perform a full computer disaster recovery with BFW, too. Because BFW is file-based, you have the luxury of options, allowing you to pick-and-choose what you want to restore if needed.
Many people use BFW to backup their Exchange Server and they want to know why BFW performs a full backup of the Exchange database each time it performs a backup. Good question.
One reason is a fall-out of how Exchange Stores it's email. Exchange Server stores email in a database called the Extensible Storage Engine. This database follows the practice of first storing new data in log files and then transferring the data into the primary databases which comprise the Message Stores. As a result, all of your email effectively gets duplicated. The first copy is in the log files and the second copy is in the Message Store. Exchange does this to prove that it can store the data before it alters the message stores. So, as all Exchange Administrators are all too familiar with, the Exchange databases get very large, very quickly. Since there is a duplicate copy in the logs, many Admins are anxious to purge the logs as frequently as they can.
When you perform a backup of your Exchange Server with Backup for Workgroups, the backup process backs up the entire Message Store and any logs that are present at the time. At the conclusion of the backup process, BFW informs Exchange Server that it have performed a full backup and that Exchange can now purge it's logs. Exchange will then delete the log files, freeing up disk space. Please note that Exchange will not purge the logs unless that it is told that a FULL backup has been performed.
Another reason why BFW performs a full backup of Exchange (as opposed to a "partial" backup) is because you cannot perform a disaster recovery of your Exchange Server with "piece parts." You have to have the FULL backup in order for Exchange to mount the Stores in a Restore. If you have ever experienced drive failure in your Exchange Server, you will appreciate the FULL backup that BFW performs, because you will get a FULL restore of your Mailbox Stores.
We get calls and emails asking the following, "My Backup for Workgroups trial installation has run out. Do I need to re-install when I purchase my license?" The answer is No. You can purchase a license, even after the 30-day trial period has expired. When you purchase a license, we email you instructions on how to continue using Backup for Workgroups without the need to un-install and re-install. When you enter the license key, Backup for Workgroups returns to normal operation and your backups continue just as you had set them up during the trial period. You can even continue using your backup data.
Let's assume that you downloaded and installed the Backup for Workgroups 30-day trial software on the First of the month, to make this easy. You ran Backup for Workgroups for 30 days, and then the month turns and realize that the evaluation period has expired. No need to panic here. When you buy your license key and enter it into the Backup Server at your location, Backup for Workgroups will resume normal operation and you can continue normal backups and restoration of your backup data. So, if you find that you need to restore a file that you had backed up on the First day of the trial period, you can do that. You can restore data that was backed up during the trial period. There is no need to start over. When you purchase the license and enter it into your Backup Server, Backup for Workgroups "picks up where it left off."
Backup for Workgroups has been shipping for a little under a decade. And we hear a LOT of stories about drive and hardware failures. And we hear a lot of "Thanks to Backup for Workgroups, I got my data back" from our customers.
But we have noticed a trend in the last 60 to 90 days. More people are experiencing hard drive failure and RAID controller failures than ever before. Throughout the years, we would hear about somebody needing to perform a disaster recovery due to hardware RAID failure where the RAID controller shredded the data. In the last couple of months - we have been hearing of this almost daily.
Drive failure is another disaster that we hear about a lot. But the rate has jumped significantly. We estimate that it could be as much as 20-times the rate we have heard in the past. Yikes! Yes, the recent floods in India that took out a lot of the hard drive manufacturing capability of the major drive makers; but the failures that we have been hearing of, the drives pre-date the floods. In other words, these are older machines exhibiting problems.
One possible explanation for this increase in hardware failure could be that computers and equipment that would normally be replaced on a 3 to 5 year schedule are no longer being replaced that often, or they are simply not getting replaced at all. Perhaps more computers are being left in service until they fail.
Another explanation could be an increase in the use of refurbished replacement parts. We have noticed that NewEgg in particular has been advertising refurbished parts more heavily than in the past.
According to information learned at a recent Intel Channel Partner Conference held here in Phoenix, Intel estimates that one-third of the parts returned to Intel are not defective at all, and that some other problem with the computer made the parts from Intel appear defective when they were not. For example, a bad power supply can make you think that you have a bad mother board - so you mistakenly return the mother board to Intel. Now this situation can cause a cascading set of events because the real problem is disguised or overlooked. The bad power supply in the meantime is causing data on the hard-drive to become unreliable. It can also cause RAID controllers to write the wrong data to the drive.
A relatively small problem can linger and lead to BIG problems later on. These small, "low-flying" problems can turn into a situation where your data is corrupt or lost. That's why it is so important to backup your data, your entire computer, your network, etc.
June 29, 2012 - More BFW Restore Speed Statistics
Just restored a data hard drive that was backed up with Backup for Workgroups. 76 GB restored in 55 minutes. That's 1.4 GB per minute!
OK, so the logical question to ask is, "Why did today's restoration go faster than the restore we performed yesterday?"
The answer is: Data restoration speed is variable. It is dependent upon a couple of factors; which can vary from situation to situation. But here's a good rule of thumb: When you restore large files you can achieve a higher throughput than when you restore little, tiny files. When restoring a large file, the restoration process is in a nice groove where all it needs to do is retrieve data, decrypt and decompress and then write the restored data to disk. When restoring a little file, the amount of data is small, but the overhead of the file attributes is larger than the data to restore. For example, let's say you have a small, 1K file. The file attributes on this small file include security settings, file date/time information, the short filename, the long file name, issues in Windows, etc. The attributes on this 1K file may be 2X - 3X the file size, PLUS, to apply these attributes requires multiple Windows functions. In other words, it takes the same amount of processing overhead to restore the attributes of a 1K as it does a 1 GB file. However the 1K file only shows up as restoring 1K of data, which looks whimpy compared to the 1 GB file.
The Windows Operating System consists of tens of thousands of TINY files. So the restoration of a LOT of TINY files takes a lot of processing overhead to restore a relatively small amount of data. When you compare this to an Exchange Server that has a single, multi-gigabyte message store, that HUGE file may restore faster than several thousand itty-bitty files; which makes the throughput computation look unattractive at times.
The lesson learned here is that you have to consider the number of files and the file sizes in the overall restoration situation.
June 28, 2012 - BFW Restore Speed Statistics
We had to restore a Developer's computer today. BFW restored at the rate of 870 MB per minute. It was 101 GB in under 2 hours. Now that's FAST!
This restoration was performed on a relatively new, decent computer, but it is not an extreme-machine. This computer, is a desktop case with a standard Intel motherboard and a Quad-Core 2.8 GHz processor, and a standard, Western Digital SATA 7200 rpm hard drive. OK it is a Western Digital Black Drive, but it is only a 3 GHz, not the newer 6 GHz SATA drives. Nothing extreme about this machine; yet Backup for Workgroups got the machine back to it's operational state in under 2 hours and restored 289,000+ files.
June 27, 2012 - What are the pros and cons of disk-image backup?
OK, this is what Acronis does and the current Windows built-in backup is also considered disk-image backup. Disk image backup tends to be good for backing up and restoring an entire hard drive - in it's entirety. Meaning, the entire hard drive is viewed as ONE THING to backup. Generally, from the disk-image point of view, the drive as a unit is backed up and if a disaster happens, the entire drive is easy to restore. This is fast and easy for whole drive replacement - ASSUMING one BIG THING - that you last disk image is VERY UP TO DATE.
Now, let's focus on the disadvantages of disk image backup software. The number one complaint is it is clunky and difficult to restore files or folders when you do not need an entire hard drive replacement. In other words, disk image software is not a good choice for day-to-day restoration situations that occur all the time when someone needs just a file or an entire folder restored. The other major complaint is that the whole disk has to be backed up each time; and as a result, people have a tendency to run disk image backup less often. When you do this, the backup data is stale, or out-of-date.
We recommend Backup for Workgroups of course to create a blended solution. This gives you lots of options. Go ahead and use the built in Windows backup software - and then supplement it with Backup for Workgroups which is a file by file backup program. The two options work great together and when it comes to data loss - you have options based on the problem at hand. If you lose the hard drive and need to replace the physical drive AND your disk image is relatively up to date, use the disk image; then you can even use Backup for Workgroups to restore files that had changed since the disk image was created. Now that's the best of both worlds. And, if you are in the most typical restore case, which is to restore specific data, BFW presents you with the easiest and fastest way to bring your data back quickly.
June 15, 2012 - Problems with the built-in Windows backup software.
We hear a lot of people's thoughts about the backup software that comes
with Windows. And this is what we learn...
Our users tell us that they like the granularity of a file-by-file backup system because it is easier to restore exactly what they want, instead of restoring an entire disk image. Microsoft probably felt the need for a whole-computer backup, and that's why they switched to a disk-image style of backup. It is also consistent with Microsoft's trend toward virtualization - effectively, the disk-image is a virtual snapshot of your drive. This is great for rapid, whole-computer restore; BUT most restore situations do not require a complete computer restoration.
June 6, 2012 - Are the current security protocols helping spammers.
One of the problems with security is that bad security can sometimes be worse than none at all. In the case of anti spam and anti virus one of the most popular approaches is to require SSL. SSL is a secure socket layer where data can be sent from one computer to another and nobody in between can see this data. SSL was designed to keep Internet transactions secretive. This is great if the data is private data that you do not want exposed such as a credit card number. For example if you purchase an item over the Internet you want to send your credit card data privately to the company you are placing the order with and you do not want anyone else on the Internet to see this information.
Since implementing SSL has been simplified, the tendency has been to throw SSL at everything and think that SSL will solve all your email security issues. This is not so. In fact, SSL can keep your antivirus and antispam programs from working. Once you introduce SSL, the only parties that can look at email are the individual senders and the recipients, which sounds good at first. But keep in mind that if you want to use anti-spam and anti-virus programs, you need to allow for a little eavesdropping. Your anti-virus and antispam services can only work if they can see the email. In the case of SSL, this encryption methods prevents the anti spam and anti virus program seeing the contents of the email and as a result your anti-virus and anti-spam programs cannot determine if the email is infected or not. When the anti virus software cannot see the contents they cannot block the bad email - and as a result, there's a higher probability of bad email getting through to you.
SSL is just part of email security, but it needs to work with other security measures and not alone.
May 24, 2012 - Revised Web Site
Today we posted a new version of our website. We have made a lot of improvements to the site of in the past 9 years, with the goal to provide relevant information about Backup for Workgroups and how it can help you simply your Windows Server Backup and save you money in that process.