Challenge

Backup Copy or Replication job is failing with errors such as, or similar to, the following:
 
Error: Source WAN accelerator error: Failed to decompress LZ4 block: Bad crc
 
Error: Source WAN accelerator error: Failed to decompress LZ4 block: Incorrect decompression result or length
 
Error: Source WAN accelerator error: Client error: Zlib decompression error: [-3].
 

Cause

Cause:    Cyclic Redundancy Check (CRC) failed while extracting data from a backup file indicates that one or more blocks in that file are corrupted.

Typically, this error indicates hardware malfunction during storage or transmission of the backup file. Most types of storage are subject to a small degree of unrecoverable failure – this is commonly called Bit Rot.

In a small sample of cases this has been seen to be caused by corruption in the WAN accelerator Cache itself rather than the backup file.

 

Solution

To isolate, start by configuring the backup copy/replication job to use the direct transport method temporarily disabling WAN acceleration.

If the job works after disabling WAN acceleration, the problem is with the WAN accelerator cache and creating a new volume/rebuilding the WAN accelerator cache may be the next step.
User-added image
If the error persists (typically seen only in a backup copy, not replication) it means the problem most likely exists with backup files not being able to be recompressed. In this case:

2. Run an active full backup on the source job.

The backup file will be completely recreated from source data. Once done, re-run the backup copy job and it will grab the data from this backup file and should bypass the error. If not:

3. Trigger an active full backup on the backup copy job.

Veeam will read the data entirely from the new backup file, bypassing all previous backup files on both the source and the target repository. This will isolate the problem from any previous backup file corruption. Should the error persist at that point:

4. Run a chkdsk or equivalent function on both the source storage, the VM datastore storage, and the target storage for this backup copy job. We’ve recreated, at this point, source and target backup files and ruled out the WAN accelerator components, so as uncommon as it is, it’s likely corruption on one of the storage volumes causing consistency errors.