Increasing availability with Error Correction

Wambo Blog | Thursday, July 27th, 2006 | Trackback

Typical peer-to-peer systems need exactly all the fragments that make up a file for the download to be complete. If one of the fragments is missing for a period of time (peer is offline, etc), then the download cannot be completed until the fragment becomes available again.

Using Error Correction. To solve this problem, Wambo uses advanced error correction techniques. With error correction, files uploaded to the WFS have extra fragments attached for quickly reconstructing the entire file even if one of its fragments is missing. There are two advantages to this technique: speed (no need to wait for exactly all the fragments to have the entire file), and availability (in case of fragment loss the file is still readable).

Optimizing the Error Correction process. Error correcting files, particularly large files, typically requires heavy computation and uses a lot of memory. Wambo uses the most sophisticated methods (Forward Error Correction) and best codes (low-density parity check) to make sure the error correcting and reconstruction processes take very little CPU and memory when running on an individual computer.