public class Deduper extends java.lang.ObjectThe Deduper implements the core deduplication algorithm used by Syncany.
The algorithm uses a
Chunkerto break files into individual
Chunks. These chunks are added to a
MultiChunkusing an implementation of a
MultiChunker. Before this multichunk is written to a file, it is transformed using one or many
Transformers (can be chained).
This class does not maintain a chunk index itself. Instead, it calls a listener to lookup a chunk, and skips further chunk processing if the chunk already exists.
For a detailed description of the algorithm, please refer to chapter 5.3 of the thesis: "Minimizing remote storage usage and synchronization time using deduplication and multichunking: Syncany as an example"
All Methods Instance Methods Concrete Methods Modifier and Type Method Description
deduplicate(java.util.List<java.io.File> files, DeduperListener listener)Deduplicates the given list of files according to the Syncany chunk algorithm.
public void deduplicate(java.util.List<java.io.File> files, DeduperListener listener) throws java.io.IOExceptionDeduplicates the given list of files according to the Syncany chunk algorithm.
A brief description of the algorithm (and further links to a detailed description) are given in the
files- List of files to be deduplicated (will be modified!)
listener- Listener to react of file/chunk/multichunk events, and to implement the chunk index
java.io.IOException- If a file cannot be read or an unexpected exception occurs