Deduper (Syncany API Documentation)

java.lang.Object
- org.syncany.chunk.Deduper

```
public class Deduper
extends java.lang.Object
```
The Deduper implements the core deduplication algorithm used by Syncany.
The algorithm uses a Chunker to break files into individual Chunks. These chunks are added to a MultiChunk using an implementation of a MultiChunker. Before this multichunk is written to a file, it is transformed using one or many Transformers (can be chained).
This class does not maintain a chunk index itself. Instead, it calls a listener to lookup a chunk, and skips further chunk processing if the chunk already exists.
For a detailed description of the algorithm, please refer to chapter 5.3 of the thesis: "Minimizing remote storage usage and synchronization time using deduplication and multichunking: Syncany as an example"

See Also:

Blog post: Minimizing remote storage usage and synchronization time using deduplication and multichunking: Syncany as an example

Constructor Summary

Constructors
Constructor	Description
`Deduper(Chunker chunker, MultiChunker multiChunker, Transformer transformer, long maxTotalSize, long maxNumberOfFiles)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`deduplicate(java.util.List<java.io.File> files, DeduperListener listener)`	Deduplicates the given list of files according to the Syncany chunk algorithm.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - Deduper
```
public Deduper(Chunker chunker,
               MultiChunker multiChunker,
               Transformer transformer,
               long maxTotalSize,
               long maxNumberOfFiles)
```
- Method Detail
  - deduplicate
```
public void deduplicate(java.util.List<java.io.File> files,
                        DeduperListener listener)
                 throws java.io.IOException
```
    Deduplicates the given list of files according to the Syncany chunk algorithm.
    A brief description of the algorithm (and further links to a detailed description) are given in the Deduper.
    
    Parameters:
    
    files - List of files to be deduplicated (will be modified!)
    
    listener - Listener to react of file/chunk/multichunk events, and to implement the chunk index
    
    Throws:
    
    java.io.IOException - If a file cannot be read or an unexpected exception occurs