Skip to content

Commit b6526e6

Browse files
authored
Update README.md
1 parent a4ae6d1 commit b6526e6

File tree

1 file changed

+21
-11
lines changed

1 file changed

+21
-11
lines changed

README.md

Lines changed: 21 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,37 @@
1-
# parallpairs
1+
# Parallel all-pairs similarity search algorithms in OCaml
22

3-
## Parallel all-pairs similarity search algorithms in OCaml
3+
##Sources
4+
5+
The repository contains the 1.0 sources, a release will be made soon.
6+
7+
##Citation
48

59
If you use this code, please cite the following paper. It is currently under review at IJPP.
610

711
https://arxiv.org/abs/1402.3010
812

9-
You may cite it as:
13+
Title: 1-D and 2-D Parallel Algorithms for All-Pairs Similarity Problem
1014

11-
Eray Özkural, Cevdet Aykanat: 1-D and 2-D Parallel Algorithms for All-Pairs Similarity Problem. CoRR abs/1402.3010 (2014)
12-
http://dblp.org/rec/html/journals/corr/OzkuralA14
15+
Authors: Eray Özkural, Cevdet Aykanat (Submitted on 13 Feb 2014)
1316

14-
1-D and 2-D Parallel Algorithms for All-Pairs Similarity Problem
17+
Abstract: All-pairs similarity problem asks to find all vector pairs in a set of vectors the similarities of which surpass a given similarity threshold, and it is a computational kernel in data mining and information retrieval for several tasks. We investigate the parallelization of a recent fast sequential algorithm. We propose effective 1-D and 2-D data distribution strategies that preserve the essential optimizations in the fast algorithm. 1-D parallel algorithms distribute either dimensions or vectors, whereas the 2-D parallel algorithm distributes data both ways. Additional contributions to the 1-D vertical distribution include a local pruning strategy to reduce the number of candidates, a recursive pruning algorithm, and block processing to reduce imbalance. The parallel algorithms were programmed in OCaml which affords much convenience. Our experiments indicate that the performance depends on the dataset, therefore a variety of parallelizations is useful.
1518

16-
Eray Özkural, Cevdet Aykanat
17-
(Submitted on 13 Feb 2014)
19+
The paper is included in the sources.
1820

19-
All-pairs similarity problem asks to find all vector pairs in a set of vectors the similarities of which surpass a given similarity threshold, and it is a computational kernel in data mining and information retrieval for several tasks. We investigate the parallelization of a recent fast sequential algorithm. We propose effective 1-D and 2-D data distribution strategies that preserve the essential optimizations in the fast algorithm. 1-D parallel algorithms distribute either dimensions or vectors, whereas the 2-D parallel algorithm distributes data both ways. Additional contributions to the 1-D vertical distribution include a local pruning strategy to reduce the number of candidates, a recursive pruning algorithm, and block processing to reduce imbalance. The parallel algorithms were programmed in OCaml which affords much convenience. Our experiments indicate that the performance depends on the dataset, therefore a variety of parallelizations is useful.
21+
##Comments
2022

23+
The code is quite interesting, as it shows how to effectively use OCaml for MPI code. There is a bunch of well-written parallel functional code that I will extract from this codebase and release separately. You need the latest ocamlmpi release as that contains the patches I made to make this code work.
2124

22-
The code is quite interesting, as it shows how to effectively use OCaml for MPI programming. There is a bunch of well-written parallel functional code that I will extract from this codebase and release separately. You need the latest ocamlmpi release as that contains the patches I made to make this code work.
25+
##Datasets
2326

2427
You can download the datasets from: https://github.com/examachine/data/tree/master/dv
2528

29+
##License
30+
31+
This code is released under AGPL-3.0. Please do not ask me to release it under BSD license. If you need a commercial license, you should purchase it.
32+
33+
Happy hacking!
34+
2635

27-
Eray Ozkural, PhD
36+
Eray Özkural, PhD
37+
Founder, Gök Us Sibernetik Ar&Ge Ltd. Şti.

0 commit comments

Comments
 (0)