Skip to content

Commit 7209543

Browse files
authored
Updated license
1 parent 0f7b746 commit 7209543

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

README.md

+4-1
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ normalized_text = normalize(
2121
apply_unicode_norm_last=True # whether to apply the unicode normalization before or after rule based replacements (default True)
2222
)
2323
```
24+
## License
25+
26+
Contents of this repository are restricted to non-commercial research purposes only under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/).
2427

2528
## Citation
2629

@@ -45,4 +48,4 @@ If you use this module in your work, please cite the following paper:
4548
pages = "2612--2623",
4649
abstract = "Despite being the seventh most widely spoken language in the world, Bengali has received much less attention in machine translation literature due to being low in resources. Most publicly available parallel corpora for Bengali are not large enough; and have rather poor quality, mostly because of incorrect sentence alignments resulting from erroneous sentence segmentation, and also because of a high volume of noise present in them. In this work, we build a customized sentence segmenter for Bengali and propose two novel methods for parallel corpus creation on low-resource setups: aligner ensembling and batch filtering. With the segmenter and the two methods combined, we compile a high-quality Bengali-English parallel corpus comprising of 2.75 million sentence pairs, more than 2 million of which were not available before. Training on neural models, we achieve an improvement of more than 9 BLEU score over previous approaches to Bengali-English machine translation. We also evaluate on a new test set of 1000 pairs made with extensive quality control. We release the segmenter, parallel corpus, and the evaluation set, thus elevating Bengali from its low-resource status. To the best of our knowledge, this is the first ever large scale study on Bengali-English machine translation. We believe our study will pave the way for future research on Bengali-English machine translation as well as other low-resource languages. Our data and code are available at https://github.com/csebuetnlp/banglanmt.",
4750
}
48-
```
51+
```

0 commit comments

Comments
 (0)