Add more dataset

Author: yuiseki

datasets/huggingface/fujiki/japanese_hh-rlhf-49k.yaml

@@ -0,0 +1,17 @@
+type: huggingface
+id: fujiki/japanese_alpaca_data
+url: https://huggingface.co/datasets/fujiki/japanese_alpaca_data
+converted_size: 13.8MB
+license: CC-BY-NC-SA-4.0
+lang: ja
+description: This dataset is based on masa3141's great work on japanese-alpaca-lora [github]. Please also refer to this repo.
+structure:
+  - id: instruction
+    type: string
+    description: Instruction
+  - id: input
+    type: string
+    description: Context
+  - id: output
+    type: string
+    description: Output

datasets/huggingface/fujiki/llm-japanese-dataset_snow.yaml

@@ -0,0 +1,17 @@
+type: huggingface
+id: fujiki/llm-japanese-dataset_snow
+url: https://huggingface.co/datasets/fujiki/llm-japanese-dataset_snow
+converted_size: 5.54MB
+license: CC-BY-4.0
+lang: ja
+description: This dataset is a subset of izumi-lab/llm-japanese-dataset only including snow tasks.
+structure:
+  - id: instruction
+    type: string
+    description: Instruction
+  - id: input
+    type: string
+    description: Context
+  - id: output
+    type: string
+    description: Output

datasets/huggingface/if001/oasst1_ja_ppl.yaml

@@ -0,0 +1,17 @@
+type: huggingface
+id: if001/oasst1_ja_ppl
+url: https://huggingface.co/datasets/if001/oasst1_ja_ppl
+converted_size: 27.2MB
+license: Apache-2.0
+lang: ja
+description: kunishou/oasst1-89k-ja のforkです。instructionとinput、outputにまとめ、kenllmでperplexityのスコアが付与してあります。
+structure:
+  - id: instruction
+    type: string
+    description: Instruction
+  - id: input
+    type: string
+    description: Context
+  - id: output
+    type: string
+    description: Output

datasets/huggingface/shunk031/livedoor-news-corpus.yaml

@@ -0,0 +1,7 @@
+type: huggingface
+id: shunk031/livedoor-news-corpus
+url: https://huggingface.co/datasets/shunk031/livedoor-news-corpus
+converted_size: Unknown
+license: CC-BY-ND-4.0
+lang: ja
+description: 本コーパスは、NHN Japan 株式会社が運営する「livedoor ニュース」のうち、下記のクリエイティブ・コモンズライセンスが適用されるニュース記事を収集し、可能な限り HTML タグを取り除いて作成したものです。