Japanese Wikipedia‑based RAG dataset for the LLM course final competition, containing ~1 million text chunks and a search index.
Description in other languages
Statistics
Statistics as of Nov 16, 2026
Hugging Face downloads (last 30 days)
13,000
Hugging Face likes
37
Listed
Nov 3, 2026
Popular in the same category
- JaGovFaqs-22kmatsuxr169A Japanese FAQ dataset extracted by hand from government websites, licensed CC‑BY‑4.0, intended for instruction tuning and RAG testing of large language models with high-quality Q&A pairs and source URLs provided.
- ClimbLab-JaKantaHayashiAI15kClimbLab‑Ja is a 300 billion‑token Japanese corpus, filtered from LLM‑jp Corpus v4 into 20 clusters for pre‑training language models, released under ODC‑BY with original source licenses and stored in parquet format.
- JMTEBsbintuitions3kJMTEB is a Japanese text‑embedding benchmark featuring 5 tasks (clustering, classification, STS, retrieval, reranking) and 28 datasets, offering a one‑line evaluation script and inviting community contributions.
- JMedBenchColdog23331kJMedBench is a Japanese biomedical LLM benchmark comprising 20 datasets across five tasks (MCQA, NER, STS, etc.) sourced from MedMCQA, PubMedQA, MMLU, and others, each with its own license, and includes a note that translations may contain biases requiring human review.
- vntl-leaderboardlmg-anon857A leaderboard that ranks large language models by their semantic accuracy in translating Japanese visual novels into English, using cosine similarity over 256 samples and reporting chrF scores for comparison with tools like Sugoi Translator, Google Translate, Naver Papago, and Alibaba Translate.
- auto-wiki-qacl-nagoya747AutoWikiQA is the largest free Japanese QA dataset, providing over 2.3 million handcrafted‑filtered question–answer pairs automatically generated from Wikipedia with Swallow‑MX and LLMs (without rule‑based templates) to support knowledge‑teaching and retrieval‑augmented generation applications.
Resources in similar research fields
- AnswerCarefullyllm-jp644AnswerCarefully Dataset supplies Japanese and multilingual data for commercial or non‑commercial LLM safety enhancement, prohibits any other use—including safety circumvention—allows derivative works with attribution, and carries a creator disclaimer of non‑liability for harms or service changes.Shared: Large language models・Corpora & datasets
- llm-japanese-datasetizumi-lab572Japanese instruction‑chat dataset for fine‑tuning LLMs (e.g., with LoRA), 9 M+ samples, recently updated to drop licensed Alpaca data, clean Wikipedia and ALT outputs, and released under CC‑BY‑SA 4.0.Shared: Large language models・Corpora & datasets
- oasst2-33k-jallm-jp551LLM‑jp provides a Japanese instruction‑tuning dataset, DeepL‑translated from an English subset of oasst2 (derived from kunishou/oasst2‑135k‑ja) and compiled by Kiyomaru and Kodama.Shared: Large language models・Corpora & datasets
- JGLUEllm-book546Dataset card for the JGLUE dataset used in the book “Large Language Model Introduction,” sourced from the original repo, with code licensed CC BY‑SA 4.0, data under the distributor’s license, citing Kurihara & Kawahara (in Japanese), and built on Shunsuke Kitada’s repository.Shared: Large language models・Corpora & datasets
ja.text8Hironsan113Japanese text8 corpus for word embedding.GitHubShared: Embeddings, search & RAG・Corpora & datasets
JapaneseWordSimilarityDatasettmu-nlp103Japanese Word Similarity DatasetGitHubShared: Embeddings, search & RAG・Corpora & datasets