The contribution of a yomi-based index to the retrieval effectiveness of a multiple-index system ... more The contribution of a yomi-based index to the retrieval effectiveness of a multiple-index system was analyzed. The results show that the yomi-based index, although insufficient as a standalone index, contributes to a slight increase in retrieval effectiveness, when combined with a bigram-and a wordbased index. However, the weight attributed to the yomi-based index must not be too high.
This paper introduces NTCIR Workshop, a series of evaluation workshops, which are designed to enh... more This paper introduces NTCIR Workshop, a series of evaluation workshops, which are designed to enhance research in information retrieval and related text-processing techniques, such as summarization, question answering and extraction, by providing large-scale test collections and a forum for researchers. A brief history, tasks, participants, test collections, CLIR evaluation at the workshops, and brief overviews at the third NTCIR workshop are described in this paper. To conclude, some thoughts on future directions are suggested.
This paper summarizes parts of the NTCIR-Geo-Time Task held in Tokyo June 15-18, 2010. This task ... more This paper summarizes parts of the NTCIR-Geo-Time Task held in Tokyo June 15-18, 2010. This task was the first evaluation specifically of search with both Geographic and Temporal constraints, i.e. it combines geographic information retrieval (GIR) with time-based search to find specific events in a multilingual collection. We describe the data collections (Japanese and English news stories), topic development, research approaches, assessment results and lessons learned from the evaluation. Median Minimum Maximum GeoTime Topic Mean Average Precision
There has been a long history of work on the application of Machine Translation (MT) to support c... more There has been a long history of work on the application of Machine Translation (MT) to support cross-language information access for patent collections. Much of this work has leveraged fairly traditional information retrieval evaluation designs as a basis for extrinsic (i.e., task-based) evaluation, but other evaluation designs are also possible. This survey reviews the work to date on extrinsic evaluation of patent MT in crosslanguage information access applications, identifying gaps in the literature, and formulating some open research questions.
For the Eighth NTCIR Workshop (NTCIR-8), we selected and organized seven research areas as "tasks... more For the Eighth NTCIR Workshop (NTCIR-8), we selected and organized seven research areas as "tasks" to investigate, test and benchmark newly constructed test collections. These areas are Complex Cross-Lingual Question Answering (CCLQA), Information Retrieval for Question Answering (IR4QA), Geographic and Temporal Search (GeoTime), Multilingual Opinion Analysis (MOAT), Patent Machine Translation (PATMT), Patent Mining (PATMN) and a pilot task for Community Question Answering (CQA). Among these, GeoTime and CQA are new to NTCIR and the others have some new elements. Each task had its own challenge to tackle and both organizers and participating research groups worked hard towards completing them. This paper provides an overview of the Eighth NTCIR Workshop, which is the latest in a series of communityoriented efforts to enhance related research by providing infrastructure for evaluation and testing, as well as a forum for researchers. The paper briefly describes the background, tasks, participants, test collections and other resources available through NTCIR-8. It is intended to serve as an introduction to the research described in detail in the full Proceedings of the Eighth NTCIR Workshop
This paper describes an overview of the first QA Lab (Question Answering Lab for Entrance Exam) t... more This paper describes an overview of the first QA Lab (Question Answering Lab for Entrance Exam) task at NTCIR 11. The goal of the QA lab is to provide a module-based platform for advanced question answering systems and comparative evaluation for solving real-world university entrance exam questions. In this task, "world history" questions are selected from The National Center Test for University Admissions and from the secondary exams at 5 universities in Japan. This paper also describes the used data, baseline systems and formal run results.
This paper introduces the outline of the first NTCIR Workshop, which is the first evaluation work... more This paper introduces the outline of the first NTCIR Workshop, which is the first evaluation workshop designed to enhance research in Japanese text retrieval and cross-lingual information retrieval. The test collection used in the Workshop consists of more than 330,000 documents with more than half are English-Japanese paired. Twenty-three groups from four countries have conducted IR tasks and submitted the search results. Various approaches were tested and reported at the Workshop. Finally some thoughts on the future directions of the NTCIR Workshop and evaluation of cross-lingual information retrieval with Asian languages are suggested.
The purpose of this study is to examine whether there is an effect on the relative evaluation of ... more The purpose of this study is to examine whether there is an effect on the relative evaluation of the IR systems using the relevance judgments made by the pooling method and additional interactive searches. Relevance judgments of NTCIR-1&2 were made using the following steps: (1) collecting candidates for relevant documents by using the pooling method, (2) judging candidate documents by
Proceedings of the ACL-2003 workshop on Patent corpus processing -, 2003
In the Fifth NTCIR Workshop, we organized the Patent Retrieval Task and performed three subtasks;... more In the Fifth NTCIR Workshop, we organized the Patent Retrieval Task and performed three subtasks; Document Retrieval, Passage Retrieval, and Classification. This paper describes the Document Retrieval Subtask and Passage Retrieval Subtask, both of which were intended for patent-to-patent invalidity search task. We show the evaluation results of the groups participating in those subtasks.
The contribution of a yomi-based index to the retrieval effectiveness of a multiple-index system ... more The contribution of a yomi-based index to the retrieval effectiveness of a multiple-index system was analyzed. The results show that the yomi-based index, although insufficient as a standalone index, contributes to a slight increase in retrieval effectiveness, when combined with a bigram-and a wordbased index. However, the weight attributed to the yomi-based index must not be too high.
This paper introduces NTCIR Workshop, a series of evaluation workshops, which are designed to enh... more This paper introduces NTCIR Workshop, a series of evaluation workshops, which are designed to enhance research in information retrieval and related text-processing techniques, such as summarization, question answering and extraction, by providing large-scale test collections and a forum for researchers. A brief history, tasks, participants, test collections, CLIR evaluation at the workshops, and brief overviews at the third NTCIR workshop are described in this paper. To conclude, some thoughts on future directions are suggested.
This paper summarizes parts of the NTCIR-Geo-Time Task held in Tokyo June 15-18, 2010. This task ... more This paper summarizes parts of the NTCIR-Geo-Time Task held in Tokyo June 15-18, 2010. This task was the first evaluation specifically of search with both Geographic and Temporal constraints, i.e. it combines geographic information retrieval (GIR) with time-based search to find specific events in a multilingual collection. We describe the data collections (Japanese and English news stories), topic development, research approaches, assessment results and lessons learned from the evaluation. Median Minimum Maximum GeoTime Topic Mean Average Precision
There has been a long history of work on the application of Machine Translation (MT) to support c... more There has been a long history of work on the application of Machine Translation (MT) to support cross-language information access for patent collections. Much of this work has leveraged fairly traditional information retrieval evaluation designs as a basis for extrinsic (i.e., task-based) evaluation, but other evaluation designs are also possible. This survey reviews the work to date on extrinsic evaluation of patent MT in crosslanguage information access applications, identifying gaps in the literature, and formulating some open research questions.
For the Eighth NTCIR Workshop (NTCIR-8), we selected and organized seven research areas as "tasks... more For the Eighth NTCIR Workshop (NTCIR-8), we selected and organized seven research areas as "tasks" to investigate, test and benchmark newly constructed test collections. These areas are Complex Cross-Lingual Question Answering (CCLQA), Information Retrieval for Question Answering (IR4QA), Geographic and Temporal Search (GeoTime), Multilingual Opinion Analysis (MOAT), Patent Machine Translation (PATMT), Patent Mining (PATMN) and a pilot task for Community Question Answering (CQA). Among these, GeoTime and CQA are new to NTCIR and the others have some new elements. Each task had its own challenge to tackle and both organizers and participating research groups worked hard towards completing them. This paper provides an overview of the Eighth NTCIR Workshop, which is the latest in a series of communityoriented efforts to enhance related research by providing infrastructure for evaluation and testing, as well as a forum for researchers. The paper briefly describes the background, tasks, participants, test collections and other resources available through NTCIR-8. It is intended to serve as an introduction to the research described in detail in the full Proceedings of the Eighth NTCIR Workshop
This paper describes an overview of the first QA Lab (Question Answering Lab for Entrance Exam) t... more This paper describes an overview of the first QA Lab (Question Answering Lab for Entrance Exam) task at NTCIR 11. The goal of the QA lab is to provide a module-based platform for advanced question answering systems and comparative evaluation for solving real-world university entrance exam questions. In this task, "world history" questions are selected from The National Center Test for University Admissions and from the secondary exams at 5 universities in Japan. This paper also describes the used data, baseline systems and formal run results.
This paper introduces the outline of the first NTCIR Workshop, which is the first evaluation work... more This paper introduces the outline of the first NTCIR Workshop, which is the first evaluation workshop designed to enhance research in Japanese text retrieval and cross-lingual information retrieval. The test collection used in the Workshop consists of more than 330,000 documents with more than half are English-Japanese paired. Twenty-three groups from four countries have conducted IR tasks and submitted the search results. Various approaches were tested and reported at the Workshop. Finally some thoughts on the future directions of the NTCIR Workshop and evaluation of cross-lingual information retrieval with Asian languages are suggested.
The purpose of this study is to examine whether there is an effect on the relative evaluation of ... more The purpose of this study is to examine whether there is an effect on the relative evaluation of the IR systems using the relevance judgments made by the pooling method and additional interactive searches. Relevance judgments of NTCIR-1&2 were made using the following steps: (1) collecting candidates for relevant documents by using the pooling method, (2) judging candidate documents by
Proceedings of the ACL-2003 workshop on Patent corpus processing -, 2003
In the Fifth NTCIR Workshop, we organized the Patent Retrieval Task and performed three subtasks;... more In the Fifth NTCIR Workshop, we organized the Patent Retrieval Task and performed three subtasks; Document Retrieval, Passage Retrieval, and Classification. This paper describes the Document Retrieval Subtask and Passage Retrieval Subtask, both of which were intended for patent-to-patent invalidity search task. We show the evaluation results of the groups participating in those subtasks.
Uploads
Papers by noriko kando