Skip to main content

Daqing He

University of Pittsburgh, School of Information Sciences, Faculty Member

Followers

39

Following

12

Co-authors

11

Mentions

1

Public Views

I am a Professor at the School of Computing and Information (iSchool) , the University of Pittsburgh. I also have academic affiliation with the Intelligent Systems Program, an interdisciplinary academic unit focusing on research and education on Artificial Intelligence. I am the director of a research lab called Information Retrieval, Integration and Synthesis. I was the chair of SIS Council, the governing body for the iSchool, between 2012 and 2014. I currently serve on the editorial board of SCI/SSCI indexed journals “Information Processing and Management”, “Internet Research” and “Aslib Journal of Information Management”.

less

InterestsView All (10)

Uploads

Papers by Daqing He

The Quality and helpfulness of answers to eating disorder questions in <i>yahoo! answers</i> : Teens speak out

Proceedings of the Association for Information Science and Technology, 2015

This research project investigated teens' perspectives on the quality and helpfulness of health i... more This research project investigated teens' perspectives on the quality and helpfulness of health information about eating disorders found on Yahoo! Answers, a Social Q&A site. A mixed methods approach was applied, using survey methods and semi-structured group interviews to gather data for the project. Eighteen teens completed a web-based questionnaire using sample question/answer sets about eating disorders from Yahoo! Answers. The teen participants were asked to choose one answer as "best" and then rank its credibility, accuracy, reliability, and helpfulness. Open-ended questions allowed teens to explain the rationale for their choice of "best" answer and to discuss why the chosen answer might (or might not) be helpful for teens. Following the questionnaire, six teens participated in a focus group interview using a semi-structured format that asked open-ended "why" questions in order to draw forth comments on criteria for evaluating the quality and and helpfulness of health information in Yahoo! Answers, as well as to reveal aspects of critical thinking. Findings suggest that, 1) teens make a distinction between health information in Social Q&A that is credible versus that which is helpful, 2) they value health information that isn't from a credible source if it addresses other needs, and, 3) when making judgments about health information on the Web, they apply an array of heuristics related to information quality, opinion, communication style, emotional support and encouragement, guidance, personal experience, and professional expertise.

Best Versus Helpful Health Information: Teens’ Assessments of the Answers to Eating Disorders Questions in Yahoo! Answers

Knowledge Acquisition and Social Support in an Online Health Community for Ovarian Cancer (Preprint)

BACKGROUND Patients and caregivers have been widely using Online Health Communities (OHCs) to acq... more BACKGROUND Patients and caregivers have been widely using Online Health Communities (OHCs) to acquire knowledge from peers. Questions posed in OHCs reflect the participant's learning objectives and are known to differ in the level of cognitive complexity. However, little is known about the topics and levels of participant learning objectives and the corresponding support they receive from other members of the OHC. OBJECTIVE This study aims to investigate patients' and caregivers’ knowledge acquisition in an OHC. Specifically, we investigate: 1) the distribution and topics of posts with learning objectives at different cognitive complexity levels; 2) the type and amount of social support provided to meet users’ learning objectives at different cognitive complexity levels; and 3) the influence of social support on the change in learning objectives. METHODS We collected 10-years of discussion threads from one of the most active OHCs for ovarian cancer (OvCa). A mixed-method app...

Knowledge Annotation for Intelligent Textbooks

Technology, Knowledge and Learning, 2021

With the increased popularity of electronic textbooks, there is a growing interests in developing... more With the increased popularity of electronic textbooks, there is a growing interests in developing a new generation of "intelligent textbooks", which have the ability to guide the readers according to their learning goals and current knowledge. The intelligent textbooks extend regular textbooks by integrating machine-manipulatable knowledge such as a knowledge map or a prerequisite-outcome relationship between sections, among which, the most popular integrated knowledge is a list of unique knowledge concepts associated with each section. With the help of this concept, multiple intelligent operations, such as content linking, content recommendation or student modeling, can be performed. However, annotating a reliable set of concepts to a textbook section is a challenge. Automatic unsupervised methods for extracting key-phrases as the concepts are known to have insufficient accuracy. Manual annotation by experts is considered as a preferred approach and can be used to produce both the target outcome and the labeled data for training supervised models. However, most researchers in education domain still consider the concept annotation process as an ad-hoc activity rather than an engineering task, resulting in low-quality annotated data. In this paper, we present a textbook knowledge engineering method to obtain reliable concept annotations. The outcomes of our work include a validated knowledge engineering procedure, a code-book for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as gold standard in further research.

Exploring Resource-Sharing Behaviors for Finding Relevant Health Resources: Analysis of an Online Ovarian Cancer Community

JMIR Cancer, 2021

Background Online health communities (OHCs) provide patients and survivors of ovarian cancer (OvC... more Background Online health communities (OHCs) provide patients and survivors of ovarian cancer (OvCa) and their caregivers with help beyond traditional support channels, such as health care providers and clinicians. OvCa OHCs promote connections and exchanges of information among users with similar experiences. Users often exchange information, which leads to the sharing of resources in the form of web links. Although OHCs are important platforms for health management, concerns exist regarding the quality and relevance of shared resources. Previous studies have examined different aspects of resource-sharing behaviors, such as the purpose of sharing, the type of shared resources, and peer user reactions to shared resources in OHCs to evaluate resource exchange scenarios. However, there is a paucity of research examining whether resource-sharing behaviors can ultimately determine the relevance of shared resources. Objective This study aimed to examine the association between OHC resourc...

Introduction to Social Information Access

Social Information Access, 2018

This chapter offers an introduction to the emerging field of social information access. Social in... more This chapter offers an introduction to the emerging field of social information access. Social information access focuses on technologies that organize users past interaction with information in order to provide future users with better access to information. These technologies have become increasingly more popular in all areas of information access, including search, browsing, and recommendation. Starting with a definition of the new field and a brief history of social information access, this chapter introduces a multi-aspect classification of social information access technologies. The two important factors for our classification are the types of information access involved and the source of the social information that has been leveraged to support information access. These two factors are the angles we use in this chapter to create a map of the field, as well as to introduce the book structure and the role of the remaining book chapters in covering social information access topics and technologies.

Confronting the Challenges of Computational and Social Perspectives of the Data Continuum

Data and Information Management, 2020

As the availability of data is increasing everyday, the need to reflect on how to make these data... more As the availability of data is increasing everyday, the need to reflect on how to make these data meaningful and impactful becomes vital. Current data paradigms have provided data life cycles that often focus on data acumen and data stewardship approaches. In an effort to examine the convergence, tensions, and harmonies of these two approaches, a group of researchers participated in an interactive panel session at the Association of Information Science and Technology Annual meeting in 2019. The panel presenters described their various research activities in which they confront the challenges of the computational and social perspectives of the data continuum. This paper provides a summary of this interactive panel.

Answer Quality Characteristics and Prediction on an Academic Q&A Site

Proceedings of the 24th International Conference on World Wide Web, 2015

Despite various studies on examining and predicting answer quality on generic social Q&A sites su... more Despite various studies on examining and predicting answer quality on generic social Q&A sites such as Yahoo! Answers, little is known about why answers on academic Q&A sites are voted on by scholars who follow the discussion threads to be high quality answers. Using 1021 answers obtained from the Q&A part of an academic social network site ResearchGate (RG), we firstly explored whether various web-captured features and human-coded features can be the critical factors that influence the peer-judged answer quality. Then using the identified critical features, we constructed three classification models to predict the peer-judged rating. Our results identify four main findings. Firstly, responders' authority, shorter response time and greater answer length are the critical features that positively associate with the peer-judged answer quality. Secondly, answers containing social elements are very likely to harm the peer-judged answer quality. Thirdly, an optimized SVM algorithm has an overwhelming advantage over other models in terms of accuracy. Finally, the prediction based on web-captured features had better performance when comparing to prediction on human-coded features. We hope that these interesting insights on ResearchGate's answer quality can help the further design of academic Q&A sites.

Challenges and Supports for Accessing Open Government Datasets

Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, 2019

The importance of open government data is often associated with increased public trust, civic eng... more The importance of open government data is often associated with increased public trust, civic engagement, and accountable administrations. While there is a myriad of benefits, the existing literature suggests that many open government datasets lack accessibility and usability for diverse users. This study seeks to explore what contextual information users require when they access these datasets. Using mixed methods, we aim to discover the challenges of accessing data, and the necessary contextual information needed by the users to overcome these challenges. As the outcome of this study, we propose a framework called "Data Guides", which is composed of the identified important contextual information. In future work, we will test the effectiveness of the Data Guide in aiding users' accessing and understanding open government data.

Student Modeling with Automatic Knowledge Component Extraction for Adaptive Textbooks

Online textbooks have become a significant component of online and blended learning environments.... more Online textbooks have become a significant component of online and blended learning environments. Taking this medium one step further, Adaptive online Textbooks (AoT) recommend the most relevant pages and practice activities based on students current knowledge state. AoT use student interaction data to infer the current state of student knowledge through student modeling (SM). The knowledge is inferred on knowledge components (KCs) associated with textbook material (sections/pages, practice activities, and quizzes). However, most of these techniques rely on expert annotated knowledge components. A challenge of student modeling in the context of adaptive textbooks is that traditional student models are constructed based on performance data (question answers or problem solving) Student interaction with online textbooks, however, produces large volume of student reading data, but a very limited amount of question-answering data. This leads to the requirement of annotating reading mater...

Dynamic Knowledge Modeling with Heterogeneous Activities for Adaptive Textbooks

Adaptive textbooks use student interaction data to infer the current state of student knowledge a... more Adaptive textbooks use student interaction data to infer the current state of student knowledge and recommend most relevant learning materials. A challenge of student mod- eling for adaptive textbooks is that conventional student models are constructed based on performance data (quiz or problem-solving), however, students' interactions with on- line textbooks may produce a large volume of student read- ing data but a limited amount of performance data. In this work, we propose a dynamic student knowledge modeling framework for online adaptive textbooks, which utilizes stu- dent reading data combined with few available quiz activi- ties to infer the students' current state of knowledge. The evaluation shows that proposed model learns more accurate students' knowledge state than Knowledge Tracing.

Concept Annotation for Intelligent Textbooks

ArXiv, 2020

With the increased popularity of electronic textbooks, there is a growing interests in developing... more With the increased popularity of electronic textbooks, there is a growing interests in developing a new generation of "intelligent textbooks", which have the ability to guide the readers according to their learning goals and current knowledge. The intelligent textbooks extend regular textbooks by integrating machine-manipulatable knowledge such as a knowledge map or a prerequisite-outcome relationship between sections, among which, the most popular integrated knowledge is a list of unique knowledge concepts associated with each section. With the help of this concept, multiple intelligent operations, such as content linking, content recommendation or student modeling, can be performed. However, annotating a reliable set of concepts to a textbook section is a challenge. Automatic unsupervised methods for extracting key-phrases as the concepts are known to have insufficient accuracy. Manual annotation by experts is considered as a preferred approach and can be used to produce...

Recommending Remedial Readings Using Student's Knowledge state

Assessment plays a vital role in learning, as it provides both instructors and students with feed... more Assessment plays a vital role in learning, as it provides both instructors and students with feedback on the overall effectiveness of their teaching or learning. However, when a student fails to correctly answer certain questions in an assessment (such as a quiz), the student needs specific recommendations that are tailored to their learning needs and to the knowledge deficiency exposed by the assessment outcomes. In this paper, we explore the methods for automatically identifying the recommended textbook materials that are most relevant and suitable to the student. In particular, we conducted experiments on how to incorporate students’ current knowledge state on domain concepts associated with the activity to recommend personalized remedial sections to each student. The results show that incorporating student knowledge states can significantly improve the quality of recommendations as compared to traditional content-based recommendations.

Is the Article Crucial to My Research? Evaluating Task-Oriented Impacts of Scientific Articles in Information Seeking

We propose a new aspect of evaluating scientific articles: crucialness, which refers to the state... more We propose a new aspect of evaluating scientific articles: crucialness, which refers to the state of articles being not only useful, but also scarce (difficult to be found or identified by scientists). Compared with the popularity-based metrics, crucialness may be a better metric in helping scientists’ information seeking and use because it identifies scientists’ difficulties in information seeking and reveals the crucial articles that may help scientists succeed in research. Some preliminary results are presented and discussed.

Semi-Supervised Techniques for Mining Learning Outcomes and Prerequisites

Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017

Educational content of today no longer only resides in textbooks and classrooms; more and more le... more Educational content of today no longer only resides in textbooks and classrooms; more and more learning material is found in a free, accessible form on the Internet. Our long-standing vision is to transform this web of educational content into an adaptive, webscale "textbook", that can guide its readers to most relevant "pages" according to their learning goal and current knowledge. In this paper, we address one core, long-standing problem towards this goal: identifying outcome and prerequisite concepts within a piece of educational content (e.g., a tutorial). Speci cally, we propose a novel approach that leverages textbooks as a source of distant supervision, but learns a model that can generalize to arbitrary documents (such as those on the web). As such, our model can take advantage of any existing textbook, without requiring expert annotation. At the task of predicting outcome and prerequisite concepts, we demonstrate improvements over a number of baselines on six textbooks, especially in the regime of li le to no ground-truth labels available 1. Finally, we demonstrate the utility of a model learned using our approach at the task of identifying prerequisite documents for adaptive content recommendation-an important step towards our vision of the "web as a textbook".

Supporting Cross-Device Web Search with Social Navigation-Based Mobile Touch Interactions

Lecture Notes in Computer Science, 2015

The wide adoption of smartphones eliminates the time and location barriers for people's daily inf... more The wide adoption of smartphones eliminates the time and location barriers for people's daily information access, but also limits users' information exploration activities due to the small mobile screen size. Thus, cross-device web search, where people initialize information needs on one device but complete them on another device, is frequently observed in modern search engines, especially for exploratory information needs. This paper aims to support the cross-device web search, on top of the commonly used context-sensitive retrieval framework, for exploratory tasks. To better model users' search context, our method not only utilizes the search history (query history and click-through) but also employs the mobile touch interactions (MTI) on mobile devices. To be more specific, we combine MTI's ability of locating relevant subdocument content [10] with the idea of social navigation that aggregates MTIs from other users who visit the same page. To demonstrate the effectiveness of our proposed approach, we designed a user study to collect cross-device web search logs on three different types of tasks from 24 participants and then compared our approach with two baselines: a traditional full text based relevance feedback approach and a self-MTI based subdocument relevance feedback approach. Our results show that the social navigation-based MTIs outperformed both baselines. A further analysis shows that the performance improvements are related to several factors, including the quality and quantity of click-through documents, task types and users' search conditions.

Automatic Concept Extraction for Domain and Student Modeling in Adaptive Textbooks

International Journal of Artificial Intelligence in Education, 2020

The increasing popularity of digital textbooks as a new learning media has resulted in a growing ... more The increasing popularity of digital textbooks as a new learning media has resulted in a growing interest in developing a new generation of adaptive textbooks that can help readers to learn better through adapting to the readers' learning goals and the current state of knowledge. These adaptive textbooks are most frequently powered by internal knowledge models, which associate a list of unique domain knowledge concepts with each section of the textbook. With this kind of concept-level knowledge representation, a number of intelligent operations could be performed, which include student modeling, adaptive navigation support, and content recommendation. However, manual indexing of each textbook section with concepts is challenging, time-consuming, and prone to errors. Modern research in the area of natural language processing offers an attractive alternative, called automatic keyphrase extraction. While a range of keyphrase and concept extraction methods have been developed over the last twenty years, few of the known approaches were applied and evaluated in a textbook context. In this paper, we present FACE, a supervised feature-based machine learning method for automatic concept extractions from digital textbooks. This method has been created for building domain and student models that form the core of intelligent textbooks. We evaluated FACE on a newly constructed full-scale dataset by assessing how well it approximates concept annotations produced by human experts and how well it supports the needs of student modeling. The results show that FACE outperforms several state-of-the-art keyphrase extraction methods.

One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Different texts shall by nature correspond to different number of keyphrases. This desideratum is... more Different texts shall by nature correspond to different number of keyphrases. This desideratum is largely missing from existing neural keyphrase generation models. In this study, we address this problem from both modeling and evaluation perspectives. We first propose a recurrent generative model that generates multiple keyphrases as delimiter-separated sequences. Generation diversity is further enhanced with two novel techniques by manipulating decoder hidden states. In contrast to previous approaches, our model is capable of generating diverse keyphrases and controlling number of outputs. We further propose two evaluation metrics tailored towards the variable-number generation. We also introduce a new dataset (ST A C KEX) that expands beyond the only existing genre (i.e., academic writing) in keyphrase generation tasks. With both previous and new evaluation metrics, our model outperforms strong baselines on all datasets.

The convergence of computational and social approaches for unveiling meaningful and valuable data

Proceedings of the Association for Information Science and Technology, 2019

The current data paradigm is seeking a more integrated and comprehensive framework to make sense ... more The current data paradigm is seeking a more integrated and comprehensive framework to make sense of data and its derived issues. From the perspective of the data life cycle, we argue that computational and social approaches complement each other to confront data challenges. Computational approaches consist of ETL (extract, transform, and load), modeling, and machine learning techniques; social approaches include policy and regulations, data sharing and reuse behavior, reproducibility, ethical and privacy issues. In this panel, we frame these two approaches as data acumen and data stewardship. The merging of these two perspectives allows data not only to become discoverable, accessible, and interoperable, but also to further the value of revealing meaningful patterns and become supportive evidence for important decision making. In this panel, the opening facilitator and three panelists will report on their recent studies in terms of this convergence of both data acumen and stewardship while sharing their recent research insights on case studies in three disciplines: agriculture, biomedicine, and archeology.

Deep Keyphrase Generation

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017

Keyphrase provides highly-summative information that can be effectively used for understanding, o... more Keyphrase provides highly-summative information that can be effectively used for understanding, organizing and retrieving text content. Though previous studies have provided many workable solutions for automated keyphrase extraction, they commonly divided the to-be-summarized content into multiple text chunks, then ranked and selected the most meaningful ones. These approaches could neither identify keyphrases that do not appear in the text, nor capture the real semantic meaning behind the text. We propose a generative model for keyphrase prediction with an encoder-decoder framework, which can effectively overcome the above drawbacks. We name it as deep keyphrase generation since it attempts to capture the deep semantic meaning of the content with a deep learning method. Empirical analysis on six datasets demonstrates that our proposed model not only achieves a significant performance boost on extracting keyphrases that appear in the source text, but also can generate absent keyphrases based on the semantic meaning of the text. Code and dataset are available at https://github.com/memray/seq2seqkeyphrase.

The Quality and helpfulness of answers to eating disorder questions in <i>yahoo! answers</i> : Teens speak out

Proceedings of the Association for Information Science and Technology, 2015

This research project investigated teens' perspectives on the quality and helpfulness of health i... more This research project investigated teens' perspectives on the quality and helpfulness of health information about eating disorders found on Yahoo! Answers, a Social Q&A site. A mixed methods approach was applied, using survey methods and semi-structured group interviews to gather data for the project. Eighteen teens completed a web-based questionnaire using sample question/answer sets about eating disorders from Yahoo! Answers. The teen participants were asked to choose one answer as "best" and then rank its credibility, accuracy, reliability, and helpfulness. Open-ended questions allowed teens to explain the rationale for their choice of "best" answer and to discuss why the chosen answer might (or might not) be helpful for teens. Following the questionnaire, six teens participated in a focus group interview using a semi-structured format that asked open-ended "why" questions in order to draw forth comments on criteria for evaluating the quality and and helpfulness of health information in Yahoo! Answers, as well as to reveal aspects of critical thinking. Findings suggest that, 1) teens make a distinction between health information in Social Q&A that is credible versus that which is helpful, 2) they value health information that isn't from a credible source if it addresses other needs, and, 3) when making judgments about health information on the Web, they apply an array of heuristics related to information quality, opinion, communication style, emotional support and encouragement, guidance, personal experience, and professional expertise.

Best Versus Helpful Health Information: Teens’ Assessments of the Answers to Eating Disorders Questions in Yahoo! Answers

Knowledge Acquisition and Social Support in an Online Health Community for Ovarian Cancer (Preprint)

BACKGROUND Patients and caregivers have been widely using Online Health Communities (OHCs) to acq... more BACKGROUND Patients and caregivers have been widely using Online Health Communities (OHCs) to acquire knowledge from peers. Questions posed in OHCs reflect the participant's learning objectives and are known to differ in the level of cognitive complexity. However, little is known about the topics and levels of participant learning objectives and the corresponding support they receive from other members of the OHC. OBJECTIVE This study aims to investigate patients' and caregivers’ knowledge acquisition in an OHC. Specifically, we investigate: 1) the distribution and topics of posts with learning objectives at different cognitive complexity levels; 2) the type and amount of social support provided to meet users’ learning objectives at different cognitive complexity levels; and 3) the influence of social support on the change in learning objectives. METHODS We collected 10-years of discussion threads from one of the most active OHCs for ovarian cancer (OvCa). A mixed-method app...

Knowledge Annotation for Intelligent Textbooks

Technology, Knowledge and Learning, 2021

With the increased popularity of electronic textbooks, there is a growing interests in developing... more With the increased popularity of electronic textbooks, there is a growing interests in developing a new generation of "intelligent textbooks", which have the ability to guide the readers according to their learning goals and current knowledge. The intelligent textbooks extend regular textbooks by integrating machine-manipulatable knowledge such as a knowledge map or a prerequisite-outcome relationship between sections, among which, the most popular integrated knowledge is a list of unique knowledge concepts associated with each section. With the help of this concept, multiple intelligent operations, such as content linking, content recommendation or student modeling, can be performed. However, annotating a reliable set of concepts to a textbook section is a challenge. Automatic unsupervised methods for extracting key-phrases as the concepts are known to have insufficient accuracy. Manual annotation by experts is considered as a preferred approach and can be used to produce both the target outcome and the labeled data for training supervised models. However, most researchers in education domain still consider the concept annotation process as an ad-hoc activity rather than an engineering task, resulting in low-quality annotated data. In this paper, we present a textbook knowledge engineering method to obtain reliable concept annotations. The outcomes of our work include a validated knowledge engineering procedure, a code-book for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as gold standard in further research.

Exploring Resource-Sharing Behaviors for Finding Relevant Health Resources: Analysis of an Online Ovarian Cancer Community

JMIR Cancer, 2021

Background Online health communities (OHCs) provide patients and survivors of ovarian cancer (OvC... more Background Online health communities (OHCs) provide patients and survivors of ovarian cancer (OvCa) and their caregivers with help beyond traditional support channels, such as health care providers and clinicians. OvCa OHCs promote connections and exchanges of information among users with similar experiences. Users often exchange information, which leads to the sharing of resources in the form of web links. Although OHCs are important platforms for health management, concerns exist regarding the quality and relevance of shared resources. Previous studies have examined different aspects of resource-sharing behaviors, such as the purpose of sharing, the type of shared resources, and peer user reactions to shared resources in OHCs to evaluate resource exchange scenarios. However, there is a paucity of research examining whether resource-sharing behaviors can ultimately determine the relevance of shared resources. Objective This study aimed to examine the association between OHC resourc...

Introduction to Social Information Access

Social Information Access, 2018

This chapter offers an introduction to the emerging field of social information access. Social in... more This chapter offers an introduction to the emerging field of social information access. Social information access focuses on technologies that organize users past interaction with information in order to provide future users with better access to information. These technologies have become increasingly more popular in all areas of information access, including search, browsing, and recommendation. Starting with a definition of the new field and a brief history of social information access, this chapter introduces a multi-aspect classification of social information access technologies. The two important factors for our classification are the types of information access involved and the source of the social information that has been leveraged to support information access. These two factors are the angles we use in this chapter to create a map of the field, as well as to introduce the book structure and the role of the remaining book chapters in covering social information access topics and technologies.

Confronting the Challenges of Computational and Social Perspectives of the Data Continuum

Data and Information Management, 2020

As the availability of data is increasing everyday, the need to reflect on how to make these data... more As the availability of data is increasing everyday, the need to reflect on how to make these data meaningful and impactful becomes vital. Current data paradigms have provided data life cycles that often focus on data acumen and data stewardship approaches. In an effort to examine the convergence, tensions, and harmonies of these two approaches, a group of researchers participated in an interactive panel session at the Association of Information Science and Technology Annual meeting in 2019. The panel presenters described their various research activities in which they confront the challenges of the computational and social perspectives of the data continuum. This paper provides a summary of this interactive panel.

Answer Quality Characteristics and Prediction on an Academic Q&A Site

Proceedings of the 24th International Conference on World Wide Web, 2015

Despite various studies on examining and predicting answer quality on generic social Q&A sites su... more Despite various studies on examining and predicting answer quality on generic social Q&A sites such as Yahoo! Answers, little is known about why answers on academic Q&A sites are voted on by scholars who follow the discussion threads to be high quality answers. Using 1021 answers obtained from the Q&A part of an academic social network site ResearchGate (RG), we firstly explored whether various web-captured features and human-coded features can be the critical factors that influence the peer-judged answer quality. Then using the identified critical features, we constructed three classification models to predict the peer-judged rating. Our results identify four main findings. Firstly, responders' authority, shorter response time and greater answer length are the critical features that positively associate with the peer-judged answer quality. Secondly, answers containing social elements are very likely to harm the peer-judged answer quality. Thirdly, an optimized SVM algorithm has an overwhelming advantage over other models in terms of accuracy. Finally, the prediction based on web-captured features had better performance when comparing to prediction on human-coded features. We hope that these interesting insights on ResearchGate's answer quality can help the further design of academic Q&A sites.

Challenges and Supports for Accessing Open Government Datasets

Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, 2019

The importance of open government data is often associated with increased public trust, civic eng... more The importance of open government data is often associated with increased public trust, civic engagement, and accountable administrations. While there is a myriad of benefits, the existing literature suggests that many open government datasets lack accessibility and usability for diverse users. This study seeks to explore what contextual information users require when they access these datasets. Using mixed methods, we aim to discover the challenges of accessing data, and the necessary contextual information needed by the users to overcome these challenges. As the outcome of this study, we propose a framework called "Data Guides", which is composed of the identified important contextual information. In future work, we will test the effectiveness of the Data Guide in aiding users' accessing and understanding open government data.

Student Modeling with Automatic Knowledge Component Extraction for Adaptive Textbooks

Online textbooks have become a significant component of online and blended learning environments.... more Online textbooks have become a significant component of online and blended learning environments. Taking this medium one step further, Adaptive online Textbooks (AoT) recommend the most relevant pages and practice activities based on students current knowledge state. AoT use student interaction data to infer the current state of student knowledge through student modeling (SM). The knowledge is inferred on knowledge components (KCs) associated with textbook material (sections/pages, practice activities, and quizzes). However, most of these techniques rely on expert annotated knowledge components. A challenge of student modeling in the context of adaptive textbooks is that traditional student models are constructed based on performance data (question answers or problem solving) Student interaction with online textbooks, however, produces large volume of student reading data, but a very limited amount of question-answering data. This leads to the requirement of annotating reading mater...

Dynamic Knowledge Modeling with Heterogeneous Activities for Adaptive Textbooks

Adaptive textbooks use student interaction data to infer the current state of student knowledge a... more Adaptive textbooks use student interaction data to infer the current state of student knowledge and recommend most relevant learning materials. A challenge of student mod- eling for adaptive textbooks is that conventional student models are constructed based on performance data (quiz or problem-solving), however, students' interactions with on- line textbooks may produce a large volume of student read- ing data but a limited amount of performance data. In this work, we propose a dynamic student knowledge modeling framework for online adaptive textbooks, which utilizes stu- dent reading data combined with few available quiz activi- ties to infer the students' current state of knowledge. The evaluation shows that proposed model learns more accurate students' knowledge state than Knowledge Tracing.

Concept Annotation for Intelligent Textbooks

ArXiv, 2020

With the increased popularity of electronic textbooks, there is a growing interests in developing... more With the increased popularity of electronic textbooks, there is a growing interests in developing a new generation of "intelligent textbooks", which have the ability to guide the readers according to their learning goals and current knowledge. The intelligent textbooks extend regular textbooks by integrating machine-manipulatable knowledge such as a knowledge map or a prerequisite-outcome relationship between sections, among which, the most popular integrated knowledge is a list of unique knowledge concepts associated with each section. With the help of this concept, multiple intelligent operations, such as content linking, content recommendation or student modeling, can be performed. However, annotating a reliable set of concepts to a textbook section is a challenge. Automatic unsupervised methods for extracting key-phrases as the concepts are known to have insufficient accuracy. Manual annotation by experts is considered as a preferred approach and can be used to produce...

Recommending Remedial Readings Using Student's Knowledge state

Assessment plays a vital role in learning, as it provides both instructors and students with feed... more Assessment plays a vital role in learning, as it provides both instructors and students with feedback on the overall effectiveness of their teaching or learning. However, when a student fails to correctly answer certain questions in an assessment (such as a quiz), the student needs specific recommendations that are tailored to their learning needs and to the knowledge deficiency exposed by the assessment outcomes. In this paper, we explore the methods for automatically identifying the recommended textbook materials that are most relevant and suitable to the student. In particular, we conducted experiments on how to incorporate students’ current knowledge state on domain concepts associated with the activity to recommend personalized remedial sections to each student. The results show that incorporating student knowledge states can significantly improve the quality of recommendations as compared to traditional content-based recommendations.

Is the Article Crucial to My Research? Evaluating Task-Oriented Impacts of Scientific Articles in Information Seeking

We propose a new aspect of evaluating scientific articles: crucialness, which refers to the state... more We propose a new aspect of evaluating scientific articles: crucialness, which refers to the state of articles being not only useful, but also scarce (difficult to be found or identified by scientists). Compared with the popularity-based metrics, crucialness may be a better metric in helping scientists’ information seeking and use because it identifies scientists’ difficulties in information seeking and reveals the crucial articles that may help scientists succeed in research. Some preliminary results are presented and discussed.

Semi-Supervised Techniques for Mining Learning Outcomes and Prerequisites

Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017

Educational content of today no longer only resides in textbooks and classrooms; more and more le... more Educational content of today no longer only resides in textbooks and classrooms; more and more learning material is found in a free, accessible form on the Internet. Our long-standing vision is to transform this web of educational content into an adaptive, webscale "textbook", that can guide its readers to most relevant "pages" according to their learning goal and current knowledge. In this paper, we address one core, long-standing problem towards this goal: identifying outcome and prerequisite concepts within a piece of educational content (e.g., a tutorial). Speci cally, we propose a novel approach that leverages textbooks as a source of distant supervision, but learns a model that can generalize to arbitrary documents (such as those on the web). As such, our model can take advantage of any existing textbook, without requiring expert annotation. At the task of predicting outcome and prerequisite concepts, we demonstrate improvements over a number of baselines on six textbooks, especially in the regime of li le to no ground-truth labels available 1. Finally, we demonstrate the utility of a model learned using our approach at the task of identifying prerequisite documents for adaptive content recommendation-an important step towards our vision of the "web as a textbook".

Supporting Cross-Device Web Search with Social Navigation-Based Mobile Touch Interactions

Lecture Notes in Computer Science, 2015

The wide adoption of smartphones eliminates the time and location barriers for people's daily inf... more The wide adoption of smartphones eliminates the time and location barriers for people's daily information access, but also limits users' information exploration activities due to the small mobile screen size. Thus, cross-device web search, where people initialize information needs on one device but complete them on another device, is frequently observed in modern search engines, especially for exploratory information needs. This paper aims to support the cross-device web search, on top of the commonly used context-sensitive retrieval framework, for exploratory tasks. To better model users' search context, our method not only utilizes the search history (query history and click-through) but also employs the mobile touch interactions (MTI) on mobile devices. To be more specific, we combine MTI's ability of locating relevant subdocument content [10] with the idea of social navigation that aggregates MTIs from other users who visit the same page. To demonstrate the effectiveness of our proposed approach, we designed a user study to collect cross-device web search logs on three different types of tasks from 24 participants and then compared our approach with two baselines: a traditional full text based relevance feedback approach and a self-MTI based subdocument relevance feedback approach. Our results show that the social navigation-based MTIs outperformed both baselines. A further analysis shows that the performance improvements are related to several factors, including the quality and quantity of click-through documents, task types and users' search conditions.

Automatic Concept Extraction for Domain and Student Modeling in Adaptive Textbooks

International Journal of Artificial Intelligence in Education, 2020

The increasing popularity of digital textbooks as a new learning media has resulted in a growing ... more The increasing popularity of digital textbooks as a new learning media has resulted in a growing interest in developing a new generation of adaptive textbooks that can help readers to learn better through adapting to the readers' learning goals and the current state of knowledge. These adaptive textbooks are most frequently powered by internal knowledge models, which associate a list of unique domain knowledge concepts with each section of the textbook. With this kind of concept-level knowledge representation, a number of intelligent operations could be performed, which include student modeling, adaptive navigation support, and content recommendation. However, manual indexing of each textbook section with concepts is challenging, time-consuming, and prone to errors. Modern research in the area of natural language processing offers an attractive alternative, called automatic keyphrase extraction. While a range of keyphrase and concept extraction methods have been developed over the last twenty years, few of the known approaches were applied and evaluated in a textbook context. In this paper, we present FACE, a supervised feature-based machine learning method for automatic concept extractions from digital textbooks. This method has been created for building domain and student models that form the core of intelligent textbooks. We evaluated FACE on a newly constructed full-scale dataset by assessing how well it approximates concept annotations produced by human experts and how well it supports the needs of student modeling. The results show that FACE outperforms several state-of-the-art keyphrase extraction methods.

One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Different texts shall by nature correspond to different number of keyphrases. This desideratum is... more Different texts shall by nature correspond to different number of keyphrases. This desideratum is largely missing from existing neural keyphrase generation models. In this study, we address this problem from both modeling and evaluation perspectives. We first propose a recurrent generative model that generates multiple keyphrases as delimiter-separated sequences. Generation diversity is further enhanced with two novel techniques by manipulating decoder hidden states. In contrast to previous approaches, our model is capable of generating diverse keyphrases and controlling number of outputs. We further propose two evaluation metrics tailored towards the variable-number generation. We also introduce a new dataset (ST A C KEX) that expands beyond the only existing genre (i.e., academic writing) in keyphrase generation tasks. With both previous and new evaluation metrics, our model outperforms strong baselines on all datasets.

The convergence of computational and social approaches for unveiling meaningful and valuable data

Proceedings of the Association for Information Science and Technology, 2019

The current data paradigm is seeking a more integrated and comprehensive framework to make sense ... more The current data paradigm is seeking a more integrated and comprehensive framework to make sense of data and its derived issues. From the perspective of the data life cycle, we argue that computational and social approaches complement each other to confront data challenges. Computational approaches consist of ETL (extract, transform, and load), modeling, and machine learning techniques; social approaches include policy and regulations, data sharing and reuse behavior, reproducibility, ethical and privacy issues. In this panel, we frame these two approaches as data acumen and data stewardship. The merging of these two perspectives allows data not only to become discoverable, accessible, and interoperable, but also to further the value of revealing meaningful patterns and become supportive evidence for important decision making. In this panel, the opening facilitator and three panelists will report on their recent studies in terms of this convergence of both data acumen and stewardship while sharing their recent research insights on case studies in three disciplines: agriculture, biomedicine, and archeology.

Deep Keyphrase Generation

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017

Keyphrase provides highly-summative information that can be effectively used for understanding, o... more Keyphrase provides highly-summative information that can be effectively used for understanding, organizing and retrieving text content. Though previous studies have provided many workable solutions for automated keyphrase extraction, they commonly divided the to-be-summarized content into multiple text chunks, then ranked and selected the most meaningful ones. These approaches could neither identify keyphrases that do not appear in the text, nor capture the real semantic meaning behind the text. We propose a generative model for keyphrase prediction with an encoder-decoder framework, which can effectively overcome the above drawbacks. We name it as deep keyphrase generation since it attempts to capture the deep semantic meaning of the content with a deep learning method. Empirical analysis on six datasets demonstrates that our proposed model not only achieves a significant performance boost on extracting keyphrases that appear in the source text, but also can generate absent keyphrases based on the semantic meaning of the text. Code and dataset are available at https://github.com/memray/seq2seqkeyphrase.