創意學習Cloze & Short Answer

# 創意學習Cloze & Short Answer [TOC] --- ## Paper 1 ### Cloze 1. Many readers actively employ typographical-cuing study methods such as underlining and ==highlighting== while reading for their classes. 2. Results of this analysis revealed that ==high-skill readers== demonstrate a significantly lower average proportion of pages marked than low-skill readers. 3. In order to determine whether or not low-skill readers marked the most relevant material in the text they were using, we examined the text material marked by students who submitted new texts that actually contained ==text marking==. 4. In order to determine if there was a significant difference between high- and ==low-skill readers== in ability to identify the most important information in a text, the reading skill groups used in the prior analyses were used as a grouping variable in an Independent- Groups t-test on the average relevance index. 5. Our results suggest that readers of ==lower skill== have a preference for previously marked textbooks and report a tendency to only study previously marked material in their used texts. 6. In addition, in our analysis of ==marked texts==, we demonstrated that as ability to identify the most relevant material in the text decreases, performance on exams in the course decreases. 7. It is purely descriptive, revealing a number of small, yet significant, relationships among ==reading skill==, text marking preferences and behaviors, and course performance. ### Short Answer 1. What is a significant relationship between reading skill, text marking preferences and behaviors? * ==course performance== 2. What type of readers prefers to study and report only studying material marked by a previous reader? * ==low-skill readers== 3. What is significant about low-skill readers? * ==previous marking== 4. What is one reason why a low-skill reader might prefer to read a book over a text? * ==They are less capable of marking the most relevant material.== 5. What do low-skill readers do? * ==They prefer previously marked textbooks.== 6. What are the two things that are important in determining a student's ability to succeed? * ==course performance and reading skill== 7. What happens after the assessment? * ==Students can then be made aware of their reading skill level.== 8. What are some of the main goals of the study? * ==marking== --- ## Paper 2 ### Cloze 1. Drawing on the concept of Innovation Diffusion Theory, we used educational data mining techniques and ==survival analysis== to examine time to adoption of highlights, notes, annotations, bookmarks, and questions in an interactive e-textbook reader. 2. E-textbooks and digital content systems are evolving into interactive platforms that enable learners to ==mark-up== material to help them retain information. 3. ==Bookmarking== is a mark-up strategy that is supported by theories of learning and metacognition. 4. We believe this research contributes important findings for the study of how college students use the ==Interactive== features of e-textbooks in their learning activities. 5. After controlling for student's grade point average, we found that number of bookmarks and its interaction with a natural log transformation of number of pages read was a significant predictor of ==learning outcome==s in the form of final grade. 6. We believe this research contributes important findings for the study of how college students use the interactive features of e-textbooks in their ==learning activities==. 7. With respect to e-textbooks, instructional technologists may need to consider communication strategies or course-based ==Intervention==s that are likely to be persuasive to learners. ### Short Answer 1. What was the findings of this study? * ==Certain measures of reading behaviors was predictive of time to adopting mark-up tools.== 2. What is the impact of the research? * ==Predict learning outcomes.== 3. What is the benefit of using a tool with frequent reading? * ==Assist students to learn content successfully.== 4. What are the benefits of these tools? * ==Assist students to learn content successfully.== 5. What is the purpose of educational technology? * ==learning with educational technology in which scholars found that technologies that better support cognition have better effects on outcomes== 6. What is the only tool used by at least 50% of the subjects * ==highlighting tool== 7. Who wanted to use the tool for regular assignments? * ==Professor Brown== 8. Which tool was used by the most students, and why? * ==The highlight tool was used by the most students, and it was the easiest tool for students to use for it did not require the addition of text to the system.== 9. What is important to instructors who provide students with educational technology that is designed to help them with their learning? * ==The time to adoption of such tools== 10. What does this research suggest about learning with tools in interactive e-textbook platforms? * ==Must examine the rate of adoption and provide interventions that address obstacles to effective adoption.== 11. What will happen when students increased usage of the bookmarking tool, coupled with more total reading? * ==Increased usage of the bookmarking tool was predictive of students’ final grades after controlling for prior learning.== --- ## Paper 3 ### Cloze 1. Digital textbook ==analytic==s are a new method of collecting student-generated data in order to build predictive models of student success. 2. ==Learning analytic==s is the "measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" (Siemens, 2010). 3. Variations in ==highlighting== and other reading strategies have been found between digital and print reading although no differences have been found in GPA between electronic and print textbooks (Daniel & Woody, 2013). 4. Therefore, it should be possible to use textbook analytics to directly measure ==engagement== with a digital textbook and to use this information to predict student course outcomes (Bossaller & Kammer, 2014). 5. Through initial beta testing and with guidance from Astin's (1984) theory of student engagement, the Engagement Index includes multiple measures of ==reading frequency== (time spent reading) and engagement (like highlighting and annotating). 6. Furthermore, based on previous research it is hypothesized that there will be differences in course ==outcome==s by the number of highlights (Daniel & Woody, 2013), but the number of notes, bookmarks, and time spent reading may not necessarily reflect a difference in academic performance (Fitzpatrick & McConnell, 2008, Hsieh & Dwyer, 2009). 7. ==Highlighting== showed an ability to predict student grades, but not as part of a linear model. 8. Highlighting and annotating are active reading strategies that help students engage with the text in order to remember and understand ==key concept==s (Bell & Limber, 2009). 9. This work suggests that transforming highlighting using a probability-based algorithm and adding that to time spent reading would yield an even more powerful ==predictor== of student course grades. ### Short Answer 1. What is digital textbook analytics? * ==A new method of collecting student-generated data.== 2. What did an engagement index based on digital textbook usage data lead to? * ==To predict student course grades.== 3. How many students printed out materials when working with someone else? * ==92%== 4. What are the differences in reading skills between digital and print textbooks? * ==No differences in reading.== 5. Which reading strategies most successfully influenced comprehension? * ==reading repeated== 6. What is an important aspect of student learning? * ==reading repeated== 7. What can be used to improved course outcomes? * ==reading repeated== 8. What are the most popular strategy? * ==reading repeated== 9. What is the primary reason why self-reported measures are not useful? * ==relating to actual behaviors== 10. What is the purpose of Highlighting and annotating strategies? * ==help students engage with the text== --- ## Paper 4 ### Cloze 1. We found the number, orientation, and complexity of questions students posed to be three indices of ==question-posing== capability. 2. An STS-oriented curriculum with a strong emphasis on the interrelationship of ==scientific concepts== and real-life phenomena may better serve nonscience students, who have limited scientiﬁc knowledge which is nevertheless necessary to prepare them as future citizens (Ben-Zvi & Gai, 1994). 3. When case studies are used as a supplement to lectures, they serve to deepen student ==knowledge== in societal, environmental, and ethical issues, as well as foster student appreciation of the interdependence of these topics (Challen & Brazdil, 1996). 4. The two objectives of our research were (a) to examine ways of using question-posing capability as an alternative evaluation tool, and (b) to investigate the effect of the case study teaching/learning approach on the question-posing capability of high school students at different ==academic level==s. 5. ==Science major==s (H-level students) studied the module for extra credit, while M and L students took it as the Science and Technology for All course. 6. To test whether the difference in the extent of increase in the average complexity of questions among the three academic levels in the pretest and posttest is signiﬁcant, we conducted yet another series of ==Kruskal-Wallis== tests. 7. The ==knowledge level== of medium and low academic level students declined signiﬁcantly and proportionately in other topics compared to the expert topic. 8. Shepardson (1993) claimed that science curricula that promote learning must be restructured to ==engage== students in higher levels of thinking about scientiﬁc information. ### Short Answer 1. What is an effective strategy for improving problem-solving ability? * ==question-posing== 2. How were students' question-posing skills evaluated? * ==the number, orientation, and complexity of questions== 3. What are two broad types of classroom interaction? * ==recitation and discussion== 4. What are the three categories of cognitive levels of questions? * ==the number, orientation, and complexity of questions== 5. What do input-level questions require students to do? * ==recall information== 6. What must be restructured to engage students in higher levels of thinking about scientific information? * ==science curricula that promote learning== 7. What was the cause of questions at a high complexity level? * ==student interaction and construction of new knowledge== 8. What is a component of thinking skills for learning tasks and a stage in the problem-solving process? * ==question asking== --- ## Paper 5 ### Cloze 1. We aimed to examine the ==testing effect== in its pure form by implementing a minimal intervention design in a university lecture. 2. One potentially effective ==review strategy== is the active retrieval of learned material from memory, which can be prompted by testing knowledge of the learned content. 3. Two factors that reliably affect the testing effect are feedback (Rowland, 2014; Adesope et al., 2017) and ==retrievability== (Rowland, 2014). 4. It has been repeatedly argued that multiple-choice questions and ==short-answer== questions differ in the effort needed to be answered correctly and given these theoretical underpinnings should consequently lead to different testing effects (e.g., Karpicke, 2017). 5. The present study investigated the testing effect in a university education setting by implementing a minimal ==intervention== in an existing university course. 6. Most research uses a repetition of the entire ==learning content== in the restudy condition, but exact repetitions are difficult to implement in real-world educational contexts because of time constraints, that is, usually only selected information is restudied. 7. To conclude, this research contributes to the literature by demonstrating a testing effect for practice tests with ==short-answer== questions in the real-world educational context of a university lecture. 8. To use the ==testing effect== to foster learning, educational practitioners should identify the most important topics of their lecture, teach these thoroughly, and use short-answer testing to solidify the knowledge about these topics. ### Short Answer 1. When do students often need to review the learned material? * ==before exam== 2. What do learners tend to do when reading or listening to it only once? * ==remember less learning content== 3. What did the results indicate for multiple-choice testing? * ==no testing effect== 4. What is defined as learning occasions that may hamper learning in the short run but enhance learning in the long run? * ==desirable difficulty== 5. What are benefits of testing vs. restudying might become visible, especially at later criterial tests? * ==assume that testing is desirable learning difficulty== 6. The superiority of testing compared to restudying might not be detected until what? * ==test== 7. What role does the testing effect emphasize during retrieval? * ==cognitive effort== 8. Questions that prompt effortful retrieval are likely to elicit what? * ==testing effect== --- ## Paper 6 ### Cloze 1. The ==testing effect== and test format In research related to the testing effect, the implications of different test formats on memory and learning have been examined over the years. 2. Most of these studies only include one ==practice test== for each format, even though it has been stated that the number of practice tests given has a large impact on a participant’s performance over time (Roediger, Putnam, & Smith, 2011); this makes it difﬁcult to examine the practice effects of testing for different formats. 3. According to the retrieval effort hypothesis, it was expected that practice testing with SA items would enhance retention ==test performance== more than would practice testing with MC items. 4. The ==practice test==s consisted of 12 SA items, 12 MC items (all MC items had four options, one correct and three lures), and 12 statements to read (the study condition). 5. The same items from the immediate test were used in the delayed ==retention test==s. 6. There are two primary explanations behind the ==testing effect==: the transfer appropriate processing hypothesis, and the retrieval effort hypothesis. 7. The aim was to test the ==hypotheses== of transfer appropriate processing and retrieval effort by comparing performance on retention tests with respect to practice testing format. 8. The result obtained whereby performance after ==practice testing== was improved when compared to studying alone is not surprising, as we provided feedback regarding the respondents’ answers. 9. Feedback has been found to be a vital factor for enhancing the testing effect in immediate retention as well as in ==long-term retention==. 10. In sum, when we corrected for guessing, we found support for the ==retrieval effort== hypothesis, where practising on SA items led to better performance than practising on MC items when the retention test items were in SA format. 11. To our knowledge, no previous study comparing ==testing effect==s across SA and MC tests have taken into account the probability of guessing. ### Short Answer 1. What is the phenomenon that shows that studying a material and then taking a test on that material promotes subsequent learning and retention more than restudy? * ==testing effect== 2. What are two common item formats used in tests in educational settings? * ==short-answer items and multiple-choice items== 3. What does practice testing with short-answer items provide support for? * ==retrieval effort hypothesis== 4. What did multiple-choice and short-answer formats improve compared to rereading? * ==short- and long-term memory== 5. What supports the assumption that short-answer items are preferred in a learning context? * ==retrieval effort hypotheses== 6. What item type do not require any effort of retrieval? * ==multiple-choice items== 7. What do short-answer items enhance? * ==memory performance== 8. What is the most important goal in educational settings? * ==To support learning among students== 9. What hypothesis predicted that practice testing with short-answer items would enhance retention test performance more than practice testing with multiple-choice items? * ==The hypothesis of transfer appropriate processing== --- ## Paper 7 ### Cloze 1. When prior knowledge is high, the effect of ==reading ability== on comprehension vanishes. 2. Test items with simple declarative answers, or fact questions, were concept-matched to ==test items== with contextualized application questions, or transfer questions, such that a concept either appeared on the pre-test or on the post-test but not both. 3. Cloze items for the three ==cloze conditions== were created either by human, randomly,or by machine using an algorithm described below. 4. Machine generated ==cloze items== were selected by using natural language processing techniques at the word, sentence, and discourse level. 5. An exploratory ANCOVA model with condition, text reading time (log transformed), pre-test proportion correct, and all interactions as predictors and post-test proportion correct as the dependent variable was created and reﬁned using backward elimination ==variable selection== based on the Akaike information criterion(AIC). ### Short Answer 1. In Paper 7, what is the effect of cloze item practice on reading? * ==Cloze item practices are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.== 2. What role does prior knowledge play in reading comprehension mentioned in Paper 7? * ==Prior knowledge plays in a central role.== 3. What are the most common reading comprehension activities mentioned in paper 7 in the educational environment? * ==Reading comprehension activities in educational contexts typically center around the instruction and practice of reading strategies.== 4. What are the main findings of the Machine Cloze condition in Paper 7? * ==The Machine Cloze condition led to superior post-test outcomes relative to other conditions.== 5. In Paper 7, what do cloze items generated by machine improve? * ==Cloze items generated by machine using natural language processing techniques are effective for enhancing reading comprehension when delivered by an adaptive practice scheduling system.== 6. In Paper 7, what will happen when comprehension to occur? * ==Vocabulary and comprehension are deeply intertwined because text must be decoded, disambiguated, and linked with prior knowledge for comprehension to occur.== 7. How many unknown words are there in a sentence mentioned in Paper 7 will not cause comprehension deficits? * ==The number of unknown words should be no lower than 1 in 20.== --- ## Paper 8 ### Cloze 1. Learners often memorize words in ==digital textbook==s while preparing for a KEYWORDS examination, and they often use the highlight function for the words. 2. ==Digital textbooks== often called just digital books or e-books are intended to serve as the textbooks for class. 3. The e-book design activities could improve students ==learning achievement==s and reduce students cognitive load. 4. However, so far these studies have not focused on supporting the ==repeated learning== by utilizing the digital textbook logs. 5. The result of the number of quizzes (Table 3) shows high SD score, so the number of the ==quizzes taken== by each learner showed a variation. ### Short Answer 1. In Paper 8, what is the best way to enhance learning achievements? * ==Taking quizzes== 2. In Paper 8, what do learners use highlighting functions for? * ==Prioritizing quizzes based on the recall method with highlighted words== 3. In Paper 8, how to support repeated learning? * ==Taking quizzes, hint== 4. What is the duration of short term memory mentioned in Paper 8? * ==15 and 30 seconds== 5. In Paper 8, How to shift from a short-term memory to a long-term memory? * ==repeated learning== 6. In Paper 8, what is the goal of repeated learning? * ==To shift knowledge from short-term memory to long-term memory== 7. What is the most important thing in a learning analytics mentioned in Paper 8? * ==Supporting repeated learning== 8. In Paper 8, what should be provided to decrease test anxiety? * ==Taking more quizzes== --- ## Paper 9 ### Cloze 1. Each type of ==distractor== indicates a different type of comprehension failure when chosen by a child instead of the correct answer. 2. For instance, a child who processes syntax and semantics but not the relation of a sentence to the context preceding it would reject the ungrammatical and ==nonsensical distractors==, but pick the plausible distractor as often as the correct answer. 3. It chose ==human-authored== distractors not yet rated by anyone who had finished the protocol. 4. Compared to their nonsensical distractors, both DQGen and humans generated significantly worse (p < 0.02) plausible distractors, with a trend (p < 0.1) toward better ==ungrammatical distractors==. ### Short Answer 1. In Paper 9, what are the types of questions generated by DQGen? * ==multiple-choice cloze question== 2. In Paper 9, what is required to answer without guessing? * ==Answering without guessing requires having relevant background knowledge and understanding the context.== 3. What can cloze questions do in language learning in Paper 9? * ==Cloze questions can test the ability to decide which word is consistent with the surrounding context.== 4. What is the focus of Paper 9's research? * ==It reports an experiment to evaluate DQGen’s performance in generating three types of distractors for diagnostic multiple-choice cloze questions to assess children’s reading comprehension processes.== 5. In Paper 9, what did a participant complete for set B? * ==The protocol by writing distractors== 6. In Paper 9, how to find the model that fits the data best? * ==They used backward model selection, starting with five predictors we expected could affect the outcome. Three were fixed effects: distractor source, intended type, and their interaction. Two were random effects: stem and rater.== 7. In Paper 9, what did Error Analysis elucidate? * ==To shed light on which distractors were rated differently than their intended type.== --- ## Paper 10 ### Cloze 1. We study automatic ==question generation== leased by photosynthesis, which uses the energy of sunlight to produce oxygen from water. 2. One key application of question generation is in the area of education — to generate questions for ==reading comprehension== materials (Heilman and Smith, 2010). 3. In addition, question generation systems can aid in the development of annotated data sets for ==natural language processing== (NLP) research in reading comprehension and question answering. 4. The ==rule-based== approaches make use of the syntactic roles of words, but not their semantic roles. ### Short Answer 1. What does Question generation aims to do in Paper 10? * ==Create natural question from a given sentence or paragraph.== 2. In Paper 10, why encoding the paragraph causes the performance to drop? * ==There are many noises in the paragraph.== 3. What modalities to measure the quality of questions generated by Paper 10 system and the H&S system? * ==Naturalness, which indicates the grammaticality and fluency; and difficulty.== 4. In Paper 10, what can be deployed as a chatbot to start a conversation or request feedback? * ==Asking questions to start a conversation or to request feedback== 5. What is needed for constructing natural problems of reasonable difficulty raised in Paper 10? * ==Require an abstractive approach that can produce fluent phrasings that do not exactly match the text from which they were drawn.== --- ## Paper 11 ### Cloze 1. We propose a maxout pointer mechanism with gated ==self-attention== encoder to address the challenges of processing long text inputs for question generation. 2. When applied to ==paragraph-level== context (Yuan et al., 2017), we observed large gaps compared to state-of-the-art results achieved by using sentence-level context. 3. The model can effectively utilize paragraph-level context, and outperformed the results with ==sentence-level== context. 4. One is built on top of heuristic rules that QG and ==question answer==ing by encoding both the creates questions with manually constructed answer and the passage with a multiperspective plate and ranks the generated results, e.g. 5. Attention mechanism (Bahdanau et al., 2014; Luong et al., 2015) has been used to improve ==sequence== to sequence models’ performance and has became a default setting for many applications. ### Short Answer 1. What is the problem with long text mentioned in Paper 11? * ==Long text can contain irrelevant information w.r.t. the answer for generating the question.== 2. What is the sequence to sequence framework also called in Paper 11? * ==encoder-decoder framework== 3. What is the maxout pointer mechanism proposed in Paper 11? * ==To address the challenges of processing long text inputs for question generation.== 4. When is the effectiveness of maxout pointer more significiant in Paper 11? * ==When paragraph is given as the model input.== --- ## Paper 12 ### Cloze 1. A Recurrent BERT-based Model for Question Generation mainly use only sentence-level information as a context text for ==question generation==. 2. ==Question generation== (QG) problem, which takes a context text and an answer phase as input and generates a question corresponding to the given answer phase. 3. The existing QG models mainly rely on recurrent neural networks (RNN), e.g. long short-term memory LSTM network (Hochreiter and Schmidhuber, 1997) or gated recurrent unit (Chung et al., 2014), augmented by ==attention mechanisms== (Luong et al., 2015). 4. When applied to a ==paragraph-level== context, the existing models show significant performance degradation. 5. The BERT model is built by a stack of multi-layer bidirectional Transformer ==encoder== (Vaswani et al., 2017). ### Short Answer 1. What QG models mainly rely on mentioned in Paper 12? * ==The existing QG models mainly rely on recur- rent neural networks.== 2. What is the latest development of pre-trained language models mentioned in Paper 12? * ==The latest development is BERT.== 3. What is the framework to train a neural network for question generation mentioned in Paper 12? * ==sequence-to-sequence framework== 4. In Paper 12, what language modeling tasks BERT can be employed in? * ==The BERT model can be employed in three language modeling tasks: sequence-level classification, span-level prediction, and token-level prediction tasks.== 5. What are the two shortcomings of BERT-SQG in Paper 12? * ==First, when processing lengthy context, we find that the generated question is often with lower quality. Second, when an answer phase appears multiple times in the context, there is ambiguity for select which one to generate questions.== 6. In Paper 12, what is the BLEU score variant that uses up to n-grams for counting cooccurrences? * ==BLEU-n== 7. What model Paper 12 propose that generate a question from the input context and the target answer? * ==A Recurrent BERT-based Model== --- ## Paper 13 ### Cloze 1. Natural language processing (NLP) has progress largely due to developments in ==deep learning==. 2. A ==Transformer== network model is proposed which consisting of the conjunction of the Transformer decoder GPT-2 model with Transformer encoder BERT. 3. BERT can easily be ﬁne-tuned for a plethora of The ==proposed approach== is evaluated on the different downstream tasks. 4. ==Question generation== often exhibits linguistic variance that is semantically admissible. 5. If the model is able to generate questions with high diversity using words with low lexical ==similarity== to the groundtruth, the QA system is also improved, it learns to generalize better. 6. BLEU and ROGUE are only of limited use therein, as they mainly measure the lexical similarity between the ==generated question== and the ground truth. 7. Given a context with ==annotated== question, a question is generated using GPT-2. 8. We leverage question generation by tying together GPT-2 and BERT facilitating ==semi-supervised== learning. ### Short Answer 1. What is the purpose of automatic question generation in Paper 13? * ==Automatic question generation aims at the generation of questions from a context, with the corresponding answers being sub-spans of the given passage.== 2. What are the two main building blocks of the Transformer architecture mentioned in Paper 13? * ==GPT-2 and BERT are the two main building blocks of the Transformer architecture mentioned in Paper 13.== 3. In Paper 13, question generation is done by tying together GPT-2 and what else? * ==Question generation is done by tying together GPT-2 and BERT in end-to-end trainable fashion facilitating semi-supervised learning.== --- ## Paper 14 ### Cloze 1. According to Chi’s ICAP framework (Chi & Wylie, 2014), practicing with short-answer questions should also provide more benefits for students’ learning than with ==multiple-choice== questions. 2. Typical ==short-answer== questions are closed-ended, namely, the correct answers are expected to match only several facts, which are also called as expectations, that are specified by the question authors. 3. To evaluate our proposed model, we conducted experiments on 7 ==reading comprehension== short-answer questions. 4. Automatic grading algorithms of this type are called ==similarity==-based grading methods in this paper. 5. The largest disadvantage of our proposed model is the requirement of many ==graded== student answers for each target question. ### Short Answer 1. In paper 14, what do reading comprehension questions used to do? * ==Reading comprehension questions are commonly used to assess students’ understanding of reading materials.== 2. In paper 14, what is the most common type of reading comprehension question? * ==Multiple-choice and short-answer reading comprehension questions are two typical types== 3. What do short-answer questions ask students to do in Paper 14? * ==Short-answer questions ask students to express their own thoughts based on various facts.== 4. What are the closed-ended questions mentioned in Paper 14? * ==This method performs well for closed-ended questions that have single or very limited numbers of correct answers.== 5. What is the most common similarity calculation in Paper 14? * ==The cosine distance is one of the most common measures.== 6. In paper 14, what are the benefits of using LSTM sequences? * ==Taking advantage of word sequence information by using LSTM improved the grading accuracy as well.== ---