Journal of Measurement and Evaluation in Education and Psychology

This research intended to examine the effects of presenting geometry items with and without shapes on the psychometric properties of the test and students’ test scores. The study was conducted on 480 eighth grade students. Within the scope of the study, two geometry tests were crafted, one with shapes and the other without shapes. Both tests consisted of 15 multiple-choice items. In the data collection process, a counterbalanced design was followed and the two tests were administered to the students three weeks apart. Analyses were carried out on 405 students who participated in both applications and whose test forms could be matched. The factor analysis results revealed that the factor loadings of the items and extracted variance were higher for the test with shapes compared to the shape-free form. The Cronbach’s alpha coefficient of the test containing shapes was found to be significantly higher than that calculated for the test without shapes. According to item difficulties, the questions with shapes were easier for the students than the shape-free questions. In terms of discrimination indices a difference in favor of the shape-containing test was observed in almost all items. Ferguson delta statistic, which is a measure of discrimination for the overall test, was higher in the shape-contaninig test. Correlation analysis denoted a strong positive relationship between student scores on the two tests. The paired sample t-test proved that there was a statistically significant difference between students’ scores on the shape-containing and shape-free tests. These results indicate that the geometry tests with and without shapes differ in terms of both psychometric properties and students’ test scores.

İslim Atçı

Mustafa İlhan

Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi

The Effect of Presenting Geometry Items with and without Shapes on the Psychometric Properties of the Test and Students’ Test Scores

几何项目有无图形呈现对测试心理测量特性和学生测试成绩的影响

This study aims to explore the intricate relationship between students' response times, item characteristics, and the effort invested during the Programme for International Student Assessment (PISA) 2015 and 2018 cycles. Through the analysis of data obtained from 69 mathematics trend items administered in a computer-based format across both cycles, this research investigates the dynamics of students' response times and their implications on effort and item characteristics. Findings reveal a significant increase in students' mean response times in the 2018 cycle compared to 2015, indicating potentially heightened effort and solution behavior. Notably, item formats exerted a substantial influence on response times, with open-ended items consistently eliciting lengthier response times compared to multiple-choice items. Additionally, a correlation between response times and item difficulty emerged, suggesting that more challenging items tend to consume more time, possibly due to the complexity of involved cognitive processes. Item based effort, assessed through Response Time Fidelity (RTF) indices, highlighted that a majority of students exhibited solution behavior across both cycles to the items.. Moreover, a decrease in the proportion of students displaying rapid-guessing behavior was observed in the 2018 cycle, potentially reflecting increased engagement with the assessment. While providing insights into the interplay of response times, item characteristics, and effort, this study emphasizes the need for further exploration into the multifaceted nature of effort in educational assessments. Overall, this research contributes valuable perspectives on nuances surrounding test performance and effort evaluation within PISA mathematics assessments.

Muhsin Polat

Hülya Kelecioğlu

The Comparison of PISA 2015-2018 Mathematics Trend Items Based on Item Response Times

基于项目反应时间的PISA 2015-2018数学趋势项目比较

The effectiveness of methods for handling missing data in educational assessments depends on understanding the underlying missing mechanisms. This study investigates the performance of the IRTree framework in detecting missing data mechanisms using a Monte Carlo simulation. Omitted responses were simulated at varying proportions according to three mechanisms: MCAR, MAR, and MNAR, across tests with different lengths and sample sizes. The IRTree was employed to model the omitted responses and detect the mechanisms based on the correlations between the propensity to omit and proficiency. Results indicate that the IRTree accurately identifies all three missing data mechanisms, with no relationship between propensity to omit and proficiency under MCAR, and negative correlations for MAR, reaching up to -0.3, and for MNAR, as high as -0.8. Furthermore, the detection of MAR and MNAR mechanisms became more pronounced with higher proportions of omitted responses, longer tests, and larger sample sizes. IRTree framework not only enables educators and researchers to accurately understand the nature of missing data but also guides them in using appropriate methods for handling it.

Yeşim Beril Soğuksu

The Efficacy of the IRTree Framework for Detecting Missing Data Mechanisms in Educational Assessments

教育评估中检测缺失数据机制的IRTree框架的有效性

The aim of the study is to examine the meta-analytic reliability generalization of the 14, 10, and 8-item forms of the Perceived Stress Scale, developed by Cohen et al. in 1983, for theses produced in Turkey. For this purpose, how different moderator variables affect the reliability coefficients and publication bias were also examined 81 Cronbach Alpha coefficients from 78 studies selected in accordance with the established criteria were included in the meta-analysis. Reliability generalization was used in the analysis of the data. Analyzes were made with the random effects model using the coefficient values converted from the Bonett method. In the study, the effect size value was found to be .82 (95% CI: .80, .83), and it was concluded that the sample type (α= .81) and the study area (α= .81) moderator variables had a statistically significant effect on the reliability estimation in terms of heterogeneity of effect sizes. This revealed that these two moderator variables affected the estimations of the reliability coefficients. In addition, it has been observed that other moderator variables such as age, gender, and the number of items in the scale are not sources of heterogeneity and have no effect on reliability estimation. From this, it was deduced that the scale works well enough to generalize to different situations with different item numbers. Finally, according to the results of the analysis for the determination of publication bias, it was seen that there was no publication bias in the study.

Ömer Doğan

Selahattin Gelbal

Meta-Analytic Reliability Generalization Study of Perceived Stress Scale In Türkiye Sample

土耳其样本中感知压力量表的元分析信度概括研究

In qualitative studies, researchers must devote a significant amount of time and effort to extracting meaningful themes from huge sets of texts and examining the links between themes, which are frequently done manually. The availability of natural language models has enabled the application of a wide range of techniques for automatically detecting hierarchy, linkages, and latent themes in texts. This paper aims to investigate the coherence of the topics acquired from the analysis with the predefined themes, the hierarchy between the topics, the similarity between the topics and the proximity-distance between the topics by means of the topic model based on BERTopic using unstructured qualitative data. The qualitative data for this study was gathered from 106 students engaged in a university-run pedagogical formation certificate program. In BERTopic procedure, paraphrase-multilingual-MiniLM-L12-v2 model was used as sentence transformer model, UMAP was used as dimension reduction method and HDBSCAN algorithm was used as clustering method. It is found that BERTopic successfully identified six topics corresponding to the six predicted themes in unstructured texts. Moreover 74% of the texts containing some themes could be classified accurately. The algorithm was also able to successfully identify which topics were similar and which topics differed significantly from the others. It was concluded that BERTopic is a procedure that can identify themes that researchers do not notice depending on the density of the data in qualitative data analysis and has the potential to enable qualitative research to reach more detailed findings.

Osman Tat

Izzettin Aydogan

Discovering Hidden Patterns: Applying Topic Modeling in Qualitative Research

发现隐藏模式：在定性研究中应用主题建模

短名	Egit. Psikol. Olcme. Deger. Derg.
Journal Impact	0.67
国际分区	PSYCHOLOGY, EDUCATIONAL(Q4)

涉及主题	心理学哲学数学艺术人文学科统计医学计算机科学政治学发展心理学法学经济物理生物数学教育机器学习
出版信息	出版商: Association of Measurement and Evaluation in Education and Psychology (EPODDER)，出版周期: ，期刊类型: journal，开源期刊: 否
基本数据	创刊年份: 2018，原创研究文献占比: 100.00%，自引率:20.00%， Gold OA占比: 100.00%
平均审稿周期	网友分享经验：10 Weeks

Journal of Measurement and Evaluation in Education and Psychology

期刊引文格式

有 1 位以上作者的期刊

有 2 位作者的期刊

有 3 位作者的期刊

有 5 位以上作者的期刊

书籍引用格式

学位论文引用格式

网页引用格式

专利引用格式

手工熬夜修改参考文献？研飞自动匹配期刊，一键轻松成稿，支持 Word/WPS

投稿经验分享

The Effect of Presenting Geometry Items with and without Shapes on the Psychometric Properties of the Test and Students’ Test Scores

The Comparison of PISA 2015-2018 Mathematics Trend Items Based on Item Response Times

The Efficacy of the IRTree Framework for Detecting Missing Data Mechanisms in Educational Assessments

Meta-Analytic Reliability Generalization Study of Perceived Stress Scale In Türkiye Sample

Discovering Hidden Patterns: Applying Topic Modeling in Qualitative Research

Meta-Analytic Reliability Generalization Study of Perceived Stress Scale In Türkiye Sample

Discovering Hidden Patterns: Applying Topic Modeling in Qualitative Research

The Efficacy of the IRTree Framework for Detecting Missing Data Mechanisms in Educational Assessments

The Effect of Presenting Geometry Items with and without Shapes on the Psychometric Properties of the Test and Students’ Test Scores

The Comparison of PISA 2015-2018 Mathematics Trend Items Based on Item Response Times

ISSN	1309-6575
h-index	5