Rethinking Metaphor Evaluation: Aptness Judgments as a Cognitive Probe for Language Models

Al-Azary Hamad, Katz Albert N. Do metaphorical sharks bite? simulation and abstraction in metaphor processing. Memory Cognit. 2021;49:557–70.

Article  Google Scholar 

Babieno Mateusz, Takeshita Masashi, Radisavljevic Dusan, Rzepka Rafal, Araki Kenji. Miss roberta wilde: Metaphor identification using masked language model with wiktionary lexical definitions. Appl Sci. 2022;12(4):2081.

Article  Google Scholar 

Birke J, Sarkar A. A clustering approach for nearly unsupervised recognition of nonliteral language. In 11th Conference of the European chapter of the association for computational linguistics; 2006, p. 329–36.

Blasko Dawn G, Connine Cynthia M. Effects of familiarity and aptness on metaphor processing. J Exper Psychol Learn Memory Cognit. 1993;19(2):295.

Article  Google Scholar 

Cambria Erik, Poria Soujanya, Gelbukh Alexander, Thelwall Mike. Sentiment analysis is a big suitcase. IEEE Intell Syst. 2017;32(6):74–80.

Article  Google Scholar 

Chiappe Dan, Kennedy John M, Smykowski Tim. Reversibility, aptness, and the conventionality of metaphors and similes. Metaphor Symbol. 2003;18(2):85–105.

Article  Google Scholar 

Chiappe Dan L, Kennedy John M. Aptness predicts preference for metaphors or similes, as well as recall bias. Psych Bull Rev. 1999;6(4):668–76.

Article  Google Scholar 

Chiappe Dan L, Kennedy John M, Chiappe Penny. Aptness is more important than comprehensibility in preference for metaphors and similes. Poetics. 2003;31(1):51–68.

Article  Google Scholar 

Choi M, Lee S, Choi E, Park H, Lee J, Lee D, Lee J. Melbert: Metaphor detection via contextualized late interaction using metaphorical identification theories. 2021. arXiv:2104.13615.

Conneau A, Lample G. Cross-lingual language model pretraining. Adv Neural Inf Process Syst. 2019;32.

Devlin J. Bert: Pre-training of deep bidirectional transformers for language understanding. 2018. arXiv:1810.04805.

Elzohbi M, Zhao R. Contrastwsd: Enhancing metaphor detection with word sense disambiguation following the metaphor identification procedure. 2023. arXiv:2309.03103.

Fellbaum C. WordNet: An electronic lexical database. MIT press; 1998.

Gagné Christina L. Metaphoric interpretations of comparison-based combinations. Metaphor Symb. 2002;17(3):161–78.

Article  MathSciNet  Google Scholar 

Gibbs Jr RW. Process and products in making sense of tropes. 1993.

Gibbs Jr RW. Taking metaphor out of our heads and putting it into the cultural world. In Metaphor in cognitive linguistics: selected papers from the 5th international cognitive linguistics conference, Amsterdam, 1997; 2011, p. 145–66. John Benjamins Publishing Company.

Glucksberg Sam, Keysar Boaz. Understanding metaphorical comparisons: Beyond similarity. Psychol Rev. 1990;97(1):3.

Article  Google Scholar 

He P, Gao J, Chen W. Debertav 3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. 2021. arXiv:2111.09543.

Jia K, Li R. Metaphor detection with context enhancement and curriculum learning. In Proceedings of the 2024 conference of the north american chapter of the association for computational linguistics: human language technologies (Volume 1: Long Papers); 2024, p. 2726–37.

Jones Lara L, Estes Zachary. Metaphor comprehension as attributive categorization. J Memory Lang. 2005;53(1):110–24.

Article  Google Scholar 

Jones Lara L, Estes Zachary. Roosters, robins, and alarm clocks: Aptness and conventionality in metaphor comprehension. J Memory Lang. 2006;55(1):18–32.

Article  Google Scholar 

Katz Albert N. On choosing the vehicles of metaphors: Referential concreteness, semantic distances, and individual differences. J Memory Lang. 1989;28(4):486–99.

Article  Google Scholar 

Lakoff G, Johnson M. Metaphors we live by. University of Chicago press; 2008.

Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. Albert: A lite bert for self-supervised learning of language representations. 2019. arXiv:1909.11942.

Leong CW, Klebanov BB, Hamill C, Stemle E, Ubale R, Chen X. A report on the 2020 vua and toefl metaphor detection shared task. In Proceedings of the second workshop on figurative language processing; 2020, p. 18–29.

Leong CW, Klebanov BB, Shutova E. A report on the 2018 vua metaphor detection shared task. In Proceedings of the workshop on figurative language processing; 2018, p. 56–66.

Li Y, Peng B, Hsu Y-Y, Huang C-R. Embodiedbert: Cognitively informed metaphor detection incorporating sensorimotor information. In Findings of the association for computational linguistics: EMNLP 2024;2024, p. 16868–76.

Li Y, Wang S, Lin C, Frank G. Metaphor detection via explicit basic meanings modelling. 2023a. arXiv:2305.17268.

Li Y, Wang S, Lin C, Guerin F, Barrault L. Framebert: Conceptual metaphor detection with frame embedding learning. 2023b. arXiv:2302.04834.

Liu Y. Roberta: A robustly optimized bert pretraining approach. 2019. arXiv:1907.11692, 364.

Malgady Robert G, Johnson Michael G. Modifiers in metaphors: Effects of constituent phrase similarity on the interpretation of figurative sentences. J Psycholinguist Res. 1976;5:43–52.

Article  Google Scholar 

Mao Rui, Li Xiao, Ge Mengshi, Cambria Erik. Metapro: A computational metaphor processing model for text pre-processing. Inf Fusion. 2022;86:30–43.

Article  Google Scholar 

Milenković Katarina, Tasić Miloš, Stamenković Dušan. Influence of translation on perceived metaphor features: quality, aptness, metaphoricity, and familiarity. Linguist Vanguard. 2024;10(1):285–96.

Article  Google Scholar 

Miller George A. Wordnet: a lexical database for english. Commun ACM. 1995;38(11):39–41.

Article  Google Scholar 

Mohammad S, Shutova E, Turney P. Metaphor as a medium for emotion: An empirical study. In Proceedings of the fifth joint conference on lexical and computational semantics; 2016, p. 23–33.

Oka Ryunosuke, Kusumi Takashi. Number of topic-vehicle shared features influences the aptness of metaphors. J Cognit Psychol. 2022;34(7):819–32.

Article  Google Scholar 

Qiao W, Zhang P, Ma Z. A quantum-inspired matching network with linguistic theories for metaphor detection. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-cOLING 2024); 2024, p. 1435–45.

Roncero Carlos, de Almeida Roberto G. Semantic properties, aptness, familiarity, conventionality, and interpretive diversity scores for 84 metaphors and similes. Beh Res Methods. 2015;47:800–12.

Article  Google Scholar 

Sanh V, Debut L, Chaumond J, Wolf T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. 2019. arXiv:1910.01108.

Shutova E, Kiela D, Maillard J. Black holes and white rabbits: Metaphor identification with visual features. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Human language technologies; 2016, p. 160–70.

Stamenković Dušan, Milenković Katarina, Ichien Nicholas, Holyoak Keith J. An individual-differences approach to poetic metaphor: Impact of aptness and familiarity. Metaphor Symb. 2023;38(2):149–61.

Article  Google Scholar 

Steen GJ, Dorst AG, Herrmann JB, Kaal AA, Krennmayr T, Pasma T. A method for linguistic metaphor identification: From MIP to MIPVU. John Benjamins Publishing Company; 2010.

Thibodeau Paul H, Durgin Frank H. Metaphor aptness and conventionality: A processing fluency account. Metaphor Symbol. 2011;26(3):206–26.

Article  Google Scholar 

Tian Y, Zhang R, Xu N, Mao W. Bridging word-pair and token-level metaphor detection with explainable domain mining. In Proceedings of the 62nd annual meeting of the association for computational linguistics (Volume 1: Long Papers); 2024, p. 13311–25.

Tourangeau Roger, Sternberg Robert J. Aptness in metaphor. Cognit Psychol. 1981;13(1):27–55.

Article  Google Scholar 

Tsvetkov Y, Mukomel E, Gershman A. Cross-lingual metaphor detection using common semantic features. In Proceedings of the first workshop on metaphor in NLP; 2013, p. 45–51.

Uduehi OO, Bunescu RC. An expectation-realization model for metaphor detection. 2023. arXiv:2311.03963.

Wang Dian, Li Yang, Wang Suge, Chen Xin, Liao Jian, Li Deyu, et al. Ckemi: Concept knowledge enhanced metaphor identification framework. Inf Process Manag. 2025;62(1):103946.

Article  Google Scholar 

Wang S, Li Y, Lin C, Barrault L, Guerin F. Metaphor detection with effective context denoising. 2023. arXiv:2302.05611.

Zhang S, Liu Y. Metaphor detection via linguistics enhanced siamese network. In Proceedings of the 29th international conference on computational linguistics. 2022, p. 4149–59.

Zhang S, Liu Y. Adversarial multi-task learning for end-to-end metaphor detection. 2023. arXiv:2305.16638.

Comments (0)

No login
gif