Natasha Johnson

Though the concept of genre has been a subject of discussion for millennia, the relatively recent emergence of genre fiction has added a new layer to this ongoing conversation. While more traditional perspectives on genre have emphasized form, contemporary scholarship has invoked both formal and institutional characteristics in its taxonomy of genre, genre fiction, and literary fiction. This project uses computational methods to explore the soundness of genre as a formal designation as opposed to an institutional one. Pulling from Andrew Piper’s CONLIT dataset of Contemporary Literature, we assemble a corpus of literary and genre fiction, with the latter category containing romance, mystery, and science fiction novels. We use Welch’s ANOVA to compare the distribution of narrative features according to author gender within each genre and within genre versus literary fiction. Then, we use logistic regression to model the effect that each feature has on literary classification and to measure how author gender moderates these effects. Finally, we analyze stylistic and semantic vector representations of our genre categories to understand the importance of form and content in literary classification. This project finds statistically significant formal markers of each literary category and illustrates how female authorship narrows and blurs the target for achieving literary status.

As language models become capable of processing increasingly long and complex texts, there has been growing interest in their application within computational literary studies. However, evaluating the usefulness of these models for such tasks remains challenging due to the cost of fine-grained annotation for long-form texts and the data contamination concerns inherent in using public-domain literature. Current embedding similarity datasets are not suitable for evaluating literary-domain tasks because of a focus on coarse-grained similarity and primarily on very short text. We assemble and release FicSim, a dataset of long-form, recently written fiction, including scores along 12 axes of similarity informed by author-produced metadata and validated by digital humanities scholars. We evaluate a suite of embedding models on this task, demonstrating a tendency across models to focus on surface-level features over semantic categories that would be useful for computational literary studies tasks. Throughout our data-collection process, we prioritize author agency and rely on continual, informed author consent.

Through technologies such as ChatGPT, generative AI has become increasingly accessible over the past few years, which has sparked concerns regarding the role that generative AI will inevitably begin to play in education. Many scholars have explored the broad affordances and limitations of ChatGPT in teaching and learning: one of its key strengths is its personal and interactive nature, while one of its most significant shortcomings is its tendency to share inaccurate information. However, little research has been done on ChatGPT’s capabilities within specific fields, such as DH. This project explores the role that ChatGPT can or should play in Digital Humanities education, particularly in helping more traditionally trained humanists understand and utilize computational methods. We break down the curriculum in Carnegie Mellon University’s Computing for Humanists class and identify the key concepts covered, which include topics such as data preparation, feature counting, and the translation of humanistic research questions into computational steps. We then test ChatGPT’s mastery of these concepts by asking it to complete or assist with class assignments in various capacities. We find that ChatGPT excels at simple computational tasks and at suggesting ways in which specific humanistic research questions can be explored computationally. However, when writing more complex functions, ChatGPT often uses out-of-date libraries and struggles to properly connect different parts of the code. Based on these results, we present the baseline level of computational education that we believe digital humanists ought to possess prior to using ChatGPT as a research tool, while also highlighting the ways in which digital humanists can be taught to use ChatGPT to help implement computational methods.