Beyond Black-Box Mutation: Grounding GPT-4 Mutant Generation in Historical Bug-Fix Patterns

=

Nome do aluno Erlon Pereira Almeida
Título do trabalho Beyond Black-Box Mutation: Grounding GPT-4 Mutant Generation in Historical Bug-Fix Patterns
Resumo do trabalho Mutation testing evaluates the effectiveness of a test suite by introducing artificial faults (mutants) into the code. While valuable, this process is a time-consuming task for developers to perform manually. Traditional tools attempt to automate it, but often produce an abundance of redundant mutants, increasing computational costs without improving test effectiveness.
While large language models (LLMs), such as GPT-4, have demonstrated high performance on code-related tasks, their potential for generating meaningful mutants similar to real faults remains underexplored.
We evaluated GPT-4.1 using a dataset of 371 bug-fix code snippets in Java to: (1) identify modification patterns that revert fixed code back to buggy versions. We then intend to (2) use these modification patterns as mutation operators and guide GPT-4 in generating mutants resembling buggy code.
The contribution of this research lies in investigating the most effective approach to leveraging a language model for mutant generation, as well as assessing the associated risks and biases.
This includes evaluating various prompt techniques and developing a replicable workflow that can be applied to any other language model, serving both as a mutant generation framework and as a benchmark for comparison with other LLMs.
Orientador Eduardo Almeida
Membro Titular 1 (com afiliação) Rohit Gheyi (UFCG)
Link para o curriculum lattes http://lattes.cnpq.br/2931270888717344
Membro Titular 2 (com afiliação) Daniel Lucredio (UFSCar)
Link para o curriculum lattes http://lattes.cnpq.br/9090396559596221
Suplente 1 (com afiliação) Ivan Machado (UFBA)
Suplente 2 (com afiliação) Leonardo Murta (UFF)
Data do exame 06 Mar, 2026
Horário do exame 9:00 AM

 

Data da Defesa: 
06/03/2026 - 09:00
Tipo de Defesa: 
Qualificação de Mestrado