Applications of Prompt Engineering for Large Language Models (LLMs) on Training of Software Engineering Tasks: A Systematic Review

Abstract

This study aimed to conduct a systematic review to identify the large language models (LLMs) that have been employed in teaching and automating software engineering tasks, analyze their research trends, identify prompts engineering techniques to improve the outputs of large language models in automating software engineering tasks, and identify the most prominent challenges of their application. Due to the novelty of the field, the review targeted all studies published in 2023 in the Web of Science database and the search engine (Google Scholar), and by following the (Kitchenham) guidelines for systematic review, and summarizing the results according to the reference list and the (PRISMA) scheme, (35) papers that met the inclusion criteria were identified, and through thematic analysis of the studies, the study concluded that the most widely used large language models in automating software engineering tasks are arranged in ascending order as follows: (ChatGPT, GPT 3.5, Codex, GPT4, Copilot), and the programming task is one of the most research trends for applications of Large language models, followed by the design task, then the task of software testing and maintenance, and that the basic prompt technique (Zero-Shot, Few-Shot) is one of the most widely used prompt engineering techniques