Natural language prompt optimization, or prompt engineering, has emerged as a
powerful technique to unlock the potential of Large Language Models (LLMs) for various tasks. While existing methods primarily focus on maximizing a single task-specific performance metric for LLM outputs, real-world applications
often require considering trade-offs between multiple objectives.

In a recent paper entitled “Pareto Prompt Optimization“, we proposed an effective technique for multi-objective prompt optimization for LLMs to address this limitation. Specifically, the proposed method called ParetoPrompt, takes a reinforcement learning (RL) approach that leverages dominance relationships between prompts to derive an effective policy model for prompt optimization using preference-based loss functions. By leveraging multi-objective dominance relationships, ParetoPrompt enables efficient exploration of the entire Pareto front without the need for a predefined (and typically heuristic) scalarization of multiple objectives. Experimental results show that ParetoPrompt consistently outperforms existing prompt optimization techniques on various benchmarks. Furthermore, ParetoPrompt demonstrates robust performance when the objective metrics differ between training and testing.

Details of the work can be found at the following link:

Guang Zhao, Byung-Jun Yoon, Gilchan Park, Shantenu Jha, Shinjae Yoo, Xiaoning Qian, “Pareto Prompt Optimization,” 13th International Conference on Learning Representations (ICLR), Singapore, Apr 24-28, 2025.