Group-Relative Attention Guidance for Image Editing
Abstract
Recently, image editing based on Diffusion-in-Transformer (DiT) models has undergone rapid development. However, existing editing methods often lack effective control over the degree of editing, limiting their ability to achieve more customized results. To address this limitation, we investigate the MM-Attention mechanism within the DiT model and observe that the Query (Q) and Key (K) tokens share a bias vector that is only layer-dependent. We interpret this bias as representing the model's inherent editing behavior, while the delta between each token and its corresponding bias encodes the content-specific editing signals. Based on this insight, we propose Group Relative Attention Guidance (GRAG), a simple yet effective method that reweights the delta values of different tokens to modulate the focus of the model on the input image relative to the editing instruction, enabling continuous and fine-grained control over editing intensity without any tuning. Extensive experiments conducted on existing image editing frameworks demonstrate that GRAG can be integrated with as few as four lines of code, consistently enhancing editing quality. Moreover, compared to the commonly used Classifier-Free Guidance, GRAG achieves smoother and more precise control over the degree of editing.
Method
The variations between individual token embeddings and the bias vector reflect how the editing content relates to the current layer's editing action. By modulating their relative relationship, it becomes possible to achieve accurate and continuous control over the editing instructions. Based on this insight, we propose Group Relative Attention Guidance (GRAG). As illustrated in the blow, we modify the cross-attention component of the MM-Attention corresponding to the query Qe. Ks is selected as a group of tokens, to which group-relative modulation is applied. An illustration of applying Group Relative Attention Guidance in the MM-DiT image editing model.
