PROMPT-BASED IMAGE RELIGHTING AND EDITING

Smart overview of the Invention

The patent describes techniques for prompt-based image relighting and editing, enabling automatic generation of edited digital images with realistic lighting and background features. A processing device takes a digital image, a lighting prompt, and a background prompt as inputs. It then generates a relit digital object with lighting conditions specified by the lighting prompt and a background that includes features specified by the background prompt. The final output is an edited digital object that includes both the relit object and the background, with enhanced shadow realism through a shadow synthesis model.

Background

Traditional image processing systems require manual editing to apply visual effects, which is time-consuming and expertise-dependent. Machine learning has automated some aspects but struggles with image fidelity and realism. Conventional techniques often result in unrealistic images due to mismatched lighting between foreground objects and backgrounds or require multiple input images, which is computationally expensive and may distort fine details.

Summary

The described techniques address these challenges by using a processing device that receives a digital image with prompts specifying lighting and background conditions. The device generates a relit digital object by removing the original background and applying the specified lighting condition using a histogram matching approach to maintain content details. It then creates a background that respects the relit object's lighting conditions, integrating both elements into an edited digital image.

Detailed Description

Conventional methods often fail to maintain consistency in lighting between objects and backgrounds or require impractical computational resources. The new system extracts the digital object from its original background, applies AI-based diffusion models to simulate desired lighting conditions, and uses histogram matching to correct any visual artifacts. This ensures high fidelity of the digital object while integrating it seamlessly into new backgrounds.

Conclusion

By generating backgrounds that align with the relit object's lighting conditions, this system overcomes conventional limitations of mismatched lighting in edited images. It maintains fine details such as textures and logos without distortion. Additionally, shadow synthesis enhances realism in the final output, supporting automatic creation of high-fidelity images with consistent lighting, backgrounds, and shadows.