Invention Title:

ELEMENT LOCALIZATION FOR ASSISTED DOCUMENT EDITING

Publication number:

US20260330123

Publication date:
Section:

Physics

Class:

G06F40/166

Inventors:

Assignee:

Applicant:

Smart overview of the Invention

Element localization for assisted document editing involves using AI to modify specific document elements based on natural language instructions. This process is challenging in documents with hierarchical structures, where elements might be nested or have complex spatial relationships. Conventional AI systems often struggle with accurately localizing these elements, particularly in text-heavy documents. Specialized models may lack the advanced language understanding of large language models, while general-purpose models may not handle document structures effectively.

Machine-Learning Model

The described techniques utilize a machine-learning model that combines the strengths of multimodal large language models with domain-specific training data. This model is trained to improve spatial localization capabilities by generating synthetic instruction-tuning data from hierarchical document representations. A reward function, incorporating intersection over union (IOU) and regression components, enhances bounding box accuracy for document elements. This training process enables the model to better interpret and localize elements based on user edit requests.

Editing Process

Upon receiving an edit request, the system uses an image of the document and a description of the object to localize the target region. The machine-learning model, trained to understand spatial and semantic dependencies, identifies the relevant document section. It then performs the requested edit, generating an updated document that incorporates the edited region. This approach eliminates the need for users to manually locate and select document elements, streamlining the editing process.

Implementation

The system can be implemented on various computing devices, from desktops to mobile phones, and integrated into different user interfaces, such as chat or voice interfaces. The editing module processes digital content, receiving input that references a document and an edit request. This module can perform a variety of edits, including text modifications, image adjustments, and formatting changes, based on natural language instructions.

Applications

These techniques apply to various document types, including PDFs and design templates, enhancing AI-assisted document editing's accuracy and efficiency. The system's ability to localize document elements precisely makes it suitable for complex editing tasks across different digital environments. By leveraging advanced language models and domain-specific training, the system bridges the gap between language understanding and spatial awareness in document editing.