A COMPREHENSIVE FRAMEWORK FOR STROKE TRAJECTORY RECOVERY FOR UNCONSTRAINED HANDWRITTEN DOCUMENTS

For a long time, handwriting analysis, such as handwriting recognition and signature verification, has been an active research area. There are two categories of handwriting, online and offline. Online handwriting is captured in real-time on a digital device such as a tablet screen with a stylus pen. In contrast, the handwritten text scanned or captured by a camera from a physical medium such as paper is referred to as offline handwriting. For offline handwriting, the input is limited to handwritten images, making handwriting analysis much more difficult. In our work, we proposed a Stroke Trajectory Recover (STR) for offline and unconstrained handwritten documents. For this purpose, we introduce large-scale word-level annotations for the English handwriting sampled from the IAM-online dataset. The current STR architectures for English handwriting use lines of text or characters of the alphabet as input. However, a word-level STR method estimates loss for each word rather than averaging DTW loss over the entire line of text. Furthermore, to avoid the stray points/artifacts in predicted stroke points, we employ a marginal Chamfer distance that penalizes large, easily noticeable deviations and artifacts. For word detection, we propose the fusion of character region scores with bounding box estimation. Since the character level annotations are not available for handwritten text, we estimate the character region scores in a weakly supervised manner. Character region scores are estimated autonomously from the word’s bounding box estimation to learn the character level information in handwriting. We propose to fuse the character region scores and images to detect words in camera-captured handwriting images. We also propose an automated evaluation to check the quality of the predicted stroke trajectory. The existing handwriting datasets have limited availability of stroke coordinates information. Hence, although the proposed system can be applied to handwriting datasets without stroke coordinates information, it is impossible to evaluate the quality of its predicted strokes using the existing methods. Therefore, in our work, we propose two measures for evaluating the quality of recovered stroke trajectories when ground truth stroke information is not given. First, we formulated an automated evaluation measure based on image matching by finding the difference between original and rendered images. We also evaluated the preservation of readability of words for original and rendered images with a transformer-based word recognition network. Since our proposed STR system works with words, we demonstrate that our method is scalable to unconstrained handwritten documents, i.e., full-page text. Finally, we present a probabilistic diffusion model conditioned on handwriting style template for generating writing strokes. In our work, we propose to learn the localized patches for handwriting style features from multiscale attention network. The multiscale attention network captures fine details about local character style and global handwriting style. Moreover, we train our diffusion model with the Dynamic Time Warping (DTW) loss function, along with the diffusion loss, which eliminates the need to train any auxiliary networks for text or writer style recognition and adversarial networks.

Grantor dc:publisher

Temple University. Libraries

Year dc:date.issued

2024

Author dc:creator

• Hanif, Sidra

Advisor dc:contributor.advisor

• Latecki, Longin

Committee members dc:contributor.committeemember

• Souvenir, Richard M.
• MacNeil, Stephen
• Ling, Haibin
• Obeid, Iyad, 1975-

• Computer science
• Diffusion model
• Dynamic time warping
• Evaluation metrics
• Handwriting stroke generation
• Multi-scale style feature
• Projected shape matching s

Statement dc:rights

• IN COPYRIGHT- This Rights Statement can be used for an Item that is in copyright. Using this statement implies that the organization making this Item available has determined that the Item is in copyright and either is the rights-holder, has obtained permission from the rights-holder(s) to make their Work(s) available, or makes the Item available under an exception or limitation to copyright (including Fair Use) that entitles it to make the Item available.

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/20.500.12613/10279

OAI identifier oai:identifier

oai:scholarshare.temple.edu:20.500.12613/10279