Inquiry Regarding Arabic and Urdu Text Support in SAM Model

[hussnainmuavia]

I’ve been experimenting with the Hi-SAM model for text-based segmentation and had a few questions regarding its use with non-Latin scripts, specifically Arabic and Urdu.

Language Support:
Is it possible to use the Hi-SAM model effectively with Arabic or Urdu text, especially when using their respective fonts? If so, are there any specific considerations or settings that should be adjusted?

Masking Issue with Tall Glyphs:
While testing, I noticed that certain Arabic and Urdu characters—especially those with tall ascenders or descenders—extend beyond the typical text line boundaries. As a result, when applying a mask, these portions often get cropped, leaving behind unwanted noise. Do you have any recommendations for handling such cases?

Annotation in JSON Format:
Finally, could you guide me on how to generate annotations (e.g., bounding boxes or masks) in JSON format for Arabic or Urdu text in images?

Any suggestions, tips, or documentation links would be greatly appreciated. Thank you in advance for your time and support!

[ymy-k]

Hi, the model may not perform well on unseen script types (it was only trained on English data). If you have Arabic or Urdu text training data, it would be better. No specific settings should be adjusted.

For the tall glyphs, try to use some external detection models to get more accurate bboxes?

For JSON format output, refer to the evaluation part README where tells how to save the hierarchical results in JSON format. As for stroke/glyph segmentation, you can transfer the mask to RLE (run-length encoding) format.

[hussnainmuavia]

Hi Ymy-K/Hi-Sam Thank you for your quick and helpful reply! I have a few follow-up questions to better understand and apply your suggestions: 1. Training on Arabic/Urdu: You mentioned that the model may not perform well on unseen scripts and that having Arabic or Urdu training data would help. Could you kindly guide me on how to fine-tune or train the model on my own dataset for these languages? Are there any specific steps or tools you recommend for this? 2. External Detection Models: Regarding the tall glyphs issue, do you have any suggestions for external detection models that work well with Arabic or Urdu scripts? If you’ve seen any successful approaches or papers, I'd appreciate some pointers. 3. JSON & RLE Output: I found the README section you referred to—thanks for that! One question: when converting masks to RLE format for glyph segmentation, is there any standard or library you recommend (e.g., COCO API)? I just want to ensure the format is compatible with your model’s expected inputs/outputs. Thanks again for your support—it’s been incredibly helpful. Best regards

…

On Fri, Apr 18, 2025, 4:31 AM Maoyuan Ye ***@***.***> wrote: Hi, the model may not perform well on unseen script types (it was only trained on English data). If you have Arabic or Urdu text training data, it would be better. No specific settings should be adjusted. For the tall glyphs, try to use some external detection models to get more accurate bboxes? For JSON format output, refer to the evaluation part README where tells how to save the hierarchical results in JSON format. As for stroke/glyph segmentation, you can transfer the mask to RLE (run-length encoding) format. — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADL3YPEDZQSCW47VJCECF6L22BPXLAVCNFSM6AAAAAB3IM4VKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJUGM3TQNJUHA> . You are receiving this because you authored the thread.Message ID: ***@***.***> *ymy-k* left a comment (ymy-k/Hi-SAM#28) <#28 (comment)> Hi, the model may not perform well on unseen script types (it was only trained on English data). If you have Arabic or Urdu text training data, it would be better. No specific settings should be adjusted. For the tall glyphs, try to use some external detection models to get more accurate bboxes? For JSON format output, refer to the evaluation part README where tells how to save the hierarchical results in JSON format. As for stroke/glyph segmentation, you can transfer the mask to RLE (run-length encoding) format. — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADL3YPEDZQSCW47VJCECF6L22BPXLAVCNFSM6AAAAAB3IM4VKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMJUGM3TQNJUHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>