Assessing the Arabic Parsing Capabilities of ChatGPT and Cloude: An Expert-Based Comparative Study

Abstract

Arabic, known for its rich morphological and syntactic complexity, poses significant challenges for natural language processing technologies. This study presents a comparative evaluation of two artificial intelligence tools, ChatGPT and Cloude, in their abilities to parse Arabic sentences accurately. Five sentences embodying diverse linguistic features were selected, and the parsing outputs were evaluated by three Arabic language experts. The results revealed a clear performance gap, with Cloude outperforming ChatGPT in overall accuracy (72.9% vs. 33.3%). Cloude excelled in handling morphological complexities and basic syntactic relationships but encountered difficulties with highly idiomatic expressions and ambiguous constructions. Conversely, ChatGPT struggled with complex morphology and long-distance dependencies. The findings highlight the importance of developing specialized Arabic language processing tools while acknowledging the potential of general-purpose language models with further fine-tuning. Recommendations include incorporating domain-specific knowledge, leveraging transfer learning, and exploring ensemble approaches to enhance the accuracy and robustness of AI-based Arabic parsing systems.