This article aims to describe the effectiveness of CIPP-based evaluation management (Context, Input, Process, Product) in Arabic vocabulary and communication mastery at Madrasah Aliyah Unggulan Nuris Jember. The research subjects were students and teachers of the Nuris Jember Madrasah Aliyah. This study used a qualitative research design with data collection techniques: in-depth interviews, document review, Focus Group Discussion (FGD), and (4) observation. Data analysis in this study used a qualitative descriptive analysis technique which consisted of the stages of data collection, data reduction, data presentation, and conclusion. The results of this study were obtained, namely; 1) "Context" evaluation management includes curriculum policies for extracurricular programs by integrating the pesantren curriculum with the K13 curriculum and supporting the madrasah environment and parental involvement. 2) the input component shows that the extracurricular program curriculum policy is consistent with graduation standards, there are Arabic language day activities, and all educators have a high professional level. 3) the evaluation dimension of the "process" of learning according to a predetermined schedule. Learning methods and media are selected by adjusting the material. Facilities and infrastructure are also used properly. 4) the results of the "product" evaluation focused on three aspects, namely cognitive, affective, and psychomotor aspects in the ability to master Arabic communication actively and the ability to master Arabic books without translation.