So far, the making of evaluation question for Arabic language lessons at MAS Ruhul Islam Anak Bangsa Aceh Besar has not been conducted in-depth qualitative research. The questions that tested to students were never through the maximum qualitative review stage. Thus, making of Arabic evaluation question in MAS Ruhul Islam Anak Bangsa Aceh Besar only relies on estimation and uniformity. So, the quality produced is unknown. Teher for, the aim of this research is to determine of the quality of the question based on the level of validity in term of evaluating Arabic lessons in MAS Ruhul Islam Anak Bangsa Aceh Besar. The question that was examined was made by Arabic Teacher in MAS Ruhul Islam Anak Bangsa Aceh Besar with 50 items. This research is quantitative research with evaluation research method. Data collection techniques used are documentation techniques, and interview techniques. The data analysis techniques used is descriptive statistical techniques with using the Anates V4.09 program. The result of this research in term of item validity show that the evaluation question on Arabic Subject in MAS Ruhul Islam Anak Bangsa Aceh Besar have 25 valid item (50%) with details of 20 items have very good signicance (very significant), 5 items have good significance (significant), 24 items have bad significance, and 1 item NaN or undefined.