The controversy of the Prophet Muhammad's caricature in Charlie Hebdo magazine resulted in Muslim communities’ protests from various parts of the world, including Indonesia. In response to this religious harassment, various steps were taken, including boycotting French products and criticizing secularism, which has been the constitutional principle of French. This article aimed at providing another perspective in viewing the controversy by looking back at the point where this problem began, the caricature of the Prophet Muhammad. This study sees that caricatures are not only inanimate objects, but they also have agency. To be able to see this agency in the particularity of space and time, we rely on semiotic ideology as the theoretical framework. Semiotic ideology is an assumption of signs that see various possibilities in signs, including their materiality. The Charlie Hebdo controversy is a clash of semiotic ideologies formed by the particularity of space and time and manifested in the responses shown in the face of the Charlie Hebdo controversy. In Indonesia, this semiotic ideology was born from political polarization, religious resurgence, and conservative turn. We see that the analysis of icon and materiality still have not really taken any significant part of religious studies and Islamic studies in Indonesia, including in studying the Charlie Hebdo controversy.