Purpose To enhance the transparency of the supply chain and ensure proper dissemination of information among the supply chain members in a timely manner, more and more companies are implementing supply chain information system (SCIS). Often the challenge among the organizations is how to go for a proper SCIS implementation and to identify the key performance indicators (KPIs) to evaluate the SCIS. The purpose of this paper is to provide a framework to evaluate the KPIs for SCIS of SCISs implementation from user’s perspectives. Design/methodology/approach In this paper, 16 KPIs were identified based on extensive literature survey. A fuzzy analytic hierarchy process (AHP) model is constructed to measure the users’ perceived importance and satisfaction for the KPIs. Subsequently, based on these two measurements, an importance-performance analysis (IPA) model along with a customer satisfaction attitude (SA) index is developed to categorize and prioritize the KPIs. As an empirical study, SCIS users across five industries belonging to different sectors were investigated to validate the model. Findings An IPA model along with a customer SA index is developed based on a fuzzy AHP model to evaluate the KPIs and provide the priorities of their improvement. Based on this result, some management implications and suggestions are proposed. Research limitations/implications The study was limited to five organizations. More representative samples which can be sector specific can ensure better confirmation of the empirical results. Originality/value The KPIs identified in the research indicate the nature and dynamics of a complex SCIS implementation. It can serve as a checklist of areas that require attention when implementing a SCIS. The KPIs are presented through grouping in a systemic way. The development of the SAs in IPA model using fuzzy AHP is a novel approach.