This study proposes the frameworks (A-HAR-AddRS and HAR-AddRS-AGARCH) to account for leverage effect in modeling and forecasting the AddRS estimator (Kumar and Maheswaran, Econ Model 38:33–44, 2014b) based on heterogeneous autoregressive (HAR) model. We evaluate the forecasting performance of the A-HAR-AddRS and HAR-AddRS-AGARCH models using the error statistic approach, the superior predictive ability (SPA) approach and the model confidence set (MCS) approach and compare the results with the corresponding results from the return based asymmetric and regime switching volatility models. To illustrate it, we use the same indices as used by Kumar and Maheswaran (Int Rev Financ Anal 34:166–176, 2014a, Econ Model 38:33–44, 2014b), that is, S&P 500, CAC 40, IBOVESPA and S&P CNX Nifty. Our findings indicate that the A-HAR-AddRS and HAR-AddRS-AGARCH models provide more accurate forecasts of realized volatility than the returns based asymmetric and regime switching volatility models.