Polypropylene (PP)/polystyrene-block-poly(ethylene-co-butylenes)-block-polystyrene (SEBS)/organo-montmorillonite (OMMT) nanocomposites of varying concentrations of maleic anhydride-grafted polypropylene (PP-g-MA) were prepared by continuous mixing assisted by ultrasonic oscillation. The structure and morphology of nanocomposites were investigated by X-ray diffraction (XRD), transmission electron microscopy, and scanning electron microscopy. It was found that both PP-g-MA and ultrasonic oscillation could enhance the intercalation and exfoliation of OMMT in PP matrix. Meanwhile, the formation of PP could be induced by ultrasonic irradiation at a power of more than 540 W. Rheological properties including complex viscosity, storage, and loss modulus of nanocomposites were increased after adding PP-g-MA or ultrasonic treatment. The results of mechanical properties showed that PP-g-MA could improve the tensile strength and tensile modulus of nanocomposites, but with the sacrifice of impact strength. This problem could be improved by ultrasound due to the reduced particle size of SEBS. However, the mechanical properties would be reduced by ultrasonic treatment with higher intensity due to the polymer degradation. Therefore, the synergistic effect of both compatibilizer and ultrasound should account for the balance between toughness and stiffness of PP/SEBS/OMMT ternary nanocomposites. © 2014 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2014, 131, 41202.