If I'm not a Halfling, can I just ignore the Stoned Golem entirely?

Yep. It's like it's not even there. The Stoned Golem makes a bad Wandering Monster because the victims can just ignore it, unless one of the combatants is a Halfling. But if you don't kill it, you don't get its level and Treasures, even if you do kill all remaining monsters!