Where's the Line?

So today, we're at this public play space at a local mall; some things to climb on, an open house-like structure in the middle with a tunnel and slide (for the kids, of course!) and a lot of children and energy, being it a Saturday.  There were 3 or 4 older boys (older meaning 4 or 5) who were running around chasing each other, seemingly uncontrollable with no adults visibly interacting with them.  I said to my wife, "I wonder if their parents just dropped them off here, I've seen no adult interaction with these kids whatsoever!"  This went on for a while.  Other kids got knocked over, adults (myself included) got ran into and toward the end, these boys began getting nearly physically aggressive with one another.

Once a guardian type was outed, in this case a grandfather with perhaps an Italian accent, one other young father approached him after his little girl was knocked over by these boys.  All this really led to was the man yelling the boys name across long distances every once in a while and the boy ignoring him.  I had to speak up on two occasions; once in the house while my daughter was nearby warning the boys to be careful and watch out for other kids, the second time I feel like I was almost breaking up a fight.  The "grandchild" was holding on to another boy somewhat aggressively right next to me, to which I spoke up and said that everyone should keep their hands to themselves.  I later identified the other boys father, who was nearby when this physical interaction was taking place, to which I thought "Why aren't you intervening here, dad?  Your son is being physically handled by another wild child in the play area!"

So where is that line, when it become necessary to intervene with other people's children when the parent's are most likely around themselves?  In hindsight, I probably could have and maybe should have spoken up on more occasions and earlier than I had, but where is that line?  When you know parents are around but aren't taking actions to control their children, when do you step in?  You know it's going to identify the parents rather quickly (although after I interjected, I surprisingly did not learn who any other of these boys' parents were) and maybe offend them?  How would I react if another parent chose to be involved in disciplining my child in my presence amongst strangers?  Perhaps it was this thought that held me back longer than it should have been.  Maybe I shouldn't have acted at all?  What do you think, dads?