Your kid is an asshole

I know harsh but seriously this kid in one of Olive's little gym classes is such a little jerk. Like full on bully. I'm guessing she is a bit older than Olive by a few months but holy shit is this kid mean. The last few classes I've watched her be a jerk to Olive, not just Olive but other kids in the class too. I usually just pull Olive away from the situation as the mom is never watching. I know toddlers have their moments. Olive is no angel but she isn't a mean kid. She will occasionally steal a toy from a kid or do whatever toddlers do best but 99% of the time I am there to tell her to apologize or to explain to her what she did wrong. I'm not super mom by any means but it's my job to be there and try and correct bad behavior.
So back to this kid in class. Last week they were playing with balls and Olive had one in her hands and the bully came out of nowhere and grabbed the ball out of Olive's hand. I kid you not Olive screamed at the top of her lungs "NOOOO TAKING." The little shit dropped that ball so fast and I just stood there not sure if I should laugh or grab her. She used her words just like she should. Later I explained to her how it was mean that the girl took her ball and bla bla bla. I felt proud how she handled herself. Today pushed me over the edge. Again Olive was playing with some pad things and the girl straight up walked up and grabbed them from her. The mom again wasn't paying attention. Normally I would tell Olive to share but this kid grabbed and was mean. Olive just looked at me and because I wasn't about to grab them back from a 2 year old I had to explain to Olive that it was ok and some kids are just mean. Obviously not a big deal but the fact that the mom wasn't there to help control her kid pissed me off. I blew it off,  I mean this is normal toddler stuff but then the kid hit Olive and I was pissed. The little shit was on one of the bars and Olive climbed up next to her  and the girl freaked out and started hitting her over and over. I ran over and grabbed Olive and sat her down and again had to explain to her that people are just assholes. The mom had her back turned of course and I'm so pissed I didn't say anything. I'm not confrontational at all like not one bit but it's been bothering me all night. The kid hit my kid and not like a silly push but like actually hit her. I feel like I failed. But what do you do in this situation? I am promising myself that next class I will speak up to the mom even if it takes everything in me. I mean if she isn't going to parent her kid then I will and I have a mouth a sailor so it's not going to be pretty.
As much as I want Olive to be polite and respectful I want her to speak up and defend herself. Obviously this is a silly situation but even so I couldn't stick up for me kid and I feel like I kinda failed. Big changes people............