Meg Tilly Hated Being Hit On in Hollywood: Why Her Quiet Life Now (on an Island!) Couldn't Be More Different

In the recent issue of PEOPLE, actress Meg Tilly, 56, opened up about what made her walk away from fame in her early 30s, opting instead to live a simple, mostly anonymous life in Canada to raise her three children.

“My mantra was that I wanted to get my kids safely to adulthood,” Tilly — who starred in acclaimed ’80s films like The Big Chill and Agnes of God — says about why she left the movie business. She instead devoted herself to her kids, taking care of them and their every need. “I don’t know if I did my son’s wives a disservice now, because the boys are like, ‘Oh moms, they make all the food and do all the laundry and all that,'” she says with a laugh about striving to be a picture-perfect parent.

But there were other reasons she didn’t want to stay in L.A. and make films anymore: She was sick of being hit on by sleazy men.

“Back then, the female lead was kind of the fire hydrant that all the men wanted to prove themselves on, so you’d get hit on a lot,” she says. “Not because they knew me or even wanted to know me. It was more a measuring post for other men. That was challenging sometimes. It was challenging dealing with the different egos and having to work with people after you’ve said, ‘No, I’m sorry, I really don’t want to.’ Then they’d still push you because they wanted to be the one to get the girl.”

She adds that while there were things she loved about acting — the travel, the immersion in her characters — she didn’t really like being famous.

“When I hear kids say things like, ‘I want to be famous,’ I tell them my story about being famous, and really needing to use the bathroom when I was working on the Jack Nicholson film [Two Jakes] … I was wearing girdles and they didn’t have toilets anywhere and I had to go. I finally get permission and ran down the hall and tried to get out of the damn girdle, and then someone started banging on the stall and slid 8×10 glossies and a Sharpie marker underneath it. I was like, ‘Not while I’m using the bathroom!’ It was horrible. So it’s not always great being famous.”

She muses, however, that actors today have it particularly tough, especially with the rise of social media. “It must be very hard to be in Brad Pitt‘s position,” she says of her costar in Netflix’s War Machine. “Especially with so much media around. It was nearly as intense then as it is now, being watched all the time.”

That doesn’t mean she doesn’t get recognized still. Even though for the last 18 years she’s been living on an island a two-hour ferry ride away from Vancouver with her writer husband Don Calame, writing books and taking on the occasional acting job, she’s recognized more often than not.

“People recognize me all the time, which really surprises me because I’m much older. Sometimes women come up to me and they’re like, ‘Oh my husband’s going to be so jealous — he had the biggest crush on you in The Big Chill!’ ” she says. “I think, well, if he saw me now … I’m a matron! I’m a middle-aged woman. But it’s still wonderful. Though I’m surprised by it.”

So did her children (Emily, 32, David, 30, and Will, 26) always know that they had a famous mom — one who walked away to put them first?

“My oldest daughter did because she’d come on sets, or we’d get tickets to Cirque du Soleil for free and things like that. But my youngest son Will had no idea. His dad is famous — he’s Colin Firth — so he knew about his dad’s fame and would go on sets with him. But I was just mom. I actually had a rule that nobody could watch my stuff until they were 18. A parent is already huge in a child’s life, but if your parent is also on a huge screen … it’s even harder.”

From Coinage: See Where 6 Stars Were Before They Were Famous

She says when Will finally saw her on set for the first time on the Canadian drama Bomb Girls, he actually just thought it was adorable.

“He’s like, ‘It’s just so cute and fun mom, to see you pretending to be an actress!’ “she says, laughing. “But I don’t even want them to feel like I gave up something and that I had regrets, because I don’t. I make the choice that I made because I needed to, and it was the best way I knew how to parent.”

Though acting was Tilly’s first profession, and one she says she’ll occasionally go back to if the project is right, she says now she’s content to stay on her little island in Canada, writing books. Her first romance novel was just released, and she’d like to write more, saying she’s done writing the “scary stuff.”

Ten years ago, she released her second novel Gemma, which spoke to her experience being physically, sexually and emotionally abused as a child and teen at the hands of male relatives and acquaintances. “I’m really proud of [the book], of being brave and standing tall in my truth,” she says of coming clean about her troubled childhood. “I’m proud of being a voice for people who don’t have one, and letting them know they aren’t alone.”