Willis' mother had left the family six years earlier. His father was blowing Willis' earnings on alcohol and drugs.
By 2007, when the 49ers made Willis the 11th player chosen in the NFL draft, Willis and his siblings were estranged from their father. They had lost a brother, Detris, to a drowning accident a year earlier.
Yeah, you could say Willis grew up tough. You could say his life experiences, detailed in Jeff Chadiha's piece for "E:60" and more recently in The New York Times, helped make Willis tough enough for 20 Hall of Famers to choose him for ESPN's Any Era team.
Just don't think Willis is looking for sympathy.
"It was sad because we all wished we could have that 'Little House on the Prairie' type family, or that good family where mom and dad are home, both parents are good and the family is happy -- just a happy household, however that may be," Willis said. "But for what it's worth, it is what helped me become the person I am today, helped make me the man I am today, and really helped my family -- my brothers and sisters, too -- just be the men and women that they are today as well."
Willis freely reflects on the hardships he endured as a child, including physical and emotional abuse from his father, because he hopes others enduring similar experiences might find inspiration in his story. He participated in the "E:60" project knowing aspects of it would reflect poorly on his father in particular. But he felt the broader message made the story worth telling.
"If one person can take something from it and better themselves, then the public was well served," Willis said.
Sympathy for Willis is probably misplaced. He doesn't want any. On-field opponents have sometimes needed some.
"Patrick Willis knocks the crap out of guys," Mike Ditka, one of the Hall of Fame panelists, said for the Any Era project.
In Week 1 this season, Willis welcomed Seahawks rookie tackle James Carpenter to the league with a collision violent enough to send Carpenter's helmet flying.
Adam Snyder, the 49ers' starting right guard, has been on the wrong end of those collisions in practice. He said the impact from a head-on collision with Willis can be great enough to stun an opponent, forcing him to gather himself.
"I can remember one against Steven Jackson," Snyder said. "He got up underneath him and it was perfect. That is the kind of game Pat plays. You could call it perfect football: takes on linemen how you are supposed to, covers backs and tight ends how you are supposed to and blitzes hard. He can do anything they ask him to do."
Willis has five Pro Bowls and five All-Pro selections (one was second team) in his first five NFL seasons. He has played through injuries, including a broken hand that became displaced so badly in 2010 that doctors told Willis he could not keep playing.
"I have endured a lot of pain early on in my life and as a kid and just growing up, whether it was wishing I had something I didn't have, wishing I had a happy home, wishing I could have team basketball shoes and not have to wear Wal-Mart shoes, or just wishing I could have electric heat instead of a wood stove, just wishing I could be a normal kid," Willis said. "But after a while, you learn to stop wishing and just make do with what you have, make the best of what you have.
"I think that is what I think about the game," he said. "We all wish we could be a little bit bigger, a little bit stronger or a little bit faster, but at the end of the day, you live with who you are and work with what you have. And so for me, I have just learned to be content with who I am as a man, who I am as a person and with who I am as a player. When I say content, that doesn't mean complacent. That means I know who I am, I know what type of player I am and I know what I can bring to the game that I play."