He would raise one child after another, sending them out into the world to make their place and raise their own. There was a social worker. A high school administrator. A postal worker. A fish market owner.
And his third and last son, one of the most highly regarded assistant football coaches in America, a humble yet proud man who also opens his home to all possibilities.
Decades ago in Williamston, N.C., Herman Johnson lost his wife and was suddenly staring at a world with six young children.
How would he possibly feed and teach and protect all of them while grieving for their mother?
Soon enough, relatives showed up to take this child or that one away, to ease his burden and give the kids a two-parent home and a so-called "better life."
But Herman Johnson refused to let even one go. Keeping his family together helped him push on and survive better than anything else ever could.
So his oldest girls, barely teenagers themselves, helped look after everyone, helped the family stay strong even when the Johnsons' baby girl died with heart problems not long after their mother.
Eventually, the two youngest boys, Larry and Aly Khan, became their father's focus.
He hardened their hands with manual labor, softened their hearts with patience and acceptance and, above all, made them understand that family comes first.
Always, he would say, make the Johnson name proud.
And how could they not, these lessons burned into them, forever molding them?
It is why this Penn State football coach needs to go about quietly changing one life after another.
Larry Johnson's reputation seems to grow in large doses each year now.
Though a successful high school head coach when he arrived at Penn State in 1996, he had no college or professional experience -- and no ties to the university, beyond working some summer football camps.
And it wasn't really until the past five years that his ability to recruit, develop defensive linemen for the NFL and serve as a father figure to his players began to be recognized beyond State College. It all reached a flattering, nervous high this past winter when Johnson was offered a defensive coordinator promotion and a suspected significant salary increase to join Ron Zook's staff at Illinois.
In the end, the timing and the place weren't right. He stayed at Penn State, pleasantly surprising supporters.
Meanwhile, his successes have been so significant that he is now mentioned among the favorites to land the Nittany Lions' head job whenever Joe Paterno retires.
"Larry's not out for Larry," Zook said. "Larry's out for the program, Larry's out for the kids. He's got all the right things in perspective. He doesn't beat his chest. He lets his work show for itself.
"There's no question in my mind we went after the right guy. ... I think one of the things you're going to see in the not-so-distant future is him running his own program, or at least his own defense."
For now, Johnson, who turned 58 last week, is patient enough being a position coach, though arguably the top defensive line coach in the nation.
Go back to that father and that town and that family to understand how it all came to be.
Herman Johnson was military tough, and maybe that's what helped him hold together after his wife, Maria, died from complications from kidney failure when their youngest boys were just starting grade school.
He did whatever he needed to support his family, grinding feed at a mill, doing auto body work, raising hogs and cows and growing tobacco and peanuts.
All of the Johnson children helped with the work, but none of them probably did more than Larry and Aly Khan.
When railcars full of lime fertilizer rolled into town, the boys helped their father shovel them clean for hours at a time after school.
When the tobacco needed picked, they were up at 5:30 each morning and in the fields, always bending and straightening as they broke off the stalks by hand. They worked until sunset, earning maybe $40 a week and giving most of it to their father.
But as hard as Herman Johnson pushed, he balanced that with a softness and a caring way. He seemed to get along with everyone, teaching his children not to become angry and bitter about the continued segregation and discrimination in the 1960s South that saw blacks still using one water fountain and whites another.
Embrace the good in others, he would preach. Pull so hard at the positive that all of the ugliness eventually fades away.
So how about that time when Larry was 13 or 14 and walking home from the movies with his brother and friends? The time a car load of white men pulled up close, spewing hate and then emptying their guns into the night air?
"The next thing you know you hear this, 'Pop, pop, pop, pop, pop,'" Larry Johnson said. "So we duck in the woods and we duck behind houses and we're running and hiding. And we're just laying on the ground for like 45 minutes. I swear we didn't move. And then we started yelling for each other to make sure everybody was OK.
"I remember walking home crying ... I was scared to death."
Or how about that all-star football game in high school, his first time lining up against white players? Larry set up both of his team's touchdowns by forcing fumbles and was the easy choice as defensive MVP.
White players, though, were picked for the awards and received the kisses from the beauty queen. Larry felt the anger and sadness tear through him as he walked away, head down, helmet in hand, barely even hearing the apologies of teammates and coaches.
That's when Herman Johnson threw his arms around his son, reminded him that he had won over everyone at that game, had become one of the first black football players to earn a scholarship offer to East Carolina University.
"My dad looked at me and said, 'You did a great job, son. You already won the award.'
"It took me years to figure out what that meant. It meant that I had crossed a barrier not many people had gone through. That was one of the defining moments of my life. ... One of the landmarks in my life that allow me to be who I am today."
Football brought everything together, it seemed.
The Johnson brothers won a state title for legendary coach Herman Boone, who became famous for being portrayed by Denzel Washington in the 2000 movie, "Remember the Titans."
The brothers played next to each other on the defensive line in high school and then roomed and played together at Elizabeth City State, 65 miles from Williamston. Herman Johnson ordered Larry to bypass that big scholarship to ECU and join his brother in college.
Family, once again, came first.
Larry could only seethe for so long at the decision. He grew to love his school and his team and realized that his father had shaped him a little more.
"When his father spoke, rats stood up in the corner and paid attention," Boone said recently. "Theses kids came along at a time when black kids stayed out of school in the fall and picked cotton. Herman Johnson said, 'My children ain't picking no cotton. My children are going to school and they're going to play.'"
Despite coming out of small Elizabeth City, Larry still got a look by the Washington Redskins.
He got cut and didn't even try other options. He knew he wanted to coach.
Soon enough he landed the head job at McDonough High in Pomfret, Md., where he eventually won three state titles. He became a folk hero of sorts in the area, not just for winning but for how deeply his players respected him, how they would have run wind sprints and hit tackle sleds until dawn, if that's what he wanted them to do.
He stayed for 16 years before moving to T.C. Williams High in Alexandria, Va., the place where Boone had worked his movie magic. But when Johnson arrived years later the program was a mess, the commute to work was draining and the fans were merciless when the victories didn't pile up right away.
He got out of coaching only to have lighting strike: Penn State defensive coordinator Jerry Sandusky called out of nowhere to ask if he was interested in a position on the staff.
He met with Paterno, accepted the job in early spring and began coaching special teams and defensive ends. Four years later he was in charge of the entire defensive line, his love, and he had a group of players who were all his own.
Meanwhile, he was reaching back into southern Maryland and D.C. and northern Virginia, pulling out the best recruits, all of them looking to him for guidance.
He could do it all, it seemed.
He still cuts an intimidating figure, a bear of a man. But he and his wife, Christine, are kind and gentle hosts, constantly opening their doors to any player who could use a plate of Larry's killer barbecued ribs or a couch to watch a movie on a Friday night -- someone who just needs a family.
He is a laid-back, comforting counselor in his office but also is renowned for his fiery pregame sermons, "guys shaking as they hear him speak."
He is as meticulous with his office organization as with his clothes, known on the staff for pulling from a closet full of fine suits and ties and perfectly pressed white shirts.
More importantly, he's often the first one players seek out to open up their hearts to about girlfriends and grade problems and brushes with the law.
"Everybody just doesn't have that kind of charisma. It's a God-given talent," his wife said. "Everybody can't just walk up to you and give you a warm and fuzzy. ... It's something you can't see, you can't teach out of a text book."
"His passion just flows through everything," said Brian Norwood, who coached defensive backs alongside him at Penn State. "I felt fired up just listening to Larry tell a kid how he wants him to play d-line for him."
So the troublemakers as well as the straight-and-narrow heroes, they all still regularly call to check in after leaving Penn State.
Take Anthony Adams, one of Johnson's defensive tackles who saw his grades plummet and his much-needed weight melt away after joining a fraternity.
"Sophomore year he was out of here, gone. He was not going to survive," Johnson said simply.
He confronted Adams and wrapped him up in tough-love and straight-talk. Not only did it work, not only did he become one of the best run-stuffers in the Big Ten, he still comes back every summer from the Chicago Bears to train with Johnson and hang out with his family.
It all goes back to one day after practice.
"I was the last guy walking off the field and Coach J sees me and says, 'Hey, man, you're toying with greatness right now. You can play in the NFL, but you have to make sure you take care of business, take care of the classroom.'
"The funny thing is, he could have just kept walking to his car and said, 'See you tomorrow.' He doesn't even know how he changed me.
"He cares about you" Adams said, "and it has nothing to do with football."
Not all of his souls are saved but enough are. And no matter who he pulls out of the projects or the suburbs, most all of them are transformed into standouts on the field.
"I think he's great at knowing when to rip you apart and when to build you up," said Jared Odrick, a senior defensive tackle. "He's a brutally honest dude."
"If you care about people and they know you care, you can reach people," Johnson said. "Really, that's what it's all about. I know them not just as a college football players but for the person they are. ... I'm always trying to find ways outside of football to reach a guy, to get him to perform better."
The thing is, the saving and teaching are never finished, not when three defensive linemen leave school in the past year because of disciplinary reasons, not even with his own sons. From Tony Johnson's DUI charge to Larry Jr.'s assault run-ins while playing for the Kansas City Chiefs.
"Are there nights you go home and cry? No doubt about it," said the father and coach. "I want to be in their hearts and souls to make a difference in their lives. My success is if I can make a difference in people's lives. If I can make a difference in your life, you're going to make a difference in somebody else's life. What I give in, you're going to give out."
The e-mail came by the dozens, then by the hundreds.
So many kind words, thanking him for staying at Penn State this winter, so much outpouring that it stunned the entire Johnson family.
Maybe he will be a defensive coordinator or a head coach some day, all the better if it were to happen in State College.
But the man soon to be 60 said it won't define him. He said he nearly got out of the profession all together a few years ago, only to be talked back in by his family who knew he still had more work to do.
Maybe he will become a minister. Maybe he will run a traveling coaching clinic, reaching out to so many more young men, building his family even larger, stretching his roots miles farther.
And that's when the conversation on a summer afternoon turned back to his father.
Herman Johnson was 68 when Alzheimer's disease robbed him of his mind and then his life. Near the end, Larry traveled back to Williamston as often as he could, the joy and the pain of those visits nearly knocking the breath from him.
"It was the most devastating thing I had to do in my life, going home to see my dad and he hadn't taken a bath. And I remember talking to my dad and saying, 'C'mon, let's get cleaned up so you and I can go hang out together.' Here I am bathing my own dad, and I can tell you it took everything I could to finish that."
He paused as he described the memory, emotions filling him.
"I was thinking, 'This is not how life is supposed to be. My dad, so strong of a man ..."
The son did what he needed to do and grew from it, grew even tougher, more determined and more compassionate, putting family first once again.
He could change lives and make his name proud, no matter the place or the people.
Just like he learned in that small town all those years ago.