Guice ran 24 times for 97 yards and one touchdown to help LSU win for the fifth time in its last six games. Williams rushed for 68 yards and two touchdowns on seven carries.
LSU (8-3, 5-2 SEC, No. 20 College Football Playoff) scored 10 points after recovering two fumbled punt returns by Tennessee's Marquez Callaway in the red zone. The Tigers also began a touchdown drive after making a fourth-down stop at Tennessee's 21.
Tennessee (4-7, 0-7) was playing its first game under interim head coach Brady Hoke after the firing of Butch Jones.
"I'm sure it was a tough week for them," LSU coach Ed Orgeron said. "Their team came out and fought hard and battled. We were expecting a battle."
The Volunteers must beat Vanderbilt next week to avoid their first eight-loss season in school history. A loss to Vanderbilt would cause Tennessee to finish a season winless in SEC competition for the first time since the league's formation in 1933.
Tennessee's Jarrett Guarantano went 13 of 23 for 239 yards with a touchdown pass.
Heavy wind surrounding Neyland Stadium caused one goalpost to lean to the side more than an hour before the start of the game, though stadium workers straightened it out. The first half was played in a steady rain that started swirling just before the start of the second half when numerous light fixtures in the upper level briefly went out.
"It literally was coming down so hard I couldn't see guys on the field," Hoke said. "It was coming right into you."
LSU's offense struggled early but still grabbed a 10-3 lead by capitalizing on Callaway's turnovers
Callaway's first fumble resulted in a Connor Culp 30-yard field goal that gave LSU a 3-0 lead. The second fumble led to a Williams 10-yard touchdown run that put LSU ahead for good 10-3 early in the second quarter.
LSU extended the lead to 17-3 when Danny Etling ran 13 yards untouched on a quarterback keeper, but Callaway partially atoned for his two turnovers by catching a 46-yard touchdown pass to make it 17-10 at halftime.
"If you go to practice and see him work in catching punts and how he worked all week on the JUGS machine because of a left-footed punter and the spin being different, he's one of the hardest workers we have, in my opinion, in this program," Hoke said about Callaway. "I felt badly for him. The good thing is he bounced back."
Then the weather started wreaking more havoc. Although the game was never delayed, the rain and wind made an immediate impact as Tennessee fumbled the second-half kickoff and recovered at its own 3.
Tennessee had to punt from deep in its own end, allowing LSU to start its next drive in Tennessee territory. That series ended with a Guice 3-yard touchdown run that extended LSU's lead to 23-10.
The weather got a little better the rest of the way, but Tennessee never recovered.
LSU: The Tigers rushed for just 9 yards on six carries in the first quarter against a Tennessee run defense that entered the weekend ranked 126th out of 129 Football Bowl Subdivision teams. But LSU ran 37 times for 191 yards the rest of the way.
Tennessee: Tennessee loved rolling the dice in its first game since the coaching change. Tennessee twice went for it on fourth-and-short from its own side of the field. A fourth-and-1 carry from the Tennessee 49 was initially ruled short of the marker but got overturned on replay, preserving a drive that resulted in Aaron Medley's 45-yard field goal. The second gamble backfired when Grant Delpit stopped Guarantano on fourth-and-1 from the Tennessee 21, leading to Williams' 6-yard touchdown run in the third quarter.
LSU hosts Texas A&M
Tennessee hosts Vanderbilt.