A scathing Richard Money has promised changes to his side that will face Mansfield Town on Tuesday night after seeing his side slip to their first away defeat in 11 matches at Crawley Town.
A double from Red Devils midfielder Ben Smith proved to be the difference with his winning goal coming just three minutes from time.
Matthew Barnes-Homer had hauled the Hatters level with 12 minutes to play cancelling out Smith's 55th-minute opener that ended a run of 852 minutes without the Town conceding away from home.
However, after the Town missed a number of chances after Barnes-Homer's strike, Smith's late header was enough to sink Money's troops.
"It was a poor performance," admitted the Hatters chief afterwards. "I don't want to dwell too much on what has happened today but it is pretty obvious to lose is disappointing to say the least.
"I have my own thoughts, and I think they will be pretty self-evident in my team selection on Tuesday at Mansfield.
"I don't want to say too much more but there are things that I've had to deal with since I've been at this football club and to some degree I'm still dealing with it."
Defending a record of eight clean sheets on the spin on their travels, the Town had earned a reputation of being hard to beat away from Kenilworth Road.
But Money was less than impressed with his player's display at the Broadfield Stadium to allow the hosts to get the better of them.
"We didn't deal what we had to today," he said. "They've done a job on us.
"They frustrated us and they did their jobs: they got nine men behind the ball and pressed and squeezed us and they've done it really well.
"We didn't handle it and they've scored from their first shot at goal and then go and win the match when we're dominating.
"You think after we make it 1-1 we'd go on to win the game but we haven't.
"They've scored [their winner] from a free-kick that shouldn't have been, but aside from that, I'm disappointed that we haven't been able to deal with what we should have done.
"Sometimes you need to see it warts n'all: to go forward you need to take one step back and maybe that's the case.
"We haven't let ourselves down too often away from home but we have today."