Jermain, who came on late in the game for Paolo Di Canio, believes the scoreline did not tell the whole story, and says: "Obviously I am disappointed about the defeat, and I think 2-2 would have been a fair result, to be honest.
"The third goal was a spawny one, it just trickled in from the deflection, and we were so unlucky really.
"Gardner was in front of me, and I was behind trying to track him back.
"The defence was obviously backing off because they didn't expect him to shoot, it got a silly deflection - and there was nothing anyone could do about it.
"I suppose that is the way your luck goes when you are at the bottom - at the moment nothing seems to be going our way.
"Watching the game from the bench I thought we were the better side and even with ten men we worked so hard.
"It just wasn't meant to be - but hopefully when the first one goes in we will start winning - it was like this last season with a slow start."
Jermain says it is all about keeping calm and focused ahead of Saturday, and he adds: "We have to just keep on working hard to see what happens and take each day as it comes.
"We need to do really well in the next three games, though you should only look to the next one.
"We need to win that one and I think if we do the confidence will be a little bit better.
"We need to get our season going because at the minute we haven't got started.
"It is going to be difficult for us now, but we have to keep working hard now to try and get out of it."
Jermain accepts that, though he is keen to play, he may yet to have to do that from the bench at the moment.
"I nearly came on a bit earlier against Spurs, then we scored and the gaffer decided to bring Michael on instead," he says.
"All I want to do is play football. I'm still young and learning the game so there is nothing I can do except be patient.
"It is up to the manager at the end of the day, and we are all in this together."