The Black Cats twice came from behind against Tottenham on Saturday to earn a share of the spoils at the Stadium of Light; Adam Johnson cancelling out Nacer Chadli's second minute goal before Harry Kane's own-goal nullified Christian Eriksen's strike to gift Sunderland a point.
Sunderland collected 13 points from their last six games to escape relegation last term, but Saturday's stalemate was only their third point of the new campaign.
While admitting points are the main focus for Gus Poyet's side, Brown understands Sunderland do have significant improvements to make.
"At the end of the day, football is all about the score," Brown told the Newcastle Chronicle. "Apart from at QPR we have not lost.
"We are not playing as well as we can, like we were at the end of last season. Yet it is still early on and hopefully that will come.
"As long as we can keep gaining points, that is the main thing. We have to start winning games, but if we can keep showing the same desire from the last 15-20 minutes against Spurs and do that at the beginning of the game we can maybe gain a few better results."
Brown admits Tottenham turned in a fine display on Wearside at the weekend, but the former Manchester United centre-back is not willing to dwell on a third stalemate of the season. Instead, he is looking ahead to facing newly-promoted Burnley next weekend.
"Spurs started very well, although I know we scored straight after their opener, and they then put a lot of pressure on us and it was difficult at times," he added.
"They played very well. They hit the post and the crossbar a couple of times. We defended quite well and even though they had opportunities we managed to get back in the game.
"We were happy with a point, but we have to look to next week now to push on and gather more points."