The visitors opened the scoring within two minutes, as Thomas Tonks placed the ball into the bottom corner from 20 yards.
Brig responded to the early setback well, and Jacob Holland-Wilkinson forced a good save out of Luke Pilling with a low driven shot from the right side of the penalty area.
On 19 minutes, Nathan Pond hit an effort on the half volley over the bar, before a sliced clearance from a Stafford defender almost found its way into the bottom corner of the net.
Adam Dawson then produced an excellent cross that whistled across the face of goal, but no Brig attacker was able to get on the end of it.
The visitors then extended their advantage on 35 minutes. Alex-Ray Harvey went on a run towards goal and placed his effort into the bottom corner.
Stafford added a third goal just after the break. On 48 minutes, Tre Pemberton scored a rebound after James Cottam had saved from earlier goalscorer Tonks.
Holland-Wilkinson was then played through for Brig but saw his well-struck effort blocked by a defender. Then, on 54 minutes, Brig were down to 10 men when Pond was shown a second yellow card.
Stafford almost extended their lead further on 68 minutes when Alexander Fletcher headed wide from close range.
It was Brig who went on to get the fourth goal of the encounter, when a defensive clearance was sliced up into the air and went over the head of Pilling in the Stafford goal.
Jamie Milligan’s side pressed late on in an attempt to reduce the deficit further, but to no avail. Substitute Paul Dawson headed narrowly over the bar, before the visitors produced a series of blocks to deny Brig from a late set piece.
Brig will look to get back to winning ways next Saturday, when they host Mickleover at the Sir Tom Finney Stadium.