The Hammers took the lead thanks to Mark Noble, but the home side responded when Joe Lolley curled an effort into the bottom corner.
However, Marko Arnautovic put West Ham in front and then created two chances for Manuel Lanzini as Moyes’ side extended their unbeaten run to five matches.
West Ham are now up to 11th in the Premier League table and Moyes is confident his troops are just getting started after a slow start to his tenure.
He told Sky Sports: “I thought today there were signs. It was a much better performance. A bigger threat and we passed it well at times.
“We’ve still got to go to another level with it, but I’ve got to say – and I know the two boys up front will get a lot of the credit – that I thought the team performance generally was really good.
“For us to beat one of our rivals is a big thing. I don’t think any of the teams are much better than the other so the games are going to be close, but we’re beginning to win on the road.
“We won at Stoke and here and we’ve started to get a wee bit of form at the London Stadium as well. Hopefully that will bode well between now and the end of the season.”