In 1971, Remy was drafted in the 8th round by the California Angels. He played three seasons for the Angels starting in 1975 before being traded to the Boston Red Sox, where he played seven more years. He was an All-Star selection in 1978, though he did not appear in the game. He was released by the Red Sox in 1985. Bill James in his Historical Abstract rated him as the 100th greatest second baseman of all time, as of 2002.

Remy, with his strong Boston-area accent and smoker's laugh, has found success in broadcasting, working for the New England Sports Network (NESN), and is the color commentator for all NESN Red Sox broadcasts along with play-by-play announcer Don Orsillo. He is a well-loved figure in Red Sox Nation, bringing his humble yet affable personality to every televised game. As such, Remy was "elected" to be the first "President" of Red Sox Nation. During slower parts of the game, the pair will carry on long non-baseball discussions, with pop culture a frequent topic. The pair also often discuss the imaginary exploits of Wally the Green Monster, a doll of which sits with the pair at the front of the broadcast booth.

Remy is affectionately known as the "Rem Dawg." He lends this nickname to his hot dog stand, "RemDawg's" on Yawkey Way outside of Fenway Park in Boston on game days. He is planning to open a second location at Boston's Logan Airport. He also runs a web site, The Remy Report, and is the author of the book Watching Baseball (Ghost written by author Corey Sandler) and the children's books Hello Wally and Wally the Green Monster and His Journey Through Red Sox Nation!. Jerry also spends much of his time on the air advertising his merchandise, constantly barraging viewers with invitations to "All Star Parties" and other events held by him, where 100% of the proceeds go to Remy and associated parties. He also sells his scorecards on his website, and merchandise with his logo on it.

Remy has also served as a color analyst for FOX's national coverage of Major League Baseball, primarily assigned to games in the Northeastern U.S.

Remy was inducted into the Boston Red Sox Hall of Fame in 2006.