James Herman Agen (April 29, 1847 – October 5, 1921) was an American businessman and politician.
Born in Montpelier, Vermont, Agen moved with his family to Wyoming County, New York. During the American Civil War, Agen served in the 1st Regiment New York Dragoons. In 1887, Agen settled in West Superior, Wisconsin. He was in the real estate, loan, and fire insurance business. He was also president of the West Superior Chamber of Commerce and as president of the Douglas Council Agricultural Society. Agen also served as commander of the Wisconsin Grand Army of the Republic chapter. In 1893 and 1894, Agen served on the West Superior Common Council. In 1897, Agen served in the Wisconsin State Assembly as a Republican. Agen died at his home in Houston, Texas.