James Matthew Whyte (ca 1788 – June 9, 1843) was a Scottish-born soldier, land owner and bank president in Upper Canada.
The son of James Whyte, he served as a lieutenant and later captain in the 1st King's Dragoon Guards, resigning in 1815. He later was lieutenant-colonel of the Surrey Regiment of Horse in Jamaica. Whyte took over a plantation in Jamaica in 1811. He served on the island's council and as a justice of assize. In 1834, he sold his assets on the island and moved to Canada, settling in Hamilton. He became involved with the establishment of the Gore Bank (later part of the Canadian Imperial Bank of Commerce) and the London and Gore Rail Road. Whyte served as president of the Gore Bank from 1836 to 1839 and was a member of the board of directors for the railroad. He also held real estate in Harwich Township in Kent County and Picton.
In 1836, Whyte build Barton Lodge, one of the city's finest homes, on the Hamilton mountain. He died in Hamilton. His home was inherited by his brother John Lionel Whyte.