From this period till the autumn of 1782 he lived in retirement at Whiteford.
Whilst in India, Call had not forgotten his parents and sister at home, and had sent to his mother priceless Indian shawls, which she, not knowing their value, cut up and turned into under-petticoats for herself and daughter and maids. A pipe of Madeira sent to the father was also as little appreciated. It was distributed among the farm-labourers during harvest time to economize the cider.
Now that he was in England and wealthy, he resolved on doing something for his sister. She had married Cadwalader Jones, the vicar of the parish, and the vicarage was a small, mean building, so Cadwalader Jones had taken the manor house that was near the church on a long lease from the Orchards, who were lords of the manor. This house had been a cell of Hartland Abbey, but at the Restoration had been given to the Chammonds. That family had died out, and now it had come to the Orchards, owners of Hartland Abbey. Call rebuilt the house, or, to be more exact, built on a modern house to the old, and installed Cadwalader and his sister in the new mansion; he also made for them a large walled garden. When he did this, he was under the impression that the property belonged to Cadwalader, and not till he had completed his building did he learn that Mr. Jones had only a lease of it. Moreover, Mrs. Jones did not live to enjoy the new house very long, as she died in 1780, and then Cadwalader married again. In course of time Cadwalader went to join his ancestors, and thereupon Mr. Hawkey saw and loved the widow and the mansion, and married her. Thus it came about that the manor house built for Mrs. Jane Jones passed into other hands. But thus it happens also that through Miss Charlotte Hawkey we have some account of Sir John Call.
Lord Shelburne, when Prime Minister, being desirous of investigating some of the existing abuses and reforming some of the public departments, fixed on Call and engaged him along with Mr. Arthur Holdsworth, of Dartmouth, to inquire into the state and management of Crown lands, woods, and forests, which had long been neglected; Call had seen this with regard to the Duchy property at his doors, and had drawn attention to it. In November, 1782, they made their first report; but a change of Ministry taking place soon after, their proceedings were interrupted till the Duke of Portland, then First Lord of the Treasury, authorized them to continue their investigation. Before they had gone far another change took place in the Ministry, and Pitt became Prime Minister. These frequent interruptions interfered with the progress of the investigation, and to obviate that, in 1785-6 Sir Charles Middleton, Call, and Holdsworth were appointed permanent Parliamentary Commissioners.