Dear Mr. Houston,
Thanks for the smiley face, it really helped out a lot.
As the house has stood for nearly 200 years without a furnace in the attic, I think it could stand a few more years without one. If I need to remove the furnace, that's what I will do. You may have misunderstood the problem. The stovepipe, as-is, is not acceptable. The fact that the government regulations are blind to the visual damage they cause does not mean there is no other way to fix the problem.
Here are my options as I see them:
1. Tap the exhaust pipe into the old chimney below the roof. This may be hard work, but it is done all the time.
2. Use the existing exhaust but add a 90 degree bend to the stovepipe and send it somewhere far enough behind the front of the house so it doesn't show
3. Move the entire attic furnace to the rear of the house where it should have been installed by the contractor in the first place.
Please try to help me out here, I cannot live with it the way it is. Thanks.