We hear a lot about how Gettysburg was as far north as the Confederate Army could get, while that may be true for the Army of Northern Virginia, it wasn't true for the entire Confederate armed forces. The actual northernmost fighting took place in northern Vermont, near the U.S. border with Canada. You can't get much further than that.
Although the Confederates did make it to Gettysburg and were stopped, there were many other places in the United States, well north of Gettysburg. During the Gettysburg campaign, another Confederate expedition was making its way up through Tennessee and Kentucky, then into Indiana and Ohio. Confederate General John Hunt Morgan led a raid that was supposed to divert men and resources from resisting the main southern thrust northward, the one at Gettysburg.
Morgan led his men, less than 3,000, through Cincinnati, Columbus, and Steubenville Ohio, only to be stopped by Union troops in Salineville, Ohio. Ambrose Burnside and his army of 40,000 relentlessly pursued Morgan up through the northern states. After they were captured, they managed to escape, retreating to Cincinnati and into Kentucky, where they took advantage of the state's neutral status.
One native Kentuckian, Bennett H. Young, was captured at Salineville and escaped but instead of sneaking down the Ohio River and into Kentucky, he moved North instead. He slipped into British-controlled, Confederate-sympathizing Canada and hatched his plan to continue fighting the Union from the other side of the Mason-Dixon line.
He decided that diverting Union troops from attacking the South was still the best way forward, so he devised a plan that served that end while funding his own expeditions: raiding Northern border towns. His first stop would be St. Albans, Vermont, just a few miles from the U.S.-Canada border.
Young's men moved into St. Albans piecemeal, coming in groups of two to three every few days, and checking into the local hotels. By Oct. 19, 1864, 21 Confederate cavalrymen had made it to the sleepy Vermont town. Once ready, they simultaneously robbed the town's three banks, fought off any resistance, forced others to swear loyalty to the Confederate States of America, and burned someone's shed. They also made off with the modern equivalent of .3 million before escaping into Canada.
The United States demanded the extradition of the soldiers, but since the men had acted as official CSA soldiers, the Canadians would not turn them over to the Americans.