Libby Prison was a Confederate prison at Richmond, Virginia, during the American Civil War. In 1862 it was designated to hold officer prisoners from the Union Army, taking in numbers from the nearby Seven Days Battles (in which nearly 16,000 Union men and officers had been killed, wounded, or captured between June 25 and July 1 alone) and other conflicts of the Union's Peninsular campaign to take Richmond and end the war only a year after it had begun. As the conflict wore on the prison gained an infamous reputation for the overcrowded and harsh conditions. Prisoners suffered high mortality from disease and malnutrition. By 1863, one thousand prisoners were crowded into large open rooms on two floors, with open, barred windows leaving them exposed to weather and temperature extremes.

The building was built before the war as a tobacco warehouse and then used for food and groceries before being converted to a prison. In 1889, Charles F. Gunther moved the structure to Chicago and renovated it as a war museum. In 1899 the building was torn down and sold for scrap.

The prison was located in a three-story brick tobacco warehouse on two levels on Tobacco Row at the waterfront of the James River. In 1861 was leased by Capt. Luther Libby and his son George W. Libby. They operated a ship's chandlery and grocery business.

The Confederate government started to use the facility as a hospital and prison in late 1861. In 1862 they reserved it to hold Union officers because of the influx of prisoners. It contained eight low-ceilinged rooms, each 103 by 42 feet (31.4 by 12.5 metres). The second and third floors were used to house prisoners. Windows were barred and open to the elements, increasing the discomfort of occupants. Lack of sanitation and overcrowding caused diseases. From holding 700 prisoners in 1862, by 1863 the facility far exceeded the maximum capacity of 1,000. Mortality rates were high in 1863 and 1864, aggravated by Confederate shortages of food and supplies. Because of the high death toll, Libby Prison is generally regarded as second only in notoriety to Andersonville Prison in Georgia.

In 1863, The New York Times published a description of "the Libby" from a purported prisoner diary entry. The next year, Kentucky Volunteer Infantry Captain I.N. Johnston, who escaped from Libby Prison, attempted to corroborate the article. (He testified that "...an African slave...did all in his power to restore us to freedom and home."

He wrote

"...the building is of brick, with a front of near one hundred and forty feet, and one hundred feet deep. It is divided into nine rooms; the ceilings are low, and ventilation imperfect; the windows are barred, through which the windings of James River and the tents of Belle Isle may be seen. Its immediate surroundings are far from being agreeable; the sentinels pacing the streets constantly are unpleasant reminders that your stay is not a matter of choice; and were it so, few would choose it long as a boarding-house."

In March 1864, Union worries about the safety of Richmond and related security of the prisons, and the scarcity of resources peaked. The next month, Union officer Harland Richardson pleaded "once more" for a "Mr. Reilly," presumably on behalf of the U.S. War Department, to send provisions to Libby. Such requests were either delayed by Libby commandants or ignored, as the U.S. War Department funneled supplies into active Union lines.