The fort was built and invested by "United States Colored Troops" -- that is, African-American troops fighting on the side of the Union. They were, of course, led by a white officer, the one-armed General Edward Wild (I can't think of a war in which more officers had less appendages!).
It had always been a truism among whites, Union and Confederate alike, not excluding abolitionists, that black soldiers could not fight as well as white soldiers. On May 24, 1864, this "theory" was put to the test when 2,500 Confederate troops under General Fitzhugh Lee, the nephew of "Massa Robert," attacked the fort, known as Wilson's Wharf at the time, expecting to easily walk over the black defenders, who were outnumbered by better than two to one. The result was a clear and humiliating Confederate defeat. After the battle, and for the remainder of the war, the fort served as a refuge for escaped slaves and as a prison for Conderates taken during the seige of Petersburg. More details are available here and here.
Equally fascinating is the fact that the property was purchased in 1996 by Harrison Ruffin Tyler, the grandson of the 10th US President, John Tyler. Yes, the grandson! Is that mind blowing or what?
John Tyler was a fire-eater who urged Virginia to secede and was then elected to the Confederate Congress, only to die before he could take office. In his sixties, he fathered Lyon Gardiner Tyler, who followed his father's footsteps by marrying twice and also siring offspring in his later years, one of whom is Harrison, now 79. So here in 2007 we have a man who is but two generations removed from the first native-born American president, who also has the distinction of being the first vice-president to ascend to the presidency, and the first president to have impeachment proceedings brought against him.
As Faulkner so rightly said, "The past isn't dead. It isn't even past."