The Cabell period at Point of Honor is important but, ironically, little documentary evidence survives to illuminate this significant era. During the quarter century of Cabell ownership, the farm was transformed into a magnificent estate. Dr. George Cabell purchased 737 acres from Lewellen Jones on December 9, 1805. The boundary of Dr. Cabell’s land holdings encompassed from the confluence of the James River and Blackwater Creek up the creek to Calloway's Mill (Hollins Mill) then across to where St. John's Church is today. From St. John’s Church, the boundary line ran northeast to the river and returns to the creek. The sale included Daniel Island (where Griffin Pipe is located today), Treasure Island, and Woodruff Island (upstream from Treasure Island) which brought Cabell's ownership to 856.5 acres.

In 1815 George Cabell paid personal property taxes on 30 enslaved individuals, most of whom likely worked on the Point of Honor plantation. Children under age nine were not counted, so the actual number of slaves living and working at Point of Honor was probably higher. Research has not yet uncovered the names of any of these individuals during the Cabell period.

George Cabell was the eldest son of Col. John Cabell and Paulina Jordan Cabell of Green Hill, Buckingham County, Virginia. Born on November 1, 1766, he attended Hampden-Sydney Academy from 1777 to 1779 and then studied medicine at the College of Philadelphia. Cabell is said to have been an early advocate of the use of ice and cool drinks to treat fevers and inflammations. He was best known as a surgeon and his most illustrious patient was Patrick Henry of Red Hill, whom he attended during Henry's last illness. 

By the time construction of Point of Honor was complete most of the Cabells’ children were grown. Their eldest daughter Paulina was married and the older boys were away at school. Alice was married in 1817 shortly after the family moved into the house. The birth and death dates of their youngest child Sarah Cabell are unknown so we are unsure if she ever lived at Point of Honor. 

There are only a few hints about the events at Point of Honor during the Cabells' residency. In 1818, George's son Edmund, also a doctor, advertised in Lynchburg that he and his partner, Paul C. Cabell, had a shop on Main Street or could be found at Point of Honor. Perhaps Cabell maintained an "office" at his home and practiced with his sons, Edmund and John Breckinridge. Little documentation has survived to describe the daily routine or significant events which took place. Research at Poplar Forest reveals that Dr. Cabell owned a number of batteau which he chartered to Thomas Jefferson and others to transport crops to Richmond.

George Cabell died in 1823 from injuries in a fall from his horse. The location of his grave and that of his wife are unknown but obituaries indicate they were buried at Point of Honor. His widow Sarah continued to live at Point of Honor until her death in 1826 at which time the estate was inherited by their son William Lewis Cabell. In 1828, William Lewis Cabell married Eliza Daniel, daughter of Judge William Daniel, Sr. and his first wife, Margaret Baldwin Daniel. William and Eliza’s residency at Point of Honor was tragically brief. He died of consumption, probably tuberculosis, in the summer of 1830 at Red Sulphur Springs where he had gone in an attempt to recover. No sooner had this happened than his young widow became ill and by October, she too had died. William and his wife were childless so Point of Honor was inherited by Judge William Daniel, Sr., Eliza's father.