Americans are likely to think of New Year’s Eve and New Year’s Day as a time to celebrate the fresh start that a new year represents, but there is also a troubling side to the holiday’s history. In the years before the Civil War, the first day of the new year was often a heartbreaking one for enslaved people in the United States.

In the African-American community, New Year’s Day used to be widely known as “Hiring Day” — or “Heartbreak Day,” as the African-American abolitionist journalist William Cooper Nell described it — because enslaved people spent New Year’s Eve waiting, wondering if their owners were going to rent them out to someone else, thus potentially splitting up their families. The renting out of slave labor was a relatively common practice in the antebellum South, and a profitable practice for white slave owners and hirers.

“‘Hiring Day’ was part of the larger economic cycle in which most debts were collected and settled on New Year’s Day,” says Alexis McCrossen, an expert on the history of New Year’s Eve and New Year’s Day and a professor of history at Southern Methodist University, who writes about Hiring Day in her forthcoming book Time’s Touchstone: The New Year in American Life.

Some enslaved people were put up for auction that day, or held under contracts that started in January. (These transactions also took place all year long and contracts could last for different amounts of time.) These deals were conducted privately among families, friends and business contacts, and slaves were handed over in town squares, on courthouse steps and sometimes simply on the side of the road, according to Divided Mastery: Slave Hiring in the American South by Jonathan D. Martin.