Denmark will pay an annual fee of €15m (£12.8m) for an initial period of five years, and will also help fund green energy in the country.
The rented cells are meant to house convicted criminals from non-EU countries due to be deported from Denmark after their sentences.
Danish laws would apply to any prisoners in the rented cells.
Kosovo has between 700 and 800 unused prison spaces.
The two governments signed a "political declaration" of intent on Monday which will run for an initial period of five years, a joint statement said.
The Balkan state unilaterally declared independence from Serbia in 2008.
In total, Kosovo is due to receive a total of €210m over the next 10 years for renting the prison in Gjilan, some 50km (30 miles) from the capital, Pristina, from 2023. Danish ministers were due to visit the jail on Tuesday.
"[The agreement] will create space in our prisons and ease the pressure on our prison officers at the same time as it also sends a clear signal to third-country nationals sentenced to deportation: Your future does not lie in Denmark, and you must therefore not serve time here," Danish Justice Minister Nick Haekkerup said in a statement.
There has been criticism of the move with activists in both countries saying Denmark should not be sending unwanted foreign convicts to other countries, or far away from their families.
"We have done what we can to ensure that this is within the rules. The exact same rules will apply as for prisons in Denmark," the minister said. "Deported convicts still be able to get visitors, although that will, of course, be difficult."
Those sent to Kosovo would not be high-risk inmates, Kosovo's Justice Minister Albulena Haxhiu said. Anyone convicted of terror offences or with a terminal illness will not be sent to Kosovo.
The agreement will need to be ratified in Kosovo's parliament by a two-thirds majority.
The idea of exporting inmates is not new in Europe, as Norway and Belgium have previously rented prison cells in the Netherlands.
prison population has grown in recent years while the number of prison guards has fallen.
Since 2015, the number of inmates in Denmark has risen from 3,400 to 4,200, while the number of prison guards has declined from 2,500 to 2,000.
Last week, Mr Haekkerup said Denmark would be short of 1,000 prison places by 2025.
In recent years, Denmark has taken a hard line on immigration and introduced dozens of restrictions, including an order to separate young asylum-seeking couples.
Last week, a former immigration minister was found guilty of illegally separating the couples in a landmark impeachment trial.
Judges ruled Inger Stoejberg's decision in 2016 to separate couples was unlawful and jailed her for 60 days.
Danish MPs voted by a substantial majority to expel Ms Stoejberg from parliament on Tuesday in response to the conviction.