I have just finished watching Someday or One Day (2019) - Taiwanese Drama and I highly recommend it.
It is romance but not considered light-hearted generally and is not really historical.
Time travel between 1998 ~ 2019. Script was really well-written and skillfully planned, probably the one of the best time travel plot out of all those I have watched. I HIGHLY SUGGEST YOU GO IN FULL BLIND without even reading the synopsis/spoilers/reviews at all if you are interested.
I only watched kdramas mainly and those that contains time-travel elements are:
365: Repeat the Year (2020)
Chicago Typewriter (2017)
Live Up to Your Name (2017)
Life on Mars (2018)
Moon Lovers: Scarlet Heart Ryeo (2016)
Mr. Queen (2020)
Queen In Hyun's Man (2012)
Sisyphus: The Myth (2021)
Splash Splash LOVE (2015)
The King: Eternal Monarch (2020)
You may check it out if you are free and see which ones they interest you, some might not have romance and might not be historical also.