While much of the allure of Starz's Outlander is the strongly developed characters and the love story that sweeps across centuries, there's no doubt that a major part of the show's richness comes from its historic locations and gorgeous set pieces. For executive producer Ronald D. Moore, nothing was more challenging — or as insanely gratifying — than turning a 600-year-old castle into one of his set pieces.

In the series, Doune Castle, which was originally built in 1390, stands in as the fictional Castle Leoch. When the series opens, Claire (Caitriona Balfe) and Frank (Tobias Menzies) visit the "picturesque ruin" in 1945 and soon after, when Claire mysteriously travels back in time to 1743, she gets taken to the Castle by Jamie (Sam Heughan) and his clan. Later in the first season, the Claire and Jamie would return to Leoch, which became the backdrop for the show's witch trial.

"It's a historic landmark so there are a lot of limitations for what you can and cannot do there," Moore shared with us during our trip to the Scotland set. "One of the rules was that we couldn't put anything down into the ground and anchor them so everything had to be sitting on top and lifted away. On the interior, we couldn't hang lights, you couldn't paint the walls, so it becomes very limiting."

VIDEO: A fascinating look at how Outlander makes its own tartan

As a result, Moore recreated much of the interior castle on a sound stage for the Great Hall scenes, and the set quickly became one of his favorite sets. "We filled it with a couple hundred extras all in their fine regalia and it was lit and beautiful," he said. "It was challenging, just the construction took a long time, and it might be the biggest set I've built in my career."

And it wasn't just the structure that was tough to work with. Executive producer Maril Davis said weather was a huge challenge. "Scotland is a high-maintenance lady," she said. "Just when you think it's going to be a really, lovely sunny day, the rain comes. They say if you don't like the weather wait five minutes."

Check out the video above for more about how the series brought Doune to life.

The Outlander finale airs on Saturday at 9/8c on Starz.