S jeho první ženou Blankou z Valois se seznámil na francouzském dvoře, kde byl v dětství vychováván. V útlém věku byli zasnoubeni a brzy došlo i k sňatku, ale jelikož jim bylo 7 let, oddělili se a dál byli vychováváni svými učiteli. Jejich pozdější soužití bylo harmonické a Blančina smrt byla pro Karla IV. citelnou ztrátou. Druhá manželka se jmenovala Anna Falcká, která porodila Karlovi syna Václava, jenž měl být zasnouben s Annou Svídnickou. Syn však ve dvou letech zemřel, a tak si Annu Svídnickou vzal sám Karel. Tento svazek dal Karlovi dědice, Václava IV. Po smrti Anny si Karel IV. vzal Elišku Pomořanskou, jež mu porodila 5 dětí, z toho tři syny a dvě dcery. Celkově se dá říct, že všechna manželství Karla IV. byla harmonická a bez problémů.

Charles met his first wife, Blanche of Valois at the French court where he was raised. Early on, they were engaged, and not long after married, but as they were only seven, they were separated and their mentors brought them up. Their life together, later on, was harmonious, and Blanche's death was a painful loss for Charles. His second wife was called Anna of Bavaria. She gave birth to Charles's son, Wenceslas, who was meant to marry Anna von Schweidnitz. However, the son died when he was two and his mother died not long after him. Charles himself then married Anna von Schweidnitz. This bond produced the heir, Wenceslas IV, to Charles. After Anna's death, Charles married Elizabeth of Pomerania, who gave birth to 5 children, three sons and two daughters. From what we know, all Charles's marriages were harmonious and had no problems.