Justin and Hailey Bieber are spending a celebratory weekend together, following the July 11 release of 'Swag,' Justin's first solo album in more than four years. A source familiar with the creation of the album tells PEOPLE that the Rhode founder supported her husband throughout the entire process.

A source tells Rolling Stone that "Breaking away from Scooter Braun and his team has been something that Justin has wanted for so long and now that he’s fully free, he could finally share this album with his fans and with the world. [It’s] something he’s been wanting to do for a long time."