Remember that your relatives are only your starting party. You can add and remove people to create the party which works for you. It's 100% up to you who you keep in your party as your adventure progresses.