While it is easy for people to condemn the family who harbored him, it is understandable that the instinct to protect the one you love and respect is difficult to fight. I have seen personally how loyalty caused people to justify wrongdoing, to protect the offender, to cover up the offense, and to brush off the offensive behavior as something one off and insignificant.
Blind loyalty to a spiritual leader often is the crux of many church division. Often there are factions within the church. Each factions are loyalists to each opposing leaderships. This shouldn't be happening in a church if our loyalty is directed appropriately to the Lord instead of to a leader.