Father of twin girls adopted by Madonna says he didn't realise it would be permanent 5 years ago