Installing Xbox One is rather long, and even longer than its direct concurente the PS4. But Phil Spencer explained himself after IGN, there is a reason for that.
According to him, having made the console usable even when not connected to the Internet has had an impact. He also explained that the drive speed is the same as on PS4 and that transfers to the memory take that long.
So why the difference that still exists today? For now, Phil Spencer said that collects data on the subject, and the goal is to be at the forefront so that people can start playing as soon as possible. It's part of their priorities, and it is out of the question for them that these installation time are lagging behind compared to the PS4.