I noticed this when using Cygwin Perl, however I suspect the issue exists with any Perl distro on Windows.
The problem occurred when I had set the HOME environment variable to point to a network share. When I ran the CPAN utility it placed all the CPAN files on the network. As a result some modules would not install (I have no idea why) and it ran very very slowly.
The solution I adopted was to to move my home directory to the local PC drive which then made CPAN use the local drive as well. Changing
/usr/lib/perl5/5.10/CPAN/Config.pm would probably have been simpler, but I was happy to use the local disk for everything.