How did it get configured this way? And how do I stop it, just uninstall Outlook
good question. You may wish to keep Outlook (which is different from Outlook Express) for the Calendar and Contacts features.
To stop Outlook from accessing your email:
use menu tools->accounts
delete all Mail accounts but the default
on the default->properties
there's a box at the bottom, clear the check mark, "Include this account ..."
To stop Outlook Express: follow the same instructions above.
Many users are recommending Thunderbird for email. What are you using?