OK, that means you are missing a driver, just as you thought. Have you tried to search for a driver on Windows Update? To do this, rightclick on the entry with exclamation mark, select "Update driver software...", then select "search automatically for updated driver software".
Can you see a name on your audio device, or just "Multimedia audio controller"?