You can do this, but you have to consider some things:
- Do you trust all your computers? Without firewalls, a worm can spread to the other computers in a matter of seconds. All it takes is a careless user on one of your clients or an infected guest laptop.
- It takes a mere hour to crack your wireless WPA security. A second line of defense can be useful against wardrivers and suchlike. Remember, there will be no firewalls between the wireless intruder, the access point and your wired PCs.
These are not serious threats, and if you have a reason to remove the desktop firewalls, then it's OK to do so. A firewall on a Linux/BSD desktop would probably be an overkill too in a home setting.