I tried networking my two home computers.. I simply connected the UTP cables to my router and I set up a home or small office network.. But the icon appears with an X-mark on it and when I view the status it says limited or no connection. What am I suppose to do?

More details please.

If youre running XP then you may have to run the set up wizard from the control panel first. If that doesn't work check the settings for your network adapter to make sure they are compatible with your router. (Device manager/ right click on the network adapter/ properties).

Do your computers have an IP ?
Go to command prompt and type ipconfig
If the output from the command looks like this

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
IP Address. . . . . . . . . . . . : 0.0.0.0
Subnet Mask . . . . . . . . . . . : 0.0.0.0
Default Gateway . . . . . . . . . :

If your output looks like that it means your computers haven’t been given an IP and more than likely DHCP is disabled on your router.
If this is the case go to your routers config and enable DHCP.
To do this open a web browser and type in the IP address of the router. If you don’t know this already it should be in the documentation that came with the router.
Once on the router look for an option that says 'DHCP' enable this and save. Now your devices will be given an IP automatically. I'm not sure if you have to restart the computers for the first time for it to work, but you could always go to command prompt and type ipconfig /renew

You didn’t given a great deal of info, so i'm not entirely sure if this is your problem.