Yep,
A Hub (Prolly Switch nowadays, they're just as cheap for a cheap 5-port job)
CAT5 cables, Ethernet (LAN) Cards or Network cards to some people..
Windows (Preferably) on all machines for better compat (Games wise)
Connect a cable to each machine then the PC's (obviously)
could do with knowing what windows you intend to run, but I'd recommend you use Static IP's for all the machines, its easier when gaming and hosting.