There are a lot of good freeware tools available now, you don't need to go shelling out loads of money for sitebuilding apps. In many cases, the freeware software is of a very high standard, and it's quite possible to build a basic site yourself without paying for software.
Graphics
Unless you're making your site text-only, you'll need some kind of graphics editor.
PaintDotNet - I tried this out recently and it's pretty good. There are a lot of third-party extras, such as filters. The brush support is very limited at the moment, though.
The Gimp - May take a little getting used to, but once you do it's a pretty powerful piece of kit. There's also a version called
Gimpshop which aims to emulate the Photoshop interface, although I've found it a little buggy.
Serif PhotoPlus - Another, slightly more traditional image editor.
JPEG Cleaner - Optimise your .jpg graphic files
ColorCop - For colour sampling. Useful for converting RGB colours to hex values for CSS.
Code
In theory, you could use Notepad for creating all the code, but it's much easier if you have a few more features
Nvu - Lots of features (WYSIWYG mode, text mode, FTP, CSS support) and it's extensible too.
Arachnophilia - Slightly more hardcore (it's a text-only editor). This will run on all platforms and is highly customisable.
Resize Browser - Handy tool for seeing how your website looks at different window sizes.
Notepad++ - I love this text editor. It has loads of functions, like autocomplete and syntax highlighting. It also handles a wide variety of languages on top of the basic XML, HTML and CSS.
Fancy Stuff
Text and graphics may be a bit too static, so here's some stuff to spice up your site.
RSS Builder - Add a news feed.
Lightbox - Javascript to handle showing images in modal boxes.
AJAX - Can't think of a short description that'd sum it up adequately, but go have a read on the site
Upload
To get your website from your computer to your webspace
FTP Explorer - It may look fairly basic, but it's fast and has all the basic features you'll need from an FTP client.
FileZilla - Installed this recently, and it seems to work pretty well.
CMS
CMS = Content Management System. Think of a CMS as a kind've web-based application that helps keep your site content organised and easy to maintain.
Textpattern - I really like this as it's easy to use but still pretty powerful.
Wordpress - One of the most widely used CMSs around. Very flexible due to the wide variety of plugins and extensions.
Drupal - Very powerful, but probably one of the more difficult CMSs to get to grips with. It includes a lot of community type modules, such as forums, polls etc.
Joomla - Pretty popular and quite easy to use and setup.