Would it make much of a difference if I omitted them? What would a suitable replacement be?
Yes, it will make a great difference. Note, however, that in most recipes anchovies should be used sparingly. You should get a hint of the taste of anchovies only. Like you taste them, but do not directly identify it as anchovies.
There is no good replacement for anchovies. They make a great ingredient for many dishes. Unfortunately they are quite expensive here in Singapore.