Rebecca,
> Once I have entered this rule, I cannot enter ANY new values, whether they are duplicates or not.
This happens because the object finds itself. You need to add another condition to check that the existing object is not the object being edited/created. The easiest way is to compare the object IDs. For example, here is the 'Uniqueness of login name' rule the system automatically adds to members of group SystemUser, which you can find on object RegularUser in your configuration:
If EXISTS SystemUser WHERE (SystemUser.LoginName = ThisRegularUser.LoginName AND SystemUser.ID <> ThisRegularUser.ID) Then
REPORT ERROR 'User with this login name already exists. Please choose a different name'