Pay no attention to the faults of others, things done or left undone by others. Consider only what by oneself is done or left undone.
For me, the constitutive element of an affair is the secrecy. It is the secrecy that leads to the lying, to the deception, to the duplicity. It is the structure of an affair - not the sexual or emotional behavior or what people actually are doing.