If you are asked

Le dà fastidio?

the person is not asking you if you are fastidious! The word fastidio means “irritation”
or “bother” so the expression means “Does it bother you?” or “Do you mind?”

Someone may ask, for example:

Le da fastidio se fumo? – Do you mind if I smoke?

Unlike in the UK, where we tend to say we do not mind even if we do, you can be honest in Italy!

You can reply:

Mi dispiace ma si – I’m sorry but it does bother me / I do mind.