No. It is not possible to deactivate contactless payment on your physical card.
For your security, you can block the card via the app and unblock it at the time of purchase. In your app, go to Cards > Physical Card or Virtual Card and select Temporarily block.