Reported in https://github.com/huggingface/huggingface_hub/runs/6894703718?check_suite_focus=true

When trying to patch `scipy.io.loadmat`:

```python
ModuleNotFoundError: No module named 'scipy'
```

Instead it shouldn't raise an error and do nothing

We use patching to extend such functions to support remote URLs and work in streaming mode
Patching fails for modules that are not installed or don't exist