I have done very careful testing and it is indeed the internet modem, only on 30m though. WSJT-X reports a 9 dB drop in noise level when I unplug the modem power supply or the phone line from the modem, when tuned to an unused frequency in that band specifically (with receiver AGC off). Another ham, who was on the committee that developed the standard for this type of modem, pointed out that it is possible for the ISP to drop the power in the modem channel(s) that are causing the problem, but I haven't tried finding the right person who can do this yet.
I would imagine this is a pretty common situation around the world. The affected bands may depend somewhat on how the different ISPs have set things up.