As someone who is strongly against UBI, and strongly supportive of welfare I can earnestly say people simply not working is not at all the reason I (or most people against UBI in my opinion) are against it.
The reason i am against it is because it causes people more harm than good. People who are in a position where they need assistance need to be given the tools to get out of their situation, and the help to get there needs to be conditional on this (and we should be spending the money that goes with that). Financial assistance should be conditional with mandatory job training or mental health therapy needed to help someone succeed, not just money.
In fact when there are underlying bad habits, as can often be the case, it is possible money can even make a persons condition worse and cause them to sleep farther into poverty.