Introducing crate training STRUGGLING

So I got a puppy for the first time about a month ago. He is about a 4 month old Corgi. Initially me and my partner were gonna crate train him but I thought that using a pen instead would be the better decision since I felt bad having him cooped up in a crate(first time dog owner bare with me here please). We also used pee pads since we noticed he would pee fairly often, we had one pee pad outside his cage inbetween the bathroom and one inside his pen. We also fed him three times a day, morning, lunch and dinner. We would take him out shortly after he ate and initially he did poop everytime we took him out which was great! We believed that as his bladder developed he could hold it in longer and we would just transition from pee pads to outside only. Except i've started having a problem where he doesn't want to poop or pee if I take him outside anymore, he only goes potty on the pads and occasionally on the carpet or where ever else he's not supposed to. His pooping has also just gotten so inconsistent that I just never have any idea of when to take him out, I swear whenever I decide to wait a while before taking him out he chooses to go right after I fed him but if I want take him out after he eat he suddenly doesn't have to go until night. I mean I am losing my head. So anyways I figured he's just getting used to the pads and going in his pen since there's enough space to go potty whenever he wants. So that brings us to today. I decided to cold turkey the pads, start using the crate and take him out every 1-2 hours. The problem im having is introducing him to the crate, I tried to lure him in the crate with treats and getting comfy walking in there but when I tried leaving him in there for a couple of minutes (with me in the room nearby) he starts freaking out and going crazy, since im sure hes used to free roaming and being able to hop in and out his pen whenever he likes. I've seen people say that you have to leave your puppy in there in order for him to get used to it and only let him out to go potty until he picks up the routine. On the other hand i've seen people say you have to introduce him to the crate slowly with short sessions so he doesn't start to dislike being in the crate, which I would like to do especially since he's a little stubborn and I know its gonna come back to bite me in the ass. My problem is that without the pads im worried he's just going to go potty whenever, especially on the carpet, since he's used to that. If anyone has advice it would be much appreciated.