How do I get over the crate guilt???

My spouse and I just got a 2.5 month old black lab/golden retriever pup and he's the sweetest boy. We adore him so much, and while we've only had him for 4 days now, he's been such a good boy. He's fine with the crate (for the most part) it's the amount of time we leave him in there. We leave him in there during night except for potty breaks, we let him have about 20-30 minutes of free time before we put him back in and go back to sleep. We both work full time, so he's also in the crate while we're away. He's done fine so far with us at work in the crate, no accidents, no destruction. So if he seems fine with it why do I feel so guilty about leaving him in there for long? I know this isn't forever but I just get overwhelmed with anxiety thinking about leaving him alone. I've helped my parents raise puppies growing up (we had 6 dogs) but this is my first time raising one while not living at home. Just looking for advice to get over this crate guilt.