How long and when can I leave my puppy alone?

Hi- my husband and I just got our 10.5 week old long haired mini doxie. We have been working on training (and surviving) for the last few days- we've only had him for 5 days! He is making strides each day but so far doesn't love being in the pen. If we walk away for just a moment he starts crying. I know this is to be expected of course and the recommendation is to work your way up. However, we do both have to work. Tues/Wed/Thurs he would be left in the house (the pen with pads, his crate, toys, etc) for the roughly 6 hours that our schedules overlap. So many of my coworkers/peers say it's totally fine-expect crying, accidents, etc. but I am extremely

Worried. We've seen him act out in the pen in distress and I'm so fearful this will happen again and he will hurt himself or worsen his mental heath. What's the appropriate recommendation? If it's not great to leave him so soon, how long is okay? What should we be working up to realistically? I feel shitty asking my family to help and feel like it is also just a bandaid but I'm open to whatever options.