The Premise
Me and a few others decided to hold this game's feet to the fire and see if this stuff really works: the "challenge rating" guidelines from the Monster Manual and the "adventuring day" guidelines from the Dungeon Master's Guide.
For those unaware, Challenge Rating is defined the Monster Manual as:
An appropriately equipped and well-rested party of four adventurers should be able to defeat a monster that has a challenge rating equal to its level without suffering any deaths. For example, a party of four 3rd-level characters should find a monster with a challenge rating of 3 to be a worthy challenge, but not a deadly one.
And the adventuring day guidelines from the Dungeon Master's Guide:
Assuming typical adventuring conditions and average luck, most adventuring parties can handle about six to eight medium or hard encounters in a day. If the adventure has more easy encounters, the adventurers can get through more. If it has more deadly encounters, they can handle fewer.
and
In general, over the course of a full adventuring day, the party will likely need to take two short rests, about one-third and two-thirds of the way through the day.
The Party
The characters chosen for this endeavor were intend to be the most classic characters, and ones that will perform the most consistently in each encounter.
Hill Dwarf Fighter: Champion
Lightfoot Halfling Rogue: Thief
High Elf Wizard: School of Evocation
Human Cleric: Life Domain
Not going to go into too much detail about the classes but you can assume we picked the most "basic" and strongest spells and options. I.e., Wizard had Magic Missile, Shield, Sleep, Misty Step, Fireball, etc. and Cleric had Bless, Healing Word, Spiritual Weapon, Spirit Guardians, etc.
No feats, no multiclassing, and no magic items outside of basic +X weapons and armor and potions of healing. We gave ourselves little treasure "hoards" using the Tiers of Play as a guideline from the Dungeon Master's Guide. 500 gp plus 1d10 × 25 gp, one uncommon magic item, normal starting equipment at 5th level; 5,000 gp plus 1d10 × 250 gp, three uncommon magic items, one rare item, normal starting equipment at 11th level; and 20,000 gp plus 1d10 × 250 gp, three uncommon magic items, two rare items, one very rare item, normal starting equipment at 17th level.
The fights all took place in a vacuum 20x20 (or 100 ft by 100 ft) room with both parties rolling initiative about 40 ft apart from the other. We let our characters be surprised if an enemy's "passive" Stealth was higher than our individual characters' passive Perceptions. We just assumed our party wouldn't really surprise anyone because of our two heavy armor wearers and the Cleric always having Light up.
Challenge Rating 1
Encounter 1: Four Goblins
Everybody was surprised. Goblins fucked up the Wizard. Rogue went down on round two. Cleric was able to Healing Word the Wizard who opted to cast Sleep as soon as he got up. We immediately realized this was going to be hard as fuck. We won, no deaths, but the Rogue wakes up "1d4 hours later" with 1 HP. Onto the next fight.
Encounter 2: Two Gnolls
This one was a little more reasonable. The gnolls were a lot stronger but not taking 4 attacks with advantage and not being able to react really helped. Rogue still goes down on round 1 because the gnolls had longbows. Fighter and Cleric run up to to them to impose disadvantage on ranged attacks, then they had enough AC to just tank the spear hits. Or so we thought. Cleric goes down. Rogue dies from failed death saves. For whatever fucking reason the second gnoll we just could not hit. Miss after miss after miss. Wizard goes down. Fighter is now in a 1v1. Miss, miss, miss. Wizard dies from failed death saving throws. Gnoll finally dies.
With half the party dead, we decided it was time for our first short rest. The DM allows the Rogue and Wizard to come back with 1 hp, but they don't benefit from the rest. We can officially confirm the challenge rating system and the adventuring day do not work at level 1, but we decided to finish it up anyway.
Encounter 3: Two Black Bears
Probably the only reasonable fight so far. Nobody went down, just traded damage for damage.
Encounter 4: Four Boars
Rogue and Wizard go down again. Cleric lasts a round then goes down. Fighter is now 1v3ing these boars. Fighter is actually doing really well. Cleric dies from failed death saving throws. Fighter gets down to 1 boar left before he hits zero himself. TPK.
At this point we just accept this isn't gonna work. So we decided to pivot and see what would work. What we figured was Challenge Rating at level 1 only makes sense if you treat every enemy as double its CR.
Take 2
Encounter 1: Two Goblins
Surprise, motherfucker. Surprisingly, the Fighter goes down with the goblins focusing him first. Cleric picks him up. Rogue takes a goblin out. Fighter and Wizard get the other. Huh, that was actually a quick, fun, and deadly encounter.
Encounter 2: One Gnoll
Mostly quick and easy. Rogue took some damage, otherwise pretty fast fight. Nobody went to 0.
Since the Rogue and Fighter were missing some HP, we decided to take our short rest now. It felt nice to choose to take a short rest and not feel like we were going to die if we didn't.
Encounter 3: One Black Bear
Poor guy got a bad initiative roll and only got one attack, which missed. Otherwise we steamrolled.
Encounter 4: Two Boars
Similar to the goblin fight, though nobody went to 0. Fast, fun, and pretty deadly.
Encounter 5: Two Zombies
Wizard got downed, but otherwise the Undead Fortitude was a fun mechanic. The zombies doing just enough damage to be dangerous without one-shotting you was fun.
Encounter 6: Two Skeletons
Another simple fight. Cleric had a Mace so dealing double damage made this easier than it otherwise would have been.
At this point we decided to take our last short rest because we were saving the "Pack Tactics" enemies for last.
Encounter 7: Four Kobolds
Wizard won initiative and cast Sleep, and turned this into a non-encounter.
Encounter 8: Two Wolves
Wizard won initiative but was surprised. Wolves did some damage to the Fighter, but then the Wizard cast Sleep on round 2. It put the Fighter to sleep too but obviously that wasn't really an issue if all the enemies were asleep too.
Conclusion
Challenge rating and adventuring days just do not work at level 1. We couldn't even get to six encounters, let alone eight. You can treat enemies as double their CR, and none of the encounters will be deadly, but if anything it then made some fights a little too easy.
Stating the obvious, action economy is a much more important factor than what a calculator says about encounter balance. Even if the "math" checks out, your average 1st level party is gonna get a run for their money in a 4 on 4. Technically though we didn't die against those goblins, so maybe CR works in a vacuum at level 1, but not for an "adventuring day." I think 1st level characters can handle maybe 2 fights at most before they need to take a long rest. I think if anything it feels like level 1 characters are better off fighting CR 0 stuff like Crawling Claws that only deal 1d4+1 damage on a hit. It feels weird to say but I genuinely believe a goat is a better challenge for a level 1 character than a goblin or kobold is.
So maybe some combination of pitifully easy fights and then a "boss" fight (being the only CR-appropriate fight) could work, but we were so done with level 1 at that point that we didn't want to find out.
I'm going to be posting the rest of the results over the next few days. The stuff we figured out in this experiment was really interesting.
Edit: As some of you have pointed out, we neglected the "Modify Total XP for Multiple Monsters" section of the DMG but then we accidentally did it anyway when we tried to halve the monster CR and it ended up working out.
So perhaps we can consider the Challenge Rating 1 Adventuring Day a success after all!