In the early summer, the first mile is a huckleberry haven (Shh, don't tell!). And you'll pretty much get awesome views for most of the trail since it starts so high up. After a hefty climb, the views totally pay off. It's 100% worth it to sit any overlook the Sweeney Creek drainage. And the lake, well, it's gorgeous year around.
The hike into Duffy Lake was approximately 7 miles with ~2600' gain. The first mile or two is fairly steep, then it will wiggle up and down the mountain until it drops down into the drainage to the lake. You can hike to Holloway Lake too if the trail isn't completely covered in snow. Late spring will still be fairly snowy, so beware and pack accordingly. Otherwise, summer and fall is a great time to hike or backpack on this trail.
You get a really nice array of open meadows, boulders, and forest through the whole hike. The trail is fairly well marked so you shouldn't have a problem navigating. Overall, this trail can definitely be strenuous at times, but it's well worth the pay off, and really nothing you probably can't handle if you hike.
For more info, check out the trip here: Snowqueenandscout.com