Crush Oreos into fine crumbs with some small chunks for texture. Place them in a ziplock bag, remove air, and crush with a rolling pin—15-20 whacks should be enough. Avoid over-crushing into dust.
Beat softened cream cheese and butter in a large bowl until completely smooth and creamy, about 2 minutes. Continue beating if lumps remain.
Add powdered sugar and beat until fully incorporated. Scrape down the sides to ensure even mixing.
In a separate bowl, whisk instant pudding mix with cold milk for about 2 minutes. Let sit for 5 minutes until it thickens to a soft pudding consistency.
Gently fold the pudding mixture into the cream cheese mixture in two or three additions until no streaks remain. Fold in Cool Whip the same way to maintain fluffiness.
Assemble in a deep serving bowl: Layer 1/3 of the Oreo crumbs, then half of the cream mixture, spreading evenly. Repeat with another third of Oreo crumbs, the remaining cream mixture, and finish with the last Oreo layer.
For a "dirt" effect, place gummy worms on top, partially buried in the crumbs.
Cover and refrigerate for at least 4 hours or overnight for the best texture.