I didn’t finish it yet but I think Metroid was always clearly on top due to better understanding the appeal of the genre. It has the most memorable power ups, the better level and encounter design, the better world design. I don’t think the Castlevania games were ever in the conversation, the rooms’ level design was much less involving and the games didn’t really use that much the whole aspect of getting new abilities to access new areas and improve your movement options. They were closer to action RPGs.
As for contemporary games, I think Hollow Knight is a great game and it was my definite second place after Super. I’m not sure whether it’s better than Dread or not. I really liked how Hollow Knight gave you multiple routes to progress, you could really feel lost in that game. However, Dread has better moment-to-moment gameplay and it really highlights HK’s flaws. In that game the power ups aren’t really the most memorable, your character looks very similar whether it’s in the start or the end of this 50 hour game. On top of that a lot of useful “utility” abilities are locked to badges, so you would end up never using them. Stuff like the map indicator, coin magnet and even the moving while you heal shouldn’t be taking badge slots. Even then, both games are pretty top notch at what they do, they have different design philosophies though. One is basically a 2D Dark Souls and the other is a new Metroid game that implements multiple aspects of past games in the series while noticeably improving on the action part of the gameplay.
As for other modern Metroidvanias, I’m missing a few (haven’t played Ori, Blasphemous, etc yet) but Dread is clearly on a different league than what I played. Dead Cells is an action roguelike, I don’t think the first few runs when you collect all runes are comparable to actual Metroidvanias. Steamworld Dig 2 is decent and one of the rare indies that actually leans closer to the Metroid side over the Vania side due to its power-ups and level design. However, it’s still essentially a souped-up version of the Flash game Motherload. It’s good but I would rather play any Metroid game over it. The third Shantae game is apparently considered a Metroidvania but I think those games are closer to sidescrolling Zelda. They might have took Metroid’s map for that one but it’s still a collection of isolated areas ending in a dungeon. I guess it can be good gaming confort food if you really miss those genres but it really doesn’t do anything that Metroid (or Zelda) didn’t already do in a better way