I think ray-tracing can look really good in simple environments, but the implementation should depend on what the art style calls for. Animal Crossing makes liberal use of diffuse (almost lambertian) materials for the environment and even many object models, for instance, so while boosting the specularity on all of the materials in the environment just to showcase ray-traced specular reflections would look cool, it could also deviate from the style a bit. On the other hand, ray-traced diffuse interreflection would be well-suited to the style.
Note the key art below:
It's vibrant, with simple, matte-like textures. The grass, trees, and characters, etc. all have a mostly diffuse, cozy look (with some specular sprinkles) that really contributes the game's aesthetic. Even the more reflective objects are primarily reflecting highlights from clear coats on top of diffusely colored surfaces, while their Fresnel reflections are more glossy than real life (likely artist-driven).
Keep in mind, the key art has the benefit of not needing to be rendered in real-time. If the artists wanted to feature specular reflections more prominently, they could have (and financial budget wouldn't have likely impeded this either). They didn't, likely because the simplicity is intentional and crucial to the style's aesthetic. Adding prominent specular reflections (ray-traced or otherwise) would introduce a level of visual complexity that wouldn't otherwise exist in this style.
That being said, as I mentioned before, there are other forms of ray-tracing (like global illumination, shadows) that would be well-suited to the style, but the point is that it needs to be used in a way that respects the style. The goal should be to enhance without overwhelming, maintaining the simple, charming look that makes Animal Crossing so appealing to casual gamers.