This is an experimental graphics engine I've been working on - on and off - for a few years now.
It's actually a voxel and polygon hybrid. The original model is a heightmap mesh, with some models for trees, buildings etc. As an offline preprocessing step the entire world is converted from meshes into voxels, which is rendered at runtime using a raycaster written in CUDA. The renderer retains the ability to render the polygon mesh as well, which it can use to render close up content so that it's not blocky. The two rendering modes actually complement each other very well. Voxels work very well for far away content (due to the octree's inherent level-of-detail mechanism) and polygons handle close up content very well. For continuous surfaces like the heightmap they fit together seamlessly.
The engine features fully dynamic lighting and shadows, water reflections, and no view distance limit. These effects fit together surprisingly seamlessly despite the rather different rendering pipelines. There's no removing far away objects or switching to lower detail meshes, because far away content is handled by the voxel renderer.
The main challenge is switching from polygon to voxel rendering without obvious seams/popping though. And some voxel inherent challenges like the "thin wall" problem.
There's a bunch of videos on my YouTube channel showing the progress over time.
In theory this approach could be applied to 3rd party graphics engines, although there's quite a few caveats, mainly around how the shaders would need to be restricted.