Do you know what kind of compression ratios you get out of curiosity? Presumably it would be lower than this format (because this format is meant to be lossy not lossless) but very curious as a baseline.
A gaussian gif. Coming to porn sites soon. The file format name works for them. But imagine whole movies shot in this format. The fly-on-the-wall fantasy of movies, without being locked in place for the whole shot. Narrative possibilities like being able to examine just how close the Tyrannosaurus Rex is to your rear view mirror. Clues in a mystery only visible on rotation. Hidden bonus scenes.
I hope this catches on just to be able to watch the evolution of cameras to capture it.
Ok but you're commenting on the general concept of animated gaussian splats. That's existed for a while and it's unrelated to what this actual post is about which is a new compression method.
I suppose it's the difficulty in recording them rather than the file size that has kept them experimental. But that's not a problem for AI generated splats. GenAI sites could have a toggle from image to video to splat4d.
>imagine whole movies shot in this format. The fly-on-the-wall fantasy of movies, without being locked in place for the whole shot.
That would be terrible. Framing is the major expressive feature in cinematography, and any interactive format needs a lot more thought put into it than just having a free camera. Literally the worst of both worlds.
Light field video streaming is a thing, however it's pretty niche. OTOY pioneered holographic codecs and light field videos with some degree of freedom and a sense of depth more than a decade ago.
It wouldn't work for a movie, but it might be good for a concert video, stage play, dance, or sports, where you're capturing something happening in a defined space. You could pick a different seat in an auditorium.
Also consider connecting a virtual space with a real one, so it looks like something happening on a virtual stage that's connected to a room in your house.
That already exists though. I believe Braindance VR uses a rig with a couple dozen cameras to capture the same scene from multiple viewpoints then converts it to a gaussian splat that can be walked around.
Its great for porn for those videos when the camera seems to be focused on the actors balls when thats definitely not the part I want to be looking at. I can just look around the room instead.
A$AP Rocky did it for a music video. Granted, it was only used for the editing process and not the final video, but it still presents some interesting opportunities, though it doesn't really look real so the applications may be limited.
Something’s way off with these numbers. The page says it encodes video at 640MB/s which is quite large even for 4D data and doesn’t match the filesize of the demo splat (7.4MB / 2sec, or ≈3.4MB/s).
In fact they say the raw file size of the demo splat was only 427MB, so maybe the 640MB/s was a statement about encode speed? Why write it that way instead of “this demo splat was encoded in 0.6sec” or even just “the time to produce the original splat took longer than the time to encode this video format”?
I feel like I did this one already: https://github.com/seanwevans/4splat
spatiotemporal data
Well there's your problem, you didn't say you wasted a million dollars doing something an expert can do in an afternoon.
I mean it’s probably in the training data too
Wow dude. That is a hell of a README. Great job!
Admittedly just scanned this but that’s lossless, right?
Do you know what kind of compression ratios you get out of curiosity? Presumably it would be lower than this format (because this format is meant to be lossy not lossless) but very curious as a baseline.
Your readme is really mobile unfriendly, FYI
A gaussian gif. Coming to porn sites soon. The file format name works for them. But imagine whole movies shot in this format. The fly-on-the-wall fantasy of movies, without being locked in place for the whole shot. Narrative possibilities like being able to examine just how close the Tyrannosaurus Rex is to your rear view mirror. Clues in a mystery only visible on rotation. Hidden bonus scenes.
I hope this catches on just to be able to watch the evolution of cameras to capture it.
Ok but you're commenting on the general concept of animated gaussian splats. That's existed for a while and it's unrelated to what this actual post is about which is a new compression method.
I suppose it's the difficulty in recording them rather than the file size that has kept them experimental. But that's not a problem for AI generated splats. GenAI sites could have a toggle from image to video to splat4d.
>imagine whole movies shot in this format. The fly-on-the-wall fantasy of movies, without being locked in place for the whole shot.
That would be terrible. Framing is the major expressive feature in cinematography, and any interactive format needs a lot more thought put into it than just having a free camera. Literally the worst of both worlds.
Light field video streaming is a thing, however it's pretty niche. OTOY pioneered holographic codecs and light field videos with some degree of freedom and a sense of depth more than a decade ago.
Not terrible, per se, but not a movie. There are other formats that fit the task better like immersive theater and theater in the round.
This has been extensively explored with VR and games, though.
The “per se” is never necessary.
I don't think their writing is supposed to be an exercise in saving bytes. Per se.
It's less bytes than "intrinsically" or "by itself", so not bad, per se.
It wouldn't work for a movie, but it might be good for a concert video, stage play, dance, or sports, where you're capturing something happening in a defined space. You could pick a different seat in an auditorium.
Also consider connecting a virtual space with a real one, so it looks like something happening on a virtual stage that's connected to a room in your house.
That already exists though. I believe Braindance VR uses a rig with a couple dozen cameras to capture the same scene from multiple viewpoints then converts it to a gaussian splat that can be walked around.
Its great for porn for those videos when the camera seems to be focused on the actors balls when thats definitely not the part I want to be looking at. I can just look around the room instead.
A$AP Rocky did it for a music video. Granted, it was only used for the editing process and not the final video, but it still presents some interesting opportunities, though it doesn't really look real so the applications may be limited.
https://radiancefields.com/a-ap-rocky-releases-helicopter-mu...
So a video game?
The application is cool, but there is little novelty here. All of the employed techniques are well-established.
I suggest removing "novel" from the title unless you wish to seriously disappoint some people.
But that’s exactly where we’re at. Disappointment is fine, attention is all you need.
I tried to apply existing mp4 H265 like compression to 3D splats. Do you have any links to other 4D splat formats?
Doesn’t with for me, iOS Safari :/
Works on my old iPhone. The video is a splat. Click the interact button to change perspective.
Moving camera completely distorts the video both on my Mac and iPhone.
Me neither on Chrome on Win10. "WEBGPU initialization failed".
Works on my iPad using Chrome or Safari.
Safari on iPhone, working as intended for me as well.
It's working on Android Chrome browser
The viewer uses WebGPU... I could support WebGL for greater compatibility.
See repo for prompt. Feel free to contribute to format or benchmarking. MIT Open Source.
Publish full session md, you can export them with `uvx claude-code-log@latest --tui`
warning: don't just run random commands in your terminal
specially not npm/python related packages with @latest
Yes please give me the latest supply-chain attacks.
too late, I already ran curl | sudo bash
Somehow that feels too private to want to share.
Software created by algorithms is public domain.
MIT license is basically that
It isn't.
https://adamraudonis.github.io/splats4D/
Does switching scenes in the demo work for anyone? There are what, 3 hidden panels in the HTML there? But none of them have a "switch scenes" button.
> a splat is static iff a single quantized value
What is up with LLMs and "iff"?
Well, iff is a useful abbreviation. I use it in personal notes at least, and would use it more often if I thought it was more widely understood.
Something’s way off with these numbers. The page says it encodes video at 640MB/s which is quite large even for 4D data and doesn’t match the filesize of the demo splat (7.4MB / 2sec, or ≈3.4MB/s).
In fact they say the raw file size of the demo splat was only 427MB, so maybe the 640MB/s was a statement about encode speed? Why write it that way instead of “this demo splat was encoded in 0.6sec” or even just “the time to produce the original splat took longer than the time to encode this video format”?
“encodes video at 640MB/s” is the common way for codec authors to talk about performance
Whats the 4 in the "4D" here? A layman like me thinks this looks 3D.
time
Why is it we don't call videogames "4D" if they also operate through time?
I guess if moving through time other than forward like normal it'd justify calling time a new dimension on which you can play on.
Otherwise, we just focus on the 3D space they represent and take time for granted
3D Gaussian splats are still images of 3D space / photogrammetry
So I'm guessing it came from representing the additional dimension
Animations are four-dimensional. Colored animations are seven-dimensional. More if they include surface normals or lighting information. Simple.
wow… this is seriously cool.
Probably shoud've noted that say Chrome requires some flags enabled
https://webgpureport.org/
Looks like crap, is it the format?
I'm so tired of HN putting sites on the top with zero human authorship. I have to listen to Claude-voice enough at my job.