Stream Video to Texture

[audovoice]

Hello, I joined basically to ask how feasible it would be to take a video stream and make it appear on a surface for in game.
I have done a certain amount of due diligence researching the topic but have found nothing that covers exactly what I want to do.
======================
Stuff I read after googling to the furthest extent of my googling skills

http://lists.ioquake.org/pipermail/ioquake3-ioquake.org/2010-April/004077.html
"In theory, this isn't too hard (well, at least the case of a camera in the corner of the screen)."
"If you want to use a video as a texture (now we're getting tricky)," he then goes on to say he does not think a swf could be used but a swf really is not the same as streaming video so I am not taking that as a solid it is impossible.

"You'd probably need to add code to the main loop to update any "video
textures", and if done properly, there shouldn't be much to change on the actual rendering."

I think this is true, well maybe true. It might be able to be done in the form of a shader.

http://www.quake3world.com/forum/viewtopic.php?f=10&t=44888

This script that is no longer around to be downloaded for me to look at uses shaders to change what the displayed texture is, well it says it does this by changing the alpha setting. So it is not exactly just switching textures mid playthrough.

http://wiki.blender.org/index.php/Dev:Source/GameEngine/2.49/VideoTexture
http://www.tutorialsforblender3d.com/Game_Engine/VideoTexture/VideoTexture_Movie_1.html
Blender's - Game Engine -Video Textures give some insight into probably how this should work in an ioquake based game. It explains that it gets a texture and replaces it with a temporary new texture and then each cycle of the game engine it calls refresh on the texture. All this method does is invalidate the texture and when it loads the texture again it loads the new texture instead. From the games perspective it just keeps loading the same texture over and over again, every frame. This seems like a much more backward compatible approach.

A large square texture is added to some map. On a standard client it is a standard texture. On a modified client the texture is just updated provided the video loads.

I have done some programing into getting raw youtube streams. It is not that hard, well given api support anyway. Either the youtube video id is used to call up the video information packet and then parsed for the list of streams or the main youtube html is loaded up as a browser would and then parsed for the list of streams. There are usually around 5 different video streams of different types an quality. The logic then goes to just check what if available and then load whatever fits the preferred quality best.

However there is still the matter of getting something to decode a video that is still downloading. FFmpeg seems to get used in a lot of projects. It is under GPL like open arena but I don't know if it is suited to reading from a buffering video file. It seems to like to take a file, decode it, and then send it elsewhere. Either to a file, or onto the internet as a video stream. I have not looked at the code for voip or saving video but I would not be surprised if one of them uses FFmpeg. Anyway if it is not the right tool for the job I know there is a right tool for the job. Just needs some digging to find it.

========================
APPLICATION OF DOING ALL THIS: In which there are fanciful delusions of everything working out perfect.

Basically I like group youtube. There are a few of them out there. A chat program hooked up to a video player for youtube videos. In the example I like you just say "tv play oO3Miysfhss" or posts the youtube link https://www.youtube.com/watch?v=oO3Miysfhss and the video plays. Upon posting a video link the video goes into a loto along with all other videos posted by users and one is chosen, max one per user, and then the video is played and the loto is emptied. The beauty of the system is that is just needs a few lines of text to be passed from the server to the clients. That being what video won by youtube ID and how far into the video it currently is. That way everyone sees the video at the same time and it is a group experience. Because some videos are long and or annoying the video only plays for a certain amount of time plus additional time if users say "tv yay". Saying "tv sux" does the opposite. A leveling system determined the power of tv yay and tv sux. But in a first person shooter it could be based on something like number of frags or ranking.

This idea could be super slick and polished but it could also just be implemented as text sent from the server that is then interpreted client side by a script so that video playback is synched. All the server would need to do is keep track of what videos are submitted and randomly pick one, and then keep track of how far into the video it is. So long as everyone calculates video timelock by the same rules videos will stay in sync.

Obviously you'd be hard pressed to watch a video and play OpenArena at the same time but you could casually frag some of your friends while that plays and when you are bored of fragging you can go back to videos. In my experience people would probably find it funny to post trolling videos, or videos that directly relate to the situation at hand. But people aslo like to post a lot of music videos. When they find a new video they like they like to share it. People get into posting the same video over and over and joking about it. Or watching some marathon of some series with other people. I just think that this social behavior would be more enjoyable with the application of more rockets.

Also
Oh and if it was figured out how to do video replacement of a texture then photos, or webcam probably would not be hard to implement from that point. And the webcam ability is something people have apparently been looking into for a while.

Also Also
Obviously webcams, photos, youtube could all be abused but people can always be kicked from a server.

[fromhell]

The issue with this is the texture loader in the renderer checks for a duplicate texture of the same name and doesn't load it.  You'd have to make a new texture loader exclusively for client's video textures.

The RoQ format video playback in the game uses the *scratch textures IIRC.  videomap also plays RoQ but without sound and only one can be played.


for OA this would be cumbersome (large video processing overhead) and isin't something I would integrate personally, especially since the codecs required for Youtube would inflate the stack greatly.  There is also a privacy concern

[audovoice]

Thanks for your reply. I get the sense this is one of those things where it is unclear how hard it will be at the start and it needs some investment of time upfront to figure out. Time spent learning more about handling incoming video steams would not be a waste of time to me because of other projects but learning OpenArena code would be if I can't with in reason do the gimmicky thing I want with it.

The unity engine could do it, and make a 3d environment but it requires a pro license for the video texture feature. 

Blender Game Engine also has the feature. But I don't know of any projects for it that I could work off of and I know I am not up to the task of working from scratch.

See I don't even know how someone normally sets a texture. I am guessing if a surface is set to a second texture it will work but the first texture, even if it is no longer needed anywhere, will not automatically be unloaded from memory as this is C or c++ code. But that does give me an idea. See I did not explain exactly how Blender Game Engine described its process. I described it as changing the texture and then loading it again because I thought that would be more easy. But what they actually said they do is change the actual texture loaded into memory. I forgot this was C++ code. Just going in and changing what is at some address directly is normally a big no no not even allowed in later languages but it can be done in C++... I think. C does not have real concepts of public and private, but c++ does. You would also have to be damned sure a texture was not read while it is being update, and on multi cores that is more tricky. If you can switch between drawing one texture while updating the second and then the reverse next frame then it would probably be ok. You would just have to be sure too always replace the new texture with one of exactly the same number of bytes.

But I could be off base entirely. I just have a feeling something like changing the address of the texture in memory to be a different one, or changing the memory that address points to, or maybe just messing with the value it checks to see if it is a new texture so it always thinks it is a new texture. IDK, my C is weak and I have a feeling the code is a bit confusing but I will try to find that part of the code today.