GLSL

[Gig]

Well, I think the markup of the old page, after the edits I did, was acceptable (well, I'm not an "artist" in wiki markup, so someone may do better). I think we may try to follow the same in the new page.

I think the page you are following should be generally okay (although Wikia may have some little difference from Wikipedia).
It should be more detailed than the short reminder I wrote here:
(DO NOT LINK) h t t p s : / / openarena . wikia . com/wiki/WikiWorks#Appendix:_Basic_MediaWiki_markup

[Hitchhiker]

I've cleaned a bit the wiki page (DO NOT LINK) h t t p s : / / openarena . wikia . com/wiki/Glsl_Tutorial_Test
Can it work like this or needs more polishing?

[Gig]

I'll look into it on Tuesday.

(Written from mobile phone)

[Gig]

For today, I checked up to DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Glsl_Tutorial_Test#Shaders_.28Quake_3.29]this section. I will continue in the next days.

[Gig]

Okay, I think I've fixed markup in the page. DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/index.php?title=Glsl_Tutorial_Test&diff=14276&oldid=14240]Are they ok?
There were many unnecessary carriage returns (e.g. in source code sections), and some "nowiki" where they were not required. I suppose they were added by Wikia's WYSIWYG editor: again, I suggest to NEVER use it, but always switch to the classic source code view before editing (you can even totally disable the WYSIWYG editor in your personal options).

In general, it looks to me a nice work. Congrats, Hitch!

I think you may better tell the difference between the two screenshots with and without glsl effect in ce1m7 map... the texture is different, but how? Does that include some yellow/greenish parts (musk maybe?)?

[Hitchhiker]

Hi Gig,
I did check the page and corrected one typo I could find. It looks OK for me too.
Do you want me to copy/paste the source to the 'GLSL Tutorial' page or simply rename this page and remove the other one? Let me know.
I think congrats go your way equally for all the testing, editing and all of the good suggestions so I think your name should also show on the tutorial page - hope you're OK for this.

[fromhell]

Do you have a simple refraction shader?

[Gig]

Do you have a simple refraction shader?

Do you mean something like this?
http://openarena.ws/board/index.php?topic=4767.msg47236#msg47236
(see video attached there; to test exactly that specific effect, see this post. The problem with that shader was when one had glsl disabled.)

Something similar is in the glsl test map...

[Gig]

Do you want me to copy/paste the source to the 'GLSL Tutorial' page or simply rename this page and remove the other one? Let me know.

I can easily delete the old version, and then rename the new version. I think we two were the only authors of that page, so no CC-BY-SA attribution issues.
What will be the final title? "GLSL" or "GLSL tutorial"? And if we choose to use "GLSL tutorial" as the title, what should I do with "GLSL" title? A redirect to the tutorial? A redirect to "Manual/Graphic options#GLSL effects"? A separate page with something (what?) in it?

PS: What about better explaining what one should notice in those screenshots?

[fromhell]

Having more GLSL fun again...  WIP 3dfx shader

[matija123]

Nice

[Gig]

Good job, Fromhell.   But with "3dfx" are you menaing the old, glorious 3dfx cards?
I didn't think 3dfx had GLSL support... latest 3dfx cards are from year 2000, and OpenGL 2.0 (the first OpenGL version that officially included GLSL, IIRC) is from 2004...  

I can guess with "3dfx" you mean "giving more depth to textures", instead... right? Not related to the old "3dfx hardware" at all...

[fromhell]

It's a shader that's meant to mimic their infamous on-chip DAC 4x1 filtering from the Voodoo Graphics and Voodoo2 cards which can play a part of the whole 3dfx nostalgia thing. (96 and 98 respectively)


The actual shader itself requires a video card supporting pixel shader 3.0.

[Hitchhiker]

Hi, was away from OA source for some time now. But just wanted to let you know I haven't abandoned yet. I think this December I'll have some time to fix those normals, escape keywords and hopefully fix few issues I'm seeing with glsl lights (just fyi..source not sent to the maintainers yet); tangent, binormal calculations (that come in handy in glsl programs); making possible to use multiple postprocessing effects at the same time(maybe send in the DOF, godrays postprocessing programs); ... hope it all works out nicely.
Hope you're all doing well.

[Gig]

Hi Hitch! How are you?   I was waiting for the official release of OACMPv1 to ask you about making a GLSL patch for it...
... and to propose you a few ideas to make GLSL support better. Well, for this last part, I'm going to tell you now, then...  

1) Current "freedom" of GLSL postprocessing mode may be theoretically used to get unfair advantage over other players (with programs that may allow to more easily see players, similar to brightskins, or adding more gamma in dark places): maybe we may put in a server-side variable (or a new DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Dmflags]dmflags or DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Videoflags]videoflags value?) to let server admins to decide preventing clients from running GLSL postprocessing programs (or, next step, even to force all of them to use the same postprocessing program -example: "connect to my glsl Edges server!"-, however that may not be completely fair anyway, considering that some players may not enable/support GLSL at all, or that a specific GLSL program may not run with some cards). What do you think?

2) The main problem with 0.8.8 implementation of GLSL seems to be the difficulty of making a certain shader look good in both "GLSL" and "standard" modes (e.g. "blendfunc" thought for a mode may completely mess up the shader look in the other one). Thanks to "glsl skip" program, it's possible to prevent extra stages designed only for "standard" shaders to be considered when working in glsl mode, right? But how to make the opposite, I mean be sure that a stage designed for GLSL mode only is not considered when running in standard mode? I can guess the best option would have been something like the "detail" keyword (according to Q3A shader manual, if you set a stage as "detail" -check here for usage infos and limitations-, that stage will not be drawn if the user uses the "r_detailtextures 0" setting), but I can guess that a similar thing would not be applicable for us, due to the lack of backwards compatibily: if the hypothetical "glslstage" keyword would be read by an older version, it would be ignored (then the rest of the stage would be computed in all cases) if behind a comment "//", or would just completely break the shader (showing a "missing texure" while in-game) if without a comment "//". Correct?
Then, how to workaround the problem? I can guess that one may make some stages executed in GLSL mode only, by setting "//GLSL" keyword in front of ALL THE LINES of that stage. Would that work? I'm not sure... IIRC with the current implementation there is some problem of GLSL lines/commands not completely ignored in some cases (when glsl support is disabled? When it is enabled, but the card does not support it? When a GLSL program fails compiling? I don't remember)... however we need some ways to use different "blendfunc"s for GLSL and standard modes.
Example:

[...]
{
//GLSL program <name>
}
[...]

for running the whole stage in GLSL mode only, would became

[...]
//GLSL {
//GLSL program <name>
//GLSL }
[...]

Would that work?
Time ago, while taking a look to a gamecode source for some reason, I noticed that some parts for code were "commented out" not by "//" on each line, but at entire blocks contained within "/*" and "*/" characters, IIRC (I'm not 100% sure of which characters they were, but I think they are these). With a quick test, it seems to me that this kind of "comment block" also works within Q3 shaders. Is it correct?
I would suggest you to foresee new "GLSL fake comment start/end" commands (e.g. "/*GLSL" and "GLSL*/"), to allow this:

[...]
/*GLSL {
program <name>
} GLSL*/
[...]

or (hopefully with the same result)

[...]
/*GLSL {
//GLSL program <name>
} GLSL*/
[...]

to have the effect of considering that stage ONLY IF you are running in GLSL mode, on a card supporting GLSL. Of course, in this example, the part to comment out is very small, but consider more advanced shaders...
Developers should be able to "GLSL fake comment (comment out if glsl disabled)" single lines or also bigger chunks.

3) Double checking how shader elaboration behaves in the various situations (glsl variable disabled, glsl variable enabled but video card not supporting, glsl enabled but one of the glsl programs failed compiling), allowing to safely display standard shader if necessary. One shader may theoretically contain more glsl programs: if even only one of them fails compiling, the result would be different that expected, so that entire shader should interrupt running in GLSL mode, and start again from the beginning, but in standard mode this time (of course, I'm not telling to disable glsl for all shaders, but only for that one).

4) Check/fix minor issues, e.g. for some reasons I ignore, postprocessing effect does not affect a few parts of the HUD interface (maybe the health meter, only when it's over 100 HP? I don't remember exactly.).

Hitch, I know I have been too "verbose", but please read!   I think this is important, for better GLSL support in future OA versions.
Of course, also Fromhell and any other one can say his/her own opinion about this stuff... and correct me if I said something wrong (as you can see, there are various things I'm not 100% sure).

Update: please read also this other post from the next day.

[fromhell]

Can there be a possibility for a second GLSL postprocess shader pass? My 3dfx shader could use it... possibly maybe as its own cvar

[Hitchhiker]

Hi fromhell,
Yes, I think it should be possible to do. Basically how it should work is to have /postprocess take few parameters - each being a separate effect so one could do something like:
/postprocess sRGB edges godrays
doing this would do all three postprocess effects in one pass.

[Gig]

I noticed that some current CVARs that hold more values use "/" to separate values...
Example: g_voteNames "/map_restart/nextmap/map/g_gametype/kick/clientkick/g_doWarmup/timelimit/fraglimit/shuffle"
(from DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Servers#Server_config_example]here).
If "/" character is not used to identify a postprocess GLSL program (due to all postprocess programs being in the same folder), I suppose you may use the same notation. If the program path (program name including subfolder: e.g. /glsl/hitch/programname or /hitch/programname) is required/supported, instead... I suppose we cannot use the same approach.

[Gig]

Hitch, in the message from yesterday, I forgot to mention another possible solution to problems regarding good working in both glsl and standard modes, for shader glsl effects.... the one based upon an idea from you, IIRC.
That idea was to use a distinct file to contain the "glsl" version of the shader. Now I try to guess how that may work:

One may keep main shader info in the standard .shader file: that would contain general shader properties (e.g. qer_editorimage, deformvertexes, surfaceparm, etc.) that are read by map editor, by map compiler and by the game itself (e.g. unlike some other "surfaceparms", it seems that "surfaceparm nomarks" does not require map re-compiling to be effective), AND the "stages" to be used when running the shader without GLSL. Right, a classic Q3 shader.

Then, one may add a new keyword somewhere (I would suggest before stages begin) in the main shader (using a "fake comment" workaround tor do not crash previous versions of the game), to tell the game (and the user) that there is a GLSL version of that shader available in a separate file. That may be just a keyword that mentions the existence of the GLSL version (then the game would automatically search for the same path/name of the main shader), or may also specify the name/path of the GLSL-powered shader (I don't know which of the two approaches would be better, let's think about that. Maybe the simpler one?). Examples: "//GLSL glslshader", or "//GLSL glslshader hitch/flames01", or "//GLSLshader", or "//GLSLshader hitch/flames01", or "//GLshader" or "//GLshader hitch/flames01" (just examples! One should also also consider that 0.8.8 will not understand the new keyword... what would happen?).

I'll call the GLSL-powered version of the shader "sub-shader" in the rest of this example, just for short. Except the keyword informing of the existence of the sub-shader, no other infos about GLSL rendering should be contained in the main shader.

Then, we would create a file with a specific extension (e.g. .glslshader, or .glshader, or .glsl, or .subshader, or something similar) and place it into either "/scripts" or "/glsl" folder in the filesystem (I don't know which one is better... we would have to decide one of the two). That file would contain one or more "sub-shaders" (like standard shader files do). Those sub-shaders files would be ignored by map editor/map compiler/older versions of the game, but would be considered by new versions of the game.
These sub-shaders would contain only the stages that would be used when displaying the shader in glsl mode: they would NOT contain general shader properties (deformvertexes, surfaceparms, etc.), and would NOT contain the stages used for standard mode. Hence, these sub-shaders would probably have less stages that main shaders.

As you can guess, the game, when having to render a shader, would just start from the main shader, read its general properties, read the keyword that informs it has got GLSL version (also considering it's not necessarily the last line before stages begin), and then:
- IF the user has GLSL disabled (or of it's not supported by the video card), game would continue computing the standard stages in the main shader; or
- IF the user has GLSL enabled, game would ignore the stages from the main shader, and go following the stage instructions of the sub-shader instead. Hence, the GLSL-powered shader would be shown on screen. If one of the glsl programs invoked by the sub-shader fails compiling (or if the sub-shader is not found), the game should abort operation and return to main shader, using the "standard" stages instead.

I suppose this one may be a relatively "clean" way to work, although it's quite different from the current implementation.
If you ask the reason for requiring to place a specific keyword just to tell the game the main shader has got a sub-shader (while it may theoretically auto-detect the thing due to simply finding a main shader and a sub-shader with the same name)... well the main reason is to let human beings reading the main shader know that a GLSL sub-shader for it does exist.

In case we decide to implement it, I don't know if we should force to use this "separate file" mode only, or also allowing the use a mode similar to the current one (letting the mapper decide which approach to use for each shader). Retro-compatibilty reasons would suggest to keep both systems, but I don't know how much 0.8.8 GLSL support has been actually used. I don't know. Should we inform users that 0.8.8 GLSL support is "EXPERIMENTAL" and its markup may drastically change in the next OA version?

[Hitchhiker]

Thanks for the hint on multiple arguments. I didn't know that. I imagine I can add a function that transforms sRGB*edges*godrays provided to the /postprocess into something readable by the postprocessing pass. Just need to agree on the character sequence that will be the separator (I imagine any character that cannot be part of a filename on dos, linux, macos, etc. can be used).
As for the shaders. Vanilla OA does not yet use //GLSL in any of them. Some maps out there might already. I would chose to use the //GLSL keywords as they are now (I imagine someone might find it easier that way) and would use as a mainstram this patch: Hardware-specific shaders (PATCH) -> http://openarena.ws/board/index.php?topic=4570.0
What do you think?

[Gig]

About other cvars that use multiple values, checking (DO NOT LINK) h t t p s : / / openarena . wikia . com/wiki/Auto_change_map and (DO NOT LINK) h t t p s : / / openarena . wikia . com/wiki/Manual/Controls it looks like that g_mappols variable uses "0\maps_dm.cfg\1\maps_tourney.cfg\3\maps_tdm.cfg\4\maps_ctf.cfg\5\maps_oneflag.cfg\6\maps_obelisk.cfg\7\maps_harvester.cfg\8\maps_elimination.cfg\9\maps_ctf.cfg\10\maps_lms.cfg\11\maps_dd.cfg\12\maps_dom.cfg\", and cg_weaponorder variable uses "/1/2/4/3/6/7/8/9/5/".
I have no idea about the reason why some of them use "\" and some others use "/"... maybe due to some error when they were created? Or maybe it is possible to use both "\" and "/" (maybe someone may check source code) I don't know.

About the patch you linked, could you please explain with words what it does? Thank you. Reading the first post there, I have the impression it is meant for something similar to this (additional hardware-specific shader files... working similarily to "sub-shaders" I thought, but without indication in main shader about the existence of sub-shader). But I'm not sure.

[fromhell]

Actually, staring at the 3dfx for awhile, I might need more than 2 passes... probably 5:


1. Dither and reduct the image to 16bpp
2. Blend all pixels just a pixel to the right, but not the ones with contrast (to keep edges)
3. Repeat 2
4. Repeat 2
5. Repeat 2


I don't have any OpenGL game or app that I know of that can do custom multipass shaders so I can't test my theory...


Once the shader is finished it could be part of OA itself as a new extra video option, since it might have practical use in debugging assets for low end machines on high end machines, visual testing for asset developers, as well as nostalgic value, since the 3dfx 4x1 DAC filter hasn't been part of mainstream PC gaming since 1999.

Other 3dfx-based limitations include bilinear filtering (forced bilinear by q3), max texture size being 256x256, and 16-bit textures at best...


Also what about postprocessing outside the game's 3d viewport, i.e. the HUD and UI interface (including the console)?

[Gig]

Just to understand... isn't it possible to make a cycle directly inside the GLSL program, to cause the elaboration repeat various times? Please forgive me if I said something stupid... I'm not inside proper GLSL language features! Maybe they did not include such kind of cycles, to avoid creation of GLSL programs slowing rendering too much? I have to admit I have no idea...

About applying glsl postprocessing to HUD, IIIRC some parts of HUD are already affected... I don't know the reason why some parts are not... I mentioned this thing as "minor issue" a few posts ago.
About extending postprocess to console, too, isn't there the risk of causing console being hard to read, with some effects? Maybe Hitch may add some keyword to be specified somewhere, to let glsl postprocessing program creator decide if the program has to affect 3D world, 2D hud, command console, or combinations of some of them. Just an idea.  

Fromhell, what's your opinion for evolution of glsl shaders markup (and behavior) for next OA versions? To fix current limitations...

[Hitchhiker]

Hi,
sorry I missed your last posts here.
The multiple postprocessing steps are limited to 6 at the moment but this can be increased. Basically each postprocess step will have to be a separate glsl source file. the entire program is assembled in memory at runtime (i.e. load init part of the shader, first fx code, second fx code, ..., final part of the shader ). I imagine a for-loop can be added like this in which case the max number of fx's will need to be increased. So in theory, r_postprocess could take an argument like 'color_LUT+for10+godrays+end_for+dither'. Though this approach is done per-pixel and only a color value of the pixel can be passed between different fx snippets so some elaborate effects might not work all that well (i.e. glow after color-correction... this because the two snippets would not know anything outside their scope.. they can just take the color of the pixel being processed and do their part on modifying the value.. anyway this is just a detail - the system should still be usable like this).

I am fighting a little with lightmap (actually since the beginning of glsl in OA) and if I manage to figure that one out then the only thing remains is lighting correctly the player models/entities (am having issues with finding/calculating their rotation in the world - is probably something very simple but is still hard for me).

Activating a GLSL pipeline in OpenGL is quite easy (once you have a program compiled) a simple glUseProgram(prog#) and everything after that, that is drawn, is drawn using that glsl program. So I imagine HUD and menus should be something that can be done. Just need to figure out where are they drawn. I'm thinking this might be in the QVM? If so, then I would need to check the QVM source and see if any opengl is mentioned there. Anyway.. to be seen. I know the background of the main menu can be drawn using glsl. So I think everything that is using Q3 shader has a good chance of working.

I think I fixed the //GLSL escape keyword (haven't tested yet but from the source seems it should work). So if vertex_shaders are off //GLSL is really a comment in the Q3 shaders. Once I've managed to finish all I'll have a look again at the patch I linked. Possibly I was mistaken.

Sorry about saying the changes would be integrated last December. I was way off.. But should not be long now.

[fromhell]

BTW I made a page for my shader explaining how it works

http://leileilol.mancubus.net/shaders/