A small guide about working with textures in Photoshop for DOTA2 Workshop with using my script. It turned more technical, but I like it…
Mostly we will talk about general principles, new features of the Source2 Workshop Tools and tricks with my texExport script for Photoshop.
For example, I chose this crazy Squig that was made for DOTA2 Warhammer Contest. I made only textures here but it will be a good sample for explanations.
For the other item types, all will be the same…
First, you should have final geometry (lod0) and all baked maps (Normal map, Ambient occlusion etc.).
Just take my template for the DOTA2 textures, place it into the project/item folder and rename it. In my case it will be squig.psd.
It is desirable that the name of the source PSD-file coincide with the name of the working session/item in the Item Tools.
Then open this file and set the desired resolution (x4 from ingame in my case), Add the Normal map to the appropriate group, save it and launch texExport script.
At the output we get the primary set of texture maps for the correct compilation asset in DOTA2 Item Tools:
Group names (suffixes output files) in the template are assigned so that the Item Tools independently pick up all the channels, as soon as you enter one of these (for example _color).
Here begin tricks:
For convenience, we can remove the restriction on the resolution output textures in order to better control the seams:
This operation must be repeated for each channel.
Of course, this is only a temporary setting, and after recompiling Item Tools through all the values will be reset.
And the most interesting… You may keep running current session or game tools.
Take our source PSD and copy it into the folder:
content\ dota_addons\ workshop_testbed\ materials\ models\ items\ courier\ squig
Inside you will find vmat-file that contains main shader settings, tga-files for each shader channel and txt-files with values of the output resolution (If you already did the previous trick then here will be no parameter).
TGA-files are renamed copy of your texture and names generated by the compiler as a [session name]_[channel].tga. (That’s why was a recommendation about the name of the PSD-file.)
workshop_testbed directory is the folder of the Item Tools modification. And the thing is that the game engine detects any changes of the source files in
content and automatically updates compiled binary resources used by the game at
game\ dota_addons\ workshop_testbed\ materials\ models\ items\ courier\ squig
Therefore, we are closing the old PSD and continue to work in the new. And now, each pass of the script will update these same source files because the names of the channels in the template are consistent with the assigned by the compiler. This means that asset will be updated in the game in real time … Well, after a few seconds =)
If you wish, you can hide or delete unused channels in the PSD, but I leave it.
All this allows you to open the item or to put a set on the hero in the Model Editor, move a window in any place and check in out how shader actually behaves in the game, in high definition, and even with animation. Without regular compilation by Item Tools or using inadequate shaders in third-party programs.
So, we have a working game asset with a solid Color, Normal map and some average values of masks from the template:
The first and most important stage – the partition by the materials. Not color, not the value, namely materials – complex entity, with special physical characteristics and response to light. (Work in color channel)
Of course, it would be better to begin working with transparency (trans), but in my case, it is not used.
It is important that the materials were not too much, otherwise, you just get confused, or the result will become a noise (do not forget about the real output texture size). That’s why, we should separate only the general entities, such as wood, metal, glass, leather and so on. Each material is denoted by a Solid Color Fill Layer (this will allow you to quickly change the color), grouped and assign it a name.
In this case, the color will be as simple indicator first. But for convenience, you can assign colors that are close to final.
Then, based on the Normal map and other baked maps, create a mask for each material. It is important that borders of each material were in line with the Normal maps and has clear junction on the UV-layout seams.
To speed up the process, you can use ID-maps.
The sequence of overlay materials on top of each other, I usually do on the basis of convenience, further creating shader masks: metal parts above.
If the material assumed sub-materials (such as different types of wood or metal) – just add a new color in the material group and add a mask to the layer. The main thing is not to get involved and do not multiply entities… All this will go into the nuances and will be redrawn after.
After that, I advise immediately to make a full pass over all shader masks for each material, as some channels will greatly affect the final result (for example, metalnessmask).
Due to the fact that we have already fixed zone for each material on the Color, we need only create a Solid Color Fill Layer for the material and copy masks.
Sub-materials easily add through a Clipping Mask.
That’s it. Now you can safely do the final color. You can use any tools depending on the task or habit – it could be a procedural or hand-paint methods, by model or by canvas, this is not so important because we have the main goal…
When the Color-map is ready, we can also use it to add depth and variety of shading masks by blending the colors in grayscale with the material layer.
Values of the solid shading masks will become a reference point for the separation of the details by levels.
At the end we should replace first PSD-file by the new, update all TGA-files using texExport and check the results in the final game resolution.
Of course, you can do without this copying back and forth, but I love it when the project is stored in a separate directory… Also I love backups =^__^=
More recommendations to the artistic and technical part you can read in the official Guides from Valve.
Thank you for attention. See you and good luck!