For the last week, I have been hacking on a new release of Mitsuba (0.3.0), which will be publicly released in another week or two. Its main new feature is a complete redesign of the material system, specifically the surface scattering models (a.k.a. BSDFs). I’ve become increasingly unhappy with the state of this very important part of the renderer and finally decided to redesign it from scratch.
I have just merged most of these developments into the main branch. Since other researches using Mitsuba might be concerned about the many changes, this post is meant to quickly summarize what is going on.
- Spectral rendering: most of the code pertaining to spectral rendering has seen a significant overhaul. It is now faster and in certain cases more accurate.
- Flexible material classes: the changes introduce a robust and very general suite of eight physically-based smooth and rough (microfacet-based) material classes. The smooth plugins are called diffuse, dielectric, conductor, and plastic. The equivalent rough microfacet models are called roughdiffuse, roughdielectric, roughconductor, and roughplastic. Please see the documentation link below for an overview of what these do.
- Material modifiers: I have added a two new “modifier” BSDFs, which change the behavior of a nested scattering model
- bump: applies a bump map specified as a grayscale displacement texture.
- coating: coats an arbitrary material with a smooth and optionally absorbing dielectric layer in the style of [Weidlich and Wilkie 07].
- Material verification: the sampling methods of all material models in Mitsuba are now automatically verified with the help of statistical hypothesis tests (using χ²-tests).
- Generated documentation: there is now a javadoc-like system, which extracts documentation directly from the plugin source code and stitches it into a LaTeX reference document.
- lookAt: Mitsuba inherited a bug from PBRT, where the <lookAt> tag changed the handedness of the coordinate system. This is now fixed—also, the syntax of this tag has changed to make it easier to read.
- Scene portability: the above changes clearly introduce severe incompatibilities in the file format. Even the old lambertian plugin now has a different name (it was renamed to diffuse to better fit into the new scheme), so one would ordinarily expect that no old scene will directly work with this release. To adress this problem, Mitsuba 0.3.0 introduces a version attribute to the scene. In other words, a scene XML file now looks something like the following:
<scene version="0.3.0">...</scene>
When it is detected that there are incompatibilities between the file version and the current release, Mitsuba will apply a set of included XSLTtransformations, which upgrade the file to the most current file format.
To upgrade to this release, simply pull from the main repository as usual (hg pull -u).
Note: you will need to update your config.py file with an appropriate file from the build directory, since there were some changes to the compilation flags.
For a peek at the upcoming documentation, take a look at the following PDF file:
Mitsuba 0.3.0 Beta Documentation
That looks promising. Will bump work for all materials ?
Btw. it seems blender plugin works ok for newest builds from graphicall. One thing thought you commented line responsible for refreshing preview of material:
context.material.preview_render_type = context.material.preview_render_type
Not sure why but uncommenting makes preview work ok.
Also there is funny bug. When I scale area lamp it’s intencity goes down. So I need to bump up intencity when scaling up area lamp. Works ok on plane emitters though.
The bump map plugin supports all shading models. You could even create multi-layer materials where each layer has a different bump map applied to it — not sure if that makes a lot of sense, but it will work :).
Regarding Blender: the plugin is in pretty bad shape, and Blender is still changing a lot. I think I’ll wait until the new render API is committed to the blender repository (I’m referring to the changes proposed by Brecht).
hi man thanks for this great renderer, i’m trying to include the latest compile on my build @ graphicall.org, i’m able to successfully compiled the x86 version, can i ask how to change the “dist” directory as i’m compiling both x86 & x64 versions.
thanks for the help
a link to my builds:
http://www.graphicall.org/bat3a
good point — I just committed an extra configuration option for the config.py file, which allows changing the name of that directory.
that helpful, thank
sorry i forgot to ask if you have a python binding for the renderer
Not currently. I think it would be relatively straightforward to cook something up with Boost::Python.
any chance in the near future? anyway i’m waiting
I’m not sure when, but it certainly would be a nice feature. Note though that just the Python API won’t do much by itself (i.e. no magical blender integration, that is a separate project)
Just looking at that documentation, the famous example of simulating refraction in a glass of liquid is given, and the solution is to use 3 different surfaces with different IORs which seems to be the most common method when looking at how people have solved the same problem with other rendering engines. I have always found this to be a rather low level and insatisfactory way of recreating the surface at which two volumes of transparent matter touch: to have to neatly separate them by hand, and create an extra in-between material for the boundary.
If the liquid inside the glass was simulated and not hand modelled, a boolean modelling operation would have to be used to calculate the correct surfaces.
I am not experienced in it, but I think that some renderers (Renderman for example), can do this sort of constructive solid geometry during render time. Although I am aware that REYES renderers calculate all of the geometry themselves before the raytracing begins.
I wanted to suggest such a feature to you because off the top of my head, I could imagine such a thing could be done quite easily using only ray-tracing. If two transparent objects intersect, one of the two inside surface’s materials could be replaced by a boundary material, and the other by a null material. The ray would have to be able to remember which object’s surfaces it has passed inside and be able to find out from the scene graph what boolean operation to perform between different objects in order to calculate the boundary material.
On second thought I can imagine this would be too many things to calculate for one little ray :S
I agree that this system can be a bit of a pain. But then ray tracing is fundamentally about what happens at surfaces, so I think it’s reasonable to expect that that is the input (i.e. it’s the job of the modeling tool to create such abstractions). It’s enough work to build a good ray tracer, so I think I’m sticking to that
Thank you for answering my question and for making this wonderful renderer!
Can we adopt the open sourced CSG implementation from PBRT?
If CSG cannot be supported, the next best thing would be adding the ability to assign materials at the polygon level.
Each boundary would have to be a separate mesh otherwise.
That really was supposed to be [two] distinct points:
* Can we port the PBRT CSG?
* Can we assign materials at the polygon level?
Even if we had CSG, you’d still have the issue of defining the IOR values at the respective boundaries.
Does PBRT support CSG operations? I don’t see anything of that sort, at least in v2.
Yeah, there are two open source CSG implementations for PBRT:
http://prideout.net/archive/projects.html#PBRT
This one seems more mature..
http://www.naadir.tk/programs/pbrt-csg-plugin
Another query I have on the subject is, say both the glass and the liquid contain different coloured absorbing media, then surely boundary material will have to be able to contain the information for the media on both sides of the surface? In order to cancel the absorption of one colour and start another. If it did not then the media colour would be inconsistent depending which side the boundary material was viewed from.
In Mitsuba currently, when a ray crosses a surface, does the surface’s material just end whatever media properties came before it?
The way I would like to imagine it would be similar to describing a paragraph of text with xml tags. The text is enclosed in a starting and ending tag that defines it as a paragraph, rather that just having single imperative newline tags. In this case each surface material is like both a starting and ending tag for describing the properties of a volume. Although I can see why only one value is needed for the index of refraction since it specifies a change of value across a boundary, rather than 2 separate values for either side.
A better system that I can imagine (off the top of my head again), is to have volume materials defined separately that describe things like absorption and scattering (and index of refraction). And surface materials just contain references to volume materials as their inside and outside mediums. The surface materials themselves can contain all the information that only pertains to the shape of the surface, such as bump, displacement, roughness and coating. That would mean you don’t need to recreate the properties of a single type of volume for lots of different boundary materials.
Hi Ian,
In Mitsuba, surfaces can specify the two media on the interior and exterior-facing side. So adjacent media with different extinction/scattering coefficients are handled without problems.
I agree that it would be nice to include the IOR directly into the medium, but I decided against it, since that will add the overhead of tracking/specifying media in scenes that do not actually use any absorption/scattering. (e.g. a clear glass object).
Best,
Wenzel
That’s great 😀
Sorry I only skimmed the documentation and misunderstood how it worked.
bat3a- I will try to update exporter when new build is ready.
Cos Wenzel will be waiting untill Brecht commits his API chages, but it may take long time, right (few moths)? I will do my best to bring at leas bump mapping and new materials. Shouldn’t be to hard.
good to hear as the currently exporter have some errors with version 0.3.0
Wenzel, you get the transmission component error when you try to apply a coating ontop of the new hk bsdf.
Yeah, it’s on the todo list. This is something that our research group needs, so it will probably happen very soon.
i tried to compile the x64 today and it gave me this error:
http://www.pasteall.org/23353
i checked the config which i copied from config-msvc2010-win64 in the build folder and it seems that it uses a lot of x32 system lib, is this okay??
Hi,
you will probably need to delete the SCons cache (I think it’s called .sconsign.dblite). The mention of x32 system libs (e.g. opengl32) is fine — these are called that way for historical reasons.
Wenzel
that’s the 1st thing i tried to do, but the same error i get, here are the console error message:
http://www.pasteall.org/23360
it seems some boost dep. are missing, i use your dep. directory, is there away around that i should do??
This message usually means that something is messed up with the compiler/build system. What does config.log contain?
i sent that in the 1st link, here it is again:
http://www.pasteall.org/23353
Ouch, sorry — not sure how I could miss that
The message indicates that something is really messed up with your build environment.
The line
cl /Fo.sconf_temp\conftest_0.obj /c .sconf_temp\conftest_0.cpp /nologo /Ox /fp:fast /D WIN32 /D WIN64 /W3 /EHsc /GS- /GL /MD /D MTS_DEBUG /D SINGLE_PRECISION /D SPECTRUM_SAMPLES=3 /D MTS_SSE /D MTS_HAS_COHERENT_RT /D _CONSOLE /D NDEBUG /openmp /nologo /D GLEW_MX /D OPENEXR_DLL /Idependencies\windows\include\colladadom /Idependencies\windows\include\colladadom\1.4 /Idependencies\windows\include\OpenEXR /Idependencies /Iinclude /Idependencies\windows\include
conftest_0.cpp
should produce a 64 bit obj file (“.sconf_temp\conftest_0.obj”). But the linker states that it’s actually 32 bit for some reason. Can you try deleting the whole .sconf_temp directory? Also, make sure that you run SCons within the right MSVC++ command shell (there is usually a different shell for the 64-bit compiler)
Best,
Wenzel
I’ve been trying to compile Mitsuba from AUR without success. I also tried to modify the PKGBUILD. I changed path to config.py, replaced glewmx with glew (because glewmx package has been removed from aur and latest glew package seems to include libGLEWmx), compiled against boost 1.43 (1.46 is not working), but nothing seems to help. I think problem is with glew, so my question is which version of glew do I need. I also tried to compile it without aur.
The AUR build documentation is pretty outdated at this point. Any recent version of glewmx will do. The boost 1.43 will be an issue! That version is *ancient* — is that something specific to your machine, or does AUR really ship with such an old version?
I build Mitsuba on ArchLinux, too. Up to now for glewmx I used an “old” version of it called “glewmx-fixed” which was of version 1.5.8-2. Like you said, this is not in AUR anymore and I tried to build with the newer glew 1.6 package without problems. Hance, your problem seems not to be related to glew(mx). For making Mitsuba work with the latest version of boost you need to add another compiler variable, telling Boost to use v2 filesystem code. This means in your config.py the line CXXFLAGS gets a new entry: ‘-DBOOST_FILESYSTEM_VERSION=2′. It should work with this change. Let me know if it doesn’t.
@Wenzel: I could update the AUR build documentation, if you want.
Thanks for offering to update the docs. I think that might be very helpful for other Arch users, as the current documentation is more of a “trap” at this point 😀
I’m also wondering: what happens when you compile with V3 of boost filesystem. Did I accidentally rely on deprecated function calls? It compiles fine on my machine, with the latest boost.
The boost v2 issue is taken care of, I have added the corresponding #ifdef to platform.h.
If you get a chance to revise the AUR build docs, I’ll be happy to include your changes.
Thanks,
Wenzel
“Can you try deleting the whole .sconf_temp directory”
i did that, i even cleaned the mitsuba directory from any unversioned files with HG tools, also i double ckecked that i use msvcsp1 x64 command.
the thing is that i can compile the x32 one very easily, but i can’t compile the x64 one, any more thoughts??
For some reason, SCons is finding the 32 bit compiler when using the 64-bit configuration. AFAIK, SCons uses the windows registry to get the configuration & path information for these two versions, so it could be a setup problem.
You could try to force SCons to use a specific version by hard-coding a path to the 64-bit “cl.exe” in the config.py file. No guarantees that that will work though, it’s just something that could potentially fix the problem.
so what should i change in the x64 config??
and what is cl.exe??
can you guide me through that
In the ‘config.py’ file (the one you get after e.g. copying build/config-msvc2010-win64.py), there are two lines (CXX and CC), which specify the name of the compiler executable (cl.exe). You could search through your Program Files folder, find the 64 bit version, and then paste the full path to the compiler into these fields.
BTW: are you by any chance using MSVC++ express? I think that might cause problems.
no i’m using msvcsp1 pro 2008, and i also have msvb.net 2010 express installed on the same machine.
i tried your solution and i get this:
http://www.pasteall.org/23416
it seems that it takes the path incorrectly, i added this path:
‘C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\cl’
and it reports this path:
‘C:\Program Files (x86)\Microsoft Visual Studio 9.0\Vinmd64\cl’
perhaps an escaping problem? You have to write “\” as “\\” in strings.
Just scrolled through the documentation. Those features are amazing! Love to see logic behind material construction, not blind paper copying.
Don’t know why I’m not wondering it is coming from an individual not a huge company.
Hope to be able to use that in blender some day.
Amazing work!
“perhaps an escaping problem? You have to write “\” as “\\” in strings.”
ok no it takes it the path ok, but still the same problem:
http://www.pasteall.org/23419
.. and what does config.log contain with this new path?
http://www.pasteall.org/23420
you need a backslash before the spaces
you mean like this:
‘C:/Program Files\ (x86)/Microsoft\ Visual\ Studio\ 9.0/VC/bin/amd64/cl’
still the same error.
i mean NOW not no
sorry
Very very nice advancements, particularly the skylights and better documentation, finally grasp what the correct way of implementing composite materials is.
hi again, i’m still not able to compile x64 bit version:
i though maybe i’m using the wrong external deps for the x64 version?
-scons 1.2.0
-python 2.6 x64
-QT 4.7.2
any thought may help?
aha! SCons 1.2 seems really old. Can you try what happens with the recent 2.x releases?
i updated to the latest scons version 2.0.1, i get the same error? anymore ideas.
maybe change boost version??
The boost version that comes with the dependencies should do fine. I’m not really sure what is going on with the builds on your system. My last suggestion would be to reinstall msvc++.
Hi again and by the way thanks for a great renderer! I got Mitsuba working in Arch with basic setup, but now I have problem with mtssrv. It gives me an error like this when I try to render with roughplastic material:
2011-08-09 19:46:01 INFO con2 [PluginManager] Loading plugin “plugins/roughplastic.so” ..
2011-08-09 19:46:02 WARN con2 [StreamBackend] Uncaught exception “2011-08-09 19:46:02 ERROR con2 [InstanceManager] Encountered an exception while unserializing an instance of “Scene”: “2011-08-09 19:46:02 ERROR con2 [InstanceManager] Encountered an exception while unserializing an instance of “SerializedMesh”: “2011-08-09 19:46:02 ERROR con2 [InstanceManager] Encountered an exception while unserializing an instance of “RoughPlastic”: “2011-08-09 19:46:02 ERROR con2 [InstanceManager] Encountered an exception while unserializing an instance of “CubicSpline”: “2011-08-09 19:46:02 ERROR con2 [class.cpp:100] RTTI error: An attempt to instantiate a class lacking the unserialization feature occurred (CubicSpline)!”!”!”!”!” – cleaning up!
When I render without any other machines it’s working just fine. (Same kind of stuff came out with sky luminaire)
thanks for the report, that sounds like a typical serialization bug. I will fix it shortly.
well, that was easy to fix
— two characters had to be changed. Could you try if it works with the most recent revision?
Now it’s working. Thanks!
Just curious when the website download page will be updated with the 0.3.0 compiled release.
I have been using the 0.2.1 release and find it really fast, and I am really looking forward to the 0.3.0 features.
0.3.0 still has a few kinks to be worked out, but it won’t be much longer
Hey, i´m waiting 0.3.0…
hi again
i reformatted my pc and installed msvcsp1 pro, and tried to build again mistuba x64 bit, the old error about cl is gone but now it don’t build anything and gives done building target:
http://www.pasteall.org/24336
Hi bat3d,
that’s kind of strange. But why are you calling “scons.py” in the mitsuba directory? Usually, one would install Scons so that it lives somewhere in the python directory, and it then gets called via “/scripts/scons.bat” (or perhaps “tools” instead of “scripts, don’t remember), which should be on the %PATH%. I’m not sure if that’s the cause of your particular problem, but perhaps something to investigate. Also, make sure that you didn’t change any of the build system files as part of your experiments in trying to get it to compile — if in doubt, check out a clean copy.
Wenzel
already done that with same result, and i did it after cleaning the repo.
Is there perhaps already some stuff in the ‘dist’ directory? Try removing that directory. Also, try removing the SCons cache file in the Mitsuba directory (named .sconsign.dblite or similar). If none of those help, what happens when you explicitly ask SCons to build a file, e.g.
C:\Mitsuba\> scons dist\mtsgui.exe
i cleaned everything and updated and still the same result, i did try (scons dist\mtsgui.exe) and got this error:
http://www.pasteall.org/24374
bizarre — what’s the content of your “config.py” file?
same as the default one, but i changed the build dir and dist dir.
aha — that must be it! Does it work with the default build and dist dir? If that turns out to have been the problem, what custom dist/build paths did you specify? Perhaps there is a bug in the build system when providing custom paths.
okay that was the case, so now i try to build the x64 bit and it starts to compile normally, but i get this error?
http://www.pasteall.org/24621
Oops, that was an error on my end, which should now be fixed. Update your repo and try again.
okay that error solved, thanks but i got another One:
http://www.pasteall.org/24629
you need to update your ‘dependencies’ repository.
Do you want to open a ticket on the bug tracker? The blog is probably not the best place for sorting out build problems.