Page 1 of 2 12 LastLast
Results 1 to 10 of 16

Thread: unpremult with maya2016 mentalray problem

  1. #1
    Join Date
    Sep 2007
    Posts
    98

    Default unpremult with maya2016 mentalray problem

    When I render with mentalray 8-bit png images intended for layering in gimp or photoshop, I turn off premult to eliminate the alpha edge fringing.
    Here is a comparison render between Maya2014 on the left and Maya2016 on the right(see attached).
    The Maya2016 render results in a light outline when layered in the paint packages using "normal" layer mode. The Maya2014 mentalray render on the other hand produces a perfect fringe free composite. As a side note in Maya2014 the premult toggle is disabled when using unified sampling and only works with legacy adaptive sampling. I did try both rendering modes in Maya2016 but got the same(bright outline) results.
    In the attached image the bright outline(on the right) is the cause.
    I dont exactly know what is causing this but it makes un-premulted mentalray renders directly from Maya2016 un-usable without some image preprocessing which is what I'm trying to avoid.
    I realize Maya2016 is still very fresh but it would be great if this could be fixed.unpremult_diff.JPG

  2. #2
    Join Date
    Sep 2007
    Posts
    98

    Default

    It seems to be directly related to turning on the color management output profile of sRGB gamma boosting but
    8-bit png images require applying sRGB on output ...
    My theory is that the divide by alpha is now happening post gamma boosting rather than before.
    Can this be fixed for Maya2016 any time soon?

  3. #3
    Join Date
    Dec 2004
    Location
    Marina Del Rey, California
    Posts
    4,143

    Default

    Hmm, I thought the color management would happen "outside" the render. Depends on its relationship to the conversion to 8-bits.

    Currently, if you look at an exported scene, you will see color management run as an output shader.
    Barton Gawboy

  4. #4
    Join Date
    Dec 2004
    Location
    Marina Del Rey, California
    Posts
    4,143

    Default

    What is your primary framebuffer datatype set to? RGBA(byte), ie 8-bit?
    Barton Gawboy

  5. #5
    Join Date
    Dec 2004
    Location
    Marina Del Rey, California
    Posts
    4,143

    Default

    Breaking it down with an example for those following along, and for my own thought process ...

    A pixel with 10 samples is filtered down (say box, ie averaged), 3 of those samples hit a red (1 0 0 1) object, and the other seven hit nothing returning black. The resulting pixel is (0.333 0 0 0.333) which represents a premultiplied alpha color value. If premultiply is turned off, the pixel becomes (1 0 0 0.3333), by dividing the r value by the alpha value as Willanie referenced above. The question is whether the output shader which affects gamma, etc. operates on the 0.333 0 0 color value or the 1 0 0 value. And does it touch alpha?
    Barton Gawboy

  6. #6
    Join Date
    Sep 2007
    Posts
    98

    Default

    HI Bart,

    Thanks for your replies. Normally for png output I set the primary buffer to RGBA byte but when this weird bright edge artifact appeared I tried RGBA float 32 bit but got the same results.
    Im trying to put together a linear workflow tutorial with the new Maya2016 colormanagment but got stuck with this new issue.
    With these new features the mentalray linear workflow pipeline is now easy and almost perfect. Frustrating.

  7. #7
    Join Date
    Dec 2004
    Location
    Marina Del Rey, California
    Posts
    4,143

    Default

    And to complete the options, if you use primary fb datatype RGBA float 32 and output to exr or tif, then convert to png using imf_copy, what then?

    Or did the output put special color management information in the exr/tif?
    Barton Gawboy

  8. #8
    Join Date
    Sep 2007
    Posts
    98

    Default

    Just tried that but as you probably assumed saving an exr with a sRGB output saves the exr gamma boosted and with the bright edge.
    I turned off mayaglow just for the heck of it ... that did nothing.
    In any case, I am trying to render directly to a 8bit image so saving to an exr and doing the gamma with a post process is not an option.
    Without any output profile unpremult (premult off) works fine.

  9. #9
    Join Date
    Dec 2004
    Location
    Marina Del Rey, California
    Posts
    4,143

    Default

    I ran some tests with Maya 2016 ...

    I made a pure green (0 1 0) torus object, no lighting, using mila_material emission.
    png output, RGBA (byte) primary fb datatype. premultiply off.

    I don't get a difference whether color management is applied to the render or not. I'm using the checkbox in the Common tab named Apply Output Transform to Renderer. As I think about it, this is expected, as gamma boost applied to 1 makes no difference.

    Next I test using roughly 20% green (0 0.2 0), expecting a green value when color managed near 50%. The green value makes it up there. But now I do get a difference in the edge. After your report, this is expected.

    The alpha is untouched in all cases.

    So, this should confirm that the color management (output shader) is applied after dividing by alpha. Will confer how to address this.
    [Wait a sec... that should still be correct. After dividing, all the green colored pixels are at 20% and they get translated to 50%. The order issue may be deeper, associated with sample filtering.]

    In the meantime, I notice that when turning on Apply Output Transform to Renderer, it doesn't turn off the view transform, eg, sRGB gamma. So it gets applied twice in the Maya Render View window. When I set the Render View (View Transform) to Raw, it all works as expected. This will probably trip up some users, and we should make a note or a little blog about it. Oops, and the setting for Use View Transform in the Common Tab doesn't seem consistent. So I just set it to sRGB gamma, and the View Transform to RAW. That was consistent.
    Last edited by bart; May 16th, 2015 at 02:02.
    Barton Gawboy

  10. #10
    Join Date
    Sep 2007
    Posts
    98

    Default

    Thanks for looking into this Bart.
    The maya2016 documentation actually mentions that you need to manually turn off ViewTranform when you have Apply Output to Renderer on.
    I could not find the View Transform pull-down in Renderview at first, the window needs to be a minimum width before its actually revealed.

    I agree this is somehow related to sample filtering.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •