Kissmy Spicoli
Snapshot Depth layer exports wrong distances
## Summary
LLViewerWindow::rawSnapshot
converts the depth buffer to linear distance with the wrong far plane, and passes window depth where the formula needs NDC depth. Depth snapshots keep the correct near-to-far order, but their values are not proportional to distance. Near the camera they read about 2 x the true distance, so a map decoded as near to draw distance gives wrong meters.
Rendering is not affected, only the exported Snapshot depth layers.
## Affected source
Found by reading the source. Stock viewers were not run.
  • Second Life viewer
    ,
    develop
    at
    373addb1cbdb
    , Depth layer:
factors,
conversion,
projection
  • Firestorm
    ,
    master
    at
    2c534084d2aa
    , Depth and Depth (24 bit) layers (FIRE-15667):
factors,
conversion, 24 bit,
conversion, 8 bit,
projection
## Cause
  1. Wrong far plane.
    The factors use
    LLViewerCamera::getFar()
    , the draw distance.
    LLViewerWindow::setup3DRender()
    builds the projection with
    MAX_FAR_CLIP*2.f
    , which is 1024 m at any draw distance.
  2. Window depth used as NDC.
    The factors come from
    1/z = (f+n)/(2fn) - ndc * (f-n)/(2fn)
    , where
    ndc
    runs from -1 to 1. The code passes the
    glReadPixels
    value
    d
    , which runs from 0 to 1. The correct input is
    ndc = 2d - 1
    .
## Effect
Computed with near = 0.1 m. "8 bit" is the Depth layer's gray value (0 to 255) over near to draw distance.
| True distance (m) | Draw 256: exported (m) | 8 bit now | 8 bit correct | Draw 1024: exported (m) | 8 bit now | 8 bit correct |
|---|---|---|---|---|---|---|
| 1 | 1.99 | 1 | 0 | 2.00 | 0 | 0 |
| 5 | 9.67 | 9 | 4 | 9.95 | 2 | 1 |
| 10 | 18.73 | 18 | 9 | 19.81 | 4 | 2 |
| 20 | 35.20 | 34 | 19 | 39.23 | 9 | 4 |
| 100 | 118.81 | 118 | 99 | 182.20 | 45 | 24 |
| 256 | 186.20 | 185 | 255 | 409.60 | 101 | 63 |
At draw distance 256, geometry at the draw limit exports as 186 m, not 256 m. Only depth 1.0, such as sky, reaches white.
## Steps to reproduce
  1. Set draw distance to 256 m.
  2. Rez a large flat panel about 10 m in front of the camera, facing it.
  3. Without moving the camera, take a Snapshot with the Depth layer, saved as PNG at the current window size. Note the gray value at the panel.
  4. With the Edit tool, move the panel exactly 10 m straight away from the camera. Take a second snapshot and note the gray value.
## Expected
The Depth layer is linear from near to draw distance, about 1 gray level per meter at draw 256. The value rises by 10 levels, give or take 1, from any starting distance.
## Actual
From the formula, the value should rise by about 16 levels (18 to 34 from a true 10 m start) and the step should depend on the starting distance. This is computed from the source, not measured on a stock viewer.
## Proposed fix
Replace the two factors. The conversion and encoding lines stay as they are. This form takes window depth directly:
1/z = 1/n - d * (f-n)/(f*n)
.
```diff
  • F32 depth_conversion_factor_1 = (LLViewerCamera::getInstance()->getFar() + LLViewerCamera::getInstance()->getNear()) / (2.f
    LLViewerCamera::getInstance()->getFar()
    LLViewerCamera::getInstance()->getNear());
  • F32 depth_conversion_factor_2 = (LLViewerCamera::getInstance()->getFar() - LLViewerCamera::getInstance()->getNear()) / (2.f
    LLViewerCamera::getInstance()->getFar()
    LLViewerCamera::getInstance()->getNear());
+ // Depth comes from setup3DRender()'s projection, whose far plane is MAX_FAR_CLIP * 2 at any
+ // draw distance, and the conversion below takes window depth d (0 to 1), not NDC (-1 to 1).
+ const F32 depth_near = LLViewerCamera::getInstance()->getNear();
+ const F32 depth_far = MAX_FAR_CLIP * 2.f;
+ F32 depth_conversion_factor_1 = 1.f / depth_near;
+ F32 depth_conversion_factor_2 = (depth_far - depth_near) / (depth_far * depth_near);
```
Behavior changes with the fix:
  • Maps get darker up close, because near distances stop reading double.
  • Geometry at or past draw distance becomes white, the same as sky, so the two can't be told apart.
  • The value is distance along the view axis, not radial distance from the camera.
## Verification
Tested on a viewer based on Firestorm, with this fix applied. Depth (24 bit), PNG, draw distance 256, camera held still, panel moved exactly 10 m:
| Snapshot | Packed value at center | Decoded distance |
|---|---|---|
| Panel at start | 641389 | 9.883 m |
| Panel moved 10 m | 1296988 | 19.883 m |
| Difference | | 10.000 m |
Sky read 16777215 (white) in both snapshots.
Not tested in the viewer: the 8-bit Depth layer, draw distance 1024 and upscaled captures. A numeric simulation of the path from true distance to export, using 24-bit depth and F32 math, matched true distance to within 5 mm up to 200 m and within about one depth-buffer step (0.6 m) near 1 km, at both draw distances 256 and 1024.