[ Previous | Next | Contents | Glossary | Home | Search ]
GL3.2 Version 4.1 for AIX: Programming Concepts

## Smoothing Jagged Lines with Antialiasing

This discussion of antialiasing includes the following topics:

Other topics that affect or are affected by antialiasing subroutines include "Performing Depth-Cueing", "Removing Hidden Surfaces", and "Working in Color Map and RGB Modes".

### List of GL Antialiasing Subroutines

 linesmooth Specifies antialiasing of lines. pntsmooth Specifies antialiasing of points. subpixel Controls placement of point, line, and polygon vertices.

### Antialiasing Introduction

Antialiasing makes lines and points drawn on the display screen appear smooth. You can draw smooth lines using antialiasing with the linesmooth subroutine.

When lines are drawn on a raster computer screen, they are often displayed as jagged, especially if they are nearly (but not quite) horizontal or vertical. The reason is that when a line is drawn on the screen, the true mathematical line is approximated by a series of points that happen to lie on the pixel grid. Except for a few special cases (horizontal, vertical, and 45-degree lines), many of the approximating pixels are not on the mathematical line connecting the two pixels.

As an example, consider the worst case: a line that connects (0, 0) to (1279, 1). It moves up one pixel for every 1280 pixels it moves sideways. The line is rendered as two horizontal segments 640 pixels long, one having y coordinate 0 and one having y coordinate 1. Although the pixels are small, you can easily detect the jump from y=0 to y=1.

The following jagged.c example program illustrates the problem of lines that are displayed as jagged on the computer screen:

```#include <gl/gl.h>

#include <gl/device.h>```
```Int32 vert1[2] = {100, 100};
Int32 vert2[2] = {500, 0};```
```main()
{
Int32 xorg, yorg;```
```   winopen("Jagged");
doublebuffer();
gconfig();
getorigin(&xorg, &yorg);
while (TRUE) {```
```      color(BLACK);
clear();
color(WHITE);
vert2[0] = getvaluator(MOUSEX) - xorg;
vert2[1] = getvaluator(MOUSEY) - yorg;```
```      bgnline();
v2i(vert1);
v2i(vert2);
endline();
swapbuffers();
}
}```

This example draws a line from the point (100, 100) to the current cursor position. Move the cursor around and notice how jagged the lines are displayed especially when they are nearly vertical or horizontal. Even at angles far from vertical or horizontal, there is some jaggedness, but it is not as noticeable. The jagged effect that you see is called aliasing, and techniques to eliminate or reduce it are called antialiasing.

Note: Only solid, single, pixel-wide lines can be antialiased, and they cannot be used in conjunction with shading processor functions such as z-buffer, light, shade, and depthcue.

### Pixel Coverage

One way to smooth a line is to vary the coloring of the pixels along the path according to how much of each pixel is covered by the line and how much is background color. This is illustrated in the following figure. This illustration shows a short line drawn between the pixels (2,2) and (5,4). Each square represents a pixel, and the ideal line segment is the tilted rectangle. This rectangle is exactly one pixel wide, and the centers of pixels C and L are one-half pixel from the end of the rectangle. This is exactly the shape of rectangle that would be drawn parallel to the axes.

Each pixel can be set to only a single color. In the figure Antialiased Line, the ideal line hits parts of 14 pixels, and none of them are covered completely. The affected pixels are labeled A through N, and the chart at the side shows the percentage of each pixel that is covered by the ideal rectangle. Here, 87% of C and L are covered, while less than 1 % of E and J are covered.

Other pixels have intermediate-sized intersections. If pure white were color 1.0 and pure black were 0.0, a reasonable antialiased line could be drawn by coloring A and B as .04051 (almost black), C as .87847 (almost white), and so on.

#### linesmooth Subroutine

GL automatically draws antialiased lines in both RGB and color map modes. The linesmooth subroutine turns this capability on and off. The syntax is as follows:

`void linesmooth(Int32 mode)`
##### RGB Mode

If antialiasing is turned on in RGB mode, the drawing of antialiased lines proceeds automatically. The drawing hardware automatically performs what is called a "read-modify-write" operation into the frame buffer. That is, it reads the color of a pixel in the frame buffer, computes a new color for that pixel, and then writes it back into the frame buffer. The color that is computed is a blend of the pixel color and the current drawing color, the blend being based on the overlap of the line with the pixel.

To draw antialiased lines in RGB mode, you need only to call the linesmooth subroutine, and then proceed to draw as normal.

Note: In order for antialiased lines and points to appear visually smooth, gamma correction must be performed. Gamma correction is performed by loading a gamma-corrected color ramp with the gammaramp subroutine. Gamma-corrected ramps are nonlinear ramps from dark colors to light, usually specified with a power function, but sometimes logarithmically. Gamma correction is required to take into account nonlinearities in the display electronics, in the phosphorescence of the display phosphors, in the human visual system, and in the pixel coverage sampling algorithm. If gamma correction is not performed, lines do not appear smooth, but exhibit a roping or braiding effect, as if the line were composed of separate, intertwining strands.

The gamma.c example program demonstrates how a gamma-corrected ramp may be constructed and loaded on the system. A gamma correction factor in the range of 2.4 to 2.7 is suggested.

##### Color Map Mode

Drawing antialiased lines in color map mode requires greater involvement from the application. In particular, a special color map, called a color ramp, must be loaded. To understand the color ramp, it helps to know what the hardware does in color map mode.

When antialiasing is turned on, the hardware replaces the low-order 4 bits of the current color index with a number ranging from 0 to 15 that represents the fractional pixel coverage. When a line is drawn, the index values stored into the frame buffer are these modified color values. As a result of this replacement of values, the images do not look correct on the screen. To alleviate this problem, the low-order 4 bits of the color map must contain a ramp from the foreground color (the color of the line) and the background color (the color to which the frame buffer was cleared).

In the following example, 16 consecutive cells in the color map are filled with colors that are uniformly spaced between black and white:

```for (i = 0; i < 16; i++)
{
mapcolor(144+i, i*17, i*17, i*17);
}```

This maps color entries 144 through 159 to shades of gray having equal red, green, and blue components of 0, 17, 34, 51, 68, 85, 102, 119, 136, 153, 170, 187, 204, 221, 238, and 255. The starting number, 144, is not completely arbitrary. For antialiasing to work, the starting number must be an exact multiple of 16. The number 144 is large enough that the color map entries affected do not conflict those lower entries in the color map used in many simple applications.

If antialiasing is turned on with the linesmooth subroutine, the percentage each pixel is covered by the ideal line is approximated by the hardware, and that number is scaled uniformly into the range of 16 consecutive shades to find the color to paint the pixel. The smooth.c example program that follows is a slight modification of the jagged.c example program. It behaves similarly except that smooth lines are drawn if the left mouse button is pressed.

```#include <gl/gl.h>

#include <gl/device.h>```
```Int32 vert1[2] = {100, 100};
Int32 vert2[2] = {500, 0};```
```main()
{
Int32 xorg, yorg, i;```
```   winopen("smooth");
doublebuffer();
gconfig();
getorigin(&xorg, &yorg);
for (i = 0; i < 16; i++)
mapcolor(144+i, i*17, i*17, i*17);```
```   while (TRUE) {
color(BLACK);
clear();
if (getbutton(LEFTMOUSE)) {
linesmooth(TRUE);
color(144);```
```      } else {
linesmooth(FALSE);
color(WHITE);
}
vert2[0] = getvaluator(MOUSEX) - xorg;
vert2[1] = getvaluator(MOUSEY) - yorg;```
```      bgnline();
v2i(vert1);
v2i(vert2);
endline();
swapbuffers();
}
}```

Before anything is drawn, the color map entries between 144 and 159 are loaded with shades of gray. Then, in the main loop, the left mouse button is examined, and the color is set either to white, if the linesmooth subroutine is turned off, or to 144 if it is turned on.

You can draw multicolor antialiased lines on arbitrary multicolored backgrounds if you load suitable ramps. The alias.c example program partitions eight adapter bitplanes into two that store the background image and two that store the line colors, with the remaining four being overridden by the antialiasing hardware. The color map is loaded with ramps from all possible line colors to all possible background colors. The alias_back.c example program illustrates the drawing of antialiased lines of multiple foreground colors on a monochrome background. The alias_fore.c example program, in contrast, illustrates drawing antialiased lines of a single foreground color on an arbitrary multicolored background.

The accurate selection of ramp intensities is imperative for effective visual appearance. The ramp must include corrections for nonlinearities in the electronics, the screen phosphors, and the human eye. Such corrected ramps are usually referred to as gamma ramps. In the alias.c example program, the colors are gamma corrected. The gamma ramp hardware, however, is not used; rather, the corrections are folded in with the color ramps.

Notes:

1. The Supergraphics Processor Subsystem does not support antialiasing in color map mode.
2. The High-Performance 3D Color Graphics Processor does not support antialiasing in RGB mode.

Other topics that affect or are affected by antialiasing subroutines include Performing Depth-Cueing, Removing Hidden Surfaces, and Working in Color Map and RGB Modes.

### Improving Intersections

When antialiased lines cross each other, there may be some undesirable presentations because the hardware is interpolating between the line color and the background color. It does not take into account any other lines that may already be drawn. Thus, if a program draws a series of smooth lines all crossing each other at the same point, you might expect the intersection to be almost entirely the color of the line. However, the hardware continues to average the original background color into each new line. The following example program, linesmooth1.c, which draws a multirayed star, illustrates the problem:

```#include <gl/gl.h>

#include <math.h>

#define PI 3.14159265
```main()
{
long i;
```   keepaspect(1, 1);
winopen("linesmooth1");
ortho2(-1.0, 1.0, -1.0, 1.0);
for (i = 0; i < 16; i++)
mapcolor(256+i, 17*i, 17*i, 17*i);
color(BLACK);
clear();
color(256);
linesmooth(TRUE);```
```   for (i = 0; i <= 180; i += 10) {
move2(-x, -y);
draw2(x, y);
}
sleep(20);
}```

The first ray drawn contains pixels blended with the background color to smooth the line. Each successive line's color is also blended with the original background color and is unaffected, even close to the intersection, by the color resulting from the previous line's blending. The pixels surrounding the center of the star are no brighter than any other line portion that has been averaged with the background color. Antialiased lines in color map mode have no effect on the color of successively drawn lines.

The interpolation problem is also apparent when two lines that are nearly parallel cross each other. One solution to this sort of problem can be obtained by using the z-buffering hardware for color comparisons. Instead of using the z-buffer to draw objects that are nearest the viewer, the z-buffering compares each new pixel color with the existing color for that pixel and causes the brightest color to be drawn. This is not a perfect solution, but it gives good results. The point at the intersection of two lines is at least as bright as the lines were originally.

### Example Program With and Without Color Comparison

The following example program, linesmooth2.c, draws a pair of intersecting lines both with and without the color comparison. As it runs, press the left mouse button to see the effects of a z-buffer type color comparison.

```#include <gl/gl.h>

#include <math.h>

#include <device.h>

#define PI 3.14159265
```main()
{
long i;
```   keepaspect(1, 1);
winopen("linesmooth2");
doublebuffer();
ortho2(-1.0, 1.0, -1.0, 1.0);
RGBmode();
gconfig();
frontbuffer(TRUE);
wmpack(0xffffffff);
cpack(0);
clear();
frontbuffer(FALSE);
cmode();
gconfig();```
```   for (i = 0; i < 16; i++)
mapcolor(48+i, 17*i, 17*i, 17*i);
linesmooth(TRUE);
for (i = 0; i < 1000000; i++) {
if (getbutton(LEFTMOUSE)) {
zbuffer(TRUE);
zsource(ZSRC_COLOR);
zfunction(ZF_GEQUAL);
} else {
zbuffer(FALSE);
}
color(BLACK);
clear();
color(48);```
```      move2(-1.0, 0.0);
draw2(1.0, 0.0);
move2(-1.0, -sin(i/30.0)*.05);
draw2(1.0, sin(i/30.0*.05);
swapbuffers();
}
}```

The linesmooth2.c example program has several interesting features. The hardware that does the comparison to find the largest color index actually compares the entire 24-bit contents of the pixel. The same physical memory is used for RGB information as is used for color indexes, but the color index data occupies only the low-order 12 bits. If there is garbage in the high-order bits, perhaps left behind by previous windows, this has no effect on the displayed values. Because the comparisons are made on the full 24 bits, the garbage contents can have an effect on the results of the comparisons.

For this reason, it is a good idea to clear out all 32 bits completely before starting. This is done by putting the system in RGB mode, and then writing a zero (for all 32 bits) with a writemask of 0xffffffff (all 32 bits enabled). This guarantees that everything in the high-order bits of the frame buffer is set to zero. Because the linesmooth2.c example program is double buffered, this must be done for both the front and back buffers by setting frontbuffer(TRUE) .

### Enabling Color Comparison

Three things must be done to enable the z-buffer color comparison properly:

• The z-buffer must be turned on by setting zbuffer(TRUE) .
• The z-buffer must be set to do a color comparison instead of a depth comparison with zsource(ZSRC_COLOR) .
• The color comparison must be changed by setting zfunction(ZF_GEQUAL) .

The comparison function is ZF_GEQUAL (it could also be ZF_GREATER), so that the new value is written into the pixel if its value is greater than or equal to the current value. (In standard z-buffer comparisons, values are written if they are closer to the eye: ZF_LESS or ZF_LEQUAL.)

#### pntsmooth Subroutine

The pntsmooth subroutine draws antialiased points. The syntax is as follows:

`void pntsmooth(Int32 mode)`

#### subpixel Subroutine

The subpixel subroutine controls the placement of point, line, and polygon vertices in screen coordinates. The default value of the bool parameter is False, causing vertices to be snapped to the center of the nearest pixel after they have been transformed to screen coordinates. The subpixel subroutine is typically set to True while smooth points or smooth lines are being drawn.

The syntax is as follows:

`void subpixel(Int32 bool)`

### Depth-Cueing

It is possible to draw lines that are both antialiased and depth-cued in color map mode, but not in RGB mode The depth-cueing hardware maps the transformed z component linearly into a region of the color map. If the color map is arranged as a series of 16-entry ramps, each beginning at a multiple of 16, and each mapping a range from the background color to a series of brighter and brighter colors, depth-cueing and antialiasing work together.

First, the depth-cueing calculation gives a position within the map corresponding to how bright a fully illuminated pixel on a line should be. Then the antialiasing hardware calculates a percentage pixel coverage, and the appropriate entry from the 16 color range is chosen. Although it is an approximation, this method gives reasonably good results.

[ Previous | Next | Contents | Glossary | Home | Search ]