Back to the previous paper Back to Maics homepage Forward to the next paper

Animating Frame-to-Frame Coherent Line Drawings for Illustrative Purposes

Maic Masuch       Lars Schumann       Stefan Schlechtweg

Department for Simulation and Graphics,
Otto-von-Guericke University of Magdeburg,
Universitätsplatz 2, D-39106 Magdeburg, Germany,

e-mail: {masuch|stefans}@isg.cs.uni-magdeburg.de

Abstract:

We present a system for rendering 3D animations in the style of line drawings. We use a highly parameterized line model in order to determine the appearance of a line. This model allows us to render characteristic line deviations that remain--in contrast to existing non-photorealistic rendering systems--frame coherent. Other inconsistencies which result from intersection and visibility changes during an animation are prevented by a path reconstruction method. Furthermore, we implemented a keyframing for linestyle parameters which enables us to extend illustration techniques like simplification of a scene or the placement of emphasis on certain objects to the field of 3D computer animation.

Keywords: non-photorealistic rendering, frame-to-frame coherence, computer generated illustrations, non-photorealistic animation, line drawings

1 Introduction

Current computer animation systems are either bound to photorealistic images or to two dimensions, what--to our opinion--is a too narrow view on the means of graphical expressions. In this paper we present an animation system that is capable of rendering 3D computer animations in the style of line drawings. The paper is organized as follows: The next section will introduce the main concept of non-photorealistic imaging. As we will see in Section 3, there are only a few approaches dealing with the creation of non-photorealistic images and even less with non-photorealistic animation. Section 4 will give a brief overview on the rendering of line drawings and Section 5 will explain why rendering an animation covers more than just the mere rendering of a number of subsequent frames. The techniques developed to preserve a strong frame-to-frame coherence facilitated the adaption of traditional illustration techniques on computer animated line drawings. This is the topic of Section 6, and finally we give an outlook on future work on line drawings (Section 7).

2 Non-photorealistic Images

Today's computer graphics research still concentrates almost exclusively on photorealistic images, although traditionally created drawings have an artistic quality that computer-generated images lack. However, up to now only very few approaches deal with the creation of non-photorealistic animations in 3D. This results on the one hand from the fact that the focus of most 3D rendering research lies on creating photorealistic images, and on the other from historical reasons and the development of 2D animation. Although in recent animated movies nearly all production steps involve the use of computers, the depicted characters remain flat shaded and the animation process is based on (digital) layers.

The use of non-photorealistic images can have essential advantages. The artist can simplify a picture by leaving out unnecessary and distorting details, and he can focus the viewer's attention on important features. Furthermore, he can stress the importance of certain parts of a depicted scene through variation of the drawing style, e.g. less important regions may be painted with bright, fading lines, while relevant parts may be depicted with strong bold lines. The resulting image is still somehow realistic, but it may differ from a photorealistic presentation in shape, color, texture and even leave out lights and shadows. The use of these techniques is very common in the field of scientific illustration [Hodges:89]. Our goal is to transfer these techniques to the field of computer animation. Here, due to the temporal nature of moving pictures, the viewer has less time to perceive a picture and it is even more important not to distract a viewer with unnecessary details.

3 Related work

First, we should investigate existing methods to create a non-photorealistic animation to see if they can be applied simply. All in all, non-photorealistic rendering systems can be subdivided into two basic types:

All 2D computer animation systems (most of them are in fact D systems), such as TIC-TAC-TOON [FBCGT:95], are strongly related to traditional animation. They are designed to create images that look like their traditional equivalents and are therefore based on two-dimensional drawings. These, however, lack exact shading and perspective correctness of the characters and objects depicted. In addition, this approach turns out to be futile, if we want to render animations using three-dimensional models.

SALESIN et al. presented an interactive system for the creation of pen-and-ink illustrations ([WinkSa:94] and [SaLiAnSa:96]). It is based on stroke textures, collections of strokes arranged in different patterns, to generate texture and tone. The PIRANESI rendering tool, introduced by SCHOFIELD et al., is based on an enriched 2D model to create ``expressive'' images ([RiScho:94] and [LanScho:95]). The extra information enables the system to apply rendering effects which are sensitive to the perspective in the image.

However, all these systems concentrate on the creation of single images and are strongly based on user interaction, which makes them unsuitable for the generation of animations, but this is just a minor hindrance. When applying one of these approaches, the user has to face one major difficulty, a lack of frame-to-frame coherence. On the one hand this results from the usage of stochastic processes to achieve a hand-crafted look. These methods are non-deterministic, i.e. no two successive frames of an animation look the same. Even if there is no motion at all, there is a disturbing distortion due to the random changes in the appearance of the depicted scene elements as no two strokes (or their digital equivalents) are drawn at exact the same position. On the other hand this effect arises if a drawing path somehow changes from one frame to the other due to, for instance, an intersection. This missing frame-to-frame coherence results in an unintended disturbance of the animation.

Up to now, there is only one solution known for the rendering of non-photorealistic animations. In 1996, MEIER presented a system for the generation of animations in a painterly style. There brush strokes, which are treated like particles, are placed on an object to form an image that resembles an oil painting. However, this approach is unsuitable for the field of scientific illustration.

4 Rendering Line Drawings

The main scope of our system, daLi!, is the generation of line drawings, i.e. graphical images which are composed of lines. More specifically, we aim for a vector-oriented description of a drawing in contrast to common pixel-oriented descriptions.

4.1 Line Styles

The visualization of computer graphics in the rendering style of line drawings is based on two crucial aspects: The placement and the appearance of the lines depicting an object. In [Schu:97] a line model was developed that allows to draw lines with different characteristics, e.g. bold lines, fading lines, lines consisting of several strokes etc. All information concerning the characteristics of a line can be summarized as style.
1. The path gives the start point, the end point and the overall course of a line. It can be attributed with information encoding width and saturation.
2. The style is a parametric description for line deviations. It is defined by a number of control vertices and an interpolation method (polyline, B-spline or chordal spline) and other attributes like the number of strokes forming a line, angle of the drawing tool, outlook of the line ending etc.

The final line results from the superposition of path and style by calculating a difference vector that specifies the deviations from the original path.


  

Figure 1: A line style is given by its start point ss an the end point se. The point l(t), , runs the scaled style definition as well as the original path. For each point on the original path the difference vector v0 describing the line deviations caused by the style is added on the corresponding point on the original path.


Furthermore, external path information can be taken into account while drawing a line, for instance if the path turns out to be cyclic or is clipped. The difference between a cyclic and a non-cyclic path is shown in Figure 2.


  Figure 2: Non-cyclic and cyclic path.



4.2 Grouping Scene Elements

Several animation techniques--such as inverse kinematics--require a tree-like hierarchical structure in order to define how groups of objects behave when they are animated. Objects within this hierarchy have defined levels of importance and can inherit certain attributes from their predecessor. The hierarchy of a scene can be built during the modeling process, or it can be generated semi-automatically later and stored in a separate hierarchy file. We extended this file in order to keep track of additional information about the drawing style of selected scene elements. For instance, the hierarchy can be used to group objects whose line style should be changed during the animation and apply the new style to all members of this group (see Figure 3).


  

Figure 3: Hierarchy of a simple scene. In contrast to the rest of the scene, the board is drawn in a line style with many strokes. Its sub-elements cone and cube inherit this drawing style.


5 Generating Subsequent Frames

As mentioned in the introduction, we concentrate on the visualization of moving non-photorealistic images. We use a high-end animation program (Autodesk 3D Studio) for modeling and setting up an animation. It is important to benefit from the power of an existing modeling and animation tool. We parse the scene information from a 3D Studio file and calculate the corresponding animation data as input for a line renderer. In order to maintain the frame coherence in an animation we have to care about two special cases: Characteristic line deviations and changes of the shape of a line in subsequent frames due to intersections.

5.1 Preserving the Frame Coherence

A first, intuitive approach of modeling different drawing styles, as for instance a hand drawing, has turned out to be unsatisfying concerning a strong frame coherence because the stochastic changes of the drawing path could not be accurately reproduced. This approach has been replaced successfully by the concept of separating path and style of a line. As the characteristic deviations of a line are now determined by a style definition and all changes in the appearance are controlled by parameters, we are able to generate subsequent frames in which the shape and appearance of all objects remain consistent. This was a substantial improvement in the visual quality of the generated animations.

5.2 Reconstructing a Segmented Line

Nevertheless, after achieving a satisfying frame-to-frame coherence on drawn lines, we encounter another disturbance of the coherence of subsequent frames, the segmentation of a drawing path from one frame to another. This effect is almost negligible in a single image but turns out to have a disturbing impact on the frame coherence of subsequent images in an animation. Let us illustrate this problem at an object, here a cone, that is moving towards a horizontal line. The path of this line consists of a single, long stroke. At some point in time, the cone will intersect with the horizontal line and break the path in two separated ones. Now the style is applied on two separated lines, what is not what we want. The smallest implication on the frame coherence appears, if we reconstruct the original path and apply the style only on the visible parts of it, which in addition requires clipping.


 

Figure 4: Reconstruction of a segmented path


The solution to this problem lies straight at hand, but turns out to be some kind of tricky: The render engine has to keep track of all lines which are segmented due to visual clipping. Then, for those lines that are drawn with a long stroke, the path has to be reconstructed, followed by a bounding correction that determines the intervals, where a stroke clipping has to be performed. This problem arises, whenever a line is drawn in a single continuous stroke. This effect is less disturbing, if many strokes are used to draw a line.

6 Animating the Drawing Style

Due to the complete parameterization of the way of drawing a line, we developed a new form of animation that allows a transformation of the visual presentation of a depicted object. In contrast to Krüger/Rist, who applied abstraction techniques on the geometrical level [KrRi:95], the geometry of the underlying 3D model remains unchanged. We extended the keyframing for the transformations of objects to animate the following line style parameters:

The changes of line style parameters are specified using keyframes similar to the keyframes used for object transformations. The animation engine--holding the entire scene data--generates modification commands for the render engine that calculates how the visual presentation of an object should behave (see Section 7). These changes allow to emphasize certain objects in a scene and are therefore well-suited for an 'explaining' animation in an illustrative context.

Line Width

As we completely control the path and style of each drawn line, we can influence the line width of a single object using the hierarchical structure introduced in Section 4.2, without influencing the line width of the others. The default line width of one can either be increased or decreased to zero, which means that the object vanishes from the scene. We encounter an effect that could hardly be reached in a photorealistic animation: The gradual simplification of a scene by the change of its visual presentation. The emphasized block in Figure 5 is clearly recognizable and remains in its spatial relationship according to the other blocks. In contrast to the first picture, where it is hardly detectable, the block in the third picture is striking by its different drawing style.
It is possible to remove elements with a line width of zero from the scene. This, however, requires an appropriate handling of objects that suddenly become visible. The opposite effect, the pronunciation of an object by doubling or multiplying its line width, can be compared to emphasizing a text by choosing a bold typeface.


   Figure 5: Gradually changing the line width for irrelevant objects in order to simplify the scene.


Line Saturation

An effect similar to the variation of the line width can be achieved by explicitly changing the saturation of the lines depicting an object. The default color of a depicted object is often black. The frame-by-frame variation of the line color allows the accentuation of certain sub-objects by displaying them in a signal-red color or by stressing the importance of an object by shifting the brightness of the drawing color for all but one scene element from black to bright gray. The pictures in Figure 6 are taken from a short animation showing the bending of the toe bones of a right foot. The focus is placed on the phalanx proximalis I [Fen:93], which is clearly visible as it is the only bone whose drawing style is not subject to changes.


  

Figure 6: Gradually emphasizing an object in a scene while maintaining its spatial relationship to other objects and deemphasizing the surrounding objects.
Unfortunately this animation is too complex for an AnimatedGif presentation in the Web. A reduced version will be available soon.


Randomness

In order to achieve a frame-consistent output, we replaced the concept of stochastic deviations to model human drawing styles by the parameterization of a line style. To our very surprise we found out that in some cases it might be useful to allow certain stochastic discontinuities when drawing a line. This, however, has been proven useful only if not all depicted objects in a scene are subject to this effect. Otherwise, as mentioned in Section 3, we encounter the disturbing effect of the missing frame coherence.


 

Figure 7: Partial release of the frame-to-frame coherence: Random changes in the presentation of the tire of the racing car creates the illusion of movement.


The hierarchical structure of the scene makes it possible to apply random changes of the drawing style only to certain objects. Each object in a scene can be selected, and its randomness can be specified by a value. This effect can take place instantly or over a certain amount of frames according to the given keyframes.

The randomness of a line is specified by assigning new coordinates to the start and end point of a line. These new points have to be placed in a certain neighborhood e0 around the original point location. For the environment e0=0 no variation is allowed, for e0>0 the point location may vary from frame to frame. The bigger e0, the more the line is subject to changes. Thus, the diameter of e0 directly controls the strength of the derivations and is scaled for the chosen output medium.

A surprising and most interesting effect is that the random variation of a line evokes the illusion of movement, even if the underlying geometry remains unchanged. We therefore call this effect an induced unsteadiness. Especially round objects that are intuitively related to movement create this impression. The effect can be intensified if the depicted object is placed in a scene without fix spatial relationships. The pictures in Figure 7 show a detailed view of four frames of an animation of a simple racing car. The tires, as well as the street are subject to random changes in their appearance. Due to the changing forms from frame to frame a viewer gets the illusion that the car is indeed moving.

This perceptual phenomenon can be categorized in terms of perception theory as apparent motion. The physiological basis for this impression can roughly be explained with certain ``quirks'' of the human visual system: The perception of motion is based on certain motion-sensitive neurons that register the presence of changes in space and time. However, the neural responses to random changes of lines are similar to those evoked by ``real'' motion and therefore trigger the impression ``movement detected''. For a deeper insight to motion perception see [Gold:96] or [SeBl:94].

Line Style

As we handle path and style of a line separately, we can as well transform one drawing style into another. In order to perform such a ``style morphing'', several preconditions have to be fulfilled. First, the two styles have to base on the same number of control vertices, otherwise one of them has to be modified by inserting spare vertices. Then, source style and target style have to be translated into the same coordinate system (same basis vectors) and some other minor parameters have to be brought into line as well. As there is no way of smoothly transforming different interpolation methods (polyline, B-splines, chordal splines) into another, the style interpolation method is the only major parameter that has to be identical.



   Figure 8: Transformation of two different styles. The geometry of the cube remains unchanged.

7 System Overview

The animation system evolved from the daLi!-system presented in [MaSchSch:96]. Basically, our system can be divided into two parts: The render engine which generates single images and the animation engine that is responsible for providing the render engine with the necessary scene information as input. The animation engine receives the following different types of data:

One part of the input for the animation engine is the 3D geometry model and possibly corresponding additional information about importance and line styles used in the rendition. The other part is a set of keyframes that describes the movement of the objects, the appearance of the lines over the time and general animation settings. An object can be of any 3D shape; it may be a light source or a camera. The animation engine can therefore be seen as a filter that provides the render engine with the necessary animation data combined with the additional information, which consist of the number of frames and settings concerning the line styles. In order to generate a sequence of images for each object, its transformations are calculated. For the given keyframes, this calculation can be done directly, whereas for the intermediate frames the object movements have to be interpolated. As a linear interpolation would lead to discontinuities that result in jerky and unsmooth object movements, the use of splines is common. In order to achieve a smooth motion, these splines have to satisfy the condition of second-derivative continuity. Furthermore, a high degree of locality is desirable in order to keep the implications for a designed motion path as small as possible when making adjustments to the animation. Here, the development of KOCHANEK is of importance. In [Kochanek:84] he introduced a spline class that satisfies the demands mentioned above using the control parameters tension, continuity and bias to specify the shape of the spline. The tension parameter controls the bending of the curve that represents the motion path. The continuity influences the ''velocity'' of the object passing through a key, i.e. the number of frames that locate the object ''near'' the keyframe, and the bias can shift the number of frames towards a keyframe or away from it.


  Figure 9: The system design for an animation system capable of rendering animated line drawings

One can think of a scene as a collection of objects. These objects may have common and distinctive features. It is possible to model common features only once and allow these features to be inherited by child objects. So, for instance, every scene object--no matter if a 3D geometry object, a camera or a light source--can be moved. Consequently, this movement can be inherited by child objects. Thus the mechanisms implemented in the animation engine can be applied to every object in the scene. Another attribute that can be inherited is an object-specific line style. This allows the user to draw objects--or sub-objects--with different line styles and to change the line style only for selected objects. As changes of linestyle parameters are quite subtle on adjacent frames, the changes can be interpolated using a linear method.

The scene is rendered by performing the line rendering pipeline, and an analytical description (technically speaking: a collection of lines) is passed back to the animation engine which paints the vector-oriented frame presentation on an abstract image. After that a special output device writes subsequent frames into a predefined directory. Currently our system supports PostScript as a resolution-independent output format and TIFF, BMP, GIF and AnimatedGIF as resolution-dependent output formats. Finally, the series of images can be combined into an animation using standard FLIC, MPEG or Quicktime encoders. All in all, we made the experience that for animations consisting solely of black and white images the FLIC format has proven to be quite suitable.

8 Summary and Outlook

In this paper we introduced a basic animation system for rendering non-photorealistic animations. An extended line model allows to preserve characteristic line deviations for subsequent frames. Visual temporal artefacts, such as a sudden change in the appearance of a drawn line due to a intersection are repaired in order to preserve the frame-to-frame-coherence of the animation.
The system extends certain illustrative techniques to the field of computer graphics. For instance it is possible to simplify a scene by gradually changing the visual presentation of scene elements. In addition, emphasis can be placed on certain objects by selectively changing their drawing style. In contrast to the former version of daLi!, the rendering algorithm is no longer restricted to non-intersecting models.

Nevertheless there are still some open problems. For instance, we would like to implement a hatching method for texturing different surfaces with different line styles. The behavior of these lines during a surface deformation should be comparable to conventional texture mapping. Furthermore, we still do not have any shadow in the scene and--as we render black and white images--with fast moving objects we encounter a temporal aliasing which is stronger than in photorealistic images. A motion blurring should be added to the images to cope with temporal aliasing effects.

Acknowledgments

The authors wishes to thank all their tutors and colleagues at the ISG, especially Andreas Raab for his crucial and inspiring work on the line rendering algorithm and Bert Schönwälder for the implementation of his work on characteristic lines.

References

[FBCGT:95] Jean-Daniel Fekete, Erick Bizouarn, Eric. Cournarie, Thierry Galas, and Frederic Taillefer. Tictactoon: A Paperless System for Professional 2d Animation.
In: Computer Graphics, Annual Conference Series, pages 79--89. ACM SIGGRAPH, ACM Press, 1995.

[Fen:93] Heinz Feneis. Anatomisches Bildwörterbuch der internationalen Nomenklatur.
Thieme, 1993. 7. überarbeitete und erweiterte Auflage.

[Gold:96] E. Bruce Goldstein. Sensation and Perception.
Brooks/Cole Publishing Company, 4th edition, 1996.

[Hodges:89] Elaine R. S. Hodges. The Guild Handbook of Scientific Illustration.
Van Nostrad Reinhold, 1989.

[Kochanek:84] D.H.U. Kochanek. Interpolating Splines with Local Tension, Continuity and Bias Control.
In: Proceedings of SIGGRAPH 84, Computer Graphics Proceedings, Annual Conference Series, pages 33--41. ACM SIGGRAPH, ACM Press, 1984.

[KrRi:95] Antonio Krüger and Thomas Rist. Since Less is often More: Methods for Stylistic Abstractions In 3D-Graphics.
In Electronic Proceedings of the ACM Workshop on Effective Abstractions in Multimedia, November 4 1995.

[LanScho:95] John Lansdown and Simon Schofield. Expressive Rendering: A Review of Nonphotorealistic Techniques.
In: IEEE Computer Graphics and Applications, 15(3):29--37, May 1995.

[MaSchSch:96] Maic Masuch, Stefan Schlechtweg, and Bert Schönwälder. dali! - Drawing Animated Lines.
In: O. Deussen and P. Lorenz, editors, Simulation und Animation '97, pages 87--96. SCS - Society for Computer Simulation Int., März 1997.

[RiScho:94] Paul Richens and Simon Schofield. Interactive Computer Rendering.
In: Architectural Research Quarterly, 1(1), 1995.

[SaLiAnSa:96] Mike Salisbury, Corin Anderson, Dani Lischinski, and David H. Salesin. Scale-Dependent Reproduction of Pen-and-Ink-Illustration.
In: Computer Graphics, Annual Conference Series, pages 461--468. ACM SIGGRAPH, ACM Press, 1996.

[SeBl:94] Robert Sekuler and Randolph Blake. Perception.
McGraw-Hill, 3rd edition, 1994.

[Schu:97] Lars Schumann. Ein parametrisierbares Modell zur Darstellung von Linien.
Master's thesis, Otto-von-Guericke-Universität Magdeburg, 1997.

[WinkSa:94] Georges Winkenbach and David H. Salesin. Computer Generated Pen-and-Ink-Illustration.
In: Computer Graphics, Annual Conference Series, pages 91--100. ACM SIGGRAPH, ACM Press, 1994.

Back to the previous paper Back to Maics homepage Forward to the next paper
Diese Seiten wurden erschaffen von Maic Masuch; masuch@isg.cs.uni-magdeburg.de.
Last modified: Febuary 1998.